logo Sign In

Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)

Author
Time
 (Edited)

Hello everyone,

Since I have a background in image analysis and enhancement, I thought I would have a go at enhancing the GOUT to high definition using a super resolution algorithm. Rather than simply upscaling the frames, a super resolution algorithm uses neighbouring frames to reconstruct a high definition frame.

The goal of this project:

Create a 1080p high definition anamorphic version of the GOUT in the form of one 25 GB mkv file for each film.

Next to having significantly increased detail, the upscales will have significantly less grain and aliasing.

The films will have the GOUT color palette, and will stay true to it’s source, the master tapes that spawned the Star Wars Trilogy - The Definitive Collection laserdiscs, and of course the GOUT DVDs themselves.

The source:

The NTSC versions of the 2006 Star Wars bonus DVDs aka the GOUT.

For more information about super resolution:

http://en.m.wikipedia.org/wiki/Superresolution

Author
Time

Sounds wonderful! I will surely follow your progress here.  :-D

Author
Time

Sounds very scientific. .....and slightly magical.  Maybe you can apply a super whizzy algorithm thingy to the PT to make them better movies.

Author
Time
 (Edited)

Doesn't Video enhancer (http://www.infognition.com/videoenhancer/) use the same technique? I tried it on a short sample from the GOUT and it did an OK job - it might make it look better on your HDTV than the built in hardware upscalers, but it's never going to create an instant OT Bluray version that I'd be happy with.

TheStarWarsTrilogy.com.
The007Dossier.com.
Donations always welcome: Paypal | Bitcoin: bc1qzr9ejyfpzm9ea2dglfegxzt59tys3uwmj26ytj

Author
Time
 (Edited)

Yes, it is a similar principle. Of course you will never be able to obtain the true HD source material. It will drastically improve the quality of the GOUT though with a frame detail somewhere between standard definition and true HD. In my view this is the closest you can get to a HD original trilogy, outside of the massive achievement of Harmy's despecialized edition. I do believe it is an interesting experiment in itself to see what the maximum quality is in terms of detail that can be squeezed from those three imperfect bonus DVDs.

Author
Time

Not to be all Negative McDoubtypants, but isn't this pretty much what DJ and You_Too are already doing (and partly already done with), except in 1080p instead of 720p?  And since even the Blu-rays don't actually have more than 720p worth of fine detail, do you really expect 1080p is going to get you that much more from the GOUT?  And finally, what about all the other stuff DJ and You_Too are doing aside from the simple upscale: image stabilization, fixing bad IVTC frames, fixing damaged frames (glue marks, etc), color correction, etc?  I mean, I'm not cheerleading their project over yours or anything--the more the merrier and all that--but I hate to see the wheel get reinvented.

Project Threepio (Star Wars OOT subtitles)

Author
Time
 (Edited)

As I'm sure DJ and You_Too have experienced there are quite a few methods of upscaling. The devil is in the details as they say... I'm mostly interested in the upscale of the GOUT, and will leave the restoration, which is a somewhat subjective process, to others that are more capable. 

Author
Time

Christ, you'll literally do anything instead of working on Detox won't you?

Will be watching your progress, haha

Author
Time
 (Edited)

The SuperResolution algorithm used in FURNACE is fantastic. You might want to take a look at that if you haven't already.

The GOUT comes with lots of baked-in aliasing which is surprisingly very good for temporal upscaling algorithms like this.

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time

Was something like this used on the pre-2010 episodes of The Simpsons for the fxx version? Granted, they also cropped and stretched them, but they definitely used some sort of super-duper upscaling algorithm to make new transfers out of the existing ones in lieu of an actual fresh hd scan of the original elements.

Speaking of the prequels, Bill Hunt theorized in his review of the bd set that something like this might come along to do a deep analysis "guess and render" for a new 4k version. It's already been almost four years since then.

Author
Time

Something similar could indeed have been done to those Simpsons episodes. If this project is successful, I may proceed with other projects, like those PT deleted scenes.

I hope I can get some screenshots on here soon... 

Author
Time

What software is being used and is there a user friendly one?

Author
Time

Nice, I wanted to do something similar, but did not have the software.

Please post some screenshots as soon as you can!

Darth Id on ‘Why “Ben”?’:

And while we’re at it, we need to figure out why they kept calling Mark Hamill’s character “Luke Skywalker,” since it’s my subjective opinion that his name is actually Schnarzle Shnuzzle.  It just doesn’t make sense!

Damn you George Lucas for never explaining why they all keep calling Schnarzle “Luke”!

Damn You!!!

Author
Time

At this point I have two options:

1) Use commercially available software

2) Write my own super resolution algorithm

I will first test the commercially available software, which in this case involves two choices: STOIK video enhancer and Infognition video enhancer. Both are relatively easy to use. My first tests indicate the Infognition algorithm clearly outperforms the STOIK algorithm. However, the Infognition algorithm sometimes results in clearly visible artifacts. I'm now working on a way around this. The preliminary results look promising. I think I will be able to show some screenshots pretty soon. To be continued...

Author
Time

One more thing for those of you that are interested. Since a super resolution algorithm generally predicts structure (details) from neighbouring frames, it generally doesn't predict noise very well, since many types of noise have no structure. So a good super resolution algorithm should also act as a noise filter...

Author
Time
 (Edited)

The problem with the artifacts has been fixed. The first screenshots for Star Wars should be up later today. I must say I'm really satisfied with the results. The amount of detail that is reconstructed is pretty impressive. It really looks like a high definition video. 

Author
Time

Cool,  this is using Infognition and the work around? What was the work around solution?

Author
Time
 (Edited)

The solution is to remove the black bars in the frames. The black bars introduce a ghosting effect under some conditions, resulting in vague horizontal lines. 

I now use the following procedure:

I start from the PAL versions of the GOUT.

1) Convert the dvd to a high quality mkv (8000kbps) without the black bars, resulting in a 720x326 video. I use Handbrake for the conversion.

2) Apply the super resolution algorithm in Infognition, and upscale to 2160x1296, with the actual video size 2160x815. This will also reintroduce the black bars (this does not affect the super resolution). The output is raw video (avi), so the file is very large (600GB). You can also use compression codecs you can download via the Infognition site. However, I prefer to post-process the upscaled video in Handbrake.

3) Convert the raw video to a mkv file with the proper size (1920x1080, 80000kbps) using Handbrake, resulting in a 20GB per movie HD GOUT. 

In all the steps I copy the original audio, so it's an exact copy of the GOUT PAL soundtrack. 

Author
Time

Advice: you should use NTSC DVD for EP IV and V, and PAL for EP VI, to have the best quality.

May you post somewhere five consecutive, lossless frames of the source before upscaling, and the third frame upscaled? I'm just curious to see the difference between the process you are using and mine! :D

Sadly my projects are lost due to an HDD crash… 😦 | [Fundamental Collection] thread | blog.spoRv.com | fan preservation forum: fanres.com

Author
Time

Well, if he's not interested in correcting bad IVTC frames, he's pretty much stuck with PAL.

Project Threepio (Star Wars OOT subtitles)

Author
Time
 (Edited)

This will look like shit and I know exactly why.

I've tried this before, more than a few times and I had to come to terms with the following as I waved my white flag.

Standard definition footage of any kind does not scale kindly to HD either through simple upsizing or any number of super-resolution techniques. It introduces shimmer and shake where the video has been de-interlaced, it provides artificially sharpened edges with no boost to detail in any other areas which further creates a strange and disconnected look to it all and unless you're using the upscaled file as a template for some serious post-processing work such as colour grading and grain addition, it's not gonna look markedly different from simply playing a carefully restored SD version.

The reason I enjoy and support projects such as Harmy's Despecialised Edition is that they are utilising the best publicly available source (the blu-ray) to pull back the various Special Edition enhancements and really focus on providing high quality, high definition versions of what audiences saw in '77, '80 and '83. They're already working with a high resolution master and bringing in high resolution elements from various other sources with a keen eye on keeping the standard definition elements to a minimum.

So I suggest, as much as I wish I could get behind this project, to give up and lend your technical and creative skills to other projects utilising the best sources we have.

Author
Time
 (Edited)

Here is the first clip. I had a few new challenges, but more about that later...

Update: Clips removed to make space for new clips.