Star Wars GOUT in HD using super resolution algorithm (* unfinished project *) — Page 15

@ Darth Lucas

The flickering of the stars is actually part of the source (when R2D2 & C3PO are in the pod). You can see the same thing in the Spline64Resize sample I posted (and in the dvd itself).

@ Zyrother

Thanks! I think it looks pretty good too. :-)

@ Spaced Ranger

Thanks for the suggestion. Although the function seems to work very well for the dark halos, the light halos are much less affected (I've tried multiple settings). Also, the edges are generally softened, which to a certain extend visually reduces the detail enhancement. I will try some other settings to see if the result improves. 

I'm genuinely amazed how much detail people are able to pull from this supbar LD scan from two decades ago.

To give you an idea how the HD GOUT is created, I will roughly describe the steps to create it.

First a raw super resolution upscale is created with the super resolution script I wrote:

http://screenshotcomparison.com/comparison/123952

This raw upscale has the enhanced detail, but also has enhanced grain and unwanted noise. Therefore in the next step the raw upscale is filtered with a non-linear filter I wrote to get rid of the grain and excess noise:

http://screenshotcomparison.com/comparison/123956

Some of the detail is removed in this intermediate step, but this can be recovered. By applying the same filter on the Spline64Resize upscale and adding the difference to the filtered super resolution upscale the final result is obtained:

http://screenshotcomparison.com/comparison/123957

As you can see below, the final result has the detail of the raw super resolution upscale and the grain and noise structure of the original GOUT:

http://screenshotcomparison.com/comparison/123815

DrDre said:

Although the function seems to work very well for the dark halos, the light halos are much less affected (I've tried multiple settings). Also, the edges are generally softened, ...

DeHalo_alpha() settings are very non-intuitive and there is little clear idea what those numbers actually do. (It's been an "alpha" for a decade, and probably will remain so 'til the end-of-the-age.  :D  So we're stuck with puzzling it out ourselves.)

Settings I tried in that link were applied to a letterboxed standard definition source. For your high definition source, I can guess the radius would be generally increased for that addition coverage -- keeping in mind the independent x,y settings to reduce influence in a direction less needed. Same with dark,light halos being independently adjustable, as needed.

The sensitivity settings probably are key. My original tests early in the THX 1138 preservations thread were way too strong (caused noticeable smear-softening). Later, I discovered what was happening and significantly back them off for better effect.

At best, it's a band-aid that might be better than a gaping wound.  :)

.

P.S. - Would you also post your non-compressed (.PNG) base-picture used for your above HD GOUT creation steps demonstration? Thanks.

P.P.S. - Also, I suggest having DeHalo_alpha() as step 2b. This will ensure it applies brightness manipulation only to legitimate targets and not tricked by the original "grain" into processing other areas. Further, when grain is restored picture-wide, that will help mask any weakness in the DeHalo_alpha() result.

@ Spaced Ranger

Which base picture are you refering to? The standard definition original?

Here's the frame for the standard definition original (712x274):

https://drive.google.com/file/d/0B8_LYKyZDiajdFBGRmxCWEtIVm8/view?usp=sharing

Yes -- the original standard-definition frame to which was applied the Super Resolution upscale. (I wanted to compare your results from the starting point.)

[Thanks for the picture -- I just saw your update w/link when ready to post this!]

.

BTW, thanks for posting that sequence of steps! While reviewing them, what came to mind was the wave-particle theory of light. So I quickly worked up a proof-of-concept in a paint program, which I thought you might find interesting.

The reason for "noise" on film (most noticeable when under-exposed) is that light can be said to behave as "particles". Light particles travel to and randomly hit the film, but in density proportional to it's "brightness". Less light means less hits or bits of exposed film -- dark and grainy (fewer exposed film grains interspersed between lots of unexposed film grains). As brightness increases, ever more hits increase the exposure of the film -- lighter and less grainy (more exposed film grains compared to fewer unexposed film grains). And the pattern of exposure is random from frame to frame, even if successive frames look the same. The result? Darker is "noisier" and lighter is "cleaner".

Why does film (like that with which we work) seem so uniformly grainy? From original negative, generational copies impresses their images, including the now built-in "graininess", on each next generation. The dark areas really don't get much more grainy (just more randomized). The light areas, however, accumulate ever more noise as part of it's picture. After enough generations, they saturate in graininess just like the dark areas.

Therefore, when "de-noising" a film to look as pristine as the original negative, the greatest strength should be applied to the lightest areas and the least strength to the darkest areas.

Also, approaching it from the opposite direction, when "re-graining" a film, the opposite is followed -- the least strength should be applied to the lightest areas and greatest strength should be applied to the darkest areas.

So ... for your re-graining, I wanted to see how it would look if the grain was added in proportion to it's brightness: most in dark; least in light. Thus ..

The 1st picture (150% magnification) is your step 3 final Super Resolution v7 -- short-labeled here as "regrain" & "regrain compare". In the paint program stack, it is the only one showing (the "open eye"). I would blink that solid layer on and off to compare before and after.

The 2nd picture has all the other "eyes" open. It used your step 2 Super Resolution v7 with grain/noise removal, here short-labeled as "no grain", and your "regrain" to isolate your added grain, to be put back a new way:

* Group inverts no grain
* Group - add grain dodges Group into no grain = isolates (added) grain from step 3
* Group - add grain burns isolated grain back into no grain

dodge and burn are paint program functions that work based on image brightness -- perfect for pulling out grain/noise and putting it back in, both in light proportion. Accuracy is not precise, but for quick test in a paint program, you can get the idea.

      
SW GOUT SuperResolution natural regrain [click for full picture]

@ Spaced Ranger

Very interesting indeed! The non-linear filter I used in the 2nd step indeed filters the lighter areas more strongly than the darker ones. Because I wanted the grain to more or less match the original grain in the GOUT, I filtered a standard upscale in the same way as the super resolution upscale. The grain I want to add is then the difference between the simple GOUT upscale and the filtered simple GOUT upscale. 

By tomorrow the first 7-8 min of Star Wars should be finished. I will post the first part here as another sample. The script now takes up so much memory I have to process the movie 10.000-15.000 frames at a time (taking about 48 hours to process). Here's one of the iconic shots from the opening from Star Wars in yet another screenshot comparison ;-):

http://screenshotcomparison.com/comparison/124203

Aside from the enhanced detail, it's interesting to note how much brighter the stars are in the super resolution version.

I am not sure if it is just my eyes, but it seems that from the start of the opening scroll, till the Star Destroyer passes, that one continuous shot seems much cleaner than the rest of the GOUT. Because that original scroll is not from the Laserdisc master.

So whoever put the original scroll in, obviously had access to a clean print, because the altered scroll was implemented in 1981, and was never on any home video release.

I might just be going crazy, ignore me if you wish. :P

@ dclarkg

Thanks, I will keep at it. Luckily there are plenty of members here with good advice that has certainly benefited the project. 

@ Intruder

Yes, I use the Infognition plugin for Avisynth, but wrote a custom script for combining the super resolution upscales in different color spaces, and in different directions to minimize artifacts, and get more details at the same time.

I actually have a question. In preparing the GOUT Star Wars for processing I cropped the black bars, which resulted in a video with 714x274 resolution. I then cropped the left of the frame with two more pixels, because of the weird black bar in the film. So my final video is 712x274. At this point I'm using 1280x544 as the resolution for the upscale (2.35:1). However, 1280x554 is closer to the actual aspect ratio of the GOUT DVD. Then again the AR for the NTSC GOUT DVD seems a bit off (2.31/2.33:1), and definitely different from the PAL DVD. I'm now wondering what the most accurate representation is for the true aspect ratio of the film? Any thoughts?

I'd say, the more picture, the better. Even if those weird anomalies exist on the sides. Granted, all DVDs are authored with about 8 pixels cut off the left and right sides. It was filmed with the foreknowledge that many theaters would playback the film with slight over scan to essentially crop out anything at the very edge of the film strip. Such as dirt, hair, or physically broken pieces of the film strip.

DVDs adhere to a standard that honors that slight over scan. But in the HD age, a 1 to 1 pixel ratio is more sought after. Totally up to you. Cropping out that small amount is actually what the cameraman/ director intended, but it is more picture that people may want to see. So the 1280x554 is a better choice, since I think even the Blu Rays and DVDs show less picture than the GOUT.

At first I did leave the over scan, only the Infognition super resolution plugin doesn't like black borders (which the over scan is for most of the film), leading to edge artifacts. So I decided to cut it. However, even with the over scan the aspect ratio of the NTSC DVD seems wrong. The planets have an oval shape, and people have slightly elongated faces. The display aspect ratio of the DVD is 2.316:1 with the over scan. So in that case I would say the true aspect ratio should be 2.35:1 without the overscan, which would mean 1280x544 is the better choice.

It was filmed in 2.35.1, and even though it was authored in a letterbox format on the GOUT, you would assume the AR would be correct. Thats weird that is was still stretched too much.

Avoiding stretching out the picture is a must. So if 1280x544 offers the correct AR that keeps everyone from having stretchy faces, then I would go with that.

Damn non-anamorphic DVDs, makes everything more difficult :P

Random question. Are you doing anything with the audio? Or are you just going to leave it? I know many people voiced complaints about the audio, but I think it sounds fine. I think the original theatrical audio on the 70mm print was 6 track stereo audio, which the GOUT is more faithful too I believe.

I was planning on leaving the audio as it is. So, it will be an exact dublicate of the audio on the DVD. I think it sounds fine as well. :-)

Here is the sample for the first 10 min of Star Wars, which includes the intro, title track, Tantive IV scenes, and the arrival on Tatooine. Hope you like it ;-):

Sample removed to make place for new samples.

CatBus said:

DrDre said:

I was planning on leaving the audio as it is. So, it will be an exact dublicate of the audio on the DVD. I think it sounds fine as well. :-)

Bah, 1993 audio with 1977 video?  Such an anachronism shouldn't be allowed to survive.  In my opinion, while I admittedly don't like the 93 audio anyway, it should at least accompany the 81 crawl to maintain some semblance of historical... uh, whatever. 77 video must have one of the 77 audio mixes.

Regarding the AR, did you crop off the left- and rightmost 8 pixels before stretching?  Because those 16 pixels of blanking (NTSC only, PAL is a little different) are never intended to make it to the display device, and the actual usable image is 704x480.  If you don't do this cropping, all DVDs with analogue blanking will make everyone just a hair too tall and skinny, circles will be ovals, etc...

 The only other home audio mixes would be from the 1982 VHS or Betamax releases. Those are closer to the original granted, but they sound terrible. The minor added sound effects in this mix are just fine, and dont detract from the enjoyment.

The 1981 crawl is in every single home video release already. This is the only home video version that has the original crawl.  :)