Star Wars GOUT in HD using super resolution algorithm (* unfinished project *) — Page 7

That's a resourceful idea, _,,,^..^,,,_ ! But DrDre is right -- using duplicated frames doesn't help for finding new information for the processed frame.

My thinking was, if there were adjustable parameters in the program, the number of new frames referenced in one direction could be increased to compensate for the lesser number of frames in the other direction:

shot change -  (Y) (Z) |  A  (B) (C) (D) (E)     +0  A  +4 == 4 sourced
change +1    -  (Y) (Z) | (A)  B  (C) (D) (E)     +1  B  +3 == 4 sourced
change +2    -  (Y) (Z) | (A) (B)  C  (D) (E)     +2  C  +2 == 4 sourced

Of course, in a running film, diminishing returns may make this unnoticeable.  :D

Here are five screenshot comparisons for the NTSC GOUT Star Wars (Avisynth Spline64Resize vs super resolution). The super resolution upscale is clearly more detailed. I'm rather pleased with the results. What do you think?

http://screenshotcomparison.com/comparison/120825

http://screenshotcomparison.com/comparison/120826

http://screenshotcomparison.com/comparison/120827

http://screenshotcomparison.com/comparison/120830

http://screenshotcomparison.com/comparison/120831

Would someone be willing to try some of these techniques on the prequel deleted scenes?

@ Darth Lucas

If you can provide the scenes, I can do it for you. I sadly only have the prequels on blu ray. Let's first try one, to see what it will look like, but I'm guessing it will look pretty good.

My computer is being really funky right now and not wanting to read my external hard drive properly, thus the deleted scene files I have cannot be accessed.  =.= fucking technology man.  Can anybody else get an example to DrDre for me?  If not I'll get something to you when I get this harddrive thing sorted out.  Or possibly just rip the scenes from my dvds again.

Here's  two more comparisons for the NTSC GOUT Star Wars (Avisynth Spline64Resize vs super resolution). The differences are most noticeable in the last frame (helmet, lips). 

http://screenshotcomparison.com/comparison/121074

http://screenshotcomparison.com/comparison/121081

To clearly show the increased detail in the super resolution upscale of the NTSC GOUT, I performed a texture analysis on a number of frames upscaled with the Avisynth Spline64Resize and super resolution. The results are very convincing (and pretty cool looking).

In the below comparisons you can see the results of the texture analysis for a number of screenshots:

http://screenshotcomparison.com/comparison/121111

http://screenshotcomparison.com/comparison/121112

http://screenshotcomparison.com/comparison/121113

http://screenshotcomparison.com/comparison/121114

In the below example you can also see the difference with sharpening the Avisynth Spline64Resize upscale to "the same level of detail":

http://screenshotcomparison.com/comparison/121116

It is clear from this example that sharpening, while enhancing some of the detail, also enhances the noise. For the super resolution on the other hand the enhancements are both more detailed and cleaner. 

While watching the super resolution upscale I noted a number of artifacts. First of all dirt in individual frames results in ghosts in the next frames. Also during fast movement ghosting of straight lines is an issue. Fortunately it turns out the super resolution handles the frames sequentially, implying that it only uses previous frames. Therefore you obtain a slightly different result when the process is done in reverse. By combining the two results, and including a spline64 upscale, you can get rid of artifacts, while enhancing detail. I wrote an Avisynth script to achieve this:

LoadPlugin("c:\Program Files (x86)\AviSynth 2.5\plugins\avssr.dll")

LoadPlugin("c:\Program Files (x86)\AviSynth 2.5\plugins\Median.dll")

orig=AviSource("Star Wars (NTSC).avi")

sr1=SR(orig,1428,548)

rev=Reverse(orig)

sr2=SR(rev,1428,548)

sr2=Reverse(sr2)

s64=Spline64Resize(orig,1428,548)

Median(sr1,sr2,s64)

The inclusion of the spline64 ensures any artifacts resulting from either of the super resolution upscales is removed when the median is taken.

The differences between AviSynth Spline64Resize and the super resolution upscale are now more subtle, because additional details are kept only when they appear in both super resolution upscales. It does result in a much cleaner upscale, though, with detail enhancement:

http://screenshotcomparison.com/comparison/121609

http://screenshotcomparison.com/comparison/121610

http://screenshotcomparison.com/comparison/121611

(Ps. these were made in preview mode in VirtualDub, so the final results should be better)

The differences are indeed smaller. One of the artifacts can be seen on Luke's arm in the following screenshot (also on the right). There was a black spot in the previous frame, resulting in ghosts in the next frames. 

http://screenshotcomparison.com/comparison/120830

I am quite excited about this.  I would love to see the prequels get this treatment as well as the 1997 SE.  

Nice work man.

Combining multiple super resolution upscales in multiple color spaces and from two different orders gives a massive enhancement of the detail, as is clear from the following comparisons:

http://screenshotcomparison.com/comparison/121679

http://screenshotcomparison.com/comparison/121686

http://screenshotcomparison.com/comparison/121687

http://screenshotcomparison.com/comparison/121688

http://screenshotcomparison.com/comparison/121689

I'm really impressed :-)

With different orders I mean, also processing the frames in reverse.

The artifact appears to also be present in the spline64, although less visible. Could this be DVNR smear?

Here are a few more comparisons for the super resolution v3. I'm still amazed this is possible using the GOUT.

http://screenshotcomparison.com/comparison/121736

http://screenshotcomparison.com/comparison/121748

http://screenshotcomparison.com/comparison/121753

http://screenshotcomparison.com/comparison/121754

http://screenshotcomparison.com/comparison/121755

Here's a a teaser for The Empire Strikes Back and Return of the Jedi:

http://screenshotcomparison.com/comparison/121758

http://screenshotcomparison.com/comparison/121759

What I'm noticing is that, while you seem to be getting a dramatic increase in detail, what your'e getting as part of that package deal is an equally dramatic increase in artifacts and noise (at least on SW.  those two from Empire and Jedi look fine).  Is there a way to soften out that noise without also losing the detail and ending up exactly where we started?

Impressive!  Most impressive!

While it is certainly true that noise is enhanced, I think this is mostly due to the nature of the source material. It seems the master for Star Wars was simply more noisy than TESB and ROTJ. I will try to filter the upscale with non local means. This filter is pretty good at retaining detail. This may take a day or two, since processing a single movie now takes 15 hours with the new settings. Then another 5 hours compressing and filtering.

For those of you that believe these results can be achieved by sharpening a single frame upscale, here's a zoomed comparison between a sharpened Avisynth Spline64Resize and super resolution v3. From this example it is very clear that super resolution can restore real detail, even if the source material is not ideal:

http://screenshotcomparison.com/comparison/121822

In my opinion, it's pretty pointless to use super resolution without doing a proper degraining first (and a proper RE-graining afterwards). SR only works really well in restoring detail on very clean material, as soon as grain is there, all you'll do is enhance the micro-contrast of the grains themselves. Also, that needs to be done on a scene by scene basis. Neat Video or Dark Energy Anti-Matter come to mind for the degraining. And THEN do a comparison. Don't be discouraged by the lack of detail on the degrained images, once you introduce the grain BACK into the footage, it will seem a lot sharper and more detailed. My experience with SR is that it does work well with VERY "un-shaky" sources, however, I think the median with regular, reverse and nnedi3+sharpen sounds genius to me, I gotta try this myself.

@ resonator

You seem to have missed the previous posts. I think the comparisons shows detail is restored, even without degraining first. Just watch the zoomed comparison of in post 173.