DrDre

@ Laserdisc Master

I don't think it is correct to say super resolution relies on artifacts. It reconstructs details from multiple frames to construct a true HD frame that is much closer to the original HD frame than most single frame upscale methods are capable of (as is evidenced in a large number of scientific papers). It is true that the reconstruction process is not perfect and will result in artifacts. However, the same is true for any upscale method. What you're saying is you prefer maintaining original shapes in the low res original and adding the illusion of detail, to imperfectly reconstructing actual detail. I prefer the latter ;-). 

Anyways, it's been an interesting discusion. Thanks!

ps. I'm on a train right now, so I can't see your screenshot comparison on my mobile. I will take a look later today.

@ RU.08

Although I see your point, it is not correct to say the super resolution algorithm combines flat detailed frames. It combines low resolution frames with depth, using statistical measures to determine the similarity between elements in the different frames. These elements have depth, so the final combined frame has depth. It probably is true that it is not possible to perfectly capture our sense of depth in statistical measures. It is also true that similar statistical measures are used to compress the original master to the dvd format, so the idea that the dvd compression somehow retains all the depth, while the super resolution completely destroys it to me seems an exaggeration. 

@ Laserdisc Man

Here's the PAL sample upscaled with super resolution v3:

Update: clip no longer available

and a screenshot comparison for frame 10 between super resolution v1 and v3:

http://screenshotcomparison.com/comparison/121899

It's clear the PAL version of the GOUT has much less detail in the source.

@ dickalan

Hahaha, I'm afraid you would need multiple photo's for that to work.

Although the degraining may help (or not), I think the idea that the increased detail in for example the edges of Obi-Wan's hood are simply "micro-contrast in the grain structure" to be pretty unconvincing:

http://screenshotcomparison.com/comparison/121822

In one of the comparisons for Leia's face for example you can actually see the blood veins in her eyes, and wrinkles in her face, which are actually not visible in the original resolution GOUT (but are visible in the SE blu ray):

http://screenshotcomparison.com/comparison/121736

Or the detail in the stormtrooper's eyes:

http://screenshotcomparison.com/comparison/121755

These real details are clearly visible in both the super resolution upscale, and the SE blu ray, but not in the GOUT.

So I would say if it looks like a duck, and it quacks like a duck, it's a duck... :-)

@ resonator

You seem to have missed the previous posts. I think the comparisons shows detail is restored, even without degraining first. Just watch the zoomed comparison of in post 173.

For those of you interested in my attempts to upscale the raw GOUT to true high definition, see the results of the super resolution v3 upscale (my own magic recipe ;-)). It's like a veil is lifted from the GOUT. Here's a comparison between one of the better single frame upscale methods (Avisynth Spline64Resize) and super resolution v3:

http://screenshotcomparison.com/comparison/121754

For those of you that believe this result can be achieved by sharpening a single frame upscale, here's a zoomed comparison between a sharpened Avisynth Spline64Resize and super resolution v3. From this example it is very clear that super resolution can restore real detail, even if the source material is not ideal:

http://screenshotcomparison.com/comparison/121822

For those of you that believe these results can be achieved by sharpening a single frame upscale, here's a zoomed comparison between a sharpened Avisynth Spline64Resize and super resolution v3. From this example it is very clear that super resolution can restore real detail, even if the source material is not ideal:

http://screenshotcomparison.com/comparison/121822

While it is certainly true that noise is enhanced, I think this is mostly due to the nature of the source material. It seems the master for Star Wars was simply more noisy than TESB and ROTJ. I will try to filter the upscale with non local means. This filter is pretty good at retaining detail. This may take a day or two, since processing a single movie now takes 15 hours with the new settings. Then another 5 hours compressing and filtering.

Here's a a teaser for The Empire Strikes Back and Return of the Jedi:

http://screenshotcomparison.com/comparison/121758

http://screenshotcomparison.com/comparison/121759

With different orders I mean, also processing the frames in reverse.

The artifact appears to also be present in the spline64, although less visible. Could this be DVNR smear?

Here are a few more comparisons for the super resolution v3. I'm still amazed this is possible using the GOUT.

http://screenshotcomparison.com/comparison/121736

http://screenshotcomparison.com/comparison/121748

http://screenshotcomparison.com/comparison/121753

http://screenshotcomparison.com/comparison/121754

http://screenshotcomparison.com/comparison/121755

Combining multiple super resolution upscales in multiple color spaces and from two different orders gives a massive enhancement of the detail, as is clear from the following comparisons:

http://screenshotcomparison.com/comparison/121679

http://screenshotcomparison.com/comparison/121686

http://screenshotcomparison.com/comparison/121687

http://screenshotcomparison.com/comparison/121688

http://screenshotcomparison.com/comparison/121689

I'm really impressed :-)

The differences are indeed smaller. One of the artifacts can be seen on Luke's arm in the following screenshot (also on the right). There was a black spot in the previous frame, resulting in ghosts in the next frames. 

http://screenshotcomparison.com/comparison/120830

While watching the super resolution upscale I noted a number of artifacts. First of all dirt in individual frames results in ghosts in the next frames. Also during fast movement ghosting of straight lines is an issue. Fortunately it turns out the super resolution handles the frames sequentially, implying that it only uses previous frames. Therefore you obtain a slightly different result when the process is done in reverse. By combining the two results, and including a spline64 upscale, you can get rid of artifacts, while enhancing detail. I wrote an Avisynth script to achieve this:

LoadPlugin("c:\Program Files (x86)\AviSynth 2.5\plugins\avssr.dll")

LoadPlugin("c:\Program Files (x86)\AviSynth 2.5\plugins\Median.dll")

orig=AviSource("Star Wars (NTSC).avi")

sr1=SR(orig,1428,548)

rev=Reverse(orig)

sr2=SR(rev,1428,548)

sr2=Reverse(sr2)

s64=Spline64Resize(orig,1428,548)

Median(sr1,sr2,s64)

The inclusion of the spline64 ensures any artifacts resulting from either of the super resolution upscales is removed when the median is taken.

The differences between AviSynth Spline64Resize and the super resolution upscale are now more subtle, because additional details are kept only when they appear in both super resolution upscales. It does result in a much cleaner upscale, though, with detail enhancement:

http://screenshotcomparison.com/comparison/121609

http://screenshotcomparison.com/comparison/121610

http://screenshotcomparison.com/comparison/121611

(Ps. these were made in preview mode in VirtualDub, so the final results should be better)

To clearly show the increased detail in the super resolution upscale of the NTSC GOUT, I performed a texture analysis on a number of frames upscaled with the Avisynth Spline64Resize and super resolution. The results are very convincing (and pretty cool looking).

In the below comparisons you can see the results of the texture analysis for a number of screenshots:

http://screenshotcomparison.com/comparison/121111

http://screenshotcomparison.com/comparison/121112

http://screenshotcomparison.com/comparison/121113

http://screenshotcomparison.com/comparison/121114

In the below example you can also see the difference with sharpening the Avisynth Spline64Resize upscale to "the same level of detail":

http://screenshotcomparison.com/comparison/121116

It is clear from this example that sharpening, while enhancing some of the detail, also enhances the noise. For the super resolution on the other hand the enhancements are both more detailed and cleaner. 

Here's  two more comparisons for the NTSC GOUT Star Wars (Avisynth Spline64Resize vs super resolution). The differences are most noticeable in the last frame (helmet, lips). 

http://screenshotcomparison.com/comparison/121074

http://screenshotcomparison.com/comparison/121081

@ Darth Lucas

If you can provide the scenes, I can do it for you. I sadly only have the prequels on blu ray. Let's first try one, to see what it will look like, but I'm guessing it will look pretty good.

Here are five screenshot comparisons for the NTSC GOUT Star Wars (Avisynth Spline64Resize vs super resolution). The super resolution upscale is clearly more detailed. I'm rather pleased with the results. What do you think?

http://screenshotcomparison.com/comparison/120825

http://screenshotcomparison.com/comparison/120826

http://screenshotcomparison.com/comparison/120827

http://screenshotcomparison.com/comparison/120830

http://screenshotcomparison.com/comparison/120831

@ Laserdisc Master

I'm afraid that won't work. I'm pretty sure the super resolution algorithm uses an equal number of frames on both sides, so reversing the order will give the same result. Replacing the frames will result in replica frames, leading to the compression artifacts being interpreted as real structure and showing up in the upscale as upscaled pixels. I noticed this effect when I erroneously left in the duplicate frames after detelecining the NTSC Star Wars. 

@ Spaced Ranger

Wow, nice gif! I guess that is a possibility. I'm pretty sure they do handle a scene change differently, as the problem is specifically mentioned in one of the scientific papers they cite. How they implemented it, is not known however, and I guess you will never completely be able to compensate for the missig information.

First comparison for the Star Wars (NTSC) upscale to 720p (Avisynth Spline64Resize vs super resolution):

http://screenshotcomparison.com/comparison/120717

Notice the increased detail in Obi-Wan's face, mouth, hair, beard, and hood. This result is far better than for the PAL version (seen on page 3, post 52, of this thread). More to follow soon...

@ Intruder

Interesting comparison! The super resolution algorithm has half the number of frames in a scene change, so you would expect it to do worse. Glad to see the differences are subtle, and not glaring. 

I'm finally almost finished with the super resolution upscale of Star Wars. I had a lot of trouble with audio synch, and getting the correct frame rate from the NTSC GOUT. The final product will be 720p (1280x544). I will post some clips and screenshot comparisons soon. Then it's on the Empire... ;-)  

@ Laserdisc Man

Denoising does remove a lot of the detail. I tried a non local means filter. Looks good in some scenes, but in others you can clearly see where random true structure (like in clothes) is smoothed away. 

What's the concept behind MagicUp?

@ Laserdisc Master

PanUp looks like a great algorithm! Very nice concept indeed. Would be interesting to combine the PanUp algorithm with super resolution. :-)

@ Laserdisc Master

I would have to agree. Is PanUp another one of your inventions? It looks good.

@ pittrek

I guess it's like with mp3. Some will hear no difference between 128kbps and 320kbps, while others will maintain the 128kbps sounds poor. For us that do hear the difference it can become an obsession...