Sign In

DrDre

User Group
Members
Join date
16-Mar-2015
Last activity
21-Jan-2020
Posts
3,709

Post History

Post
#771028
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

The AA filters I've tested do better, but the loss in terms of detail is still apparent, so some progress, but not so much that I would use any of them at this point. I think I will now first focus on releasing a version of this that goes for maximum detail recovery, with a mild grain reduction, and leave the antialiasing for the next iteration. I never considered doing multiple iterations, but I've discovered that once you're down this path, forever will it dominate your destiny... ;-)

The laserdisc captures I've seen so far have a significantly lower resolution than the GOUT. However, I've seen some pretty promising developments on this forum with respect to laserdisc captures, so if one approaches the GOUT in terms of detail, it is certainly worth a try.

Post
#770435
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

I'm posting a single comparison with Team Blu, to highlight the differences between the two approaches. The methods chosen to enhance the film leads to a very different end result. In my humble opinion there are positives and negatives to be found in both methods, which is why they can co-exist as two different visions on what the GOUT can be.

http://screenshotcomparison.com/comparison/127122

Anyone interested in other comparisons, please send me a PM or post on this thread, 

Post
#770413
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

Yep, that's the one. Here is a long list of screenshot comparisons for the debugged SRV8:

http://screenshotcomparison.com/comparison/127093

http://screenshotcomparison.com/comparison/127094

http://screenshotcomparison.com/comparison/127095

http://screenshotcomparison.com/comparison/127096

http://screenshotcomparison.com/comparison/127097

http://screenshotcomparison.com/comparison/127098

http://screenshotcomparison.com/comparison/127100

http://screenshotcomparison.com/comparison/127102

http://screenshotcomparison.com/comparison/127103

http://screenshotcomparison.com/comparison/127105

Update:

Here are the comparisons to the simple upscales for the frames used in the MagicSR comparisons:

http://screenshotcomparison.com/comparison/127115

http://screenshotcomparison.com/comparison/127116

http://screenshotcomparison.com/comparison/127118

Post
#770328
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

Andrea, I can't thank you enough. There actually was a bug in the SRV8 script, causing some of the detail reconstruction to be undone. So, the full potential of SRV8 has not yet been seen. It sadly also means I have to process the movie again... :-( 

Here are new comparisons to MagicSR for the previous frames:

http://screenshotcomparison.com/comparison/127050

http://screenshotcomparison.com/comparison/127051

http://screenshotcomparison.com/comparison/127052

There are actually tiny new details to be found here and there...

Update: 

Removed another bug, so there's a little more detail.

Post
#770281
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

This screenshot comparison shows very clearly that super resolution has some clear advantages over sharpening. In this case I used the sharpen filter in Virtualdub for the comparison. From this example is it very clear that compared to sharpening, super resolution has more detail, while having less noise enhancement, less ringing, and less aliasing. 

Avisynth Spline64Resize vs Avisynth Spline64Resize + sharpen

http://screenshotcomparison.com/comparison/126997

Avisynth Spline64Resize vs super resolution (raw)

http://screenshotcomparison.com/comparison/126998

Avisynth Spline64Resize + sharpen vs super resolution (raw)

http://screenshotcomparison.com/comparison/126999

So, unlike some critics stated in the past, super resolution does indeed enhance detail, while simultaneously reducing noise, ringing, and aliasing compared to standard enhancement techniques. 

Post
#770128
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

I hope to get another video sample online this weekend. In the mean time here are a couple of screenshot comparisons for the sample. Notice the detail recovery for R2D2 in the first screenshot.

http://screenshotcomparison.com/comparison/126843

http://screenshotcomparison.com/comparison/126844

http://screenshotcomparison.com/comparison/126847

Update:

Here's a screenshot for one of the scenes I'm working on now:

http://screenshotcomparison.com/comparison/126862

This is a standout example of super resolution detail reconstruction, especially noticeable on C3PO.

Post
#770007
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

@ Zyrother

This is one of the challenges with this type of upscale. Some of the scenes are so grainy, the amount of detail enhancement is matched by an equal amount of grain amplification. The standard filters weren't quite up to par, so I wrote a non-linear filter that keeps as much of the detail, while removing the enhanced noise, and replacing it with a reduced GOUT grain. 

Post
#770000
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

@ zee944

The differences between the simple upscale and raw super resolution are still pretty easy to spot (at least to my eye). 

Update:

Here's the comparison between SR raw and SRV8:

http://screenshotcomparison.com/comparison/126617

update 2:

I now see it's not the same frame. It's difficult to see on my mobile. I used a different frame for the Team Blu comparisons than the ones I posted earlier. 

Post
#769960
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

Here's the same comparison between sharpening, deblurring, and super resolution for the Obi-Wan frame I posted a while ago.

Simple upscale vs simple upscale + sharpen

http://screenshotcomparison.com/comparison/126592

Simple upscale vs simple upscale + deblur

http://screenshotcomparison.com/comparison/126594

Simple upscale + sharpen vs simple upscale + deblur

http://screenshotcomparison.com/comparison/126599

Simple upscale vs raw super resolution

http://screenshotcomparison.com/comparison/126602

Simple upscale + deblur vs raw super resolution

http://screenshotcomparison.com/comparison/126603

Post
#769925
Topic
Star Wars GOUT in HD using super resolution algorithm
Time

To clarify the difference between sharpening, deblurring, and super resolution, I will use another example programmed in MATLAB.

When an image (or video frame) is downscaled two things happen: the image is blurred and compressed. Blurring causes the information content to be distributed differently among the pixels, resulting in loss of detail, but in principle no information is lost. Compression results in loss of information.  We can subsequently upscale the downscaled image back to its original size through interpolation. When we compare the upscale to the original, we obviously get a blurred image with less detail:

http://screenshotcomparison.com/comparison/126569

Sharpening is the same as applying an unsharp mask filter. An unsharp mask filter enhances the high frequency content in the image. In other words sharpening results in edge enhancement. No lost details are recovered, the existing details are simply made more visible:

http://screenshotcomparison.com/comparison/126570

A side effect of sharpening is ringing. Although sharpening enhances detail, it does not necessarily result in a better representation of the original high res image. It simply represents a subjective reimagining of the low resolution image. 

Deblurring is the process of undoing the effects of blurring. Provided a reasonable estimate of the blur function is known, many details can be recovered:

http://screenshotcomparison.com/comparison/126571

The following comparison shows the difference between sharpening and deblurring. The deblurred image obviously has more detail, and is a better representation of the high res image. The deblurred image also has less artifacts, such as reduced ringing:

http://screenshotcomparison.com/comparison/126572

However, the loss of detail due to compression cannot be undone by deblurring. If we have multiple images with subpixel shifts the images can be super resolved, and part of the detail lost due to compression can be recovered. The objects in the images are registered, aligned, averaged, and deblurred:

http://screenshotcomparison.com/comparison/126585

The super resolved image has more (accurate) detail, less artifacts, such as reduced ringing, and is an even better representation of the high res image, when compared to the deblurred single image:

http://screenshotcomparison.com/comparison/126586

Summarizing, sharpening enhances existing details, deblurring reconstructs details lost as a result of blurring, super resolution reconstructs details lost as a result of blurring and compression. Any of these methods results in artifacts, the foremost being ringing. However, sharpening generally has the most artifacts, while super resolution has the least. 

The sucess of super resolution in practise, hinges on accurate data fusion, and the accuracy of the estimated point spread function (blur function). Over the last decades increasingly accurate approximate solutions have been defined for both these problems, resulting in accurate and patented applications in image/video processing and forensics.

For more information on the limits of super resolution, I refer you to this scientific paper:

http://homepage.tudelft.nl/e3q6n/publications/2005/SPIE2005_SanJose_TPLVKS.pdf