logo Sign In

DrDre

User Group
Members
Join date
16-Mar-2015
Last activity
6-Sep-2024
Posts
3,989

Post History

Post
#769960
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

Here's the same comparison between sharpening, deblurring, and super resolution for the Obi-Wan frame I posted a while ago.

Simple upscale vs simple upscale + sharpen

http://screenshotcomparison.com/comparison/126592

Simple upscale vs simple upscale + deblur

http://screenshotcomparison.com/comparison/126594

Simple upscale + sharpen vs simple upscale + deblur

http://screenshotcomparison.com/comparison/126599

Simple upscale vs raw super resolution

http://screenshotcomparison.com/comparison/126602

Simple upscale + deblur vs raw super resolution

http://screenshotcomparison.com/comparison/126603

Post
#769925
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

To clarify the difference between sharpening, deblurring, and super resolution, I will use another example programmed in MATLAB.

When an image (or video frame) is downscaled two things happen: the image is blurred and compressed. Blurring causes the information content to be distributed differently among the pixels, resulting in loss of detail, but in principle no information is lost. Compression results in loss of information.  We can subsequently upscale the downscaled image back to its original size through interpolation. When we compare the upscale to the original, we obviously get a blurred image with less detail:

http://screenshotcomparison.com/comparison/126569

Sharpening is the same as applying an unsharp mask filter. An unsharp mask filter enhances the high frequency content in the image. In other words sharpening results in edge enhancement. No lost details are recovered, the existing details are simply made more visible:

http://screenshotcomparison.com/comparison/126570

A side effect of sharpening is ringing. Although sharpening enhances detail, it does not necessarily result in a better representation of the original high res image. It simply represents a subjective reimagining of the low resolution image. 

Deblurring is the process of undoing the effects of blurring. Provided a reasonable estimate of the blur function is known, many details can be recovered:

http://screenshotcomparison.com/comparison/126571

The following comparison shows the difference between sharpening and deblurring. The deblurred image obviously has more detail, and is a better representation of the high res image. The deblurred image also has less artifacts, such as reduced ringing:

http://screenshotcomparison.com/comparison/126572

However, the loss of detail due to compression cannot be undone by deblurring. If we have multiple images with subpixel shifts the images can be super resolved, and part of the detail lost due to compression can be recovered. The objects in the images are registered, aligned, averaged, and deblurred:

http://screenshotcomparison.com/comparison/126585

The super resolved image has more (accurate) detail, less artifacts, such as reduced ringing, and is an even better representation of the high res image, when compared to the deblurred single image:

http://screenshotcomparison.com/comparison/126586

Summarizing, sharpening enhances existing details, deblurring reconstructs details lost as a result of blurring, super resolution reconstructs details lost as a result of blurring and compression. Any of these methods results in artifacts, the foremost being ringing. However, sharpening generally has the most artifacts, while super resolution has the least. 

The sucess of super resolution in practise, hinges on accurate data fusion, and the accuracy of the estimated point spread function (blur function). Over the last decades increasingly accurate approximate solutions have been defined for both these problems, resulting in accurate and patented applications in image/video processing and forensics.

For more information on the limits of super resolution, I refer you to this scientific paper:

http://homepage.tudelft.nl/e3q6n/publications/2005/SPIE2005_SanJose_TPLVKS.pdf

Post
#769891
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

@ Laserdisc Master

Sorry I forgot to reply to your post. Now I was being rude. I will ignore myself. :-P

The example I used is highly idealized, but the SR algorithm I use, uses the same principle. This is the registration step, where similar objects in different frames are identified and aligned.

Post
#769787
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

Is rudeness a common feature among (former) gurus on this forum? 

In order to avoid another unpleasant discussion, and safeguard the positive and constructive atmosphere on this thread, I will adhere to the following rule:

Rude posts will be ignored!

Any constructive critisism is of course very much appreciated. 

Post
#769786
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

@ g-force

That's like saying a car is not the wheels, just the engine. The definition of super resolution has long since been defined by the scientific community (aka registration + fusion + deblurring). So, there's no sense debating semantics and trying to sound clever. 

You're also wrong about equating sharpening to deblurring.

Sharpening vs deblurring = unsharp masking vs deconvolution

Unsharp masking: a linear operation that increases the apparent sharpness of an image in ignorance of the manner in which the image was acquired. 

Deconvolution: a solution to an inverse problem that increases the apparent sharpness of an image, but based on information describing some of the likely origins of the distortions of the light path used in capturing the image.

The fact that there is ringing is due to the approximate nature of the point-spread-function. It is actually a very common artifact, as can be read in numerous publications. This in no way changes the fact that the deblurred frame is much closer to the original high resolution frame than the low resolution source.  

Post
#769762
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

@ g-force

Could you enlighten the rest of us?

Chapter 1 of this phd thesis makes my point pretty clear:

https://www.google.nl/url?sa=t&source=web&rct=j&ei=9i5SVfrpI8WYsgHGkYCYCg&url=http://repository.tudelft.nl/assets/uuid:43eed463-1a44-41fc-96d0-b8edcf9570d0/thesis_vaneekeren.pdf&ved=0CEgQFjAI&usg=AFQjCNEl6r6W7nlriDkq81lYp5BwXvxBcQ&sig2=b5ztLeOggrnKz1VnkahjgA

Super resolution = registration + fusion + deblurring

Post
#769753
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

@ g-force

The last step in a SR algorithm is deblurring. In fast moving scenes little additional detail can be recovered. In that case the deblurred and sharpened results will be similar. Also, although deblurring and sharpening are technically different, many socalled sharpeners are actually deblurrers. 

Post
#769744
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

@ ChibiMAK

I've extensively tested the script (with some variations). Although it is extremely effective at reducing aliasing, there are a number of downsides to this method:

1) The detail is significantly reduced by halving the vertical resolution. It isn't as bad as you might expect, especially with sharpening, but the difference is certainly noticable.

2) Details appear and disappear between different frames. This leads to artifacts in the video, that are worsened by the sharpening applied to compensate for the loss of detail. 

So, I suppose some might take those artifacts and the loss of detail for granted. However, I choose to keep as much of the detail intact, and avoid new artifacts.

There is a promising new way of antialiasing called "Subpixel Morphological Antialiasing". Sadly, it is not available for Virtualdub or Avisynth.

Post
#769735
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

@ zee944

You're welcome! The Obi-Wan shot always was one of my favourites. My guess would be that for about 50% of the frames super resolution and sharpening will be very similar. These are usually the fast moving scenes or any out of focus scenes. For about 30% you will get noticable detail enhancement, and for 20% you will get significant detail enhancement. 

As the movie is still processing, and I'm kind of adjusting filter settings on a scene by scene basis, I will send a new batch of comparisons when I''m further into the movie. 

Post
#769524
Topic
Star Wars GOUT in HD using super resolution algorithm (* unfinished project *)
Time

I've finally watched Team Blu's Star Wars. I have to say it's a stunning restoration. It looks great and it sounds great. I'm also glad to say there is added value to super resolution, and that both upscales are substantially different. Both methodologies have their advantages, and disadvantages. Of course I prefer mine, but my opinion is hardly objective ;-).

I've decided to go for a compromise with respect to the demand for screenshot comparisons, and also to respect DJ's wishes. I will send some screenshot comparisons via PM to anyone who wishes to see them. Of course those members that have already expressed such a wish, will recieve them as soon as I get around to it. If anyone else wants to have them: ask and you shall recieve. You can then discuss the differences via PM or on this thread. I hope this makes everybody happy...