logo Sign In

Star Wars GOUT in HD using super resolution algorithm (* unfinished project *) — Page 23

Author
Time
 (Edited)

DrDre said:

I will use EEDI2 on the source, and then use SR to reconstruct the lost detail in the individual frames. Not sure how well it will work, but still worth a try.

That may not work well since SR relies on aliasing. It's effectively a temporal sharpener for high frequency details. When those details fall between the cracks (aliasing) in adjacent frames, they can be recovered using temporal data if the local neighborhood has a sufficiently-high correlation (i.e., little motion). Basically, more aliasing => more details recovered by SR.

See: http://en.wikipedia.org/wiki/Superresolution#Aliasing

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time

DrDre, thanks for the article on fanres!!!

An idea: why don't you try the subpixel shift - that you used for your example - also on GOUT? Maybe it will work not, but trying is harmless! (^^,)

Sadly my projects are lost due to an HDD crash… 😦 | [Fundamental Collection] thread | blog.spoRv.com | fan preservation forum: fanres.com

Author
Time

@ AntcuFaalb

Good point! I guess antialiasing is a synonym for anti-SR. So it seems you either accept the aliasing or you accept a loss of detail. Bummer...

Author
Time

DrDre said:

@ zee944

You're welcome! The Obi-Wan shot always was one of my favourites. My guess would be that for about 50% of the frames super resolution and sharpening will be very similar. These are usually the fast moving scenes or any out of focus scenes. For about 30% you will get noticable detail enhancement, and for 20% you will get significant detail enhancement. 

As the movie is still processing, and I'm kind of adjusting filter settings on a scene by scene basis, I will send a new batch of comparisons when I''m further into the movie. 

 

I just don't understand how SR could be the same as sharpening for 50% of the movie. Shouldn't it be the same as doing nothing? Isn't this an admission that you are also sharpening?

-G

Author
Time
 (Edited)

@ g-force

The last step in a SR algorithm is deblurring. In fast moving scenes little additional detail can be recovered. In that case the deblurred and sharpened results will be similar. Also, although deblurring and sharpening are technically different, many socalled sharpeners are actually deblurrers. 

Author
Time
 (Edited)

@ g-force

Could you enlighten the rest of us?

Chapter 1 of this phd thesis makes my point pretty clear:

https://www.google.nl/url?sa=t&source=web&rct=j&ei=9i5SVfrpI8WYsgHGkYCYCg&url=http://repository.tudelft.nl/assets/uuid:43eed463-1a44-41fc-96d0-b8edcf9570d0/thesis_vaneekeren.pdf&ved=0CEgQFjAI&usg=AFQjCNEl6r6W7nlriDkq81lYp5BwXvxBcQ&sig2=b5ztLeOggrnKz1VnkahjgA

Super resolution = registration + fusion + deblurring

Author
Time

No. SR is the alignment and sum/median part. Deblurring is just deblurring, which is also called sharpening. There are a million things you can call sharpening, but it's all spatial filtering, either implemented in the linear domain via a convolution, or in frequency space by a multiplication. Some do a better job of approximating the inverse transfer function of a blurring operation, but it's all the same thing. And whatever you are applying is not even close to the inverse of any blur that was ever applied, as evidenced by the increased ringing.

You really gotta lay off the thesis papers. They're just thesis papers.

-G

Author
Time
 (Edited)

@ g-force

That's like saying a car is not the wheels, just the engine. The definition of super resolution has long since been defined by the scientific community (aka registration + fusion + deblurring). So, there's no sense debating semantics and trying to sound clever. 

You're also wrong about equating sharpening to deblurring.

Sharpening vs deblurring = unsharp masking vs deconvolution

Unsharp masking: a linear operation that increases the apparent sharpness of an image in ignorance of the manner in which the image was acquired. 

Deconvolution: a solution to an inverse problem that increases the apparent sharpness of an image, but based on information describing some of the likely origins of the distortions of the light path used in capturing the image.

The fact that there is ringing is due to the approximate nature of the point-spread-function. It is actually a very common artifact, as can be read in numerous publications. This in no way changes the fact that the deblurred frame is much closer to the original high resolution frame than the low resolution source.  

Author
Time
 (Edited)

Is rudeness a common feature among (former) gurus on this forum? 

In order to avoid another unpleasant discussion, and safeguard the positive and constructive atmosphere on this thread, I will adhere to the following rule:

Rude posts will be ignored!

Any constructive critisism is of course very much appreciated. 

Author
Time

DrDre said:

Sharpening vs deblurring = unsharp masking vs deconvolution

Does = in this context mean "is analogous to"?

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time

DrDre said:

Is rudeness a common feature among (former) gurus on this forum? 

It's just common on forums. 

Author
Time
 (Edited)

DrDre said:

Is rudeness a common feature among former gurus on this forum? 

You've gone meta and ad hominem (and, in consequence, nearly circular!) at the same time. I can't figure out which meme to use.

Man... where's TV's Frink when you need him?

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time

@ AntcuFaalb

Yep, that's what I meant. I'm aware that one of the goals of deblurring is of course to sharpen the image. However, in most of the literature I've read on the subject a clear distinction is made between the two. In a sense it's like the difference between reimagening and restoration. 

Author
Time
 (Edited)

Anyway, if you must go down the SR route, then I suggest AAing afterward. The goal is to recover as much temporal data as possible and THEN AA. EEDI2().TurnLeft().EEDI2().TurnRight() usually yields fantastic results, but I haven't kept up with the latest Sangnom2, etc. developments.

If you're willing to go all-the-way, then I'd suggest SRing and only AAing shots with annoying visible aliasing. Aliasing is always present, but it's not always easy to see. AAing only the most painful parts is a good compromise.

Also, get your hands on a really good SR algorithm. The one that comes with TheFoundry's FURNACE for NukeX (F_SmartZoom) is fantastic.

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time
 (Edited)

@ AnctuFaalb

Yeah sorry, I got a little worked up. 

Thanks for the advice. I guess that's indeed the best way to go. I'm downloading TheFoundry software now. You've made me curious.

Author
Time

AntcuFaalb said:


Man... where's TV's Frink when you need him?

 You just cost yourself a soup!

Author
Time

@ TV's Frink

I think you forgot my bread... 

Author
Time

Puggo - Jar Jar's Yoda said:

I like the new Frink!

 You're pushing your luck, little dog.

Author
Time

I miss the old Ric Olie Frink. This new one is a jerkhole.

-G

Author
Time
 (Edited)

g-force said:

I miss the old Ric Olie Frink. This new one is a jerkhole.

-G

 Oh yeah, g-force?  Well, the jerkhole store called, they're running out of you!

Author
Time
 (Edited)

@ Laserdisc Master

Sorry I forgot to reply to your post. Now I was being rude. I will ignore myself. :-P

The example I used is highly idealized, but the SR algorithm I use, uses the same principle. This is the registration step, where similar objects in different frames are identified and aligned.

Author
Time
 (Edited)

To clarify the difference between sharpening, deblurring, and super resolution, I will use another example programmed in MATLAB.

When an image (or video frame) is downscaled two things happen: the image is blurred and compressed. Blurring causes the information content to be distributed differently among the pixels, resulting in loss of detail, but in principle no information is lost. Compression results in loss of information.  We can subsequently upscale the downscaled image back to its original size through interpolation. When we compare the upscale to the original, we obviously get a blurred image with less detail:

http://screenshotcomparison.com/comparison/126569

Sharpening is the same as applying an unsharp mask filter. An unsharp mask filter enhances the high frequency content in the image. In other words sharpening results in edge enhancement. No lost details are recovered, the existing details are simply made more visible:

http://screenshotcomparison.com/comparison/126570

A side effect of sharpening is ringing. Although sharpening enhances detail, it does not necessarily result in a better representation of the original high res image. It simply represents a subjective reimagining of the low resolution image. 

Deblurring is the process of undoing the effects of blurring. Provided a reasonable estimate of the blur function is known, many details can be recovered:

http://screenshotcomparison.com/comparison/126571

The following comparison shows the difference between sharpening and deblurring. The deblurred image obviously has more detail, and is a better representation of the high res image. The deblurred image also has less artifacts, such as reduced ringing:

http://screenshotcomparison.com/comparison/126572

However, the loss of detail due to compression cannot be undone by deblurring. If we have multiple images with subpixel shifts the images can be super resolved, and part of the detail lost due to compression can be recovered. The objects in the images are registered, aligned, averaged, and deblurred:

http://screenshotcomparison.com/comparison/126585

The super resolved image has more (accurate) detail, less artifacts, such as reduced ringing, and is an even better representation of the high res image, when compared to the deblurred single image:

http://screenshotcomparison.com/comparison/126586

Summarizing, sharpening enhances existing details, deblurring reconstructs details lost as a result of blurring, super resolution reconstructs details lost as a result of blurring and compression. Any of these methods results in artifacts, the foremost being ringing. However, sharpening generally has the most artifacts, while super resolution has the least. 

The sucess of super resolution in practise, hinges on accurate data fusion, and the accuracy of the estimated point spread function (blur function). Over the last decades increasingly accurate approximate solutions have been defined for both these problems, resulting in accurate and patented applications in image/video processing and forensics.

For more information on the limits of super resolution, I refer you to this scientific paper:

http://homepage.tudelft.nl/e3q6n/publications/2005/SPIE2005_SanJose_TPLVKS.pdf