Star Wars GOUT in HD using super resolution algorithm (* unfinished project *) — Page 18

DrDre said:

... frame 6167. I tried to clean it in the past, but didn't get very positive feedback. It's a really difficult frame, because of the amount of grain.

With Super Resolution being a temporal process, using temporal noise reducers will directly work against the SR effect. One must use spacial noise reduction, but those tend to smear over everything, even those having withholding parameters (edge-preserving smoothers).

I don't know if there is an Avisynth equivalent to this, but I've found a paint program's JPEG artifact removal to be an amazing noise reducer by targeting "out of place pixels" that happens with such things as compression and random noise. If a pixel is in keeping with it's surroundings, it isn't processed. As a result, the noise is reduced (strength settings: low, normal, high, maximum) but the detail is not ..

I have been meaning to ask this, but I just wanted to be sure that you guys are doing 3:2 pulldown which converts it from 29.97 FPS to 23.976 FPS.  That should get rid of the interlacing in theory.

Edit:  I just looked and you are both providing 23.976 FPS versions so that answers that question.

In my opinion the SR7 stuff looks like it recovers more details but the combing is bad.  If you could do SR7 plus decombing, it might do the trick.

http://avisynth.nl/index.php/Decomb

Not sure if there are other plugins that do decombing by looking at the picture and finding problem areas but leaving the rest untouched.

Hope that helps.

Edit 2: this might do the trick: http://neuron2.net/decomb/decombnew.html

@ thorr

The problem is that the aliasing is in the source (watch the dvd and see for yourself). I'm afraid decombing is not going to help, as the video has been properly decombed. The only solution is a some super antialiasing script, but the ones I tested, destroyed to much detail (according to my tastes). 

From my second link: "Frames that are detected as combed are smart deinterlaced. The deinterlacing is adaptive, i.e., only the portions of a frame that are combed will be deinterlaced. This means that full detail is retained in static picture areas."

It seems like that would help because the frames are still combed (in the source as you mentioned). 

@ thorr

I tested the plugin. Sadly it gives the same result as the deinterlacing method I use. Although it is true that poor deinterlacing can result in aliasing, I don't think deinterlacing will improve material with aliasing in the source, before it was interlaced.

While this probably doesn't help us, I found it to be super interesting.

I played back the GOUT on my CRT television that I use primarily for classic gaming, and the GOUT looked incredible on it. The obtrusive grain/noise is pretty much gone, and all that is left is the natural grain field and it looks incredibly clean. Aliasing is nearly gone as well. Its still there, but it is very very minimal. Color reproduction looks much better than anticipated, but that is less of a concern. And that 6167 frame? Looks very clean, grain is almost non existent.

Since LaserDisc is an analogue format, it needs scan lines to properly display. Its the same reason why LaserDisc and VHS players look so horrible on digital TVs. The analogue to digital conversion sucks. Yes, DVDs are digital, but since the GOUT was authored so terribly, it might as well be analogue.

Just in case you want to know the equipment used, its a Toshiba DVD recorder, played back with S-Video onto a flatscreen 19 inch CRT.

Sorry I cant provide screenshots, unless I point my camera at it and take pictures, but that would not be an accurate representation.

_,,,^..^,,,_ said:

I'm aware that CRT image is clear, but this is due to lower resolution, and the fact CRT has not pixels but dots and scanlines... and, due to the Kell factor, effective resolution is quite low - the smaller the TV, the smaller the resolution!

 Yeah, I knew that it would help some, but I was surprised by just how much it did.

Its hard to work with source material so bad. Only so much can be pulled out, but the results so far are still above and beyond what many of us thought was possible. :)

It took a little time, but here is super resolution v8. The ringing is reduced to the level of the original GOUT. The grain is also reduced, while avoiding a painted look. Here are 7 screenshot comparisons:

http://screenshotcomparison.com/comparison/125446

http://screenshotcomparison.com/comparison/125447

http://screenshotcomparison.com/comparison/125448

http://screenshotcomparison.com/comparison/125449

http://screenshotcomparison.com/comparison/125450

http://screenshotcomparison.com/comparison/125452

http://screenshotcomparison.com/comparison/125453

Guys, did you think about using your techniques for the low quality deleted scenes (from the Making Of ebooks)? Or the behind the scenes footage? It would come in handy for a project I'm working on

@ pittrek 

We did use it on one of the AOTC deleted scenes. It worked really well. Here's the screemshot comparison I posted before:

http://screenshotcomparison.com/comparison/124536

and the upscaled video sample:

https://drive.google.com/file/d/0B8_LYKyZDiajUWFhcEt6aGpVSjA/view?usp=sharing

Looks nice and one day I'll definitely get to the prequels, but now I'm still working on the OT :) That's why I asked specifically about the scenes which were exclusive to the Rinzler ebooks

The result of your "superresolution" method does look better than the original GOUT, but I'm unsure if it comes from the actual superresolution part.

You guys forget (or avoid, I don't know) one important thing to do: to post comparsions between your, Team Blu's or some other denoising+sharpening method that was done with at least some effort. It's very misleading to post comparsions with a simple AviSynth resizing (one line of actual scripting?); it should be a good resizing *AND* a good sharpening to make it fair.

I'm not saying there's no improvement. There is indeed some. I'm saying there is little to gain from the "superresolution" theory and it's more about temporal denoising and sharpening. It'd be important to see where the gains are coming from, and if those are bigger than in the earlier projects.

zee944 said:

The result of your "superresolution" method does look better than the original GOUT, but I'm unsure if it comes from the actual superresolution part.

You guys forget (or avoid, I don't know) one important thing to do: to post comparsions between your, Team Blu's or some other denoising+sharpening method that was done with at least some effort. It's very misleading to post comparsions with a simple AviSynth resizing (one line of actual scripting?); it should be a good resizing *AND* a good sharpening to make it fair.

I'm not saying there's no improvement. There is indeed some. I'm saying there is little to gain from the "superresolution" theory and it's more about temporal denoising and sharpening. It'd be important to see where the gains are coming from, and if those are bigger than in the earlier projects.

I discussed some comparison frames from Team Blu's release with DrDre via PM. I figured it may upset some people to make this thread into a 'how is or isn't this better than Team Blu's work' thread. 

But even aside from the filters, I think Team Blu put a lot of work into cleaning up the GOUT as far as damage, glue, etc. It would be wonderful to see a collaboration with them, if they were interested, once these scripts are worked out. Especially if Blu's scripts in tandem with Dre's would produce superior results. But that might be expecting a bit too much or they might just prefer their own methods.

@ zee944

Although I certainly would like to compare super resolution to Team Blu's method or another, I simply don't have Team Blu's or another video available. So that complicates matters. Also Team Bu did a lot more to enhance the GOUT, like enhancing the colors for example. To make a fair comparison you would have to compare the results to Team Blu's upscale before the color corrections, which complicates matters further. As towne32 stated, he did provide a few screenshots for some comparisons (for which I'm very grateful). However, it's difficult to draw conclusions from those, considering the above. 

We did however post examples of frames where details are recovered by super resolutions that could never have been recovered with sharpening or (temporal) denoising. Take this screenshot for example that was created with an older super resolution script:

http://screenshotcomparison.com/comparison/122740

You can clearly see details reconstructed in Tarkin's face and Vader's chest plate. This cannot be done with sharpening and denoising. The same is true for a more recent example:

http://screenshotcomparison.com/comparison/124983

Again details are reconstructed in Vader's chest plate that simply are not visible in the GOUT. As a rule of thumb sharpening and denoising can only enhance details that are already visible. Details that are not visible are beyond retrieval with such methods. Super resolution uses information from multiple frames to reconstruct these details, increasing the true resolution of the frame. With sharpening and denoising details may be enhanced, but the resolution stays the same. That's not to say super resolution doesn't have it's drawbacks, but as far as detail reconstruction there is no comparison. 

In any case, this thread was not started to compete with any of the other methods seen on this forum. In a sense it was inspired by them. I was just interested to see what super resolution could do with the GOUT. So far I'm pleased with the results, and if Team Blu is interested in a comparison or cooperation I'm certainly in for it. I'm hoping to get Star Wars finished soon (although the constant tweaking has delayed the release unfortunately). Then TESB is next in line. As Team Blu is also gearing up for TESB, we may see some interesting comparisons in the near future. 

@ pittrek

If you can get me one of the scenes, I'll certainly give it a try :-).

@ NeverarGreat

Thanks, I'm hoping I can stop myself from more tweaking, so I can finally complete Star Wars ;-). 

DrDre said:

Again details are reconstructed in Vader's chest plate that simply are not visible in the GOUT. As a rule of thumb sharpening and denoising can only enhance details that are already visible. Details that are not visible are beyond retrieval with such methods. Super resolution uses information from multiple frames to reconstruct these details, increasing the true resolution of the frame. With sharpening and denoising details may be enhanced, but the resolution stays the same. That's not to say super resolution doesn't have it's drawbacks, but as far as detail reconstruction there is no comparison.

I did not say that super resolution is not working at all.

http://screenshotcomparison.com/comparison/125491

http://screenshotcomparison.com/comparison/125492

What do you think of these?

@ zee944

They look much improved. There seems to have been some deblurring done among other things, but I may be mistaken. Some detail has been lost on the walls, but very impressive nonetheless. Did you develop this methodology yourself?

DrDre said:

@ zee944

They look much improved. There seems to have been some deblurring done among other things, but I may be mistaken. Some detail has been lost on the walls, but very impressive nonetheless. Did you develop this methodology yourself?

Some details on the wall appear to be lost, yeah, although it may be mostly noise that looks like texture on still images. But there are new details weren't there on the originals, do we agree?

As for I developed the method... yes and no. I've used functions and plugins written by others.

@ zee944

Deblurring can indeed restore details that are normally not visible. In many cases however when the source is not blurry retrieving details can be difficult. Have you tried your method on the GOUT? 

However the best test is to downscale a real 1080p sample to 540p, and then upscale back to 1080p.

Here's a screenshot comparison for the Star Wars blu ray. In this case the raw super resolution upscale is shown. For a true upscale some denoising and regraining would be performed, but as an example for detail reconstruction it is good enough.

Original 1080p vs super resolution v8:

http://screenshotcomparison.com/comparison/125526

Avisynth Spline64Resize vs super resolution v8:

http://screenshotcomparison.com/comparison/125527

Here's the 540p video sample:

https://drive.google.com/file/d/0B8_LYKyZDiajV0pRbUFYeW1NM28/view?usp=sharing

I used frame 600. Why don't you try your method for a fair comparison?

@ DrDre,

You were careful enough not to acknowledge anything specific. :) I've posted those Twin Dragons screencaps to prove my point. The improvement on them is very impressive, as you stated too. If I'd have started a thread here similar to yours about superresolution with those caps, no doubt I'd have got reactions like "whoa, superresolution is at work, awesome". Especially if I would've done the same thing to GOUT.

And I couldn't even blame those posters. It indeed looks like resolution has improved and tiny new details appeared that weren't there before. And yet no superresolution was used. It's just denoising and sharpening. Of course, it needed lots of tweaking and experimenting. My point is, that it's nearly impossible to draw the line where denoising and sharpening ends and where superresolution begins. That's why fair comparsions are important.

I can't try out my "method", these are old caps. That's why I suggested Team Blu's project, they have done similar things, already put a lot of effort into it and they've done all of it on the GOUT. They have everything at hand for a fair comparsion.

It doesn't matter they've altered the colours or the contrast; the detail enhancement would be still comparable.

In your last post, you once again compare with your results with a very simple AviSynth resizing. This is not fair at all! One line of actual coding versus days of work? That's not a proof of superresolution concept. It's just a proof of more work.

@ zee944

The problem is that for this methodology no sharpening has been applied. In fact in the first iterations no denoising was applied. So, yes the detail enhancement is due to super resolution. You don't need to compare a Tesla to a Ferrari to know you have a supercar. Your argument is like arguing the Tesla's performance is most likely due to the number of cylinders (update for those that don't know this: Tesla produces electric supercars). If course it is interesting to compare them, but it is in no way misleading to show a method is able to retrieve details that are not visible in the original or a simple upscale. The fact that another method may be able to also retrieve these details doesn't take away from this fact. 

Now if you want to argue that super resolution has very little to offer, and that it is mostly due to sharpening and denoising, I suggest you write a scientific paper that proofs the last 20 years of scientific research is wrong. It's been scientifically proven time and time again that super resolution can retrieve details better than most other methods. 

As I've stated I'm very interested in comparing super resolution to Team Blu's upscale. Maybe their upscale is better, maybe not. However, if you argue you can reproduce my results with sharpening and denoising, you should be able to back that up with a direct comparison. Otherwise your words simply ring hollow. 

Edit:

You argue my comparison in my last post to the Avisynth Spline64Resize is unfair, but you forgot to mention that I also compared it to the true HD frame. This to me seems like the fairest comparison of all.