I'm finally almost finished with the super resolution upscale of Star Wars. I had a lot of trouble with audio synch, and getting the correct frame rate from the NTSC GOUT. The final product will be 720p (1280x544). I will post some clips and screenshot comparisons soon. Then it's on the Empire... ;-)
@ Laserdisc Man
Denoising does remove a lot of the detail. I tried a non local means filter. Looks good in some scenes, but in others you can clearly see where random true structure (like in clothes) is smoothed away.
What's the concept behind MagicUp?
@ Laserdisc Master
PanUp looks like a great algorithm! Very nice concept indeed. Would be interesting to combine the PanUp algorithm with super resolution. :-)
@ Laserdisc Master
I would have to agree. Is PanUp another one of your inventions? It looks good.
@ pittrek
I guess it's like with mp3. Some will hear no difference between 128kbps and 320kbps, while others will maintain the 128kbps sounds poor. For us that do hear the difference it can become an obsession...
@ Laserdisc Man
Here's the comparison of MagicUp vs super resolution compressed to 10Mbps:
@ Laserdisc Master
Yes, I will make the comparison at 10mbps. Hopefully I will have it done by tomorrow.
@ Padawan Learner
Perhaps you should check out the screenshots for the downscaled and then upscaled blu ray of Star Wars. I think this pretty convincingly shows that super resolution works under a wide variety of consitions. Of course it all depends on the quality of the source material, but in principle it can be a very powerful tool that, if the quality is good, easily outperforms single frame upscaling methods.
Also, you don't need aliasing for super resolution to work. In principle each pixel in a image or frame contains information about the object filmed or photographed. In case of subpixel shifts of the object (which is exactly what happens during movement in a video), the information in each pixel changes. By combining these different bits of information, subpixel detail is recovered. Although it is not easy to implement mathematically, the science is sound.
Infognition has a large number of video examples on there website (including Hollywood productions), where they objectively compare a large number of single frame and a few super resolution upscaling methods to their algorithm, including nnedi3. The infognition algorithm outperforms any of these methods by a significant margin for a magnification factor of 400%.
Link to the comparison of upscaling methods:
http://www.infognition.com/articles/video_resize_shootout.html
@ Spaced Ranger
Glad to hear you enjoy the work! More coming soon...
I actually noticed the NTSC GOUT's aspect ratio is a little bit off (as are my screenshots), 2.33:1 rather than 2.35:1. Since the super resolution algorithm only upscales by factors of 2, I get a 1428x548 output. I guess I either have to upscale to 1920x816 or downscale to 1280x544. Since the latter implies a reduction of the resolution, I guess I'll upscale with the Avisynth Spline64Resize.
Here are a couple of more comparisons for the Avisynth Spline64Resize and super resolution upscales of the NTSC GOUT Star Wars.
http://screenshotcomparison.com/comparison/119288
I guess this is a first. A comparison for the 4K upscale of the Star Wars blu ray for the Avisynth Spline64Resize and super resolution. The differences are much smaller, but you can still notice some improved detail in the stormtrooper helmets and Leia's face:
http://screenshotcomparison.com/comparison/119231
@ nightstalkerpoet
Here's are two comparisons for the 1080p original and the 4K super resolution downscale:
http://screenshotcomparison.com/comparison/119252
http://screenshotcomparison.com/comparison/119255
The differences seem to be very small.
@ Laserdisc Master The super resolution version indeed looks sharper and more detailed, but it's pretty close. What method do you use?
@ dclarkg In essence this is because the MagicUp version has less detail/noise. Both detailed and noisy images are more difficult to compress, so the png-file ends up being bigger.
Update: Clips removed to make space for new clips.
Here are two more screenshot comparisons (Avisynth Spline64Resize vs super resolution).
Frame 1:
http://screenshotcomparison.com/comparison/119148
Frame 2:
http://screenshotcomparison.com/comparison/119153
Note the enhanced detail in the walls for frame 1, and Luke's eyes, lips and hair for frame 2.
@ Laserdisc Master
Update: Clips removed to make space for new clips.
@ Laserdisc Master
Do you mean a lossless clip at 540p, 1080p or of the super resolution 1080p?
@Intruder
I noticed the mkv for the NTSC GOUT are jerky on my smart tv, but I guess that maybe more to do with the refresh rate. Also, I work with raw avi, and avi doesn't handle variable frame rates, although it apparently uses some kind of workaround. So, I'm in two minds about this. But I guess assuming the GOUT is 23.976fps originally, setting it to same as source with constant frame rate and a detelecine filter should do the trick.
I have a question. I hope one of you knows the answer. I've noted some issues with the frame rate while converting the NTSC GOUT (29.97fps, 23.976 after conversion) with Handbrake that I didn't seem to have when I converted the PAL GOUT (25fps). Now, I'm a bit confused. Should I use constant or variable frame rate in the conversion for the PAL and NTSC GOUT?
Update:
As far as I understand it PAL uses either 25fps or 50fps (interlaced). So always multiples of 25. NTSC uses 23.976fps or 29.97fps (interlaced). However, it can also be a combination of both if film footage is combined with video, leading to the variable frame rate.
@ FrankT
If you're refering to the blu ray I would say the difference is quite substantial. For the GOUT the differences are smaller (especially in the first screenshot), but you can see that Darth Vader's mask in the second screenshot is much less blurry in the details for the super resolution upscale. Other examples are the commanders face and hair in the same screenshot. I will post some screenshot comparisons where the differences are more substantial.
@Laserdisc Master
I assumed you wanted to see the 540p upscale. Here are the comparisons. There are a few tiny differences between Video Enhancer and the Virtualdub plugin. I think this is due to the fact that I used RGB mode. This option is not available in Virtualdub (the default is YUV mode). Also I don't have a license for the VD version, so there onfortunately are watermarks in that one. If you want to see other comparisons, please let me know.
Video Enhancer SR vs Virualdub SR
http://screenshotcomparison.com/comparison/118981
1080p original vs Virualdub SR:
http://screenshotcomparison.com/comparison/118982
Avisynth Spline64Resize vs Virualdub SR:
First comparison upscale NTSC GOUT. The difference for the first frame is very small (hair and eyes), however for the second frame the reconstructed detail is more substantial, which is particulary visible for Darth Vader's helmet.
Frame 1:
Avisynth Spline64Resize vs super resolution:
http://screenshotcomparison.com/comparison/118947
Frame 2:
Avisynth Spline64Resize vs super resolution:
@You_Too
Here is a comparison of two frames between the Avisynth Spline64Resize and super resolution for the Star Wars blu ray. As before the 1080p sample was dowscaled to 540p, and then upscaled back to 1080p.
Frame 1:
1080p original vs Avisynth Spline64Resize:
http://screenshotcomparison.com/comparison/118932
1080p original vs super resolution:
http://screenshotcomparison.com/comparison/118497
Avisynth Spline64Resize vs super resolution:
http://screenshotcomparison.com/comparison/118935
Frame 2:
1080p original vs Avisynth Spline64Resize:
http://screenshotcomparison.com/comparison/118936
1080p original vs super resolution:
http://screenshotcomparison.com/comparison/118506
Avisynth Spline64Resize vs super resolution:
http://screenshotcomparison.com/comparison/118938
It seems to me that the super resolution upscale is much more detailed. The Avisynth Spline64Resize upscale looks more more blurry.
The super resolution upscale of the NTSC version of Star Wars is indeed looking much better than the PAL version. Much more detail is recovered due to the obviously better source material. Clips coming soon...
@intruder I haven't yet looked in detail at the results after a scene change. I'm pretty sure it won't be as good, but by how much I don't know. Since we are dealing with commercial software the precise way they implement the super resolution concept is not known. It is one of the fastest implementations though, with upscaling speeds that approach real time.
@towne32 The lables are correct. Any upscaling algorithm will have some artifacts. Some of the "details" reconstructed will be phony details caused by apparent structure in noise. Whether these are acceptable I guess is a matter of taste. In the end it's all about the improvement compared to the low resolution version, and to what extend you can approach the 1080p original. Personally I think the amount of detail that is reconstructed is pretty impressive, and that it is a massive improvement over the bicubic upscale. All things considered, I find the artifacts to be pretty minor.
@LaserDisc Master I used handbrake to downscale the 1080p original using the h264 codec at 8000kps.
No problem, I will upscale the blu ray and then downscale it back to 1080p and will hopefully have a sample by next week.