logo Sign In

Post #1568397

Author
zenarch
Parent topic
VR.5 - AI Upscaled Edition
Link to post in topic
https://originaltrilogy.com/post/id/1568397/action/topic#1568397
Date created
9-Dec-2023, 2:16 PM

This has been taking a good deal longer than I thought it would, but I’m finally getting really good results. But I have been able to enhance faces, sharpen lines, smooth out curves, and enhance colors. I’m using HitPaw Video Enhancer for face enhancement and Microsoft Clipchamp for color enhancement. HitPaw is very very slow compared with Topaz, even when using GPU instead of CPU… but it makes far fewer mistakes on denoising and faces. However, it hallucinates a lot of detail if you try to upscale low quality video with it. But if you use HitPaw to get your video up to solid medium-quality, Topaz’s “Proteus” AI does a great job with upscaling it from there.

Here is the full set of steps I used on the first episode:

  1. Had to add a lossless video codec to Topaz. This is pretty easy to do, since you just have to edit an XML file. I used FFV1, since that’s what the guide I found on Google recommended.
  2. Run the video through HitPaw’s Face Quality model at 100 percent scale (no upscaling yet).
  3. Run your exported file through a color enhancer. The one that worked best for me on this video was Vaporwave, from Microsoft Clipchamp. Turn the Effect and Grain all the way down on this preset and adjust the intensity slowly, and it’s like the yellow cast is magically removed from the video. Now you can see (for example) how blue Louise Fletcher’s eyes are. I also had to play around a little with the sliders for color temperature, contrast, exposure, and saturation, to get rid of all the yellow tint. I had ClipChamp save the file in 720P resolution, so that HitPaw and Topaz won’t have to do as much guessing.
  4. Changing contrast and exposure revealed a moderate amount of new noise in the dark areas of the image, so I waited to de-noise and upscale the whole picture until this stage. The video was already clean enough for Topaz to treat it as medium quality video, so I was able to use Iris with Manual settings:
    a. First, crop to remove the black bars on the right and left of the image. (Microsoft ClipChamp had no option for 4:3 video, so I used 16:9.)
    b. Set a custom resolution, with height priority on. Set the height to 1080.
    c. Set Iris’s video quality setting to Medium.
    d. The rest of the Iris settings should vary some from video to video. For this one, they were Fix Compression 28, Improve Detail 75, Sharpen 40, Reduce Noise 1, Dehalo 65, Anti-Alias/Deblur 1, Recover Detail 90. The easiest way to find these settings is to set Iris to Manual and use the Estimate button on several scenes that look very different. I suggest using one bright and one dark indoor scene, one bright and one dark outdoor scene, and one VR scene. Then set each slider to the median of the estimates for those five scenes, and fine-tune using 5 or 10 second previews from there.
    e. Play around with adding a little noise under the Iris settings. This helps remove color banding that makes people look cartoonish. Also try the Grain setting, below that. These two together can help to add shading back to faces that have been over-smoothed by the upscaling process. My settings were Add Noise 4 under Iris, and Amount 3 and Size 5 under Grain.
    f. Export your file using h.264 or h.265.
  5. Run the exported file through HitPaw’s “Eliminate Flickering” tool under Repair. This got rid of the lingering VHS artifacts that weren’t removed by noise reduction earlier. Maybe this is the first thing I should have done. Will test it on other episodes and update this post.

Now what I have is a decent approximation of 1080p. Still need to experiment with interpolation to get the framerate up to 60, but it looks really good. PM me if you’d like to see.