logo Sign In

HFR 3D Restoration

Author
Time

I wanted to get a start on preserving 3D HFR formats, due to the recent success of Avatar The Way of Water. While yes, the format has garnered polarized reactions, I am of the belief that we should still have access to what the filmmakers originally released in theaters, very much like the case of the theatrical cuts of Star Wars. The two films I intended on focusing the restoration on is Gemini Man at 4K 3D 120fps and The Hobbit trilogy (extended of course) with 4K 3D 48fps.

Considering MVC encodes only can output 24fps with not the highest bit rate, I was thinking that it would be easier to output a full SBS encode. The crux of my idea is this, my main goal right now is to get Gemini Man back to 120fps 3D as it is one of two native HFR sources out there with the 4K release. What I was thinking is that if you can create a depth map just from the Bluray 3D version, you could apply it to the 4K version and create a new second image in the process. From there, you would be able to double the frame rate in whichever preferred motion interpolation software, reel to reel. Now the main question is, which software could be able to generate depth maps from a Bluray 3D source? The second question is which motion interpolation software works best to increase frame rate?

Thanks for reading and any thoughts or suggestions would be much appreciated

Author
Time
 (Edited)

Wow. I’d love to see if this is possible. I had to read your post twice to understand it.

You might want to try checking if the UHD HFR release is right or left eye only. If it is, you might possibly downscale the 4k source, and then take the missing eye side from the 3D BD, and then use an AI framerate increase on it.

Then you can skip depth maps, etc. Sure the extrapolate side will have lower quality, but when viewed in 3D the difference should smooth out.

I’d love to see this… assuming I have hardware capable of playing it.
Of course my preference would be the Hobbit films, since GM isn’t supposed to be very good and I skipped seeing it.

Edit: As far as the Hobbit goes, I’m fairly certain there is no such thing as 1080p48 on anything, 2D or 3D.

I always figured a HFR 2D release could be created (if you had a HFR source) by doing a double telecine to 60 fps and then decombing it. I suppose the same idea might hold for a 3D version.

Edit 2: Apparently my 8 year old Sony with Panasonic active 3d glasses handles 1080p60 3D just fine. I had my doubts.

I’m assuming you’ll keep the frame rate at the 59.94 that the UHD BD uses so you don’t have to re-encode the audio.

For anyone curious, here is a test I created. The source is the ‘Big Buck Bunny’ short (which has a Creative Commons license, so should be okay to post here). I split it, resized, restacked, re-encoded and slowed it to 1080p59.94 Half Side By Side 3D with Dolby 5.1 audio.
https://www.mediafire.com/file/e3xx8s70lc1terx/Big_Buck_Bunny_Half-SBS_3D_1080p59.94.mkv/file

Dr. M

Author
Time
 (Edited)

Thanks for the feedback! I was also surprised at how much detail you were able to retain with just the butterfly wings flapping, there were some test runs for The Hobbit (specifically the barrel sequence) were that look didn’t fully come together on my end, so very well done! I do want to clarify some of my statements based off your response.

Then you can skip depth maps, etc. Sure, the extrapolate side will have lower quality, but when viewed in 3D the difference should smooth out.

I did test this out (with Gemini Man) and unfortunately, the result is a depth mess. Since the frame rates aren’t fully synced, there are constant moments where frames don’t line up. The process for using a depth map would cause less headaches for preservation purposes and there won’t be any sync issues. Not to mention that with a few tweeks in davinci resolve’s node tree, you can get a depth map applied seamlessly to the original source. Sure, it’s not the native image they used, but the process is used in every other Hollywood 3D conversion, it would be barely noticed. The only issue is finding a way to get a depth map from the original 3D bluray.

I always figured a HFR 2D release could be created (if you had a HFR source) by doing a double telecine to 60 fps and then decombing it. I suppose the same idea might hold for a 3D version.

That probably would work, however I am not familiar with the process of decombing, how would that look?

I’m assuming you’ll keep the frame rate at the 59.94 that the UHD BD uses so you don’t have to re-encode the audio.

Initially for the Ang Lee HFR films, that is the plan. But I do want to motion interpolate later in the process to bring the image back to 120fps. He made a big stink about that format, so I want to try to give peeps the opportunity to experience it, regardless if it seems unnecessary.

Author
Time

Yeah you’ll likely end up just post converting whatever you get from your interpolation efforts of the UHD source. I also imagine that you’ll be doing multiple passes through some frames as the last time I played around with ML based interpolation the artifacting on transition and fast motion frames was pretty bad.

Author
Time

Hey y’all, have some updates.

I did a V1 pass with just the normal rife interpolation and removing end frame manually, but the results were inconsistent to say the least. Majority of the results were good, but they need more cleanup. So I started from scratch for V2 and doing a complete manual cleanup of the original 3D bluray source. I have yet to find the means on fusion to convert the 4K footage, so I’m just going to upscale the 1080p to 4K.

There lies some other issues with the 4k as it is different color timing altogether (ie flashback scenes now all have a sepia tone filter to indicate their date) and some of the 4k footage is a bit too soft when converted to 4444 prores, possibly done in service of removing digital sharpness to make the 4k image comparable to the Lord of the Rings remaster.

This time around, I worked with 4 variations of interpolation, 96 and 120fps with and without motion blur, from both speed warp and rife.

Here are the first ten minutes with this new method. Keep in mind, plenty of work to be done for frame work and I am still working out the kinks on finding a balance between sharpness and retaining detail for the upscale.

Other than that, let me know what you think and I hope you enjoy!

https://youtu.be/v3pA7M9Sa3A

Author
Time

You did a commendable efforts to preserve 3D HFR formats. Its really a Impressive initiative to preserve 3D HFR formats! Your plan to restore Gemini Man and The Hobbit trilogy to their native HFR formats is commendable. Utilizing MVC encodes and generating depth maps from Bluray 3D sources show innovative thinking. Best of luck with your restoration efforts!
Thankyou