logo Sign In

Phantom Menace '99 - HD Theatrical Version by Chewielewis (a WIP) — Page 2

Author
Time
 (Edited)

x264ing it now, this is my ffmpeg command, missing anything?

ffmpeg -i /Volumes/Video\ 4/TPM99\ THEATRICAL\ CUT\ PREVIEW\ RC05.mov
-c:v libx264 -pix_fmt yuv420p -preset veryslow -tune film -profile:v high -level 4.1 -crf 17 -c:a copy
TPM99\ THEATRICAL\ CUT\ PREVIEW\ RC05.mp4

Just got back from solo, very good.

Author
Time

I’d normally do a 2 pass encoding, but not necessarily for a preview version.

Author
Time
 (Edited)

Hi,

Congrats for your work, it seems to be a huge step forward to a full HD theatrical version of TPM.
As you know, I’m currently working on my own HD reconstruction for almost 1 year. It seems that I took a different way to get a theatrical reconstruction in HD quality (some of my sources and tools are different than yours + some “missing” scenes like the arrival on Coruscant or the Senate will be in true HD in my version, not in upscaled SD).

Examples :
http://www.framecompare.com/screenshotcomparison/KD7LNNNX
http://www.framecompare.com/screenshotcomparison/FJ9JNNNU

It seems also that I took some different decisions than yours about theatrical audio sources, color grading, missing frames or subtitles, and that I detected some issues with the TV sources that you use (HDTV and ORF1), mainly about missing frames, colors and cropped images.

I still have several weeks of work before achieving something watchable. Perhaps the simplest way is that I also finish my version, and eventually we get together to propose an “ultimate” version based on your work and mine ? What do you think ?
Anyway, I do not want to give up a work I’ve been to for a year, and in the same time, I do not want to make a “concurrent version” of yours. So when I’ll have finished my version, I’ll suggest, if you’re interested, a collaboration to achieve the best result.

In the meantime, thank you and congratulations for the quality work you share!

Author
Time

ZigZig said:

Hi,

Congrats for your work, it seems to be a huge step forward to a full HD theatrical version of TPM.
As you know, I’m currently working on my own HD reconstruction for almost 1 year. It seems that I took a different way to get a theatrical reconstruction in HD quality (some of my sources and tools are different than yours + some “missing” scenes like the arrival on Coruscant or the Senate will be in true HD in my version, not in upscaled SD).

Thanks. I wanted to get this out quickly before the 35mm was released.

Yeah looks like your Senate shot is slightly better looking that mine. Im interested to know what sources you have that I don’t have and how you will be getting true HD for the Arrival on Coruscant scene, 35mm scan?

We could definitely consider collaborating on this, I’m pretty much ready to release this as a v1.0 in the next week or so. Ive had it sitting, ready to be finished since mid last year, I hit a stumbling block with audio sync issues and just ended up syncing directly to the LD audio rip.

As for missing or duped frames, I’m pretty sure I’ve “accounted” for all of them, any future (v2.0) release would have them restored and the audio patched to match it but since all the dropped/duped frames are on side/reel changes it’s not the end of the world to me.

Author
Time

I’ve just sent out links to the preview to all who offered their time, thanks people.

Author
Time

Chewielewis said:

I’ve just sent out links to the preview to all who offered their time, thanks people.

I messaged you earlier but didn’t receive a link.

Author
Time

My bad, was looking at the names in the thread, not my inbox

Author
Time
 (Edited)

Chewielewis said:

ZigZig said:

Hi,

Congrats for your work, it seems to be a huge step forward to a full HD theatrical version of TPM.
As you know, I’m currently working on my own HD reconstruction for almost 1 year. It seems that I took a different way to get a theatrical reconstruction in HD quality (some of my sources and tools are different than yours + some “missing” scenes like the arrival on Coruscant or the Senate will be in true HD in my version, not in upscaled SD).

Thanks. I wanted to get this out quickly before the 35mm was released.

Yeah looks like your Senate shot is slightly better looking that mine. Im interested to know what sources you have that I don’t have and how you will be getting true HD for the Arrival on Coruscant scene, 35mm scan?

We could definitely consider collaborating on this, I’m pretty much ready to release this as a v1.0 in the next week or so. Ive had it sitting, ready to be finished since mid last year, I hit a stumbling block with audio sync issues and just ended up syncing directly to the LD audio rip.

As for missing or duped frames, I’m pretty sure I’ve “accounted” for all of them, any future (v2.0) release would have them restored and the audio patched to match it but since all the dropped/duped frames are on side/reel changes it’s not the end of the world to me.

Thank you for your answer!

Feel free to release your v1.0 whenever you want, as I won’t finish my version before several weeks.
But IMHO, wanting to get this out quickly is maybe not a smart move, especially as we know that poita will probably not deliver his TPM 35 mm scan before a long time.
IMHO, it would be a pity if, by wanting to be the first to release, the work you propose would be partly incorrect or incomplete.

About the missing frames, there is maybe another issue than just reel changes (there is a global issue in some early digital transfers - maybe the ones you use? - so that every 4000th frame is dropped).
There are also missing frames around “de-subtitled” scenes in some HDTV versions.

Another problem concerns color corrections: there is more than just a general “too much red” correction to do: some scenes (including the entire first reel) were differently regraded than others on the DVD and the HDTV versions. So I guess that a thorough scene-by-scene regrading is necessary to recover real theatrical colors (actually, that is what I’m currently working on).

There is also a difference of framing/cropping for the couple of scenes that were redone or rescanned a posteriori: those scenes need to be recropped and skewed to fit the theatrical original format.

[JEDIT : I removed the paragraph about DTS audio, since Schorman brilliantly detailed everything below 😃 ]

About using other sources, a very good result can be achieved by using ^…^'s PaNup technique (https://originaltrilogy.com/topic/PaNup-or-how-to-upscale-PAL-NTSC-capture-and-live-quite-happy/id/15928) and merging as many NTSC and PAL sources as possible (there is a lot more theatrical sources than just LD and ORF1), then using an AI-based cloud service like letsenhance.io and finally precisely rotoscoping the result on a frame by frame basis.

I do not say that to criticize, far from it: I’m really happy that a TPM restoration work is made.
I just want to draw your attention to the fact that maybe we do not have to rush if we want the best result…

About the Arrival on Coruscant scene, I’ll be happy to make a frame comparison between my version and yours if you post one of your frames, you’ll see that my version is in true HD.

Author
Time

I’ve synced all six of the Films’ theatrical DTS mixes. I’ve also shared that with Chewielewis. It syncs perfectly with the other two LD tracks posted.

Of note:

Hal9000 discovered, and I have verified that the old winamp APT-X100 decoder has a problem with decoding the front right and surround right channels. Basically, there is intermittent noise in those two channels throughout, which can be especially noticeable during quieter passages.

For this reason I’ve reconstructed each mix using the Foobar2000 decoder, which does have a different set of decoding issues. Basically, the surround channels are decoded 3dB too loud. This can cause clipping in the audio during louder portions, but I have been able to correct this using de-clip in Izotope RX.

Details on the process:
Steps:

  1. Convert to 16bit/44.1kHz 5.1 wav with Foobar2000 apt-x100 DTS Decoder.

  2. Split to 6 mono wavs with ffmpeg.

  3. Pad each wav with 16 samples at the beginning using SoX to match decoder offset in the winamp decoder. (This is done to allow me to use my old Adobe Audition projects, originally created for use with the winamp decoder, in order to sync the tracks.)

  4. Edit Reels together in Adobe Audition and output full film as 6 mono wavs at 44.1kHz/16bit.

  5. Declip and attenuate the Ls and Rs channels by 3dB using Izotope Rx 6 Advanced (64Bit).

(For 24fps/44.1kHz/16bit)
6a. Dither Ls and Rs channels using Izotope Rx 6 Advanced (64bit) MBIT+ (noise shaping off, dither amount set to “Normal”, Auto-blanking on).

7a. Combine into a 5.1 channel wav using ffmpeg.

8a. Encode to 5.1 channel FLAC using eac3to.

(Result)
9a. 5.1 Channel flac file @ 44.1kHz/16bit/24fps

(For 23.976fps/48kHz/24bit)
(Convert to 23.976 fps)

6b. Leave Ls and Rs channels at 32bit float, then, for all channels, interpret Sampling Rate as 44,056Hz using Izotope RX 6 Advanced (64Bit).

7b. Resample to 48kHz using Izotope RX 6 Advanced (64Bit) SRC (Steepness=64, Cutoff Shift=1.00, Pre-ringing=0.50).

8b. Dither to 24bit using Izotope RX 6 Advanced (64Bit) MBIT+ (noise shaping off, dither amount set to “Normal”, Auto-blanking on)

9b. Combine tracks into a 5.1 channel wav using ffmpeg.

10b. Encode to 5.1 channel FLAC using eac3to.

11b. Encode to DTS-HD MA using the official encoder suite.

12b. Strip superfluous 2 frames and DTS header by running through eac3to with -21ms delay. Ex: “eac3to -input.dtshd -output.dtshd -21ms”

(Results)
13. 5.1 Channel flac file @ 48kHz/24bit/23.976fps

  1. 5.1 Channel DTS-HD MA file @ 48kHz/24bit/23.976fps

If I had some gum, I’d chew a hole into the sun…

Author
Time
 (Edited)

Waow, thank you for all these details.
I knew about your amazing job with LD audio archive, but not about this one!

schorman13 said:

Basically, the surround channels are decoded 3dB too loud. This can cause clipping in the audio during louder portions, but I have been able to correct this using de-clip in Izotope RX.

Isn’t that linked to the fact that in theatrical DTS, the LFE is added to each of the surround channels, and need to be filtered with a steep lowpass at 80 Hz ? Maybe the foobar2000 APT-X100 decoder doesn’t treat this particularity ? Basically, you shouldn’t have any sound below 80 Hz in the surround channels after having filtered the LFE signal.

Author
Time

The Foobar decoder does outputs a true 5.1 stream, whereas the old winamp decoder would only allow you to decode a single channel at a time. In addition to that, the winamp filter did not decode the LFE and left it as part of the surround channels, as it was encoded. The Foobar decoder does use lowpass/highpass filtering to create the LFE channel and Surround channels, but but does not take into account that the hardware DTS decoders are set to attenuate the surround channels by 3dB. The result is that the surround channels are 3dB too hot which can lead to digital clipping in those channels during loud passages, and of course over emphasizes the surround channels in the mix. The solution is to simply declip and attenuate the surround channels after decoding.

When using the Winamp decoder, the Surround and LFE filtering must all be done in stages using software, while it’s mostly automated with Foobar, except for the 3dB gain eduction. The drawback is that the filtering used by the decoder is a bit of a black box. There’s no way to know the quality of the filtering being done, the cutoff being used or it’s steepness, or whether it follows the DTS white paper. It would probably be preferable to use higher quality software like iZotope Ozone to do the EQ filtering, but since the Winamp decoder has shown those audio problems, the method described seems to be the best possible solution.

If I had some gum, I’d chew a hole into the sun…

Author
Time

schorman13 said:

The Foobar decoder does outputs a true 5.1 stream, whereas the old winamp decoder would only allow you to decode a single channel at a time. In addition to that, the winamp filter did not decode the LFE and left it as part of the surround channels, as it was encoded. The Foobar decoder does use lowpass/highpass filtering to create the LFE channel and Surround channels, but but does not take into account that the hardware DTS decoders are set to attenuate the surround channels by 3dB. The result is that the surround channels are 3dB too hot which can lead to digital clipping in those channels during loud passages, and of course over emphasizes the surround channels in the mix. The solution is to simply declip and attenuate the surround channels after decoding.

When using the Winamp decoder, the Surround and LFE filtering must all be done in stages using software, while it’s mostly automated with Foobar, except for the 3dB gain eduction. The drawback is that the filtering used by the decoder is a bit of a black box. There’s no way to know the quality of the filtering being done, the cutoff being used or it’s steepness, or whether it follows the DTS white paper. It would probably be preferable to use higher quality software like iZotope Ozone to do the EQ filtering, but since the Winamp decoder has shown those audio problems, the method described seems to be the best possible solution.

That is why I love this forum.

Author
Time

I’d be interested in seeing a comparison between Chewie’s Naboo bridge removal compared to yours, you can see that the area where the bridge was is softer on his screenshot. Just judging from your senate scene, you can make the image slightly clearer and have harder edges so I think it would blend in perfectly then.

The user formerly known as “Trillary Dump” (it was the 2016 election…)

Author
Time

trillary dump said:

I’d be interested in seeing a comparison between Chewie’s Naboo bridge removal compared to yours, you can see that the area where the bridge was is softer on his screenshot. Just judging from your senate scene, you can make the image slightly clearer and have harder edges so I think it would blend in perfectly then.

Indeed. I am not at home today but I’ll post a screenshot within the next 24 hours.

Author
Time

I guess I should be clear about the scope of this project. Really I’m just trying to recreate what they would have released on DVD had they not done the extended version. Somewhere out there on a shelf sits 3 D-5 HD video cassettes, holding a rough as guts telecine of the interpositive, used to create the laserdisc and the VHS. Thats what im trying to achieve, with the theatrical subs and cinema DTS as a fun extra.

I’ve actually gone and redid, from scratch, all 3 upscaled/patched shots. I think they look much better than they did before. I can’t say they are as good as yours, ZigZig, but they are good enough for this project. I used all 4 sources available to me (TB, ORF1, and two LD rips) then blended them in after effects. I haven’t quite got my head around the PaNup thing yet but this seems to do ok.

Imgur
Imgur
Imgur

As for color. Yeah they are a bit all over the place, blow out highlights in pink and yellow and blue. But I have no interest in regrading the film without a true color reference, like the 35mm. Currently this release is regraded to the theatrical video master using a known reference to do so (The THX DVD). When the 35mm comes out I might consider doing a full color correction but for now this will do. That being said, seeing how long it took for Despecialized to get its color right and then watching 4K77 fuss over it for two whole years and people still fighting over it, I think I’ll pass for now.

As for missing frames, I have done a lot of checks and everything seems to be accounted for. This is direct from a broadcast tape, not a capture off air so it’s pretty accurate. Only one frame issue which is present on the DVD too, as mentioned above, it’s been corrected.

Not worrying about cropping, The bridge shot is as close as I can get to the original crop, its fine. Senate shot I cant get anywhere near it since it’s shifted considerably. Everything else is fine.

Audio is taken care of, thanks Schorman.

Release Candidate 6 is rendering now.

Author
Time

Chewielewis said:

I guess I should be clear about the scope of this project. Really I’m just trying to recreate what they would have released on DVD had they not done the extended version. Somewhere out there on a shelf sits 3 D-5 HD video cassettes, holding a rough as guts telecine of the interpositive, used to create the laserdisc and the VHS. Thats what im trying to achieve, with the theatrical subs and cinema DTS as a fun extra.

That’s something I’ve always wanted to create by myself but with my very limited skills haven’t been able to do so. Answering the question “What the DVD would have looked like if it was released at the same time with VHS, VCD and Laserdisc?” I’m definitely looking forward to your version but also ZigZig’s and poita’s version because everyone has a slightly different goal and the end result will be different. It’s really nice that there’s finally three ACTIVE theatrical TPM HD projects going on!

Author
Time
 (Edited)

Skippy The Jedi Droid said:

I’m definitely looking forward to your version but also ZigZig’s and poita’s version because everyone has a slightly different goal and the end result will be different. It’s really nice that there’s finally three ACTIVE theatrical TPM HD projects going on!

Thank you! I fully agree with you: after years of famine, what a joy to see a craze for the restoration of theatrical TPM all around this forum!

Chewielewis said:

I’ve actually gone and redid, from scratch, all 3 upscaled/patched shots. I think they look much better than they did before.

Indeed, they look really great now!

trillary dump said:

I’d be interested in seeing a comparison between Chewie’s Naboo bridge removal compared to yours, you can see that the area where the bridge was is softer on his screenshot. Just judging from your senate scene, you can make the image slightly clearer and have harder edges so I think it would blend in perfectly then.

Here is the comparison you asked for (this is one of the scenes I have not finished yet). The area where the bridge was is maybe slightly less soft in my version, but it is still soft (I guess that the initial HD rendering was already softened) :

PALACE OF NABOO BY NIGHT (aka “there was no bridge”)
FrameCompare : http://www.framecompare.com/screenshotcomparison/FJ1CNNNU

Chewielewis
Imgur

ZigZig
Imgur

PALPATINE’S OFFICE
(Here, the difference between SD upscaled and true HD is more obvious)
FrameCompare : http://www.framecompare.com/image-compare/screenshotcomparison/KDY7NNNX

Chewielewis
Imgur

ZigZig
Imgur

I won’t make any other comparisons in this thread so as not to encroach further on Chewielewis’s work, and I’ll create my own thread when my own version will be ready.

Chewielewis said:
I can’t say they are as good as yours, ZigZig, but they are good enough for this project.

It is your project, it is (of course) up to you to estimate what is good enough, I do not want to interfere with my own quality requirements and I will not further parasitize your thread 😃 .

Author
Time

ZigZig said:
PALPATINE’S OFFICE
(Here, the difference between SD upscaled and true HD is more obvious)
FrameCompare : http://www.framecompare.com/image-compare/screenshotcomparison/KDY7NNNX

You definitely have a source that I don’t have, there’s a little more picture on either side and top and bottom that I don’t have on any of mine. One or two pixels worth. I’ve scoured these forums but I cant find any mention of a source other than the LD, the TB, ad the ORF1.

Although I’m not sure I would call this “true hd” Since you can scale it down to SD and back up to HD and not lose any detail.

Author
Time
 (Edited)

Chewielewis said:

Although I’m not sure I would call this “true hd” Since you can scale it down to SD and back up to HD and not lose any detail.

True HD it is 😃
But I guess that in 1999, the CGI rendering of this view of Palpatine’s office was not made in true HD, so that the 35mm IP eventually has no more details than what you see here, even on a “true” HD digital transfer.

JEDIT : anyway, your version is very compelling (based on your latest screenshots), I’m pretty sure that it is better than mine on other scenes that this particular one.
And, as I said, I won’t finish mine before several weeks.

Author
Time

Can you tell me what sources you have? You don’t have to give them to me.

Author
Time

Damn I think you guys should definitely collaborate soon! I don’t know how you did it with that air taxi sequence and the rotoscope elements but bravo! Can’t wait to see yours when it releases or maybe a collab in the distant future 😉

The user formerly known as “Trillary Dump” (it was the 2016 election…)

Author
Time

I second a collaboration between the both of you. Pool your sources and talents!

she/her
mwah

Author
Time

This looks fantastic! I’ve wanted to see a project like this done for a while now, and I’m excited to see the results. Thanks so much for your work.

Author
Time
 (Edited)

Chewielewis said:

Can you tell me what sources you have? You don’t have to give them to me.

You know, I already offered you to collaborate a few posts ago and I mean it, but I’m not sure that telling you my sources without giving them to you and then letting you do the job is the way of collaboration I expected 😃

To be fair, most of my sources are the result of one year’s job by deeply searching every possible official or unofficial video and photo source (and sometimes buying items like the VHS, VideoCD and Laserdisc), including early 35mm photos made 19 years ago and unofficial CAMS (i.a. the famous Z bootleg, but also another NY VHS CAM), contacting people on several forums, merging multiple sources to get a new one (cf. ^…^'s PaNup), learning a lot about video and audio restoration, acquiring softwares and hardwares sometimes expensive (Photoshop CC 2017, Video Enhancer 2.2, Izotope RX 6 audio editor Advanced), making my own LD captures - color-calibrated to NTSC-J colors with 7.5 IRE correction (and acquiring 3 LaserDisc turntables, a Leitch DSP-575 and a 300$ video card), financially participating to poita’s 35mm scan and even involving myself in Harmy’s 3.0 project, subscribing to a paid subscription on letsenhance.io cloud platform, eventually comparing every versions that I found to clean them on a frame by frame basis in Photoshop, regrade the colors of each scene, and then start all over because I found a new source with better quality…

So for now, if I share my sources today, I will have the impression that I give up all this journey just before finishing it, and I’m not sure that it will help you since your way and mine are quite different.

Honestly, when I tried a few posts ago (maybe clumsily, because English is not my mother tongue) to tell you about other sources than yours, about color regrading and about ^…^'s PaNup technique, you answered that your sources were good enough for your project, that you had no interest in regrading the film without a true reference, and that you “hadn’t quite got your head around the PaNup thing”. Actually, you didn’t seem to want to know more and I felt a little disappointed.

On the other hand, I already asked you details about one of your sources 10 days ago but you did not even answer me:

ZigZig said:

Chewielewis said:

SilverWook said:

Danfun128 said:

“no home release ever had the subs burned in.” So the VHS releases don’t count then?

Yes they would, but with the OT they didn’t always use the theatrical font. I presume the OP wants to see what was actually on release prints.

Yep. The Old HDTV has burn in that has the wrong font, style and timing, Theres also a new HDTV version with the correct timing, very similar style but not quite, and a few corretions.

Getting a close look at the 35mm is what i need for my TPM restoration (yeah, im doing one too.)

Could you please tell me more about this new HDTV version with correct timing ?

So for now, I think that the best move is that both of us release our own v1, and then that we benefit from this sporty “competition” to eventually unite our efforts in a v2 if it seems then to be feasible. I sincerely hope it will be!

Anyway, I’ll tell about the sources I have and explain everything in a new thread on this forum when my version will be done, I promise!

Author
Time

That’s fine, I’m not asking for copies of your work. But you say you have a “True HD” source but as far as anyone is aware this source does not exists. If it does exists I want to know about it. If you want to keep it to yourself thats totally understandable.

And I think that’s a fair question when you come into my thread and show how all your work is much better than my work.

On the other hand, I already asked you for one of your sources 10 days ago (https://originaltrilogy.com/topic/The-Phantom-Menace-on-35mm/id/56841/page/8#1208819) but you did not even answer me.

Sorry I didn’t follow up on that. I was in the middle of drafting this thread which would have that info. This is just a TV version I got of the 2011 master but panned and scanned, it’s not online. It’s no good for preservation but the subtitle timing is correct so it was an easier reference to use for dissolves than the Z master.