Joel Hruska

FrankB said:

Animaxx said:

Another little note here: My old device had a mode (unfortunately I can’t remember the exact name, let’s just call it “Pure SD-Resolution”) which played files in their natural ratio/size, only going to a max of 1080p, which - funny as it sounds - looked better than the 4K upscale my present device does. So the only way around that is to use a separate player, that can play back the older files in original resolution, since the TV always upscales (which is as I have said fine for everything else, just not the older shows).

So the technical evolution of playback devices may “force” fans to take action in order to keep enjoying their shows because they look worse through enhancements they can’t change manually on the devices themselves.

This certainly has to do with bad deinterlacing… You are right: Especially NTSC-DVDs can be watched best, when you set HDMI to 576i. That may be the mode your player used.

I just looked into the upscaled pilot. Well, I don’t want to be negative, but it’s not for me. And I still think, there is not REALLY much done about new details. Everything Topaz did, seem to me to be also possible with slight sharpenings (if you know how) and simple resizing.

But: This can be the future, and everything new needs pioneers who just do it, become better and better, inspire even better techniques, a. s. o.

Maybe it’s also about youth and age. I watched my first Star Trek episodes in Germany in 1972, so I think I can also wait a few years longer to see something really revolutionary.
But keep on to move forward, where no man has gone before!

FrankB,

The problem is DS9 itself. There’s a reason I didn’t use Emissary (1x01) to show off my work. Early Deep Space 9 does not respond to upscaling nearly as well as later DS9 does. It’s not as clear. It’s not as sharp. It makes early greenscreen effects look really fake, and the footage can almost look as though it’s being spliced in from VHS. There’s moire on everything, the models are dull, and it scarcely looks like an upgrade. Some of that is baked right into the source and looks bad there (like the early holodeck scene), but gets magnified by the upscaler. But the show looks bad – really bad – on DVD.

It’s Deep Space Nine’s fault. And the reason I know that is because later episodes of the show, filmed with different techniques, upscale beautifully using the exact same settings.

I do not yet know how to meaningfully clean up Emissary, and Animaxx will tell you that my QTGMC script that I shared with him is excellent for denoising and fixing artifacts. It’s not good enough. Sharpening isn’t enough. The rainbow filters I’ve run against the content haven’t worked, and it’s very hard to draw detail out of scenes.

The level of improvement I get in Emissary is less than half the improvement I get in episodes from Season 4 forward.

Animaxx said:

@ Frank: Certainly I would want best quality. I will try on a short sample clip and see.

@Joel: Loved your latest article over on extremetech on fans getting impatient and doing there own upscaling. Did we inspire you’re latest journalistic endeavor a bit? 😃

You are one of the people I was referring to, certainly. There are at least a half-dozen or so DS9 upscaling projects. There’s a big Babylon 5 project. I’ve met people on the Topaz forums working on all sorts of varied stuff.

I do not condone piracy and will not be distributing my own work illegally, even though that means my version of the show will be the one virtually no one ever sees. But that was the price of getting permission to work on the story in the first place. Hopefully people who decide to undertake these efforts will pay for the DVD sets of the discs they want to work on. If you feel a show is worth remastering, it’s worth investing in to do it right.

FrankB said:

I see. “true” means audio resampling. I never used this, we handle audio mostly with Audition. I hope the quality will be the same with your “true”-method.

Frank,

If it isn’t, there are other ways to slow the audio. Bolting the NTSC tracks on works perfectly, for example, and this function allows you to set the interval properly.

http://avisynth.nl/index.php/DelayAudio

FrankB said:

I don’t know anything about Staxrip, but this might work:
Staxrip-Framerate AssumeFps

Like Frankb said. Just run:

AssumeFPS(24000,1001,true)

That will slow your video down. You need to make a call to sync_audio to also shift the audio. I didn’t do this because I decided to just use NTSC audio. The “True” command tells the filter to synchronize and slow the audio by 4%.

If you make a clip available to me, I’ll verify your audio is properly slowed. Emissary or Sacrifice of Angels preferred, because I have those ripped already and I know you have Emissary finished.

Also, Frank: You are absolutely correct re: the precise amount of slowdown, but AssumeFPS handles the calculation after the decimal correctly, as far as I know. Animaxx shouldn’t need to set anything beyond “24000,1001,true”

FrankB said:

Animaxx said:
Strangely enough, the tv broadcasts and dvd versions are higher pitched around germany than the original vhs tapes, which had lower pitch.

Interesting, I didn’t know that. That means:
By the time the series was brought to Germany it was often practised technique to convert from NTSC to PAL in a mix of blending and keeping the original interlacing, especially with such mix-content of pulldowned and native 29.97 material.
So they seem to have dubbed it in its original length. This is what is also on the VHS cassettes.
The newer masters for the DVDs were carefully IVTCed (and the 29.97 portions brought to 23.976 in I-don’t -know-what-way), and then sped up to 25fps. Then they sped up the sound without correcting the pitch to its lower original. So you are damned right when you pitch down - even better would have been to slow down to 23.976 and then simply resample the German dub down, pitch would be corrected automatically and everything fine!

As for the speed down to 23,976 FPS: That would cause motion stutter again, which I was happy to have avoided with PAL at 25 FPS.

That’s an error. Slowing down does not at all cause any stutter, if you do it right - that means the very simplest way. Try it with avisynth with, as I said:
new=assumefps(old,24000,1001)
This will just change the SPEED, no t one frame will be added or dropped.

I can confirm what FrankB has said here. In fact, I experimented with doing this.

You can slow a PAL broadcast from 25 to 23.976 fps and never notice that you’ve done it. Then you can pitch-shift the audio down by 4% or get yourself the NTSC discs. Either works. I tried stapling the NTSC audio to a PAL rip that I’d performed this on, and while I had to add a 1.2 second offset to the track, it worked perfectly for alignment once I did. No stuttering.

This is why I seriously considered using PAL instead of NTSC for my project.

Animaxx said:

Guys, I have done it. The 4K-pilot is finished. It was rendered at 15 Mbps in 4:3 format at a display size of 2846x2160p (4:3).

I also used the original NTSC audio and adapted to the PAL-running time while maintaining the original pitch; also, I have pitch corrected the german audio so the PAL-Speedup has been taken care of (no more high pitched voices).

The motion is nice and smooth (as it is at 25 FPS).

There are only minor imperfections: Slight block artefacts with explosions and fast moving scenes (barely noticeable), sometimes the added grain is more visible but also adds nice detail, so I will keep that for it looks less “oversmoothened/arteficial”. There is still some minor rainbowing and there is that one frame were the deinterlacer couldn’t handle it all, but it’s again minor. Unfortunately with the pilot, there is a sort of “double border” on top/bottom, but that is only present with the pilot and I didn’t want to crop further to avoid losing parts of the image.

Overall, I am happy with the result. I think it is a really nice compromise between upscaled/enhanced quality, retained detail and filtered image properties.
The 1080p-version is being rendered right now and should be done in 12 hours. I will then upload that as well.

Finally, I will do comparison shots and write the text for the release topic. I will invite you guys and then you can get your hands on the pilot.

As for specs: The 4K-Version will have a size of roughly 10 GB (9,9) and the 1080p-Version will have about 6 GB. Since the pilot (double episode) runs 87 minutes (PAL) and the single episodes clock in at about 43 minutes (PAL) we can assume that the size for the singles will be about 5 GB at 4K and 3 GB at 1080p.

I think size-wise those figures would be ok, since a complete 26 episode season at 4K would take up 130 GB and at 1080p it would take 75-80 GB. In comparison, if I were to copy the DVD-files with original sizes (1,8 GB for singles at SD-Resolution) I would end up with a season count of 45-50 GB, so that appears reasonable to me.

I hope you’ll agree.

Are you using actual 3840x2160, or are you using 4K loosely? (Meaning, like, a 4x upscale instead of a true 4K upscale) Assuming you mean it literally – have you compared the quality gain from jumping that high?

The reason I’ve stuck to 2560x1920 is because I don’t get any benefit from going higher. I do get a benefit from using Topaz in 4x mode, because the 2x modes seem to blend poorly, but I don’t see an improvement from anything more.

I have also found that using x265 with a CRF=19 is pretty solid. I can barely see the smallest difference between CRF=6 and CRF=20, while I can’t see a difference at all, really, at CRF=19.

Running H.265 in “Slow” or “Very Slow” will also improve your compression. Compression is much more improvable than I ever believed. But this is also why I ask about the benefits of 3840x2160 – because if you get the same quality at, say, 2560x1440 than you do at 3840x2160, you could save yourself a lot of bandwidth just by getting rid of pixels people don’t actually meaningfully benefit from.

If you do benefit from 4K, I recommend trying H.265 with Slow encode speed to see what kind of compression boost you get.

FrankB said:

Joel Hruska said:
Seriously, Frank. In no way am I trying to either imply you are unhelpful or crap on your expertise.

Never mind, so didn’t I with yours. Maybe my English is not that good, and you feel wrong subtle “vibrations”, not meant. It is not really easy to help, if someone of high intelligence like you is in the job for months, already did hundreds of experiments a. s. o.
But in spite of this you - sorry - seem not to really understand what really happens with the original 24fps-film when being pulldowned.
Ok, you see RELATIVE smooth motion in your 60fps result, because you use interpolated frames, but this artificially extended motion cannot be really smooth, it’s sometimes faster, sometimes slower. This can’t be called “stutter”, ok, but it is not the original smooth motion, where one frame of the original celluloid-film is one frame (or at least a multiple of one for EACH frame) in the end result. This is simply not possible, if you have 24fps film and 60fps end result.

I am preparing some clips to show you final output of the method. I want to make sure I am clear about what I expect you to see:

1). Recovery of the same progressive frames as in the 23.976 fps Rio Grande version.
2). Interpolated frames that provide the smoothness required to make the 59.94 version move smoothly in places where the 23.976 fps version fails to recover ideal motion.

The 60 fps motion of Orinoco is smoother than the 119.88 fps conversions that I’ve tested (I tested 119.88 as a way to equalize the 23.976 fps and 29.97 fps content). The motion is less irregular. There are still absolutely interpolated frames being used to smooth out the presentation and I won’t claim there aren’t, but the subtle, hitching jerking that I spent so many months trying to repair has vanished. I’ve messed around with changing the frame rate from 23.976 to 29.97, 47.952, 59.94, even 71.9128 (a little) and 119.88 (a lot).

The encode method I call Orinoco is the best 59.94 conversion method I have found. I’d prefer not to use it. It takes a very long time to upscale 59.94 content. But it does solve problems in the places where Rio Grande leaves bad motion – and it does so without causing visible frame rate stutter. Are the interpolated frames still there? Absolutely. Do I want them there? No. I’m just willing to tolerate them if I have to.

Pulldown is done by doubling several fields (several ways to do so, doesn’t matter here). These have to be decimated to get back the clean progressive, original content, which is necessary to fill the AI upscaler, and to get really smooth and good results. > Your interpolated frames are not really good anyway concerning picture quality, but make also motion slower and faster.

Just the portions where you might have native 29.97fps, (interlaced or not, not important because you deinterlace), will work, because there you have simply doubled frames, when going to 60 (59.94)fps. This is smooth, ok.
I don’t think, it’s the way to go, to keep any double frames, and even worse to add interpolated frames to make it a bit(!) smoother. So your decision to get to 23.976 was the only way to go.

I will show you Orinoco in sections that are originally 29.97 and originally 23.976. That’ll be the easiest way to compare.

Again: If you like to see a IVTC-by-hand-script, send me some portions of the NTSC-DVDs. Also the opening credits are interesting for me - is this field-shifted? Interlaced? Pulldowned AND field-shifted?

I am going to send you the credits and the first part of an episode with multiple 23.976 --> 29.97 transitions in it. About 7:30 worth of footage. This will be a direct copy of the DVD. Prepping these today.

FrankB said:

This is not a creative art, as one could think sometimes. It’s just preservation of something others created, and which is worth to be preserved the best possible way. The result counts, that’s all!
The real big thing is to create something like Star Trek, compared to this, we are all small, but it’s always fun and very rewarding to somehow do something good for this worthy content, I think Joel feels it similar.

I agree with you that it is fun and rewarding to do some good for this worthy content, no question. I think of the people who do this kind of work – amateur and professional, provided you really care about it – as being in the field of “art restoration” if you wanted to call it that.

A person who paints paintings is not the same as the person who restores them, but you cannot restore art if you do not understand something of the artist. The creativity in restoration comes in choosing which techniques to apply, and maybe in finding / inventing new techniques that handle something better than a previous solution did.

FrankB said:

This is not a creative art, as one could think sometimes. It’s just preservation of something others created, and which is worth to be preserved the best possible way. The result counts, that’s all!
The real big thing is to create something like Star Trek, compared to this, we are all small, but it’s always fun and very rewarding to somehow do something good for this worthy content, I think Joel feels it similar.

Frank,

I will get you some samples made today. Do you have a Microsoft OneDrive account or do you prefer Google Drive?

Animaxx said:

You guys know when I was satisfied? When I was finally able to see the freaking Aztec-Pattern on the ships hull (last shot) - I know, I am so ready for the looney bin, but I used to/still do build starship models, so details are a point of obsession for me …

Just wanted to let you know: I’m definitely getting a quality boost off some of these changes. I need to tweak them to fit the somewhat different characteristics of the DVD source, but you have meaningfully boosted my project along. 😃

FrankB said:

Just tried to help. I also do this for 20 years, professionally for more than 12 years. And I do not have to boast somehow either. And 60/24 makes 2.5. Good luck, and high respect for what you’re doing!

I’m not saying you aren’t helping. I’m saying that when I say the output is smooth, I’m not wrong about it. In TDeint’s case, we’re interpolating frames to get there. I said as much, at multiple times. That’s why I gave you an extensive explanation for what scripts I run and how I process the file.

The reason the method works is because the interpolated frames are near-duplicates of the originals. The reason they don’t get noticed is because we repair the frame boundary issues that make them noticeable. The end result is an image that maintains the same smoothness as the original.

All data is stored on an NTSC DVD as 59.94 interlaced fields, including these DVDs. We keep the three progressive frames and repair the interlaced ones, either by weaving them, repairing via the “Repair” command, or outright borrowing them from a second file. These methods can be used to eliminate the distorted frames at frame boundaries.

Because all data is stored on these NTSC DVDs in 59.94 interlaced fields, 59.94 fps is not a problematic target as far as motion is concerned. A D2V file, processed in this method, runs at 29.97 fps not 23.976 fps by default. Progressive frames are retained, interlaced frames are deinterlaced, and new frames are interpolated.

https://www.extremetech.com/wp-content/uploads/2020/09/Sacrifice-Correct.jpg

This is what a proper Staxrip file looks like before you run a program like TDeint. That’s what you see when you pull a D2V file into the application for editing. Then you run TDeint against this file and you double the output rate to 60 fps. There is no invocation of the 2.5x multiplier. At 60 fps (having used TDeint for interpolation), the video remains perfectly synced with the audio and neither 23.976 nor 29.97 fps content moves too quickly.

I will upload you video clips if you would like to see it.

EDIT:

Seriously, Frank. In no way am I trying to either imply you are unhelpful or crap on your expertise.

FrankB said:

Joel Hruska said:
Alright. Now I see it. If that’s what you call a major anti-aliasing problem, I strongly suggest you never watch the NTSC credits. I cannot upload them to YouTube in SD because the YT algorithm utterly destroys SD content, so I have to give you screenshots – but take a look.

Looks like shifted fields. Can you upload the credits, or a few seconds of it, without conversion somewhere?
I do not understand where the problem realy lies?

Look at the degree of aliasing in those images, and I think you’ll understand why I’m experimenting with TR2=4 or TR2=5. TR2=3 creates a ripple across the front of the station that TR2=4 (at least) helps fix.

Seems not the right way to handle this, but let me see it first - maybe I will have to admit it IS that horrible. But hard to believe.

The reason to write an article telling people how to deal with PAL is so that PAL people know how to best convert the show. I am not advocating for some kind of code of honor. I want to produce the best overall version of Deep Space Nine. But the goal is to provide a “Best-in-class” improvement method to everyone, which means I’m also interested in the best way to handle PAL.

Not much difference to NTSC. After you made NTSC sources progressive and the anti-aliased possible rest of aliasing which came from reordering fields it should be pretty much the same procedure plus a slowdown at the very end.

Doom9 is correct that IVTC is complicated. The only way to perform it perfectly, as far as I know, is to hand-comb every scene and manually tune the frame order method via a TFM OVR file.

It is even better not to write a file for TFM, but to IVTC completely by hand. Not very complicated, I do this all the time, if the pattern doesn’t change too often. With the amount of episodes it is quite a job, though, but writing files for TFM also… If you like, I can post a typical script for by-hand-IVTC.

However – it turns out that the following actually works pretty damn well:

TFM()
TDecimate()

Would be easy then, but if you do so with native 29.97(i or p) content there will be missing frames and skipping (is that the right word?) all the time.

I have also developed a secondary method of creating a 60 fps method of the show that matches the quality of what I’ve shown you.

No good idea, because it doesn’t remove stuttering from telecine-double-fields. That is the whole problem with content like this. Progressive 30, 60 will stutter with originally film=24fps. How could this be avoided? There have to be double frames, at least fields, or can you divide 60 through 24?

Still searching for a way to automate it perfectly (the commands above are automatic, but not perfect).

Don’t exactly know, what’s your goal. But simply the first necessary, to IVTC, is not perfectly automatically possible. That’s why we do it by hand if ever possible.

Initial evaluation of naive implementation of AniMaxx’s algorithm suggests it’s oversharpening in my case. I like the overall output otherwise. Going to adjust some variables, then toss in the rest and see what it looks like. 😃

I wouldn’t sharpen at all, only a bit unsharpmasking, but that’s another question… 😉

Would be easy then, but if you do so with native 29.97(i or p) content there will be missing frames and skipping (is that the right word?) all the time.

There isn’t.

While I have self-identified as a noob, I need to ask you to trust me to some extent. I am not always right about what I am seeing, or why I’m seeing it, and I don’t always understand how to fix a problem I encounter, but I have spent thousands of hours staring at this show in the last nine months. I have accelerated it to every single frame multiple between 23.976 and well above 1000. I have interpolated it with at least six different filters. I have tested source derived from MakeMKV, Handbrake, and DVD Decrypter. When I decided I wanted to test the impact of using Handbrake to process the DVD, I ripped the same DVD more than 250 times in a several-day period. I ran every single potential combination of filters, frame rates, and output file types that I was interested in, and then I started watching them – at between 3-5 minutes per watch – to evaluate the impact of various settings.

I won’t tell you that I watched the entire set of videos, because I didn’t. Once I was able to start predicting what output would look like, I was able to speed through my testing more quickly. Eventually I realized these methods would not serve me, and I quit after looking at ~120 videos of the 250 or so I rendered.

I’m not explaining this to boast or brag or to come off as thin-skinned. You don’t know me from Adam, and you don’t have any specific reason to think I know what I’m talking about, especially when I openly identify as being very new to this work. I figure it’s on me to explain the level of detail and rigor with which I have approached the process.

Here is what it means when I say that the TFM / TDecimate combination works. It means that I’ve run it against multiple episodes with a high film percentage, but have seen only very isolated incidents of motion judder. I’m comfortable asking people to accept 3 seconds of bad motion in a 45-minute episode of TV. If I have to ask for 5 seconds, I’ll wince, but I’ll do it. If I have to ask someone to tolerate 10 seconds of bad motion, total, in one 45 minute video, I will do so only after I have made every effort I can make to fix it. If I had to ask someone to tolerate 60 seconds of bad footage in an episode, I’d refuse to publish my methods. That’s why I didn’t publish my methods in April or May. They weren’t worth publishing.

Here. If you want to see the difference between my old methods and my new ones, watch this other version of the DS9 credits that I did (I’m going to give you the time codes in the episode to show you the differences directly):

Watch the following: https://www.youtube.com/watch?v=YbjqaVHZvAA

0:21 - 0:48
0:59 - 1:14
1:27 - 1:36

Now, compare those exact same sections to the credits I gave you this morning. Ignore any differences in image quality and just look at the motion.

https://www.youtube.com/watch?v=zMaMHT4skn0

The first video shows you what my original approach looked like. It’s very nearly as pretty in terms of achieved image quality, but the motion is jerky and poor at multiple points. Too many points.

The video I sent you this morning is the video created when you use my script and preface it with: TFM() / TDecimate()

After the blue flash, the credits are in 29.97 until the end. And I’ve tested the method in more than just the credits.

I’m basically willing to tolerate less-than-great motion in a small portion of one scene, maybe two scenes tops. So far, that TFM / Tdecimate approach has hit that quality level. It might prove unable to do so in certain places. Where it can’t, I’ll find a different method and publish it, or nudge people towards Orinoco.

No good idea, because it doesn’t remove stuttering from telecine-double-fields. That is the whole problem with content like this. Progressive 30, 60 will stutter with originally film=24fps. How could this be avoided? There have to be double frames, at least fields, or can you divide 60 through 24?

Here’s how my method of doing this works.

We create two files, not just one. First script, for first file:

TDeint(mode=1, type=2, tryweave=true, mtnmode=3, full=false, ap=10, aptype=2, slow=2)

This script orders TDeint to produce doubled frame output, to perform kernel interpolation, to attempt to repair a frame by weaving if the result is fewer combing artifacts than deinterlacing would create, and to only deinterlace interlaced frames. This preserves the progressive frames baked into the NTSC source. Type=5 was the only option that came close to Type=2 in overall image quality, and the bi-directional blending Type=5 introduces causes more frame blending issues at scene boundaries. Type 5 occasionally fixes a problem in Type 2, but it causes them far more often than it repairs them.

Now we set this clip aside and turn our attention to the other. We run QTGMC against the second clip with the exact same processing steps I use otherwise except, we don’t use progressive repair mode. We use interpolation mode, and we take the frame rate up to 59.94 fps.

Here’s why we do this: The deinterlaced clip created by TDeint will have perfectly smooth motion thanks to the frames it interpolates. Those frames will be well-blended because the script above performs very good blending during the same scene. Scene boundaries, however, present a problem. It is not unusual to find a character with a fireball blowing out of her forehead at a scene boundary because the previous scene was an explosion.

But there’s a way to fix this: Repair.

We now take our two files and write the following script:

clip1=FFVideoSource(“C:\DS9S6D2\Sacrifice-TDeint.mkv”)
clip2=FFVideoSource(“C:\DS9S6D2\Sacrifice-QTGMC-ToPairWithTDeint.mkv”)
Repair(clip1, clip2, 9)

This process repairs the broken frames that had bad interpolated data in them.

This method – which I nicknamed Orinoco – is motion-perfect and quality-identical to the episode videos you have already watched. I do not have footage to show you straight to hand, though I could get some with a bit of processing time, but I have no reason to lie. I have published my methods and staked my reputation on them.

I was going to go to press with Orinoco, and only Orinoco, before I found my 23.976 fps solution at the last second. When I say the image quality is identical to what you have seen with perfect motion, I do not mean “Except in the places I don’t want to admit exist.” I mean “In every single instance I am aware of thus far.”

I have high quality standards. I don’t like to compromise on them.

Artan42 said:

I’m enjoying all this technical detail. I have no bloody clue what is being said (it may as well be in Klingonese for all I know) but it looks like there’s several people who really know what needs to be done here.

As for CBS, every time somebody uploads a test video, I go watch a couple of episodes on Netflix. I think it’ll be the viewing figures for DS9 and VGR that make the case to CBS that they might have to pull their fingers out eventually as they’ll be the only two of the nine produced TV shows without a HD release.

As for CBS, every time somebody uploads a test video, I go watch a couple of episodes on Netflix.

I uploaded 16 videos this week. (I’m not actually shamelessly pumping my channel, so feel free to ignore it. But I’m also not kidding).

https://www.youtube.com/user/Dputiger

If you’ve already seen my story from earlier this week, apologies, but if you didn’t: I rendered out major scenes from the show, both dramatic and combat-oriented, using a variety of different models and presets from Topaz Video Enhance AI. The scene with the largest number of available presets (to compare various AI Upscale models) is “Second Fleet.” Gaia-CG is the most common model I use. Do not use the Artemis models on DS9, ever. You don’t actually have to watch 32 episodes of DS9 in exchange for watching. 😉

FrankB said:

Joel Hruska said:
I’m experimenting with renoise options at the moment, to see how they impact things. Also, yes, trying to calibrate the proper amount of processing and denoising to do in the front-end before letting an AI program have a go at it. Some AI models include denoising that works effectively but I’d rather not have to use them in the first place.

My opinion: Better choice. Because it will kill details rather than create “new ones”, if you let the AI denoise. Just my experience, there may be other scenarios. Renoising - to say it again - is best if you use the original noise. This is not common practice, by the way. Mostly they use some more or less random algorithms, as you surely know.

It’s not the FX scenes that are automatically in 29.97. In fact, in the first season, at least some episodes are basically 100% film. I don’t know when this stops. In others, like Sacrifice of Angels, most of the battle scenes are 23.976 fps, though there’s one post-credits scene that has preserved incidents of 3:2 pulldown in a 29.97 fps stream. That one threw me for awhile, trying to figure out how that could happen. Baked-in source error is awesome.

So there ARE pulldowned 23.976 and native 29.97 scenes, right? Just this fact makes it A LOT harder to deal with the NTSC sources instead of PAL, if you finally want
-progressive (to use with AI)
-stutter-free
results.
In addition the PAL sources have less aliasing. So two very strong reasons to use PAL as sources.
And are there also scenes where they overlayed both? Am I right with this speculation? We had this in a series I worked at a few years ago, also SciFi, made about the same time as DS9, a bit earlier. The overlayed scenes are definitely not stutter-free, and there is no way to IVTC it 100% correctly.
We handled, by the way, the native 29.97 scenes different from the well IVTCed 23.976-scenes and had to convert it with Alchemist optical flow, which was the best option at that time (today also AI is somehow better in these cases…).

Sorry, but in screenshot 2 there is more aliasing than in 1. Look at the shoulder.

I’m not seeing it. I see one pattern that might be what you are talking about, but doesn’t come across as aliased when the actor is in motion:

The most left part with the light background. A big difference concerning “staircases”.

Alright. Now I see it. If that’s what you call a major anti-aliasing problem, I strongly suggest you never watch the NTSC credits. I cannot upload them to YouTube in SD because the YT algorithm utterly destroys SD content, so I have to give you screenshots – but take a look.

https://i.imgur.com/JBV6gtu.png

That’s a close-in zoom on the station and the runabout passing it. Here’s a few frames in sequence. I’m going to give you each frame so you can see what I’m working with:

https://i.imgur.com/t0QiE1a.png
https://i.imgur.com/qGbQlBX.png
https://i.imgur.com/r9PcDef.png
https://i.imgur.com/uILnJdN.png
https://i.imgur.com/cHK9fCC.png
https://i.imgur.com/BeHLD4Z.png
https://i.imgur.com/gmUSqFp.png

Look at the degree of aliasing in those images, and I think you’ll understand why I’m experimenting with TR2=4 or TR2=5. TR2=3 creates a ripple across the front of the station that TR2=4 (at least) helps fix.

Deinterlacing is enabled in these images. That’s not what the output looks like without deinterlacing. That’s what the output looks like with deinterlacing, although it is being assembled from the 29.97 fps D2V file. If you watch a MakeMKV file it’s not so terrible looking, though I was never able to find a way to start this process with a MakeMKV-derived file and wind up with smooth motion at the end of it. I’m not saying it isn’t possible – but I searched for a way to create smooth motion out of a VFR MakeMKV base-file for about six of the nine months I worked on this project, and found a way to create first a perfectly smooth 60 fps version and then a 23.976 fps version (with help from Cyril, who is also working on the PAL version of the show). Regardless, the credits are the absolute worst-looking part of the show on NTSC.

I don’t see why. I have access to the whole PAL show if I want it, but I also have episodes on-hand from S1 and S6. The PAL quality, as near as I can tell, is virtually identical to NTSC quality with the following differences:

Reasons are above. Of course, your decision.

1). Motion is intrinsically smoother and easier to deal with. NTSC can be brought back to PAL quality in this regard, but it’s taken me more work to do it.

I will have to check this once myself. I only read that IVTC seems complicated at doom9.

2). There’s a very slight color shift, at least in S6. Colors that are slightly more blue in NTSC are slightly more purple in PAL.
3). PAL is stretched slightly and just slightly blurrier by default. Compared this frame-by-frame in NTSC vs. PAL editions of S6.
4). PAL, of course, has the 4% audio shift.

These are of course no significant reasons to take the PAL sources, I agree.

Because I want to create a project for people to do at home with legal source, asking people to buy PAL is pretty tricky.

Come on… I do love this code of honour that people obey here, but the difference between PAL and NTSC sources of the same show is purely technical.

I want to write an article about the best way to deal with PAL

What do you mean by “deal with PAL”? To convert it (back) to NTSC? For me it’s no question, I am in PAL-country, for me the question always is “how to deal with NTSC”…

The reason to write an article telling people how to deal with PAL is so that PAL people know how to best convert the show. I am not advocating for some kind of code of honor. I want to produce the best overall version of Deep Space Nine. But the goal is to provide a “Best-in-class” improvement method to everyone, which means I’m also interested in the best way to handle PAL.

Doom9 is correct that IVTC is complicated. The only way to perform it perfectly, as far as I know, is to hand-comb every scene and manually tune the frame order method via a TFM OVR file.

However – it turns out that the following actually works pretty damn well:

TFM()
TDecimate()

I have also developed a secondary method of creating a 60 fps method of the show that matches the quality of what I’ve shown you. The same Repair command I mention above will get rid of the frame blends and interpolations created by using TDeinterlace in double-output method and raising the frame rate to 60 fps. I may have even found a method of doing it all in a single script without a quality loss (right now, you have to use two different files to create a third using my 60 fps conversion method, IF you want the final quality to match the 23.976 fps method and to show no blended frames). I may have found a way to do it in a single ingestion with no loss of quality.

My Rio Grande encode uses 23.976 but is not perfect. It handles the majority of video content correctly but not all of it. Right now, my solution to that problem is to keep looking for ways to improve it while offering Orinoco at 60 fps as an alternative if someone runs into a problem episode. If I have to ask people to accept a few seconds of jerky motion every few episodes as a cost of getting this kind of quality improvement (or, alternately, to build that episode in 60 fps), that was a fair-sounding tradeoff to me.

One of the major goals of my project is an automated process. I want people to need to make as few script changes as possible and while I’m willing to do a custom script to fix an episode if I must, I’m trying to avoid that being necessary. The hand-combing and tuning that works for IVTC if you do it manually will unwind DS9 in perfection, but I didn’t want to do the scene-by-scene combing that it apparently requires. Still searching for a way to automate it perfectly (the commands above are automatic, but not perfect).

Initial evaluation of naive implementation of AniMaxx’s algorithm suggests it’s oversharpening in my case. I like the overall output otherwise. Going to adjust some variables, then toss in the rest and see what it looks like. 😃

Animaxx,

Going with the following initial run to blend our two approaches:

QTGMC2=QTGMC(“Preset=Very Slow,” InputType=2, SourceMatch=3, Lossless=2, MatchEnhance=1.0, MatchPreset=“Placebo,” MatchPreset2=“Placebo,” sharpness=1.0, SMode=2, TR2=4, Rep0=11, Rep1=9, Rep2=9, RepChroma=True, EdiMode=“EEDI3+NNEDI3”, Sbb=0, NoiseProcess=1, ChromaNoise=True, DenoiseMC=True, NoiseTR=2, GrainRestore=1.0, NoiseRestore=0.1, NoiseDeint=“Generate,” StabilizeNoise=True)

QTGMC3=QTGMC(“Preset=Very Slow,” InputType=3, SourceMatch=3, Lossless=2, MatchEnhance=1.0, MatchPreset=“Placebo,” MatchPreset2=“Placebo,” sharpness=1.0, SMode=2, TR2=4, Rep0=11, Rep1=9, Rep2=9, RepChroma=True, EdiMode=“EEDI3+NNEDI3”, Sbb=0, NoiseProcess=1, ChromaNoise=True, DenoiseMC=True, NoiseTR=2, GrainRestore=1.0, NoiseRestore=0.1, NoiseDeint=“Generate,” StabilizeNoise=True)
Repair (QTGMC2, QTGMC3, 9)

One thing I want to make sure you know is that using the command structure I was using, with variables loaded into QTGMC2 and “ReuseGlobals” called – doesn’t work. I mean, it works, as in, it delivers output, but QTGMC3 does not inherit variables from QTGMC. The “global” call is still confined to the QTGMC2 function.

So if you write a script like this:

QTGMC=QTGMC2(Placebo, Everything tweaked, tuned, and precisely chosen)

And you pair it with this:

QTGMC=QTGMC3(InputType=3, Reuse Previous Globals)

What you actually get, in terms of output, is your QTGMC2 data run against a bog-standard default QTGMC3 run with only InputType=3 as an assigned variable.

Writing out both lines changes the output.

FrankB said:

Joel Hruska said:
FrankB,

Pleasure to meet you. Before we discuss relative processing technique I should probably provide you some samples. For example:

Nice to meet you, too. You are right: Theoretical discussions are always a bit too - theoretical. Your results are astonishing, especially the captions! I am still sceptical against the whole AI-upsizing (why I wrote here in another thread, if you are interested in pure theory I can search for it), but it seems that maybe I am too old meanwhile - maybe a mix of that fact and some true facts…
But it looks great!
Critics and proposal: For my taste a bit too LESS noise. Maybe you should consider to
-denoise in avisynth (as you did), because the AI-denoising may be worse in quality, thus having full control of the denoising
-scale up denoised, which is necessary for the AI in order not to produce too much “new details from noise”
New:
-mix back some of the original noise(!) - which makes it more natural. F. e. just resize the original in avisynth with nnedi3 or so and mix it back with overlay(…, opacity=0.2) or similar. We do this very often, and it’s common practice in studios to re-noise.

I’m experimenting with renoise options at the moment, to see how they impact things. Also, yes, trying to calibrate the proper amount of processing and denoising to do in the front-end before letting an AI program have a go at it. Some AI models include denoising that works effectively but I’d rather not have to use them in the first place.

The net effect of TR2=4 or TR2=5 is a substantial improvement in the final output.

You are right concerning aliasing. But you have to pay with less detail before AI (I suppose).
I don’t like the QTGMC “input type” > 0, also because in some scenes it works pretty well, and sometimes suddenly there is quite no effect.

I have spent 20-40 hours per week for the past nine months running thousands of encodes of Deep Space Nine. DS9, however, is also my first project.

I wish I had the time for my private projects, too. Hats off to all your efforts, great that there are still people who really pull off something.

Because I was able to work at this project as part of my job, I was able to treat it equivalently to part of my job. This is not to say I haven’t invested huge amounts of time, because I had everything else to do with my job while working on this as well. None of my work responsibilities got shifted. But I also wanted to try and build a new coverage area for ExtremeTech in this space, so… mutual priority alignment.

QTGMC2 = QTGMC(Preset=“Very Slow”, SourceMatch=3, TR2=5, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset=“Very Slow”, MatchPreset2=“Very Slow”)
QTGMC3 = QTGMC(preset=“Very Slow”, SourceMatch=3, Lossless=2, Sharpness=0.5, MatchEnhance=0.75, InputType=3, TR2=5)

After a lot of experiments some years ago I decided not to use “placebo” and “very slow” any more, because you lose too many details. In this special case (to feed the AI upscaler) it may be good - but as I said before: You should consider to put SOME of the noise back in the end…

Repair(QTGMC2, QTGMC3, 9)

That seems interesting, I never had this idea!

If you want 23.976 fps output, just throw TFM() and Tdecimate() ahead of the QTGMC calls.

But this would ruin the original 29.97i (cgi) sequences? Or aren’t there any? I am sure there must be, I never checked this myself up to now, just picked it up from doom9 postings.

It’s not the FX scenes that are automatically in 29.97. In fact, in the first season, at least some episodes are basically 100% film. I don’t know when this stops. In others, like Sacrifice of Angels, most of the battle scenes are 23.976 fps, though there’s one post-credits scene that has preserved incidents of 3:2 pulldown in a 29.97 fps stream. That one threw me for awhile, trying to figure out how that could happen. Baked-in source error is awesome.

Baseline DVD. From PastPrologue.
Identical screenshot after processing. Zero upscale:

Sorry, but in screenshot 2 there is more aliasing than in 1. Look at the shoulder.

I’m not seeing it. I see one pattern that might be what you are talking about, but doesn’t come across as aliased when the actor is in motion:

Here’s a pair of enlarged shoulder shots from the same base pair of images.

https://i.imgur.com/OnEo5w3.png (DVD)
https://i.imgur.com/rOgxQEE.png (upscale)

Are you referring to the very faint line above Bashir’s right (left from our perspective) shoulder?

But maybe this is all obsolete with the PAL sources? I am ashamed not to find time for even look at it (apart from watching some epissodes in the late evening, when my brain doesn’t want to think any more…)

If you know a better way to clean up the former into the latter – possibly by preserving more detail on Bashir’s forehead, where my method is losing some of it – I’d love to incorporate it.

We should postpone everything else until you tried the PAL sources, shouldn’t we?
But again: Astonishing!

I don’t see why. I have access to the whole PAL show if I want it, but I also have episodes on-hand from S1 and S6. The PAL quality, as near as I can tell, is virtually identical to NTSC quality with the following differences:

1). Motion is intrinsically smoother and easier to deal with. NTSC can be brought back to PAL quality in this regard, but it’s taken me more work to do it.
2). There’s a very slight color shift, at least in S6. Colors that are slightly more blue in NTSC are slightly more purple in PAL.
3). PAL is stretched slightly and just slightly blurrier by default. Compared this frame-by-frame in NTSC vs. PAL editions of S6.
4). PAL, of course, has the 4% audio shift.

Because I want to create a project for people to do at home with legal source, asking people to buy PAL is pretty tricky. I want to write an article about the best way to deal with PAL, possibly in partnership with the folks like yourselves and my friend Cyril who have worked on it, if people were amenable to that. Either way, though, I want to be able to give solutions for both.

snip
(or maybe SNIP, because that’s a big bit I’m clipping) 😉

So that’s it. I hope that was useful for you; perhaps you could adapt some of my suggestions to your work.
I would like to give something back for you have really helped me with mine.

Just wanted to say: I’m going to read this in the morning because it’s very late here (or very early), but I’m really glad I was able to help you.

There are something like 3-6 remastering projects going on right now for various Star Trek properties. Maybe even more. I’m aware of you, myself, Project Defiant, and QueerWorm’s project over on Github. There are at least a dozen other people I know of who aren’t really working on public “projects” per se, but have been quietly building their own remasters / upscale collections. And that’s just the folks I know of who are specifically working with AI, as opposed to people who were performing all the other work you can use AviSynth for over the years.

We all have different short-term goals in terms of preferred filters or AI models, but ultimately, we all want the same thing: The best possible version of Deep Space Nine, for whatever device/region/bit-rate/quality-level anybody wants. I’m happy to contribute to that effort.

I want our collective efforts to stand as a counter-weight to the absolutely terrible condition of the show on Netflix. I want people who see the show on Netflix, and like it, and wonder if a better version might exist somewhere, to discover that one absolutely does, and that if they’re willing to buy a set of DVDs and invest some time and patience, they can have a version of the show that approaches HD quality.

Thanks for sharing your code. Going to experiment tomorrow. 😃

Animaxx said:

Ok guys, as I had promised, here are the new sample images from the updated/improved work I have done.
As before, images on the left are source (PAL-DVD), images in the right are upscaled/enhanced.

I think you will note there is a little more sharpness as well as detail (compared to my previous version).

I really think this is the way to go.

What’s your current script?

BTW, I compared our Sisko shots. Here’s my version.

https://i.imgur.com/P0xfEAM.png

I’m not sure why yours is so stretched, but the quality there looks about the same to me. I took the liberty of cropping your image to show just a 4:3 ratio, and the one thing I noticed is that you might be using an additional sharpening method compared to my own: Your Sisko’s ears are slightly sharper.

I also compared our starship shots. You on the left, me on the right:

https://i.imgur.com/Rf5tjAM.png

We’re very close, but you’ve got a distinctly sharper edge at the moment than I do. How are you getting it? Different QTGMC settings, or a different filter altogether?

PS – Sharpening filters should be applied after QTGMC if you intend to use anything but QTGMC. In a lot of cases, the Repair run I do will wipe out sharpening changes applied by other filters. Also, if you set a “Sharpness” variable when using my approach (if you experiment with it), make sure you set one in both QTGMC2 and QTGMC3. Variables are not passed between them. If you do not set the variable in both runs, no level of sharpness you attempt to apply in QTGMC2 will ever work. Neither will the vast majority of other sharpening filters if they are run before QTGMC. I believe this may be because QTGMC has its own functions to limit sharpness, or it may be the result of the Repair command. Either way, I wanted to make you aware of it.

FrankB said:

Joel Hruska said:
One difference I do know between PAL and NTSC is that PAL doesn’t have the same problems with aliasing that I’m trying to clean up with commands like TR2=5 or TR2=4. Some of the issues I have spent time fixing for NTSC just aren’t in the PAL copy.

If the NTSC sources are THAT bad, that you have to anti-alias by QTGMC-temporal-filtering with a window of 4 or 5, it’s really time for the PAL sources… I would never use this. If QTGMC, then always TR2=0 and StabilizeNoise=false. There are better ways to filter noise.

FrankB,

Pleasure to meet you. Before we discuss relative processing technique I should probably provide you some samples. For example:

https://www.youtube.com/watch?v=zMaMHT4skn0

That’s the most recent version of the NTSC credits that I’ve done (set YT to 4K for actual quality). Check two things, specifically:

1). The degree of antialiasing in the nacelles on the runabouts as they fly past just before the blue light burst.
2). The station pan immediately following the blue light burst.

Compare against the following in the same two areas:

https://www.youtube.com/watch?v=OqG_A72Q5fM

You will find that the antialiasing is improved in both areas, with the last ripples of QTGMC’s earlier run quashed now. These are visible in the station pan. Now, quashing the ripple has caused a buzzing artifact later on in the credits that I’m exploring methods to get rid of. I’m confident I’ll nuke it. The net effect of TR2=4 or TR2=5 is a substantial improvement in the final output.

If the NTSC sources are THAT bad, that you have to anti-alias by QTGMC-temporal-filtering with a window of 4 or 5, it’s really time for the PAL sources… I would never use this. If QTGMC, then always TR2=0 and StabilizeNoise=false. There are better ways to filter noise.

Then please, by all means, check the output links I’ve posted or any of the others on my channel. I would love to hear your improvements for further improving the quality of my work. I just posted 15 videos on Monday in various Topaz modes and across multiple episodes. I can provide actual clips of these rather than links as well, but YT is probably easiest for speed and to give you a good idea. If you set YT to 4K I don’t lose much quality, even if it is compressed.

https://www.youtube.com/user/Dputiger

I have spent 20-40 hours per week for the past nine months running thousands of encodes of Deep Space Nine. DS9, however, is also my first project. I have never found a formula to beat the following (with the exception of the TR2=4 / TR2=5 settings, which are new introductions to my method of handling the project and are still under evaluation).

If you run the script below without calling TR2=4 / 5 in either line – leaving it at default in both calls – you get the second video. If you call it as below as TR2=5 in both scripts, you get the first video.

QTGMC2 = QTGMC(Preset=“Very Slow”, SourceMatch=3, TR2=5, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset=“Very Slow”, MatchPreset2=“Very Slow”)
QTGMC3 = QTGMC(preset=“Very Slow”, SourceMatch=3, Lossless=2, Sharpness=0.5, MatchEnhance=0.75, InputType=3, TR2=5)
Repair(QTGMC2, QTGMC3, 9)

I’m actively testing the impact of only calling TR2=4 /5 in one of the two runs before the repair. Oftentimes, this still provides a subtler version of the same effect and I prefer to use the lightest touch possible. The script above is the one that actually produced the output you watched, so that’s the one I’ll stick with for now.

If you want 23.976 fps output, just throw TFM() and Tdecimate() ahead of the QTGMC calls.

I almost forgot. I can give you screenshots of the difference between the baseline DVD and the results of my method with the TR2=4/5 call in it.

Baseline DVD. From PastPrologue.

https://www.extremetech.com/wp-content/uploads/2020/09/Screenshot-705.jpg

Identical screenshot after processing. Zero upscale:

https://www.extremetech.com/wp-content/uploads/2020/09/Screenshot-708.jpg

Finally, the image after upscale.

https://i.imgur.com/AZaBHLI.png

The slight color shift is something I know how to unwind. Just hadn’t done it yet.

So that’s a demonstration of the level of quality and noise processing I currently achieve, at both the frame level and in motion.

If you know a better way to clean up the former into the latter – possibly by preserving more detail on Bashir’s forehead, where my method is losing some of it – I’d love to incorporate it. I’m willing to trade a little forehead wrinkles on a 27 year-old actor in exchange for the improvements the above generates, but I’d prefer to get more improvement without trading anything in the process. 😃

If you’d like to see clips via a different method I can also make that happen. Let me know if you have a Microsoft account.

It does.

I am not certain if it is capable to clean the PAL source up quite as much as the NTSC source, but the PAL source seems to respond well to similar filters. The PAL conversions looked good to me. I prefer the NTSC audio, having grown up with it, and there’s a very slight color difference between PAL and NTSC, but it seems to me like those conversions were done really well.

Until recently, I thought I might use the PAL source myself.

Ani,

One difference I do know between PAL and NTSC is that PAL doesn’t have the same problems with aliasing that I’m trying to clean up with commands like TR2=5 or TR2=4. Some of the issues I have spent time fixing for NTSC just aren’t in the PAL copy.

The native DVD credits for NTSC look terrible compared to PAL.

I have considered working with the PAL footage myself. It’s an easy solution to the 29.97 / 23.976 fps problem, and while the quality is intrinsically lower than the NTSC version, the difference looks minimal in Season 6. (I have S6 in PAL at the moment, but only S6).

The AviSynth script I call Rio Grande – the one based around QTGMC – runs well against PAL source as well, if you care to try it.