Animaxx

FrankB said:

Animaxx said:

Guys, I have done it. The 4K-pilot is finished. It was rendered at 15 Mbps in 4:3 format at a display size of 2846x2160p (4:3).

First: Congratulations!

I also used the original NTSC audio and adapted to the PAL-running time while maintaining the original pitch; also, I have pitch corrected the german audio so the PAL-Speedup has been taken care of (no more high pitched voices).

Critics: (as always…)

1. For the German sound there is no pitch to correct if you leave it at 25fps. Or do I misunderstand? It is originally made for 25fps, and your result is also 25fps. Why correct?
2. As I said before: You should simply slow down Video to 23.976, and the English audio will fit.
3. IF you do so, THEN the German audio you would have to slow down, too, and in this case you should pitch correct it about 1/4 to 1/2 tone up.

By the way: These pitch corrections always produce a quality loss, even with the best algorithms, be aware of that fact.

Strangely enough, the tv broadcasts and dvd versions are higher pitched around germany than the original vhs tapes, which had lower pitch. It also sounded better, because music and sound effects now match the original ntsc music and sound effect pitch once lowered.

As for the speed down to 23,976 FPS: That would cause motion stutter again, which I was happy to have avoided with PAL at 25 FPS.

Are you using actual 3840x2160, or are you using 4K loosely? (Meaning, like, a 4x upscale instead of a true 4K upscale) Assuming you mean it literally – have you compared the quality gain from jumping that high?
The reason I’ve stuck to 2560x1920 is because I don’t get any benefit from going higher. I do get a benefit from using Topaz in 4x mode, because the 2x modes seem to blend poorly, but I don’t see an improvement from anything more.
I have also found that using x265 with a CRF=19 is pretty solid. I can barely see the smallest difference between CRF=6 and CRF=20, while I can’t see a difference at all, really, at CRF=19.
Running H.265 in “Slow” or “Very Slow” will also improve your compression. Compression is much more improvable than I ever believed. But this is also why I ask about the benefits of 3840x2160 – because if you get the same quality at, say, 2560x1440 than you do at 3840x2160, you could save yourself a lot of bandwidth just by getting rid of pixels people don’t actually meaningfully benefit from.
If you do benefit from 4K, I recommend trying H.265 with Slow encode speed to see what kind of compression boost you get.

I am using Topaz 4K-mode but switch to custom settings to avoid black bars left/right, so I end up with an automatic display size of the values I had specified.

I am not enocoding with a CFR as you suggested, I use Adobe Premiere Pro with a CBR of 15 Mbps in x264; while I know x265 could increase compressibility, I ran into issues with that in the past, so I need to stick to x264 for now.

As for the display size itself, I will let the automatic setting do that, for it scales up while maintaining the aspect ratio.

By the way, since the rules here do not allow for link posting to complete material, please send me a private message to get the link for the current 4K-version of the pilot (Joel and FrankB).
It will be the same as in the release topic I am preparing right now, but this way you’ll get it sooner, since the topic will take a little bit.

Guys, I have done it. The 4K-pilot is finished. It was rendered at 15 Mbps in 4:3 format at a display size of 2846x2160p (4:3).

I also used the original NTSC audio and adapted to the PAL-running time while maintaining the original pitch; also, I have pitch corrected the german audio so the PAL-Speedup has been taken care of (no more high pitched voices).

The motion is nice and smooth (as it is at 25 FPS).

There are only minor imperfections: Slight block artefacts with explosions and fast moving scenes (barely noticeable), sometimes the added grain is more visible but also adds nice detail, so I will keep that for it looks less “oversmoothened/arteficial”. There is still some minor rainbowing and there is that one frame were the deinterlacer couldn’t handle it all, but it’s again minor. Unfortunately with the pilot, there is a sort of “double border” on top/bottom, but that is only present with the pilot and I didn’t want to crop further to avoid losing parts of the image.

Overall, I am happy with the result. I think it is a really nice compromise between upscaled/enhanced quality, retained detail and filtered image properties.
The 1080p-version is being rendered right now and should be done in 12 hours. I will then upload that as well.

Finally, I will do comparison shots and write the text for the release topic. I will invite you guys and then you can get your hands on the pilot.

As for specs: The 4K-Version will have a size of roughly 10 GB (9,9) and the 1080p-Version will have about 6 GB. Since the pilot (double episode) runs 87 minutes (PAL) and the single episodes clock in at about 43 minutes (PAL) we can assume that the size for the singles will be about 5 GB at 4K and 3 GB at 1080p.

I think size-wise those figures would be ok, since a complete 26 episode season at 4K would take up 130 GB and at 1080p it would take 75-80 GB. In comparison, if I were to copy the DVD-files with original sizes (1,8 GB for singles at SD-Resolution) I would end up with a season count of 45-50 GB, so that appears reasonable to me.

I hope you’ll agree.

Man, you guys (Frank and Joel) have so much technical knowledge, I salute you. I am beginning to feel small, very small.

😃

Joel Hruska said:

Animaxx said:

You guys know when I was satisfied? When I was finally able to see the freaking Aztec-Pattern on the ships hull (last shot) - I know, I am so ready for the looney bin, but I used to/still do build starship models, so details are a point of obsession for me …

Just wanted to let you know: I’m definitely getting a quality boost off some of these changes. I need to tweak them to fit the somewhat different characteristics of the DVD source, but you have meaningfully boosted my project along. 😃

Glad to hear it. Since I had worked for month with the NTSC and didn’t find a pleasing enough solution for motion issues, I know what you mean by adapting to the source.

Some valid points there. It all boils down to this: Hybrid materials with baked-in properties are “The Stuff from Hell” …

Artan42 said:

I’m enjoying all this technical detail. I have no bloody clue what is being said (it may as well be in Klingonese for all I know) but it looks like there’s several people who really know what needs to be done here.

As for CBS, every time somebody uploads a test video, I go watch a couple of episodes on Netflix. I think it’ll be the viewing figures for DS9 and VGR that make the case to CBS that they might have to pull their fingers out eventually as they’ll be the only two of the nine produced TV shows without a HD release.

Believe me, this is what I feel like when people start talking avisynth. I am still learning while doing but to be honest if there is a way to use a gui/user interface which is visually pleasing I prefer that.

I guess if you were to do it all the time, it becomes second nature, but I think everything in the general direction of avisynth isn’t for me.

Also Joel, I think it is really kind and generous of you to try to offer solutions for both standards.

That would probably help a lot of people, who for some reason may just have access to one of the formats or simply don’t want to spend money again on something they already own, but still would like to “spice it up a bit”.

And besides: In the end every attempt benefits us all, for I have the dream that our persistence and idealism will someday shame someone at CBS into thinking “Oops, fans outsmarted and outworked us, now everyone can have it for free? Let’s jump on that train and make some cash!”
And if they did it well, I would gladly pay.

One can dreeam, right?

Initial evaluation of naive implementation of AniMaxx’s algorithm suggests it’s oversharpening in my case. I like the overall output otherwise. Going to adjust some variables, then toss in the rest and see what it looks like.

I am happy to see that a few pointers from my approach worked, but I was actually afraid it would oversharpen in your case. My initial NTSC sharpening approach was enough for that medium, but PAL is softer, so I had to adjust - in your case you will have to reduce.

Oh Frank, I see I hadn’t posted my sample images around here, I will now, so you can see a PAL-Version with some of Joels key points as well as my own tweaking, so here goes:

EDIT: Sorry for the double post, I had skipped to far/fast. My bad.

FrankB said:

Joel Hruska said:
FrankB,

Pleasure to meet you. Before we discuss relative processing technique I should probably provide you some samples. For example:

Nice to meet you, too. You are right: Theoretical discussions are always a bit too - theoretical. Your results are astonishing, especially the captions! I am still sceptical against the whole AI-upsizing (why I wrote here in another thread, if you are interested in pure theory I can search for it), but it seems that maybe I am too old meanwhile - maybe a mix of that fact and some true facts…
But it looks great!
Critics and proposal: For my taste a bit too LESS noise. Maybe you should consider to
-denoise in avisynth (as you did), because the AI-denoising may be worse in quality, thus having full control of the denoising
-scale up denoised, which is necessary for the AI in order not to produce too much “new details from noise”
New:
-mix back some of the original noise(!) - which makes it more natural. F. e. just resize the original in avisynth with nnedi3 or so and mix it back with overlay(…, opacity=0.2) or similar. We do this very often, and it’s common practice in studios to re-noise.

The net effect of TR2=4 or TR2=5 is a substantial improvement in the final output.

You are right concerning aliasing. But you have to pay with less detail before AI (I suppose).
I don’t like the QTGMC “input type” > 0, also because in some scenes it works pretty well, and sometimes suddenly there is quite no effect.

I have spent 20-40 hours per week for the past nine months running thousands of encodes of Deep Space Nine. DS9, however, is also my first project.

I wish I had the time for my private projects, too. Hats off to all your efforts, great that there are still people who really pull off something.

QTGMC2 = QTGMC(Preset=“Very Slow”, SourceMatch=3, TR2=5, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset=“Very Slow”, MatchPreset2=“Very Slow”)
QTGMC3 = QTGMC(preset=“Very Slow”, SourceMatch=3, Lossless=2, Sharpness=0.5, MatchEnhance=0.75, InputType=3, TR2=5)

After a lot of experiments some years ago I decided not to use “placebo” and “very slow” any more, because you lose too many details. In this special case (to feed the AI upscaler) it may be good - but as I said before: You should consider to put SOME of the noise back in the end…

Repair(QTGMC2, QTGMC3, 9)

That seems interesting, I never had this idea!

If you want 23.976 fps output, just throw TFM() and Tdecimate() ahead of the QTGMC calls.

But this would ruin the original 29.97i (cgi) sequences? Or aren’t there any? I am sure there must be, I never checked this myself up to now, just picked it up from doom9 postings.

Baseline DVD. From PastPrologue.
Identical screenshot after processing. Zero upscale:

Sorry, but in screenshot 2 there is more aliasing than in 1. Look at the shoulder.
But maybe this is all obsolete with the PAL sources? I am ashamed not to find time for even look at it (apart from watching some epissodes in the late evening, when my brain doesn’t want to think any more…)

If you know a better way to clean up the former into the latter – possibly by preserving more detail on Bashir’s forehead, where my method is losing some of it – I’d love to incorporate it.

We should postpone everything else until you tried the PAL sources, shouldn’t we?
But again: Astonishing!

I will provide my PAL-version soon in the release topic where I will invite both of you - perhaps together we can come up with something.

Completely understandable. Time zones bite. Read and experiment whenever you have time.

I can’t always work on this project when I want to either. After all we all have day jobs, many or at least some probably have families and then there is always that little thing (like an email from a friend or someone asking you to give them a hand) that blows up to be much more and … bam - you’re side tracked again.

But since we are all doing this for free, I think we’re coming ahead pretty well.

And yes, the netflix version is … I don’t think a word has been invented yet to accurately describe my feelings on that one, but I would like to paraphrase Garak: “To see that one burn wouldn’t exactly be tragic”.

But then again, times move fast and physical storage mediums are becoming more and more a thing from the past, streaming and online cloud storage is where we are headed with warp speed.
A friend of mine who used to be the manager of a classical “video store” (yes, they even had VHS tapes until about 2012) said “The way things are going, DVDs will suffer the same fate as VHS; BluRay will probably indure still but streaming will replace that as well some day.” The rate things are going, he expects the DVD to be more of an collectors thing within the next 5-10 years.

And that is (in my eyes) something to be sad about, since there are many show and movies that never made the jump from dvd to bluray, and even if they are available at streaming services, they are being transferred so carelessly/badly, that the original source is still a better way to go (if you can still find it).

The same said friend had the opportunity to take any vhs tapes he wanted for free when they were cleared out.
I can still remember him carrying boxes into his small car, afterwards telling me he couldn’t belive how insane it was by the company that owned the store chain to go like “Well, that’s that, no one wants that old stuff, so collect it and throw it out”.
Thankfully, he managed to “sneak out” a lot before that happened and luckily no one noticed (or perhaps even cared). He still feels sorry today for not being able to save them all - he rescued about 3.000 tapes, among them old disney classics, movies and so on (can you imagine what his place looks like?).
But he told me that pales in comparison to the total stock they had (which was about 8.000 with the last inventory he did back then).

And there are still those gems sometimes, like extended versions or “prints” that were done once and forgotten.

Anyway, let’s hope our efforts will matter to people/fans.

I’m not sure why yours is so stretched, but the quality there looks about the same to me. I took the liberty of cropping your image to show just a 4:3 ratio, and the one thing I noticed is that you might be using an additional sharpening method compared to my own: Your Sisko’s ears are slightly sharper.

Well, that is the result of StaxRip doing some cropping, my player wanting to stretch to fill the screen. I assure you, the final product won’t look as “stretched”, furthermore I cut of the black bars on top/bottom, so the image appears larger (so you can compare more closely without resorting to zoom.

What’s your current script?
We’re very close, but you’ve got a distinctly sharper edge at the moment than I do. How are you getting it? Different QTGMC settings, or a different filter altogether?

You’re updated settings were my inspiration and a sort of “jumping-of-point” for me, but I realised that PAL had it’s rather unique properties in terms of noise, artefacts and so on; after I had applied most of your settings, I quickly realized that I hadn’t jumped far enough and decided to do a little experimenting on my own.

The basis for my work are the vob-files I got from ripping the PAL-Disc with DVD Shrink.

Then I dumped that into StaxRip.
Aside from the automatic source settings, the “first” filter is QTGMC in Progressive Full Repair Mode.
Afterwards, my customized string reads/writes as follows:

QTGMC(preset=“Placebo”, InputType=3, sourceMatch=3, lossless=2, MatchEnhance=1.0, MatchPreset=“Placebo”, MatchPreset2=“Placebo”, sharpness=1.0, SMode=2, tr2=2, Rep0=11, Rep1=9, Rep2=9, RepChroma=true, EdiMode=“EEDI3+NNEDI3”, Sbb=0, NoiseProcess=1, ChromaNoise=true, DenoiseMC=true, NoiseTR=2, GrainRestore=1.0, NoiseRestore=0.1, NoiseDeint=“Generate”, StabilizeNoise=true, Border=true, ediThreads=8)

A few words on that:
I increased the MatchEnhance to 1.0 and while I know it could create more noise, it also provides a bit more detail.
I pushed the sharpness setting to 1.0 but also specified the SMode=2 to allow for “vertical max/min average + 3x3 kernel” to get better results; in addition I did not set a clear value for sharpness limiting - I know, it’s risky and rarely my output can look close to the edge of oversharpening, but it’s an ok trade-off having one or two seconds of that compared to having a better look overall throughout the episode.
I adjusted the rep-settings to minimize blur for motion search, initial and final output after temporal smooth, despite not having set TR0 and TR1 values - I have found that this produces a clearer image and better motion quality with quick camera pans, I also included the ChromaRep.
Furthermore, I added the interpolation with “EEDI3+NNEDI3” to help with the “half-frame-to-full-frame-issue” during deinterlacing, since StaxRip still shows an interlaced source from the PAL, despite being flagged as progressive. And while I was comparing, it also seemed to better handle the visual flow after deinterlacing (less image stutter).
The “Sbb=0” is usually a default setting, but I wrote it in anyway, so QTGMC knows not to back-blend-in the blurr-difference pre-post-sharpening.
When it comes to Noise (which is related to the increased MatchEnhance and sharpness), I chose to use the stronger NoiseProcess=1 to adapt for the previously increased settings, I also included ChromaNoise here.
The DenoiseMC was modified to have a better noise vs. detail detection and in combination with NoiseTR helps to better identify what is actually noise and what detail (that I want to keep, naturally).
More on that, I decided to raise the value of “GrainRestore” to 1.0, cause with the PAL it preserves things better, albeit keeping a little more grain, but that’s visually ok for me, I don’t need things to be “super smooth” in that regard.
Finally the Border=true setting is added to remove flickering on top/bottom borders, which happens with PAL when explosions or very fast moving objects are in those areas. It doesn’t handle it all the time, but reduces the effects.

Then I used a second filter, “Deblock_QED”, because the PAL is quite blocky, especially when explosions or quick camera pans happen. I modified the string so the “quant1=35” (default would be 24) increases the removal of outer edges on blocks. I don’t dare go higher, for then other lines will become distorted or have weird visual effects happening - unfortunately it doesn’t get rid of all blocks, but reduces them enough to barely see them (mostly with quick movement and/or explosions).

Then I used a third filter, “pSharpen”. Now I don’t know how familiar you are with that one, but I love it. It uses a “two-point approach” on every pixel, comparing to the min/max of it’s spatial neighbor whilst avoiding overshoot and also giving you the option when to use compensation for that avoidance.
I also raised the super sampling factors to reduce aliasing.
Now I know, doing additional sharpening can lead to terrible results (considering I also pushed it with qtgmc), but I think I have made a reasonable compromise.
My specific pSharpen string as follows: pSharpen(strength=75, threshold=90, ss_x=2.0, ss_y=2.0)

Also (4th filter) AntiAliasing comes into play with MMA2/MAA2.
I know aliasing is not present “as much” with PAL, but the instances it happens it get’s distracting very fast. So I really pushed it around here as well, including chroma.
String: MAA2(mask=1, chroma=true, ss=2.0, aa=128, aac=128, threads=4, show=0)

The 5th and (almost) final filter is to handle the Color Grading, I went for GradFun3, since it gives to most pleasing results. I only do that because with PAL you can sometimes see banding, especially with stronger colors.
The string here as follows: GradFun3(smode=2).

And the last “filter” is the automatic crop.

When it comes to Topaz VEAI I know you prefer Gaia CG and while I worked with it as well for some time, I do not use it for DS9 - here I use Gaia HQ.
It’s a personal preference for the following reasons:
Gaia CG smoothens/cleans out to much detail (in my opinion) and makes the image look a little “too artifical”, faces are a particular issue here (wax effect).
Furthermore, HQ adds a little more sharpening while keeping a bit more grain/noise, which is nice for I tried to preserve some when doing QTGMC.
And: Sometimes Gaia CG has an issue with introducing a visual “grid pattern” that hasn’t been solved yet and especially during nature shots (see the holodeck scene with Jake and his father) the trees don’t look right, that really is something Topaz should take care of.

So that’s it. I hope that was useful for you; perhaps you could adapt some of my suggestions to your work.
I would like to give something back for you have really helped me with mine.

You guys know when I was satisfied? When I was finally able to see the freaking Aztec-Pattern on the ships hull (last shot) - I know, I am so ready for the looney bin, but I used to/still do build starship models, so details are a point of obsession for me …

And also the new 4K-Clip as Preview-Video:

https://drive.google.com/file/d/1dkC_LTDR5j2M0OuZyuEnqioGbz8ZoqV1/view?usp=sharing

Ok guys, as I had promised, here are the new sample images from the updated/improved work I have done.
As before, images on the left are source (PAL-DVD), images in the right are upscaled/enhanced.

I think you will note there is a little more sharpness as well as detail (compared to my previous version).

I really think this is the way to go.

Joel Hruska said:

It does.

I am not certain if it is capable to clean the PAL source up quite as much as the NTSC source, but the PAL source seems to respond well to similar filters. The PAL conversions looked good to me. I prefer the NTSC audio, having grown up with it, and there’s a very slight color difference between PAL and NTSC, but it seems to me like those conversions were done really well.

Until recently, I thought I might use the PAL source myself.

Ani,

One difference I do know between PAL and NTSC is that PAL doesn’t have the same problems with aliasing that I’m trying to clean up with commands like TR2=5 or TR2=4. Some of the issues I have spent time fixing for NTSC just aren’t in the PAL copy.

The native DVD credits for NTSC look terrible compared to PAL.

The audio I will be able to handle (I know how to adjust the original NTSC audio so it fits/syncs with PAL without any pitch issues (so the original will be preserved, for I prefer it as well, besides the NTSC is there in 448 kbits, while the english PAl only has 384).

Also, I have developed a modified qtgmc string to better accomodate PAL and I also added a couple of filter tweaks, so the PAL will look “cleaner/sharper”.

I have already done a sample according to my setup and will upload comparison shots as well as a sample soon.

And here are the video files; please not that I haven’t tweaked the audio pitch yet, so it does sound to high, but I will handle that later, for now it’s just a visual preview, ok?

First the link for the original SD-Quality video from the PAL-DVD (source):
https://drive.google.com/file/d/1RzZISla7xctAzI9zQ9PsogxxKwUlznCo/view?usp=sharing

And now to compare the filtered and AI-enhanced 4K-clip:
https://drive.google.com/file/d/1_L8a5gtwftVfMROA01WiYLBl_jWF8_vX/view?usp=sharing

Here are a few sample image to compare: Images on the left are from the PAL-DVD without any work (source), images on the right are after filtering and AI enhance.

Looks good, doesn’t it?

FrankB said:

Animaxx said:

… specifically how to properly deinterlace source material and restore detail to source footage.

Can you specify a bit?

As soon as I get to it, right now I am busy doing a sample. But I will do it, promised.

I must once again thank Mr. Joel Hruska for his extremely valuable insights into several aspects of this project, specifically how to properly deinterlace source material and restore detail to source footage.

With those pointers, I am really able to get a lot out of the PAL-Source, even before utilizing AI.

Once I am done, I will provide samples (including the source to compare).

Vidmaster said:

This is a really nice project. Noughsaid.

Thanks. Once I get around to actually releasing the episodes, I will include you into my invite-list for the topic.

pahuffman said:

Here’s another DS9 upscale project. Looks like he’s achieved good results and moving on to upscaling the whole series:

https://www.extremetech.com/extreme/314653-remastering-deep-space-nine?fbclid=IwAR0M0qhlJZ8qm95SCzE-UduVZv-0lL-rDni48DdD-bfuc3-wKWX-iepe61g

That is the project from Joel, who just joined us around here. But thanks for looking around.

Joel Hruska said:

I have considered working with the PAL footage myself. It’s an easy solution to the 29.97 / 23.976 fps problem, and while the quality is intrinsically lower than the NTSC version, the difference looks minimal in Season 6. (I have S6 in PAL at the moment, but only S6).

The AviSynth script I call Rio Grande – the one based around QTGMC – runs well against PAL source as well, if you care to try it.

A big and happy welcome to you, Sir. Thank you for joining us.

I am picking up my new PAL-Sets of DS9 and Voyager and will start working on them asap and try your suggestions.

Baobab Archiver said:

In case it’s of use or relevance, there are easier ways of reversing PAL speed up in an audio file.
Admittedly it’s command line, but there are gui interfaces available.

Eac3to has a simple flag -slowdown which does it, so just add that to the end of the command line.

Example to convert a file called ep1.ac3 to a new file called ep1fixed.ac3 you would type:

eac3to ep1.ac3 ep1fixed.ac3 - slowdown

You can also add the required bitrate as a further option (e.g. -384 or -448). I’m not sure what the default is.

I use this for any PAL speedup reversal, although I won’t always go to AC3. You can just use filename ep1fixed.wav if it want uncompressed output, or if you get the required plug ins installed .dts .aac etc.

Hope that’s of help to someone.

Also, vobsub is a quick little program for changing frame rate of subs (i.e. Removing PAL speed up)

Thanks, you made some nice suggestions.

My PAL-Discs should arrive today, depending on the postal service (or in this case when the delivery guy arrives). I hope (since I am not there the whole day) someone will take it and keep it safe in case I am not there, otherwise I would have to pick it up tomorrow at the local postal office.

Fingers crossed. I’d like to get to work asap (meaning this evening).