Sign In

Star Trek - DVD-Restoration and Upscale Project

Author
Time
 (Edited)

With this project I want to showcase to fans of DS9 what can be possible in the universe of (AI-based) upscaling.

Those of you who have watched Deep Space Nine recently on modern/bigger screens know what the quality of the DVDs, streaming services and WEB-files look like: Most of the episodes (especially during earlier seasons) display problems like blurriness, areas out of focus, issues with grain (luma and chroma), washed out colors, a non-HD-conforming color palette (which makes sense given the technology back then), compression or block artefacts, interlacing issues and problems with motion fluidity originating from variable frame rates.

The approach of this project (in a nutshell) is to do the following steps to assure that the visual quality improves and the result is better “watchable” on a 4K screen:

  1. Take the source material and extract the raw data (working from PAL sources here)
  2. Adjust the audio track so it will fit with the new constant frame rate of 23,976 FPS (finished video)
  3. Re-Encode and filter the source material to get rid of several issues, like:
    • Rainbowing, Interlacing, Artefacts, Block Compression, Haloing, Ringing, Aliasing, Softness, Banding
  4. AI-based image enhancements and upscaling, like:
    • Detail enhancements, increase of focus, upscaling of the picture dimensions (to 1080p or even 4K)
  5. Final Encode/Rendering with additional image improvements. like:
    • Custom Aspect Ratio (original 4:3 on HD-resolution without black bars left and right
    • Correction of Pixel Aspect Ratio
    • Adjustment of the SD-color-palette to more closely resemble an HD-like look
      (the work here is based on what I have seen on the DS9 Documentary “What We Left Behind”)
    • Working in grain to give a “less plastic/waxy look”
    • Sharpening an Unsharpening to create a more “pseudo-detailed look”
    • Encoding in a more HD-compatible, modern file-format

If you wish for more detailed information (like the complete guide/workflow on “how to do the project”, comparison images, samples & more) and other stuff, please contact me via private message and I shall do my best to provide what you are looking for 😉

I will do regular updates with images on what I am currently working on, so you can see what is possible - I will also actively discuss the project with you and will try to answer questions (if I can).

Stay tuned.

EDIT:
Ok, the general password to access all the files around here is: “DS9Enhanced” (without the quotation marks).
Please be advised, that all comparison images have the following characteristics:

  • Left Side: Original SD-resolution image (PAL-DVD) without crop oder modifications
  • Right Side: Upscaled 4K-resolution image (2880x2160p) with cropping left and right
  • Images are displayed through a media player that stretches the image to fit the screen, so the right
    sided image looks a little “wider” than the left, but the original aspect ratio is maintained (4:3)

The Ends DO Justify The Means …

Author
Time

Currently doing upscale work on the Pilot-Episode “Emissary” - comparison images to follow soon.

The Ends DO Justify The Means …

Author
Time
 (Edited)

And here is (exception, only for this episode to see the quality in a 4K videostream) a sample from the pilot episode. It contains the first 16 minutes of the pilot in 4K (folder size about 2,5 GB; format h265 in mkv).

Please be advised that no further video samples will follow, only image comparisons. For more samples and additional information please contact me via private message.

Link:
https://www.dropbox.com/sh/pawv5udlr7g5t1e/AABPHoORvlW2yyoZwpgiX6-la?dl=0

The Ends DO Justify The Means …

Author
Time

Do you have any examples of how your process looks on episodes from Seasons 5+? The video quality of the DVDs gets much better in the later seasons so the results tend to be better.

By the way, what’s your methodology for conforming to 23.976fps? I am working off of the NTSC DVDs rather than the PAL ones, but I’ve found I ultimately have the most success retaining the variable framerate to allow the 29.97 sections to remain at that while the unCGI’d film stuff is at 23.976. Kodi copes with it just fine, in my experience, so no need for CFR for my purposes. Joel Hruska at ExtremeTech spent a long while trying to get the series to a constant 23.976fps but his avisynth script is a) time-consuming and b) imperfect, producing rippling, hence I stuck with VFR.

I’d also be interested to know about any anti-rainbowing you did; and whether or not you’ve found any good way to combat the smearing often apparent on faces in the early and middle seasons.

Author
Time

@sade1212:

Regarding Season(s) 5+ :
I will do a test and let you know soon, ok? But I did a sample earlier with previous settings and the results looked encouraging (but as you have said, the souce quality does get better, so the results will also look better).

Regarding the CFR at 23,976 FPS :
As I have already stated, I am working from PAL, since I have already had my experiences trying to establish a CFR from the NTSC DVDs and found it to be a horrible experience.
But still, I will explain (although I am not sure if this is applicable to NTSC sources with VFR, so please let me know what you have found out on that end).
I am using StaxRip to handle the Framerate. Here, I apply QTGMC (with a custom made and rather complicated script) for Deinterlacing in Progressive Full Repair Mode and then I use “Assume FPS” towards 23,976 FPS. Works well for PAL.

As for the rainbowing :
That’s also done with StaxRip, the filter here is called “DeCross” (which needs to be manually added as a dll.-file to the AviSynth directory and has to be applied before the Deinterlacing, otherwise it doesn’t work).
I must admit, it does not catch everything, but it reduces a lot of rainbows well enough to be “less noticeable”.

As for smearing and detail loss with faces :
That’s kind of hit and miss, depending on the source. Often, faces in the background are a real pain, since there is little detail to begin with. Still, in my case, a combination of these factors helped:

  • StaxRip Filters (QTGMC with adjustments on detail improvement and sharpness increase)
  • Using a multi-step-approach with the AI-based enhancement software “Topaz Video Enhance AI”
    (First Pass: A Model called “Gaia HQ” at 100%, purpose: Deblock/Denoise & Detail Recovery
    Second Pass: A Model called "Artemis LQ at 1080pHD, purpose: resolution upscale on low quality input
    Third Pass: A Model called "Gaia CG at 4K, purpose: resolution upscale with Detail Enhancement)
  • DaVinci Resolve
    (Adding Grain, Texture and Feature Recovery)

Not all the faces (especially in the background) look great, but they do look like faces with little to no distortion, which was important to me.

Also, I fairly recently started working exclusively with higher bit color spaces (10 bit in StaxRip and 16 bit in Topaz, back to 10bit in DaVinci), since the original 8 bit had banding issues which were handled quite nicely once I switched/encoded/rendered to/in/with higher bit ranges, which is important on DS9 whenever there are larger areas of the same color (like uniforms, walls or space) or sudden changes in light levels (like with flickering lights or explosions, something which almost drove me crazy during the pilot when the station lights on the promenade kept cutting in and out, causing hard lines on everything they touched).

Hope I could help, I will post two samples from my favorite season 6 episode “Sacrifice of Angels” here soon so we can compare quality between season 1 and 6 - that way we can see, if my workflow can be applied there as well.

The Ends DO Justify The Means …

Author
Time

Ok guys, here are some comparisons for two scenes of the season 6 episode “Sacrifice of Angels” - I must admit the Defiant, the Galaxy ships and the Miranda ships look gorgeous.

Since it is well known that the later seasons of Star Trek DS9 (and Voyager as well) had better quality when it comes to the production values of those shows and conclusively their DVDs, I wanted to see if my workflow would hold up against the later seasons, so I decided to do two test scenes from the DS9 season 6 episode “Sacrifice of Angels”.
Both scenes focus more on the cgi/vfx elements, but I included a few shots of live action/actors/faces as well.
Please be advised that both scenes had a lot of camera shaking (which is logical, since they’re both battle scenes), so it was hard to find images that were actually “still”.

First scene:

Comparison Images (7 MB ; SD-DVD left, 4K upscale right ; Password: DS9Enhanced):
https://www.dropbox.com/s/n44dny0vrm761xr/SoA - Clip 1 (SD vs HD Images).rar?dl=0

Videofile in 4K (120 MB ; Password: DS9Enhanced):
https://www.dropbox.com/s/26hrbpy1r88oww3/SoA - Clip 1 (4K-Test).rar?dl=0

Second scene:

Comparison Images (18 MB ; SD-DVD left, 4K upscale right ; Password: DS9Enhanced):
https://www.dropbox.com/s/07wupkwg0fzuxof/SoA - Clip 2 (SD vs HD Images).rar?dl=0

Videofile in 4K (263 MB ; Password: DS9Enhanced):
https://www.dropbox.com/s/nsozbqelje8ccjv/SoA - Clip 2 (4K-Test).rar?dl=0

Have fun watching and let me know what you think.

The Ends DO Justify The Means …

Author
Time

Ah, thanks for the explanation and the clips. I’ve been upscaling the series (just to 1080p; not 4K: diminishing returns and all) for my own personal consumption and have made it to mid-season 5 already, and I wish I’d known about de-rainbowing filters earlier on when that was more of an issue. Running three passes of Topaz must take forever - I’m on a 1070, and I’ve found running even a single pass of the Gaia models prohibitively time consuming, hence I’ve settled with a single 200% Dione Interlaced Robust pass. Despite the name I’ve found it works impressively well for progressive footage; even recent 1080p Blu-rays. To get from the 1440x960 Topaz gives me to 1440x1080, I use Red Giant Instant 4K in Premiere (but just because I already had it; I bet bilinear filtering or the built-in ‘detail-preserving upscale’ or whatever Resolve has would work just the same) and then add a little noise and sharpening.

I processed the same bit of S06E06 as you did for your second clip so I could decide whether to nick borrow your methodology or not. Here’s my attempt if you’re curious. (I unfortunately don’t have access to the PAL DVDs and am stuck wrangling the framerate fun of the NTSC ones, so this is in VFR. That does however luckily mean I get to avoid the extra processing time of Avisynth/Staxrip and QTGMC. I think the frame skip with Odo at the very beginning is a consequence of me separating this bit out as that sort of thing doesn’t occur when I process full episodes.) Yours has better sharpness with the large, bold details - things like the Defiant’s text - and generally looks cleaner, thus avoiding issues mine sometimes suffers from like ‘dirty’ looking eyeballs. I think mine better retains the fine details, however. Could this be a consequence of the PAL vs NTSC sources? Yours is also noticeably cropped at the sides. I know the very edges shouldn’t really be visible and often have irritating green or yellow lines etc., but the crop you’ve gone with cuts off quite a bit. Is that intentional, or is it inherent to the PAL DVDs?

Author
Time
 (Edited)

Thanks for your sample - and since I used to work with NTSC Discs as well, I can see that the PAL Discs were definitely “down-mastered” from NTSC, which accounts for the detail and sharpness loss.

Your sample has definitely more detail and looks really nice, so let me take a bow and say “Fascinating!” 😉

I did some trial runs with Dione but found that with the PAL Discs (mixed material in the worst possible sense) it doesn’t “catch” all the problems, QTGMC however does. That unfortunately also means extrapolating image information which does not always help much with detail preservation.
Also, I am doing a little color regrading in DaVinci which darkens things slightly and reduces the shift toward the more warmer colors and gives it a bit of a cooler look (speaking about color temp here) - I was inspired by the DS9 Doc “What We Left Behind”. In rare cases (dark scenes) that can make a little detail disappear here and there, but not much.

As for the crop on the sides: My bad, sorry. I am usually doing manual cropping but for my samples I leave things on automatic with StaxRip. The problem lies with the PAL DVDs here: The image is not consistently centered, so there is no proper way to crop, since the values keep shifting (during one scene the image can have a black bar of about 2 pixels width on the left, the next scene can have 4 pixels width on the left).
I had it on automatic, so it made a bad choice. Usually, I skip through the episode and cut the smallest range. Again, sorry.

I only made the decision to work from PAL for the “smooth and stable” 23,976 FPS, otherwise I would have gladly used the NTSC.

On my workflow: Yes, it takes forever. Example for a standard 45 minute episode:

  1. StaxRip - 7 hours
  2. Topaz Gaia HQ 100% - 4 hours
  3. Topaz Artemis LQ 1080pHD - 3 hours
  4. Topaz Gaia CG 4K - 26 hours
  5. DaVinci Resolve Videostream rendering - 8 hours.

So it comes down to a total of about 48 hours, so 2 days for a single episode, providing I am there to make the switch between programs/input. I am crazy enough to invest that, running on a 2060 RTX.

Hope I could clear up all your questions.

The Ends DO Justify The Means …

Author
Time

Hey guys, sorry I have been so absent lately, but a lot of stuff kept interfering with my work here.

However, an updated version of the DS9 pilot is on the way (I will post samples here soon) - that includes some tweaking to compensate for oversharpening and better motion flow. Stay tuned.

The Ends DO Justify The Means …

Author
Time

Hello there people, huge updates coming your way. Here they are in a nutshell:

New Workflow: https://www.dropbox.com/s/7xlq13flzan9g40/Detailed Workflow.rar?dl=0

Template for StaxRip: https://www.dropbox.com/s/kp6y09fxhugovht/StaxRip Template - Restoration %26 Cleanup.rar?dl=0

Color Regrading LUT (works with Adobe, DaVinci Resolve): https://www.dropbox.com/s/ezm128n0nyfhjx9/DS9 Restoration Lut .rar?dl=0

Comparison Images for the Pilot Episode: https://www.dropbox.com/s/eqffxpkjyc3610w/1x01%3B02 - E %3B DA (SD vs 4K Images).rar?dl=0

4K-Video Samples for the Pilot Episode: https://www.dropbox.com/sh/riwdi0lb9w0s5kg/AADiY-spYBjJz2MMW_MYuKaLa?dl=0

Have fun and let me know what you think!

The Ends DO Justify The Means …

Author
Time

Black Alert said:

With this project I want to showcase to fans of DS9 what can be possible in the universe of (AI-based) upscaling.

Those of you who have watched Deep Space Nine recently on modern/bigger screens know what the quality of the DVDs, streaming services and WEB-files look like: Most of the episodes (especially during earlier seasons) display problems like blurriness, areas out of focus, issues with grain (luma and chroma), washed out colors, a non-HD-conforming color palette (which makes sense given the technology back then), compression or block artefacts, interlacing issues and problems with motion fluidity originating from variable frame rates.

The approach of this project (in a nutshell) is to do the following steps to assure that the visual quality improves and the result is better “watchable” on a 4K screen:

  1. Take the source material and extract the raw data (working from PAL sources here)
  2. Adjust the audio track so it will fit with the new constant frame rate of 23,976 FPS (finished video)
  3. Re-Encode and filter the source material to get rid of several issues, like:
    • Rainbowing, Interlacing, Artefacts, Block Compression, Haloing, Ringing, Aliasing, Softness, Banding
  4. AI-based image enhancements and upscaling, like:
    • Detail enhancements, increase of focus, upscaling of the picture dimensions (to 1080p or even 4K)
  5. Final Encode/Rendering with additional image improvements. like:
    • Custom Aspect Ratio (original 4:3 on HD-resolution without black bars left and right
    • Correction of Pixel Aspect Ratio
    • Adjustment of the SD-color-palette to more closely resemble an HD-like look
      (the work here is based on what I have seen on the DS9 Documentary “What We Left Behind”)
    • Working in grain to give a “less plastic/waxy look”
    • Sharpening an Unsharpening to create a more “pseudo-detailed look”
    • Encoding in a more HD-compatible, modern file-format

If you wish for more detailed information (like the complete guide/workflow on “how to do the project”, comparison images, samples & more) and other stuff, please contact me via private message and I shall do my best to provide what you are looking for 😉

I will do regular updates with images on what I am currently working on, so you can see what is possible - I will also actively discuss the project with you and will try to answer questions (if I can).

Stay tuned.

EDIT:
Ok, the general password to access all the files around here is: “DS9Enhanced” (without the quotation marks).
Please be advised, that all comparison images have the following characteristics:

  • Left Side: Original SD-resolution image (PAL-DVD) without crop oder modifications
  • Right Side: Upscaled 4K-resolution image (2880x2160p) with cropping left and right
  • Images are displayed through a media player that stretches the image to fit the screen, so the right
    sided image looks a little “wider” than the left, but the original aspect ratio is maintained (4:3)

I have a copy of “In the Pale Moonlight” HD .MP4 1080p on my HDD that I upscaled from a DVD… They need to be updated. Beware there is a clickbait “4K” news story out there so watch out! No one else is talking about such news…

Author
Time

@RedRust1987:

Thanks for the information about the clickbait 4K article around the net - if it is the same one I saw (it was just on one site which I found by pure coincidence) then it talked about some studio doing a 4K remaster of DS9 which was supposedly connected to some Trek producers.

However, when I looked further, I found no evidence that there is any truth to it.

So I gues we are talking about the same thing here.

The Ends DO Justify The Means …

Author
Time

I think the latest iteration looks great, but I have to say that I’m not a fan of the color grading. DS9 always had this “warm” look to it in general, and I think the color grading takes something away from it. It’s most obvious with skintones, in some of the comparison shots Patrick Stewart’s skin is so pale and gray he looks like a Cardassian, and Colm Meaney loses that “rosy” look he usually has, if that makes any sense. I understand it’s a personal choice and you’re doing all of the work, just thought I’d offer up my opinion here.

Author
Time

@lowkey:

Thanks for the input, I have actually been reconsidering my color grading lately, so your advice gave me the boost I needed to take another shot at it.
Also, I have been tinkering around again with my settings, I will post updated samples soon.

The Ends DO Justify The Means …

Author
Time

I quite liked the colour grading. It added to the feeling of the dilapidation of the station, though it did wash out Dax’s spots.

Author
Time

I just modified it a little - a bit more saturation (about 10%) and an added 0.10 in the reds, so the faces don’t look “too pale”. But I have to wait and see it on the big screen, images on the pc don’t give the same vibe as the real thing on the 4K OLED.
Also, I switched out one model in t he workflow, since I still couldn’t get rid of the grainy pattern look the sometimes appears on faces. Once I got that covered, I should also be able to deliver a sharper output, since I had to reduce it to avoid that look I mentioned.

Will report back soon.

The Ends DO Justify The Means …