Star Wars: A New Hope [SET ruLes 1.0] - AVCHD & BD RELEASED! — Page 2

g-force said:

Just watched the OUT sample you posted. Large halos, trails of grain.

I thought you were giving up on the "enhancement". You are using a horrible denoiser, and an even worse sharpener.

Would you like to give me a gift, in form of denoiser/sharpener/upscaler avisynth script?

G,

I didn't want to be sarcastic, really! After looking *carefully*, at a lot of frames processed with my script, I must admit that there are good things, but bad ones too...

So, I used the plugins you suggested, and it was better; yes, my script retain more details, but also more noise... please download the new clip, resized, and then LimitedSharpenFaster and MCTemporalDenoise:

SENDSPACE

and let me know if could be a good choice for OUT ruLes 2.0 -> by the way, the processing time was incredibly high! 0.05 to 0.09FPS!!! Many hundred (up to one thousand) hour to process the whole movie? NO WAY!

So, I must find a way to accelerate the dub time - my PC will be fried after 40 days! - I thought to save the unprocessed upscaled version in lagarith, then let the script use this last file, instead of doing all in real time (load captures, median/average, upscale the result, then sharpen/denoise)...

OR I should find another script.

What about "Avisynth best script to enhance Star Wars restoration project that uses only laserdisc captures" contest? (^^,)

CapableMetal said:

Wow, what processing is happening in your script? Could you post an example? That's certainly very slow, and I'm curious as to how you're applying filters.

First, more than 200 lines of getting rid of black frames, aligning pixel perfect, median and average - I avoid to paste it here... then, at the end:

ROUGH.nnedi3_rpow2(rfactor=2, cshift="spline64resize",\

fwidth=ROUGH.width*2, fheight=ROUGH.height*2).\

LimitedSharpenFaster(strength=200).\

I tested the filter before nneed3, in between, and after resamplehq, and "in media stat virtus", so...

My advice is to keep it simple. I've watched the SET sample you posted before and its not as clean as I expected, there seems to be a lot of noise in the picture and I'm unsure as to what has caused it. What format do you capture to?

HuffYUV YUV.

It also suffers from the 'tiled' effect that is inherent to the PAL SE laserdiscs (a crude description, but I don't know what else to call it), its visible in your sample mostly as faint vertical lines, but appears as little tiles if you purposely boost the brightness too high on a raw capture; I usually use DeGrainMedian as a noise filter and it tends to fix this.

I'll test it.

Would doing that not take the same/similar amount of time, but using two scripts?

I used only the first two lines of the preceding script to save lagarith video (2.70 FPS); then the resulting upscaled video is processed with the last three lines (0.86 FPS)... not bad this time!

You_Too said:

If you have a multicore processor you could check what the possibilities are of accelerating that render time.

There's multithread-supporting versions of avisynth and some plugins support it too. It could help a lot.

I think that new sample looks pretty good!

I use the multicore avisynth DLL; I applied SetMemoryMax and SetMTMode with different settings, but the speed didn't improve. Any hint wil be highly appreciated (my PC has a Core 2 Duo E6300 with 3GB RAM)

At the end, I found "the peace of mind" thanks also to all your feedbacks, and now I want to follow a new philosophy, "less is more"... in other words, use as few filters as I can - to not overprocessing as I did with my former script.

But, anyway, should I add a derainbow filter? And what about a smoother? Will be they useful? Will be they really needed?

The last words to the experts.

g-force said:

Dude, that looks so much better! I think it's still a little waxy, as in too denoised. I wonder if you could back off the denoising a bit and gain more processing speed.

The sharpness looks good!

Thanks, I'm happy you appreciate my work!

I must confess that I like this last "less is more" script way more than the long script I used before. To me, if not perfect, it's almost perfect - and, because "perfection is not of this world", I could be happy anyway!

I know it's a little waxy as you wrote... I tried also MCTemporalDenoise(settings="low) (this is the default setting) but the result was still too noised; I like also the settings="high", but the processing time was incredibly higher, and the result, if carefully inspected, too waxy, so...

Maybe the script I tested is the "right" compromise - and, I repeat, I tested A LOT of filters in a year of testing, and I prefer to re-release soon the SW:ANH [OUT ruLes] with improved audio (and in sync french soundtrack) plus improved video, than waiting and testing for weeks (or months) to achieve a visual result only slightly better than the actual one: avisynth is WONDERFUL, but with so many filters, plugins, scripts, the possibilities are unlimited, but my time unfortunately is not! (°°,)

CapableMetal said:

If there's rainbowing, then I'd give it a try. I've had limited/poor results using some of these filters in the past, and I believe if your comb filter is doing a good job then you should have minimal rainbowing already.

I used the 925 S-Video because, in my comparison tests, its comb filter resulted better than my capture card. I didn't noticed rainbowing, but you know, "four eyes are better than two"... and six are better than four etc.

So, if anyone else would like to download the OUT ruLes test clip, and let me know its problems, it will be really helpful!

With laserdiscs, after median/averaging captures, I simply use DeGrainMedian for noise reduction (and it does a nice job for the most part), Msharpen used subtly to bring back some small details, if I upscale then a combination of nnedi3_rpow2 and a Spline64 or 36 resize, depending on my needs. If there is visible colour banding then a dithering filter (like GradFun3) can help.

Today was "capture day #2"... hard work... I finished to capture all the three editions, and carefully aligned them both temporally and spatially, and then medianed... now I'm testing the script I tested yesterday on OUT

here you are the usual clip at SENDSPACE (70MB)

download it, and tell me what do you think.

I'm still testing a basic color correction, as the laserdisc picture is still pinkish - but not as GKar... - here few examples:

Not perfect, but not too bad... (^^,)

Next days I will test the filters you suggested - if you want to help me, could you please post an example script?

I occasionally add and remove filters as needed, but those tend to do a nice enough job. Using an MT version of AviSynth helps a lot. If you have a dual core CPU then I'd try it, it should greatly increase your render speed and give that idle core something to do ;)

I use avisynthMT too - but some filters still does not improve speed with it...

Hope to release the ANH [SET ruLes] .m2ts file before the end of this month!

CapableMetal said:

Less certainly is more. I wish some of the studios would realise this when applying DVNR to blu-ray masters!

I'm happy I changed idea and gone into the "less is more" philosophy also about avisynth script - I always follow that philosophy when I build web sites.

I thought you were using a DVD recorder pass-through for comb filtering? I know that the s-video on my CLD-2950 is horrible compared to the composite output (which I tend to use with the "HQ Filter" off, but cannot disable it on the s-video output), I thought that was the case with most LD players, with the exception of the very best NTSC players?

When I used the composite out of the CLD-D925, I saw the rainbow effects around the white, superimposed display info; when I used the S-Video, the rainbow was not there anymore, so I think with this player the comb filter is better than my capture card. About CLD-2950, maybe, as it is older, the comb filter is not that good as the D925; and I set the HQ always off, and it stay off also using S-Video.

Watched it and its looking good, however there is a serious misalignment of one of your captures on the first shot (look at the small moon, the Tantive's engines, then the Star Destroyer). There is also a combed frame as Threepio looks up and says "What's that?".

You are a keen observer; about the misalignment, I think now it's fixed - the german capture was off only in this scene, but as it was really dark I made the wrong alignment...

About the combed frame(s): they are frames from 4663 to 4666, but they must be in the master, at least in the PAL one used with these THX laserdiscs, as they are present in all the three laserdisc editions, and in all of the several test captures I made with different players and connections, so... maybe if they are good in the NTSC SE, I could use those frames from there...

Looks great! Its difficult with the SE because all of the transfers suffer from colour issues, you're doing a nice job.

Thanks, I think it's not perfect, but surely better than the "original" pinkish!

I just stick with SetMTMode(2,0) at the start of a script which is maybe not as fast as manually assigning filters to threads but is enough of a performance boost. My scripts tend to be quite basic because I've found filters that I prefer and that do a good job, all of which seem to work nicely multi-threaded. I depends on the filters you are using and how much filtering you're doing, of course. As you say, some filters just can't benefit. I tend to use the default filter settings a lot of the time too, only changing settings if something doesn't look right (for fine adjustment). I rarely have scripts that occupy more than 30 lines, usually they're only about 10 lines, but then I haven't tried layering different sources! The results I get from the filters I use are very good, too. Sometimes keeping it simple is enough, I'm sure I read on another thread that the upscaling scripts for DJ and You_too's project was only a few lines...

I use also SetMTMode(2,0); yes, it improves speed, but I discovered that the best thing is to save the aligned/median/upscaled result (with lagarith) and then feed it to sharpener/denoiser/color correction script, and save with the preferred codec (e.g. AVC1 for the test clip). With this trick, the total processing time would be a lot less, and, I want to change something in the enhancement script, I have not to process the whole script, but only the enhancement, as the aligned/median/upscaled is saved as .avi - got the idea?

About the lenght of the aviscript: only to align the three captures, I used about 30 lines of code - more than 50 are used for OUT; for median, maybe five or six; for color correction, sharpener, denoiser, upscaler, about ten.

Right now my PC is processing another test clip; it's the same, but fixed at the beginning (hope so) and color corrected; now I must go to work, but I'll post it in the evening.

***

Last thing: anybody has the french 1997 SE THX laserdisc trilogy - separate movies, not in a box, with the FRENCH audio? I'd like to add the french soundtrack...

Understand that I'm not on a calibrated monitor but shots from your projects always seem to look kind of green-ish to me. It's especially evident on that Leia shot. I think it looks better than the red from your sources but you might want to take another look at your color correction process.

The color correction tests are going on... it's really difficult to find the right settings!

I changed a bit the color correction settings; here the last test - upscaled, sharpen, denoised, color corrected: SENDSPACE

Waiting for feedbacks.

You_Too said:

To comment on the latest clip, it looks like the shadow detail has been crushed.

I don't know... because in the last clip I uploaded, that scene has NO color correction! To be sure, I checked all the three original captures, and they look like the same as the median result, as the upscaled and the sharpened/denoised results...

Maybe is due to the fact that the first screenshot came from the first tests I made, using three different players - and two through dvd recorder's comb filter; maybe there are different gamma settings on the laserdisc players, or on dvd recorders, or in the combination of two... who knows?!?

***

Well, let's change the clip! Now I took Mos Eisley scenes; on the left, the original colors, on the right the color corrected. And this time I used a low bitrate (2000kbps) as this is only a test clip, and for that purpose, the quality is good enough... so, at 12MB, anyone would be happy to download and review it:

SENDSPACE

***

About color correction, some thoughts came into my mind... "I made this simple color correction in Avisynth - that is a powerful tools, and free - in, counting only time devoted to "invent" the simple script and testing it, let's say ten hours; this is my first attempt at color correcting a movie, and I'm only a young old padawan learner, but the result is decent, surely better than the original pinkish mess, so... why THEY didn't do color correction with the laserdisc master at the time (1997) with all their powerful hardware - computers and also analog gears - and software, or why THE OTHERS didn't make something similar when they broadcast the DVB version (2004-5) when surely digital color correction was easy and not so expensive as, for example, could be in 1997?"

By the way, IIRC, avisynth were released around 2002, and could be used for the DVB broadcasting color correction... but I digress! (++_)

CapableMetal said:

BTW, the latest clips look great!

Thank you very much! It's great when someone shows his appreciation for my work, and keep me going on with my project!

You_Too said:

To comment on the latest clip, it looks like the shadow detail has been crushed.

Still thinking about what may be the cause of it... I used two different VDub versions for capturing the first clip (1.9.11 for D925, 1.8.8 for 4300 and 600), and only the newest 1.9.11 for the second clip, using the same D925 player for all the three editions. Is any difference in quality and reliability, when capturing analog video, between different VDub versions?

Or maybe it is because I didn't flag neither the "extend luma black point" nor the "extend luma white point"? I don't think the problem lies here.

Or (and I think the problem, and the solution too, is here) it is because in the first clip I passed the laserdisc signal through the Pioneer DVD recorder, using it as comb filter, with its white level set high?

If it's the latter, it's just a matter of recapture everything (sic!) with the right white/black settings - find the right contrast and brightness on VirtualDub video proc amp settings during capture, this time looking carefully at the histogram, instead using the defaul settings...

OK, I'm ready to recapture all (Xx/) ..."errare humanum est"!

Some updates - for the few still interested in this project...

Black crushing & white clipping (supposed) problems

I thought I captured the laserdiscs with wrong settings (default)... so I took one of the laserdisc I captured before, and did some tests. For example, I took the scene right after the opening crawl: I let the laserdisc player play that scene over and over, and with the Virtualdub histogram in capture mode I discovered that the image histogram (luma values) are inside the left and right borders, except when there are explosions, with flashes the image with totally white screen - only in that case, the right border (white luma) stay a bit out of border; I don't think this is a problem, so I decided to not recapture the laserdiscs again, as the default settings are 99% accurate.

But I realized that my captures' luma range is 0-255, and not 16-235 as I thought before... it's true that the "extend luma white point" and "extend luma black point" in Virtualdub capture mode make the image "tv compliant" - infact, when flagged, the image range is "compressed" and forced to stay in the 16-235 luma range, but the exceeding black and white parts are clipped... so, if I use Avisynth to convert my capture from full range 0-255 to tv range 16-235 will be the same, if not better - and if I will decide to release a full range version, I will just not do the conversion! Obviously this fact applies also to my OUT captures... hope this fact will help someone else!

Color correction

At the beginning, I was sure that all the movie had wrong colors, so I was testing the same color correction settings with different scenes, and in some it was perfect, in others so-so, and in others completely wrong... so I realized that only some scenes are affected by the pinkish tint, and many are good "as is"...

To test my assumption, I ran the whole movie through an avisynth script: on the left, the unprocessed movie, on the right, the movie with autowhite corretion; the second part of the movie is almost all color correct, as both images are quite the same. Of course, the initial scenes inside the Tantive corridors and in Mos Eisley were afflicted by the pinkish tones.

But not all the scenes with wrong colors were corrected in the right manner by autowhite (obviously, as it is a "stupid" filter, as written also in the avisynth wiki)... for example, in the desert scenes, sometimes the sand were yellow - instead of pink, so it was right, but sometimes it was grey!!! If the autogain is applied, many times things get better, but the contrast fluctuated a lot, and in few cases there are on overly exagerated grain in the whole picture.

It's safe to say that autogain CAN'T be used for color correction, but autowhite can, in limited cases - at least, this is what I have discovered during my tests. But, HOW can I use autowhite, where it make its work good for a part of a scene, and bad for another? Must I use it for the whole movie, check where the job is wrong, and change the wrong scenes with my own color correction?

At the end, the usual little 25-watt light bulb appeared over my head... what if I use a median of three color version of the same image? the first will be the unprocessed, the second will be the autowhite corrected, and the third the manual corrected... if the first and second would be (quite) the same, the manual correction will be discarded, if the autowhite is overly exagerated or plain wrong, the result will not contain that...

Well, I did some tests along the whole movie, using four quadrants (unprocessed, autowhite, manual correction, median) and I exclaimed: "in media(n) stat virtus!" (virtue stands in the middle)... it corrected all the autowhite problems.

I'm also thinking about getting an hardware color corrector (I'm after a cheap Sony XV-C900, PAL) and use it instead of avisynth... it will color correct real time, with its two joysticks for black and white adjust... what do you all think?

Last thing: as my captures are YUY2, I used ColorYUV (and ColorYUV2) avisynth filter for color correction, to avoid any conversion to RGB and back... but at the beginning, I applied the color correction to YV12 video! And discovered that, despite the fact that the filter works with YV12 too, the results sometimes are completely wrong! So, the lesson learned is: color correct YUY2 video with ColorYUV BEFORE converting the video to YV12! (^^!)

Comb filter and rainbowing

On my OUT project, I used the same capture chain used here: Pioneer CLD-D925 laserdisc player, S-Video out to the capture card, with Philips SAA-713x chipset. I chose to use S-Video instead of composite, because, after all the tests I've done, it seems that the result is slightly better; also if the capture card comb filter for PAL signals should be 4-line, Vs CLD-D925's 3-line, the composite output is just a recombination of the S-Video signal, so I think it's wiser to use the player S-Video directly.

But I tested also other laserdisc players and units - CLD-600 and LD-V4300D, both directly, via composite to capture card, and through DVD recorders - Pioneer DVR-320S and Panasonic DMR-HS15 - used as external comb filter.

It's difficult to get information about the comb filter type present on these DVD recorders; I found that the DVR-320S has a 3D comb filter, but unfortunately it works only with NTSC... with PAL, it *could* be a 5-line type, that *should* be better than capture card's one... about the DMR-HS15, many praise its TBC, but it's not needed when capturing laserdisc, as any laserdisc player has one built-in...

By the way, I remeber that when I tested those players through the DVR-320S, using Video Essentials' Snell & Wilcox zone plate, its comb filter was really good - as a good 3D comb filter it's better than 4 or 5 line one. But using the DVR-320S with PAL signal didn't improve the quality... at least, my own eyes didn't see any difference between its comb filter and the capture card's... if only I could have a PAL test card on laserdisc to test it... OR... if I take a PAL test card, 720x576, and let a computer play it through its composite output, then capture its composite output with my capture card composite in, and also with the DVR-320S and DMR-HS15, then I could clearly see which has the best comb filter, right?

Other reasons I avoided using a DVD recorder as external comb filter are:

longer video path - with everything in between that could worsen the quality instead improve it... like

LD -> cable -> A/D -> DVD digital processing -> D/A -> cable -> capture card

VS

LD -> cable -> capture card

...and the unknown processing inside the DVD recorder... it could include, apart comb filter, video DNR, color DNR, color AGC, MPEG2 conversion, and quite surely TBC, and YUY2 to YV12 conversion. Someone could argue that the latter would be not so bad for quality, but I prefer to stay inside the YUY2 colorspace as long as I can, including color correction and resizing... infact, if I have (for example) 700x340 YUY2 video, the chroma will be (at best) 350x340; useful if I want to convert to 1280x568 - where the chroma resolution will be 640x284; if I have 700x340 YV12, chroma will be only (at best) 350x170... do you think it's still the best option for resizing - also for a "simple" letterbox to anamorphic resize?

I'm also thinking at a "crazy" alternative capture technique... using the LD-V4300D RGB output, capture the separate red, green and blue image, then recover the RGB image with avisynth. I'm sure someone else has done it before, I read it a long time ago, but could not find that technique info again anymore... I know that laserdisc is composite, and the chroma resolution of a PAL laserdisc will be *at best* around 212*576, but who knows how good could be a composite-to-RGB converter inside an old industrial-grade laserdisc player... maybe it will be an interesting discover!

At the end...

Laserdisc captures done; spatio-temporally alignment to median the captures done; upscaling, denoising, sharpening scripts tested - found the best ones; median color correction tested - found good result. Now I must "mix" all these elements, and the video will be ready.

I decided - to reduce processing time, and to not stress a lot my old pc - to process things in distinct steps: tomorrow evening the median/2x upscale will be finished; after that, I'll apply color correction median and then sharpening/denoising/resizing.

Next things to do: capture audio from italian VHS, align PCM, AC3, DTS soundtracks to video, try to find the PAL french laserdisc soundtrack, convert audio to 23.976fps; correct and align the subtitles; convert to suitable release formats; mux all.

Latest news: I decided to buy another PC ("only for Star Wars project?" - "well, *not* only for that...") and it will be... another Sony VGX XL! This time with a faster CPU and a BD BURNER!!! Hope to upgrade soon the 2.13GHz E6400 (no, the Sony VGX-XL series don't permit overclock...) to a faster X6800, or better to a QX6800 - if only I could be absolutely sure it will work into my new PC!

         "...do you think anyone will read all your post until here?" (OO_)

(u°°) "...surely someone will read these last lines first!"

Project update:

the median/2x uspcale file is ready.

I improved a bit the color correction scripts; I used the median and average mix method I explained before here, but I was still not enterely happy with the result... so I was "forced" to use the autowhite in some (really few) scenes... now the colors are consistent (almost everytime) during the whole movie.

Now the PC is rendering the color corrected file - it will be ready before midnight; then, I will process the color corrected file with the sharpen/denoise filter, and downscale it to the right resolution - it will take further 3 days of PC work!

After reading through this whole thread, I'm actually pretty excited to see how this turns out. Thanks for the hard work!