Info: Digging up those blacks - using the STAR WARS Blu-ray for preservations — Page 5

Here's my whole plugin folder: http://www.zshare.net/download/99393704c8cd6a0b/

Better than trying to dig up which exact ones were used for this script! haha... And all the other ones can come in handy anyway.

Awesome! You're a gentleman and a scholar. :)

I think I found a fix for that deepblue issue I was having. Some of the code appears to have been repeated on one line, which led to an error. If I successfully test tonight I'll post the fixed code... and then be rendering out this bad boy!

The code is not repeated, I double checked it. Gotta have happened if you pasted it in notepad.

Weird... yeah it must be something with that then. Didn't think to compare the line with your original code. Damned Microsoft! ;)

Looks like I got deep blue working, but now it's saying I'm missing the curve file. Checked your most recent link and it doesn't appear to be on there? Would you happen to still have that? Sorry about the hassle.

Here's the curve file: http://www.zshare.net/download/99414424f1f64331/

Don't worry, it's not your fault there's been trouble with those file hosting sites!

I would increase the saturation. The movie was originally very saturated

Well, the settings I posted here back then are very outdated now.

Having better references now, I'm sure I could do better if I remade the settings today. Currently I'm a bit too busy to do it though.

Here's an experimental script I made a while ago, trying to make make the movie look more like the technicolor version:

DirectShowSource("Starwars.mkv") # Replace with the filename of the movie.

ConvertToRGB32()

GiCoCu("BD1.cur")

selectivecolour(last,\
0,-0.05,0,\
-0.4,0,0,\
-0.05,0.25,0,\
0,0.5,-0.5,\
0,0,0,\
-0.25,-0.5,0,\
false)

GiCoCu("BD4.cur")

ConvertToYV12()

 

Function SelSah_v3 (clip clp, float "Sat", float "Hue", int "Luma_Low", int "Luma_High", int "Low_OS", int "High_OS",
\                    Float "minSat", float "maxSat", float "startHue", float "endHue", float "interp", clip "clp2", 
\                    bool "Exclude", bool "Preview") 
{

#  Extends Tweak to allow selective control of saturation and hue over a defined luma range. 
#  Assembled by WorBry. MaskTools computations kindly provided by Didee. Thanks to Gavino for advice on script syntax.   
#  Requires YV12 clp input and assumes full (pc) luma range 0-255.
#  Requires MaskTools v2. Tweak should be v2.58 or higher.
#
#  Parameters:
#  Clp       - Is the input clip that will be processed.
#  Clp2      - Provides option to use a second clip with different luma curve for the saturation/hue control.
#  Sat       - Saturation. Desaturation 0.0 =< 1.0 > Increased saturation. See Tweak 2.58 doc 
#  Hue       - Hue. Range -180 to +180. Default 0.0. See Tweak 2.58 doc 
#  startHue  - See Tweak 2.58 doc for details. Range 0 - <360. Default 0 
#  endHue    - See Tweak 2.58 doc for details. Range >0 - 360. Default 360
#  minSat    - See Tweak 2.58 doc for details. Range 0 - <150 ; must be < MaxSat. Default 0
#  maxSat    - See Tweak 2.58 doc for details. Range >0 -150. Default 150
#  interp    - See Tweak 2.58 doc for details. Range 0 - 32. Default 16
#  Luma_Low  - Lower luma range point. Range 0 - 255*  Default 0
#  Luma_High - Upper luma range point. Range 0 - 255*  Default 255
#  Low_OS    - Lower over-shoot point. Provides soft roll-off at lower range boundary. Range 0 - 255*
#              Default is Luma_Low+20 for normal (inclusive) range mode and Luma_low-20 for Exclude mode. 
#  High_OS   - Upper over-shoot point. Provides soft roll-off at upper range boundary. Range 0 - 255*
#              Default is Luma_High-20 for normal (inclusive) mode and Luma_High+20 for Exclude mode.
#  Exclude     True/False. True excludes the range selected by the luma set points. False (default) includes the selected range.  
#  Preview   - True/False. True gives preview of input clip, output and luma masks with original and output chroma.
#              Default is false
# 
#  *Note: the luma set points are limited with respect to each other to ensure correct orientation and avoid overlap.   
#
#Defaults:
Sat       = default(Sat, 1.0)
Hue       = default(Hue, 0.0)
startHue  = default(startHue, 0)
endHue    = default(endHue, 360)
minSat    = default(minSat, 0)
maxSat    = default(maxSat, 150)
interp    = default(interp, 16) 
Exclude   = default(Exclude, False)
Luma_Low  = default(Luma_Low, 0.0)
Luma_High = default(Luma_High, 255) 
Low_OS    = Exclude ? default(Low_OS,  Luma_Low + 20)
\         : default(Low_OS,  Luma_Low - 20)   
High_OS   = Exclude ? default(High_OS, Luma_High - 20)
\         : default(High_OS, Luma_High + 20)
clp2      = default(clp2, clp)
Preview   = default(Preview, False)        
#
#Limits: 
Sat       = Sat <0.0 ? 0.0 : Sat
Hue       = Hue <-180 ? -180 : Hue > 180 ? 180 : Hue
Luma_Low  = Luma_Low  <0.0 ? 0.0 : Luma_Low  >255 ? 255 : Luma_Low
Luma_High = Luma_High <0.0 ? 0.0 : Luma_High >255 ? 255 : Luma_High
Low_OS    = Exclude &&  Low_OS \         : Exclude &&  Low_OS > High_OS ? ((Luma_Low + Luma_High)/2)                 
\         : Low_OS  <0.0 ? 0.0 : Low_OS >Luma_Low ? Luma_Low : Low_OS      
High_OS   = Exclude &&  High_OS >Luma_High ? Luma_High
\         : Exclude && High_OS < Low_OS ? ((Luma_Low + Luma_High + 2)/2)
\         : High_OS 255 ? 255 : High_OS
#
#Limits for startHue, endHue, minSat and maxSat set by Tweak 
#
#Create Saturation and Hue control clip:
twclp = clp2.tweak(sat=Sat, hue=Hue, startHue=startHue, endHue=endHue,
\       minSat=minSat, maxSat=maxSat, interp=interp, coring=false)
#
#Create luma mask with low/high range points and roll-off gradient at boundaries:  
#"Include" [0]<-black->[Low_OS]<-gradient->[Luma_Low]<-white->[Luma_High]<-gradient->[High_OS]<-black->[255]
#"Exclude" [0]<-white->[Luma_Low]<-gradient->[Low_OS]<-black->[High_OS]<-gradient->[Luma_High]<-white->[255]
#
Low_OS    = string(Low_OS) Luma_Low  = String(Luma_Low)
Luma_High = String(Luma_High) High_OS = String(High_OS)
#Create "Include" range expression for mask:
blkI      = "x "+Low_OS+" < x "+High_OS+" > | 0 "
whtI      = "x "+Luma_Low+" >= x "+Luma_High+" <= & 255 "
grLI      = "x "+Luma_Low+" < 255 x "+Low_OS+" - "+Luma_Low+" "+Low_OS+" - / * "
grHI      = "x "+Luma_High+" > 255 1 x "+Luma_High+" - "+High_OS+" "+Luma_High+" - / - * "
exprI     = blkI + whtI + grLI + grHI + "x ? ? ? ?"
#Create "Exclude" range expression for mask:
blkE      = "x "+Low_OS+" >= x "+High_OS+" <= & 0 "
whtE      = "x "+Luma_Low+" < x "+Luma_High+" > | 255 "
grLE      = "x "+Low_OS+" <= 255 x "+Low_OS+" - "+Luma_Low+" "+Low_OS+" - / * "
grHE      = "x "+Luma_High+" < 255 x "+High_OS+" - "+Luma_High+" "+High_OS+" - / * "   
exprE     = blkE + whtE + grLE + grHE +  "x ? ? ? ?"
#Luma Mask (greyscale for output):
lmask     = twclp.mt_lut(expr = (Exclude ? ExprE : ExprI))
lmaskg    = lmask.Greyscale()
# Apply luma mask:
Output    = mt_merge(clp,twclp,lmaskg,luma=true,Y=2,U=3,V=3)
#
#PREVIEW set-up
clps      =clp.Subtitle("Original Clip").LanczosResize(640,360)
outs      =output.Subtitle("Final Output").LanczosResize(640,360)
#Luma Mask with chroma from tweaked clp
lmcts     = lmask.Subtitle("Luma mask + Tweaked Chroma").LanczosResize(640,360)
#Luma Mask with chroma from original clp
lmcc      = Exclude ? clp2.mt_lut(expr=ExprE) : clp2.mt_lut(expr=exprI) 
lmccs     = lmcc.Subtitle("Luma Mask + Original Chroma").LanczosResize(640,360)
#Create composite view
Clpout    = StackVertical(clps,outs)
Masks     = StackVertical(lmccs, lmcts)
Prev      = StackHorizontal(Clpout,Masks)
#
Final     = Preview ? Prev : Output
#
Return(Final)

}

SelSah_v3(Luma_Low=0, Luma_High=25, startHue=310, endHue=80, Sat=0.50)

ConvertToRGB32()

GiCoCu("BDtech.cur")

selectivecolour(last,\
0.2,0,0,\
0,0.05,-0.05,\
0,0.20,0,\
0,0,0,\
0,0,0,\
-0.25,-0.5,0,\
false)

GiCoCu("BDtech2.cur")

Levels(0, 0.85, 255, 0, 255)

ConvertToYV12()

 

You'll need the custom curve files for this script. (and the following one)

Here's an alternate version of the script, aimed more for the 70's projection look as seen in Mike Verta's photos he posted:

DirectShowSource("Starwars.mkv") # Replace with the filename of the movie.

ConvertToRGB32()

GiCoCu("BD1.cur")

selectivecolour(last,\
0,-0.05,0,\
-0.4,0.3,0,\
-0.05,0.25,0,\
0,0.5,-0.5,\
0,0,0,\
-0.25,-0.5,0,\
false)

GiCoCu("BD4.cur")

ConvertToYV12()

 

Function SelSah_v3 (clip clp, float "Sat", float "Hue", int "Luma_Low", int "Luma_High", int "Low_OS", int "High_OS",
\                    Float "minSat", float "maxSat", float "startHue", float "endHue", float "interp", clip "clp2", 
\                    bool "Exclude", bool "Preview") 
{

#  Extends Tweak to allow selective control of saturation and hue over a defined luma range. 
#  Assembled by WorBry. MaskTools computations kindly provided by Didee. Thanks to Gavino for advice on script syntax.   
#  Requires YV12 clp input and assumes full (pc) luma range 0-255.
#  Requires MaskTools v2. Tweak should be v2.58 or higher.
#
#  Parameters:
#  Clp       - Is the input clip that will be processed.
#  Clp2      - Provides option to use a second clip with different luma curve for the saturation/hue control.
#  Sat       - Saturation. Desaturation 0.0 =< 1.0 > Increased saturation. See Tweak 2.58 doc 
#  Hue       - Hue. Range -180 to +180. Default 0.0. See Tweak 2.58 doc 
#  startHue  - See Tweak 2.58 doc for details. Range 0 - <360. Default 0 
#  endHue    - See Tweak 2.58 doc for details. Range >0 - 360. Default 360
#  minSat    - See Tweak 2.58 doc for details. Range 0 - <150 ; must be < MaxSat. Default 0
#  maxSat    - See Tweak 2.58 doc for details. Range >0 -150. Default 150
#  interp    - See Tweak 2.58 doc for details. Range 0 - 32. Default 16
#  Luma_Low  - Lower luma range point. Range 0 - 255*  Default 0
#  Luma_High - Upper luma range point. Range 0 - 255*  Default 255
#  Low_OS    - Lower over-shoot point. Provides soft roll-off at lower range boundary. Range 0 - 255*
#              Default is Luma_Low+20 for normal (inclusive) range mode and Luma_low-20 for Exclude mode. 
#  High_OS   - Upper over-shoot point. Provides soft roll-off at upper range boundary. Range 0 - 255*
#              Default is Luma_High-20 for normal (inclusive) mode and Luma_High+20 for Exclude mode.
#  Exclude     True/False. True excludes the range selected by the luma set points. False (default) includes the selected range.  
#  Preview   - True/False. True gives preview of input clip, output and luma masks with original and output chroma.
#              Default is false
# 
#  *Note: the luma set points are limited with respect to each other to ensure correct orientation and avoid overlap.   
#
#Defaults:
Sat       = default(Sat, 1.0)
Hue       = default(Hue, 0.0)
startHue  = default(startHue, 0)
endHue    = default(endHue, 360)
minSat    = default(minSat, 0)
maxSat    = default(maxSat, 150)
interp    = default(interp, 16) 
Exclude   = default(Exclude, False)
Luma_Low  = default(Luma_Low, 0.0)
Luma_High = default(Luma_High, 255) 
Low_OS    = Exclude ? default(Low_OS,  Luma_Low + 20)
\         : default(Low_OS,  Luma_Low - 20)   
High_OS   = Exclude ? default(High_OS, Luma_High - 20)
\         : default(High_OS, Luma_High + 20)
clp2      = default(clp2, clp)
Preview   = default(Preview, False)        
#
#Limits: 
Sat       = Sat <0.0 ? 0.0 : Sat
Hue       = Hue <-180 ? -180 : Hue > 180 ? 180 : Hue
Luma_Low  = Luma_Low  <0.0 ? 0.0 : Luma_Low  >255 ? 255 : Luma_Low
Luma_High = Luma_High <0.0 ? 0.0 : Luma_High >255 ? 255 : Luma_High
Low_OS    = Exclude &&  Low_OS \         : Exclude &&  Low_OS > High_OS ? ((Luma_Low + Luma_High)/2)                 
\         : Low_OS  <0.0 ? 0.0 : Low_OS >Luma_Low ? Luma_Low : Low_OS      
High_OS   = Exclude &&  High_OS >Luma_High ? Luma_High
\         : Exclude && High_OS < Low_OS ? ((Luma_Low + Luma_High + 2)/2)
\         : High_OS 255 ? 255 : High_OS
#
#Limits for startHue, endHue, minSat and maxSat set by Tweak 
#
#Create Saturation and Hue control clip:
twclp = clp2.tweak(sat=Sat, hue=Hue, startHue=startHue, endHue=endHue,
\       minSat=minSat, maxSat=maxSat, interp=interp, coring=false)
#
#Create luma mask with low/high range points and roll-off gradient at boundaries:  
#"Include" [0]<-black->[Low_OS]<-gradient->[Luma_Low]<-white->[Luma_High]<-gradient->[High_OS]<-black->[255]
#"Exclude" [0]<-white->[Luma_Low]<-gradient->[Low_OS]<-black->[High_OS]<-gradient->[Luma_High]<-white->[255]
#
Low_OS    = string(Low_OS) Luma_Low  = String(Luma_Low)
Luma_High = String(Luma_High) High_OS = String(High_OS)
#Create "Include" range expression for mask:
blkI      = "x "+Low_OS+" < x "+High_OS+" > | 0 "
whtI      = "x "+Luma_Low+" >= x "+Luma_High+" <= & 255 "
grLI      = "x "+Luma_Low+" < 255 x "+Low_OS+" - "+Luma_Low+" "+Low_OS+" - / * "
grHI      = "x "+Luma_High+" > 255 1 x "+Luma_High+" - "+High_OS+" "+Luma_High+" - / - * "
exprI     = blkI + whtI + grLI + grHI + "x ? ? ? ?"
#Create "Exclude" range expression for mask:
blkE      = "x "+Low_OS+" >= x "+High_OS+" <= & 0 "
whtE      = "x "+Luma_Low+" < x "+Luma_High+" > | 255 "
grLE      = "x "+Low_OS+" <= 255 x "+Low_OS+" - "+Luma_Low+" "+Low_OS+" - / * "
grHE      = "x "+Luma_High+" < 255 x "+High_OS+" - "+Luma_High+" "+High_OS+" - / * "   
exprE     = blkE + whtE + grLE + grHE +  "x ? ? ? ?"
#Luma Mask (greyscale for output):
lmask     = twclp.mt_lut(expr = (Exclude ? ExprE : ExprI))
lmaskg    = lmask.Greyscale()
# Apply luma mask:
Output    = mt_merge(clp,twclp,lmaskg,luma=true,Y=2,U=3,V=3)
#
#PREVIEW set-up
clps      =clp.Subtitle("Original Clip").LanczosResize(640,360)
outs      =output.Subtitle("Final Output").LanczosResize(640,360)
#Luma Mask with chroma from tweaked clp
lmcts     = lmask.Subtitle("Luma mask + Tweaked Chroma").LanczosResize(640,360)
#Luma Mask with chroma from original clp
lmcc      = Exclude ? clp2.mt_lut(expr=ExprE) : clp2.mt_lut(expr=exprI) 
lmccs     = lmcc.Subtitle("Luma Mask + Original Chroma").LanczosResize(640,360)
#Create composite view
Clpout    = StackVertical(clps,outs)
Masks     = StackVertical(lmccs, lmcts)
Prev      = StackHorizontal(Clpout,Masks)
#
Final     = Preview ? Prev : Output
#
Return(Final)

}

SelSah_v3(Luma_Low=0, Luma_High=25, startHue=310, endHue=80, Sat=0.50)

ConvertToRGB32()

GiCoCu("BDtech.cur")

selectivecolour(last,\
0.2,0,0,\
0,0.05,-0.05,\
0,0.20,0,\
0,0,0,\
0,0,0,\
-0.25,-0.5,0,\
false)

ConvertToYV12()

SelSah_v3(Luma_Low=0, Luma_High=25, startHue=310, endHue=80, Sat=0.50)

ConvertToRGB32()

Levels(0, 0.90, 255, 0, 255)

selectivecolour(last,\
0,0,0,\
0,0,-0.1,\
0,0.5,0,\
0,0,0,\
0,0,0.2,\
0,0,0,\
false)

GiCoCu("technew.cur")

Levels(0, 0.90, 255, 0, 255)

ConvertToYV12()

 

And note that these are only experimental scripts. They will have their ups and downs. Feel free to play around with them and change settings. :)

By the way, for those who wants to make sure the scripts work like they should, get this file.

I've included my whole plugins folder for avisynth there, which should usually be put in C:\Program files (x86)\AviSynth 2.5\

I've also included some system files. The latest version of avisynth which is also the most stable version I've come across, and a couple other files which are used by avisynth and it's plugins. If you have a 32-bit OS, put them in C:\Windows\System32\  , in 64-bit OS, put them in C:\Windows\SysWOW64\

That scene's color was retimed and boosted with absolutely no thought given to the people running past the doorway in the background. So when the red levels were jacked up, it played havoc with the skintones of those extras.

Even in the foreground stuff with 3PO you can tell that the red was pumped up too much. The little reflections off of his body have red haloes - someone compared them to Christmas lights. I think the red lights on the walls are also visibly clipped.

frank678 said:

For those that don't know how to run scripts could you post up a frames of each just to see how much you can dig out of the Blu-ray source (not for feedback purposes since they are experimental scripts)

Script 1 gallery

Script 2 gallery

(If you want to see them full-sized, click "direct links" on the right side and paste in notepad, then you can use those as links for the full images.)

I also want to point out again before anybody criticizes the colors in these new scripts, that they're just experimental scripts I wrote kinda fast when I had nothing else to do some weeks ago. I know the shadows are blue and all that. I simply did that to try and replicate what I've seen in the shots from the Senator and what Harmy posted in the color timing thread.

And most likely I won't be updating these scripts for a long time. There is much work ahead on DJ's Blu V2.

About lobster-man: It was simply something that happened due to the red boost applied to the special edition. Someone at Lucasfilm apparently thought the movie didn't have enough red so they boosted them so much that the brightest reds got clipped, and skintones would sometimes appear too saturated, as with this guy.

Here's how he looks in the raw blu-ray:

And just for fun, here's the same using the first of the two experimental scripts, which is my favorite of those scripts. :)

Wow although I prefer the understated naturalness of the first script - the second really does achieve that golden 1970s look. The 1970s look would be the version that I would want to see first. Both palettes far superior to the original Blu Ray again. Its good to see how much of the heavy redness of the original Blu Ray can be reduced.

Thanks for posting up these screenshots!

That frame, and the one of Leia peeking behind the corner, really make the red clipping obvious. In the 3PO/lobster-man frame, look at the reflection on 3PO's left leg. Even in the improved version, it's still an oversaturated blob that doesn't mesh with the rest of the image at all. You can see several other oversaturated red points among the wall lights, the reflections on 3PO, etc. There's blooming and pixellation around the edges - it look nasty even when the rest of the image is improved.

I'm absolutely not faulting you, Y_T - the red information in the transfer is just that screwed up.

Hey snicker, have you done any more work on your project to fix the clipped reds?

TServo2049 said:

I'm absolutely not faulting you, Y_T - the red information in the transfer is just that screwed up.

No, of course I understand you're not! It sure is screwed up.

frank678 said:

Can someone help me with my knowledge on this - how much of the blu-ray redness is red damage/red shift now in the original damaged negative and how much is redness boosted in the color correction moves done by Lucasfilm.

It is all by Lucasfilm. They call it "George's vision".

Check this, a 70mm frame:

And the same from the blu-ray:

Now let's compare the part at the upper left:

In the original it can clearly be seen that there's no red clipping. You can see that the red light shines through some glass/plastic which has it's own reflection. In the blu-ray the red light shines stronger than the reflection on the outside of it.

EDIT:

Thought about another thing that shows it even more.

Harmy posted this re-timed 70mm frame in the color timing thread, which should look like the original technicolor version:

Here's the same from the blu-ray:

TServo2049 said:

That frame, and the one of Leia peeking behind the corner, really make the red clipping obvious. In the 3PO/lobster-man frame, look at the reflection on 3PO's left leg. Even in the improved version, it's still an oversaturated blob that doesn't mesh with the rest of the image at all. You can see several other oversaturated red points among the wall lights, the reflections on 3PO, etc. There's blooming and pixellation around the edges - it look nasty even when the rest of the image is improved.

I'm absolutely not faulting you, Y_T - the red information in the transfer is just that screwed up.

Hey snicker, have you done any more work on your project to fix the clipped reds?

I've almost achieved a workflow I'm happy with. Had a massive tax problem to sort out (which I now have) so I can go back to this. To answer your question, yes, I have fixed the problem with the red channel clipping. Single channel clipping is not a major problem because there are two good channels to pull detail from. I know it's way past when I said I would have something ready but I really (no, really) hope to have a thread dedicated to all these colour problems up soon. I've also discovered through hue mapping that the Blu-ray transfer reproduces only 1/2 to 2/3 of all possible hues at any time. This range is usually cyan/blue (hue 210) to red/yellow (hue 30) and right in the middle of that range is magenta (hue 300) contaminating roughly half of the total hue range. If you scrub through the transfer in a video editor it's very rare to find green clipping but the instances of red and blue (magenta) clipping and colour shifts are numerous.

I would like to see some Back to the Future Blu-ray frames with the Script 2. By the way, Script 2 is amazing!

Wow, both comparisons are shocking. That red light looks heinous in the modern transfer, especially when you see it up close - the grid pattern thing is much darker than the original, and the reflection that was basically white is now this ugly dark purple.

And I know snicker says there aren't many green clipping issues, but I'm noticing that the green reflection on R2's translucent circle also seems to be overpowering compared to the 70mm frame. Is that evidence of green clipping?

And that laser...awful. That shot is recomposited, notice how the difference in grain between the 70mm and the Blu-ray is much more visible than in the other comparison. A lot of the recomposited lasers in that sequence have the same problem - the glow element is way too saturated and fat, there's no difference in opacity/density between the beam and the surrounding glow, and the actual beam shape just looks like a stencil on top of the glow.

The recomposited lightsaber shots had a lot of the same issues. Whenever animation was recomposited, there seems to have been no attempt to duplicate the way the various elements were exposed and layered in the original composites, or even to get the colors right. Thus we have things like Luke's saber looking white or greenish, even though it was most definitely BLUE on the original theatrical prints.