Spaced Ranger

darth_ender said:

Isn't that just Jambe Davdar's three Star Wars documentaries put together?

YouTube - Star Wars Documentary:

http://www.youtube.com/watch?v=jeiL3Eru1A0

Uploader Comments (AstonisherVision)

    Jamie Benning 3 months ago
    Hi,? I am the creator of these documentaries.

    AstonisherVision 3 months ago
    What happened to your channel? I got this sometime ago from you and decided to upload it? so others could watch it. Why did you take the series down?

FanEdit.org - Jambe Davdar:

http://fanedit.org/ifdb/component/jreviews/tag/faneditorname/jambe-davdar/

Brief Synopsis:

    After the success of Star Wars Begins, filmmaker Jamie Benning (aka Jambe Davdar) embarked upon a new adventure.

Yep, that's what it looks like. So even with his incredible amount of "missing footage", there's still more? Come'on George, spill it all!  :)

Well, in that case, you definitely want to check out:

Star Wars Documentary
http://www.youtube.com/watch?v=jeiL3Eru1A0
"Over seven hours of behind the scenes action and information, from one of our greatest science fiction sagas of all time."


... re-posted from the author who took it down (or had it taken down) and dropped by for a nod. Get it while it's hot!

From The Hobbit blog about 3D with Peter Jackson:

http://www.youtube.com/watch?v=gHF536TJ0iE
mentions of coloring begin @ 5:41

Peter Jackson: "If you look at the ungraded footage, the trees look incredibly psychedelic. They look like they were painted in 1967. In the movie, they won't look anything like that. They'll be graded down and you'll just get the barest hint of color in the finished film."

Prosthetics Superior: "we have to change our whole way about coloring these things, 'cause we found out in early tests that if there wasn't enough red in the pieces they would punch-up yellow and react differently than normal skin with blood running through it"

Frankly, all this sounds like they are "laboring under a misconception", unless their state-of-the-art cameras are "color challenged". Even so, there still is no excuse for green-tint clouds or, worse green-tint titles ... here, or in any of TLOTR movies:

http://www.cheapassgamer.com/topic/295388-lord-of-the-rings-extended-edition-trilogy-6585-fs/#entry9847547

Wow! From green ...

... to clean!

What is wrong with Jackson & crew -- shifted green spectrum on the brain? With their high tech, bleeding edge, massive budget company, they can't get right something so simple?

How's the THX 1138 preservation coming along?

While over at the Song Of The South preservation thread, something came up that I thought you might like to try here. They are using DaVinci Resolve lite (a freeware version of the commercial program, with limited functionality) on their faded 16mm.
http://originaltrilogy.com/forum/topic.cfm/Song-Of-The-South/post/643982/#TopicPost643982
One function in their processing stream is for luminance stabilization. We hadn't tried this but it might be something to consider. (There might be an Avisynth filter/script for this to use if DaVinci Resolve isn't convenient.) It would make a single color correction work better over the length of the movie.

Thanks, everyone! And of course thanks to you-all for making the project happen (and happen so well) in the first place.  ;)

Also, please see the Addendum just added to my previous post.

Here's something interesting. The look of the latest complexity of Curves was reminiscent of a simple Histogram. So I tried it (because I don't know when to stop):

                           Curves                                                         Histogram

HISTOGRAM
RED       Gamma 1.0  Midtones 0    = no adjustment
GREEN   Gamma 0.9  Midtones 15
BLUE     Gamma 0.8  Midtones 20

:)

edit:
ADDENDUM

I wanted to add this note to explain why the above part of this post was important (at least, to me) for the "de-fading" of film fade, which is not necessarily that same as "movie restoration". When motion picture film fades, it essentially loses contrast -- the picture as a whole becomes lighter as the film grain, which holds the darkness of colors, is bleached away. This happens in different degrees to the different color layers due to varied exposure to the environment that produces characteristic changes ("turning red").

For example, this faded 35mm film frame [center] from the 2001: A Space Odyssey trailer, when compared to the original picture [top] (a later "restored / remastered" DVD), demonstrates fade -- loss of contrast and spectrum shift at the color layer level:

Once these characteristics are identified, it is a procedurally simple matter to "re-contrast" the remaining color in the layers (there may be some unrecoverable loss of completely faded away highlights, but not as much as may appear at first examination) and "re-shift" their positions into a mostly corrected picture [bottom]. Histogram's functions (Low & High for contrast, Gamma for shifting, and Midtones for weighting the shift) are almost perfect to quickly and easily do this.

As gamma is only a close correction (it wasn't designed for film fading), the greater flexibility (and greater complexity) of Curves would be used for fine-tuning with "problem" corrections.

As is the case with the Song Of The South correction above, using Curves (and other such tools) is tricky. When one is too close to the color-trees, one loses sight of the picture-forest. Such micro-management results in squiggly-line adjustment curves, which is not how film fades nor is it the way to correct such fading. Applying manual smoothing reduces this +/- "ringing" of the adjustment. After my smoothing, I noticed the characteristic curves I see when using Histogram only. Going back to Histogram, I quickly adjusted it's functions to produce similar-looking curves and ... lo ... the mostly-same Curves-corrected picture appeared!

This is important as it indicates that the faded film (with normal fade variations, per reel) can be simply and globally corrected, with tweaks added thereafter where needed.

* As a side note, consider how luminance stabilization should be applied. Per R-G-B separation? To maximum values? Averages? To minimum values? Remember, it's not the luminance that is varying, but the degree of fade (the varying of contrast). As a practical matter, it may not make much difference, but tests should be conducted on a worst-case shot for that determination.

After more self-critical review, I tried more tests, R-G-B separation comparisons, & tweaks. Mainly, RED was little hot on the live-weighted correction, Curves needed more manual smoothing (resulting in a straightened RED curve, which was the average of it's squiggling back & forth), and better balance on the highlights & shadows. And fox's shirt got (a little) it's groove mint-green back!  :)

                 universal live-weighted                           universal (cartoon & live)

CURVES: universal both
RGB       (0,0) (255,255) = no adjustment
RED       (0,0) (255,255) = no adjustment
GREEN   (0,0),(64,60),(116,88),(167,131),(217,197),(255,255)
BLUE     (0,0),(64,60),(113,79),(183,122),(211,175),(238,211),(255,255)

I thought I'd try further tweaking, working off the last correction, working towards the 35mm coloring while mostly disregarding the cartoon color. Primary was to further reduce the green cast, which consequently reduces the mint green shirt color. Also, I smoothed the curves, reducing hard turns that creates hard-boundaried hotspots of color (like the prominent tears on girl's face):

            universal cartoon-weighted                               universal live-weighted

CURVES: universal live-weighted
RGB       (0,0) (255,255) = no adjustment
RED       (0,0) (24,10) (59,58) (99,104) (155,161) (203,182) (231,209) (255,255)
GREEN   (0,0) (16,6) (64,54) (118,91) (162,126) (195,165) (220,197) (255,255)
BLUE      (0,0) (16,20) (48,48) (106,75) (176,118) (216,189) (238,237) (255,255)

Speaking of procedural record-keeping, I've duplicated my above color correction steps into a single RGB Curves adjustment -- using the previously demonstrated one-to-one color-compare between the starting capture and the previous correction to get me started. While at it, I tried reducing some of the green cast (but preserving the mint green in the shirt-spectrum-area) while maintaining the stronger colors and higher contrast of the 35mm.

BTW, starting with the cel instead of the 35mm probably didn't make much difference. But it was the cel's initial correction that began with better looking skin-tones (such as it was).

Anyway, let me know of problems using these numbers and, or course, how it looks across the entire reel/film (posting some before-after picture pair examples would be cool). If you can take it from here ... great! If you need closer tweaking, work up something in both cartoon and live-action (per reel?) to demonstrate where it might need to go. The same settings, with proportional adjustments, probably could compensate for different degrees of fading in different reels.

CURVES -- points (in,out)
RGB       (0,0) (255,255) = no adjustment
RED       (0,0) (88,76) (110,97) (144,166) (202,197) (231,236) (255,255)
GREEN   (0,0) (57,60) (115,93) (172,143) (201,188) (220,228) (255,255)
BLUE      (0,0) (64,58) (106,76) (181,120) (213,169) (238,237) (255,255)

If your Master::Curves for each R-G-B channel has minimum=0 and maximum=1.0, divide each of the above numbers by 255, for your use.

@ AntcuFaalb

After watching poita's link for the DaVinci color adjustments demo, for it's professional nomenclature, I see that the "Y" in YRGB is merely luminance ... the linked +/- movement of individual R-G-B separations as a whole or on spectrum areas. That function is in the paint program I'm using and in the Gradation Curves plug-in for VirtualDub (both simply identified as "RGB"). Nothing proprietary about it.

In fact, there is no such thing as YRGB per se. It all starts as RGB --> is manipulated by luminance and/or saturation and/or tint and/or the individual R-G-Bs and/or any other function --> ends as RGB (now modified). So, there is no translation for it, nor is one needed.

@ poita

Thanks for the ACES links -- I approve of generalized future-proofing.  :)

Thanks for the info, guys! I hadn't heard of "YRGB" or "ACES". Then again, I'm using a paint program. Still it's something to look into and ... wait. Did someone say ... proprietary?

Back! Back, color abomination! Back to copyright hell from where you came!
:O

@ poita & ww12345 - Thanks! Shortcomings aside, I was surprised, but pleased, this had the most color-fullness depth thus far.

The correction at this stage is something of a compromise -- fox's mint-green shirt color area overlaps that of girl's white lace. The more the lace goes to neutral, the more the shirt moves away from green. Since the mint-green is a spike of color, it might be tolerable to keep a lesser strength of it, in it's narrow area, as long as surrounding areas are more on-color.

I'm trying to duplicate in Curves (for the R-G-B separations) what the paint program tools added. BTW, is that necessary for me to do, or can you use your similar-function tools on it?

In the meantime, I'll work up & post my previous step-by-step numbers in case those become helpful recreating it in your software.

P.S. I mis-numbered the pictures in my previous post, which are now rearranged to be in the correct sequence with the corresponding text.  :)

Thanks for all the info, guys! It all rang true to good bits I've picked up here & there on Star Trek 1960's film history, it's restorations for HD, and all that.
It got me to thinking, which led to this -- the possibility of a single, sweet-spot color correction across both live action & cartoon! How?

Well, I cheated on my R-G-B separations technique by using some of those other functions in the paint program. I'm sure this could've worked solely in the separations but there is no un-do when testing the input numbers (and I change them back & forth to see how they do). But I could un-do specific applications of those tools, to try again with other settings or with something else entirely.

It started when confirming that contorting the graph to get fox's shirt to show as "mint green" would not work well with the live action. Confirmed:

  35mm targets                                                  for 16mm areas                                             results

Setting specific color areas of the 35mm to the 16mm produced such spikes in the R-G-B graphs that it could not be useful elsewhere (and even affected inappropriate, other areas in the same picture).

From the previous discussion, it was clear that even the 35mm source was inaccurate of the actual production and cartoon cels. Cels! poita had just posted a really good cel with, dare I say it, real good color. So, this time, the 16mm color was set using the cel, on fox again. The graphs were somewhat rolling but not as spiked as from the 35mm. It turned out very dark but, oddly enough, the live action shots didn't look quite so bad. Manually smoothing out the graph lines improved not only the cartoon, but also the live action -- but still dark. To compensate, the control points were pushed upward to produce a lighter picture. It began to look amazingly better. A little more moving points gently upward produced this picture strip:

[EDIT: corrected the sequencing of the mis-numbered pictures]

               16mm LUT capture                                  with cel color - manually smoothed
 

Woah ... not only "mint green" cartoon shirt, but good looking white and black actors (a hard photographic task) ... all with the same setting! At this point, I stop manual manipulation and continued with the other paint program tools.

As this produced a strong orange tinge, Greyworld color balance color temperature was set to 6500K ("sunlight") to move the color cast cooler (bluer). Keeping in mind the stronger color of the 35mm, saturation was increased +25. Done:

       & greyworld color balance @ 6500K                        & saturation +25
 

ww12345 said:

I'd be interested to see if you could make some sense out of my color correction software ...

What I do in the paint program looks like it falls under the main category
MASTER (PFC picture 1)

My settings (shown here from an older posting) are
Low .. Gamma .. High .. Midtones

Your MASTER :: HISTOGRAM (PFC picture 4) has my Histogram settings of Low and High, for each channel R-G-B. Normally, I would set these to stretch out the ends of the shrunken spectrum due to fading. In this case, each R-G-B channel already has been expanded in the capture. So Low will remain at it's default of 0 and High at it's default. In the paint program, the High maximum is hard-wired to 255. In PFClean, it is set to the relative value of 1.0 to allow for more than 8-bit range (0 to 255). When needed to convert my numbers to yours, divide mine by the maximum of 255. It's that simple.

Your MASTER :: COLOUR (PFC picture 2) has my Histogram setting of Gamma for each channel R-G-B. This adjusts the low end in proportion to the rest of the spectrum, with the higher end least unaffected. This mostly adjusts the shadows for more visibility while changing R-G-B color strength that moves into the mid area. From the tick-marks, it looks like my numbers, default = 1.0, minimum = 0, and maximum = 5, may correspond directly to yours.

Unfortunately, I don't see anything here that looks like Histogram's Midtones, unless there is a screen not posted here that produces it's characteristic "S" curves. Histogram has default = 0, minimum = -50, and maximum = 50. If need be, I can just post my Histogram graph with Midtones only for you to recreate the shape in your MASTER :: CURVES for each channel R-G-B.

That's all I use ... at this point. From the PFClean demo video, you could try the auto-color-correction(s) to determine how useful it is. As much as you can, go for general manual settings and/or automatic settings that do the acceptable job (because this is a BIG job).

ww12345 said:

Here's the same frame grab from the BBC broadcast, so I guess that rules out the different coloring, ...

Also, here are my color correcting tools ...

Green shirts. Why did it have to be green shirts!  :D

Thanks for the software screenshots. Pretty cool!
I'm not "a colorist"; just some guy analyzing film fade and trying to un-do it based on the way it's faded (which might then allow techniques for automatic or semi-automatic correction). I recognize some of your program's settings; others are new to me, too. At least, after some review, I could see if the numbers I had supplied might present a problem for your use.

poita said:

A quick go at the fox scene in davinci, correcting secondaries (and I do mean quick) gets me to here ... Which looks close to the references I have.

Thanks for the green shirt  ...  :O

Those "secondaries" you mentioned are YCM (Yellow, Cyan, Magenta -- the inverse of RGB)? I cringe at approaching the film fade problem with anything besides RGB (Red, Green, Blue), which is why I try to stay away from HSL (Hue, Saturation, Lightness) or variant HSV (Hue, Saturation, Value), or the other spectrum segmentation/augmentation tools (I fear such corrections will produce the Hollywood cringe-worthy, visually broken Blu-ray releases we've seen of late ... right GL?). Just a personal research approach.

Can you list numbers for settings, which might be available in a paint program, for your nice green shirt correction? Alternately, could you reproduce it on the LUT-capture frame-strip (above) to see the effect of that cartoon correction on the live action sections?

BTW, awesome looking cel! I suggest the next archive and/or distribution medium incorporate vector frame-reproduction video!!

Although I can play/watch the .MP4 files, VirtualDub doesn't know the format and won't load them. I'll search if there's a driver to fix that. But, in the meantime, I'm working on stills (the counterparts of the ones Zip Doodah posted) in the paint program.

Fox's green shirt has been bothering me. I just can't get green to show stronger without noticeably throwing off the rest of the picture. Also, I was suspicious of the 35mm's shaded shirt color compared to the 16mm's properly flat, painted cel color:

                        16mm                                                        35mm

When the pictures were split to their R-G-B elements, something jumped out from the RED separation:

                     16mm RED                                                 35mm RED

Do you see it?
Although the values look comparable overall, the green shirt compared to the head is value reversed in the 35mm clip! That is impossible, and that is why the green doesn't come out from the 16mm clip. One of them has been altered and my bet is on the 35mm with it's non-cel-like shading. Now that we can safely ignore the infamous green shirt, on to the color correction.  :)

Using the 35mm as a guide (even though I still think it looks over-saturated -- see the skin colors), the pictures were arranged into one strip to see how adjustments for one picture affected the rest:

                  35mm reference                                                 16mm LUT capture
 

Keeping in mind poita's admonition to watch the range of the picture, I compressed the midtones across R-G-B to show more of the detail in girl's highlighted face. Then using gamma, the R-G-B spectrums were moved to assure the white doily on the dress remained mostly neutral white. I use a stand-alone "eye-dropper" program to check the colors. (BTW, if all R-G-B gammas are adjusted upward, the picture becomes lighter; downward, darker; and one down, one at 1.0, and one higher, remains overall the same with color variations.) The result, trying to make them all look good with one setting, is this:

1st correction
             low  gamma   high  midtones
RED         0      1.1      255           10
GREEN     0      1.0      255           15
BLUE        0      1.75    255           10

               16mm LUT capture                                                color correction 1

Put those numbers into your program and check how other scenes look with this correction. Your experimenting should be easier with the LUT capture's pre-normalized RGBs -- it's only gamma & midtones twiddling now. Of course, the settings are inter-dependent -- changing one usually requires compensation on the other, within each RGB separation.

Zip Doodah said:

 

From the end of reel 4 ... From the beginning of reel 5- Technicolor variance is displayed pretty well between these two scenes!

Wow! Thanks for posting those snapshots.

Previously, I thought film fade was pretty uniform over time and the medium. Now that I've discovered it's all over the place, it blows up my thought of a fade fingerprint(tm) based restoration. Oh well ... back to the drawing board.  :)

ww12345 said:

I figured I'd share this with others, too ... Totally SD, but still worth it to play with... Paging Spaced Ranger...  :)

Did I mention I hate green shirts? Al least, I hate fox's green shirt (see Zip Doodah's fox & rabbit picture). I can also see complications trying to match similar shots across reels (again, Zip Doodah's pictures).

I'll do what I can with the present method, test it's viability on samplings from this file, and post some numbers to test. Don't be afraid to play around with them in your way-better software (if you can directly input low/gamma/high/midtones numbers). If you can work better reconstructing my resultant R-G-B graphs, I can post those, too.

In the meantime, try using my previous numbers just to see what they do. I'll probably make those my starting point anyway.

Thinking out loud:
My present approach using a paint program's standard spectrum manipulation has some not-easily-resolved deficiencies. But I know, to paraphrase Yoda, that "you can un-fade what has been faded"! (It's okay to mix similes here, this being the Original Trilogy forum.) I have some ideas to develop that should also yield post-able numbers but I'm not sure if the standard manipulations will work to easily produce them.

poita said:

The cartoon shot on the right actually looks more washed out to me though?

Oh, I didn't mean the contrast of the cartoon ... rather the color strength improvement over the previous page's attempt (like in trying to see more of that green in fox's shirt). But right now I have only a little idea what everything should look like.  :)

Thanks for all the information! Will definitely look into it.

New programs will be problematic as I'm using older equipment with older OS. For now, I'll just spew out the theories and approaches (which I test and refine in these instances) ... and the relevant numbers as a starting points for the project(s). And, yes, they should definitely use the high bit processing, which I think they can with that expensive software they use (ouch, I think I hurt my wallet again).

Now that I'm not in a rush, I went back to correct the slight over-brightness, which didn't look so bad in the live action, but did show it's deficiency in the cartoon.

When working the live action shot, I used mostly the girl's hair and face for the color correction. This time I reduced the bright areas of her face (the 35mm target was a different shot) for a now subtly better correction (the background opened up a bit, too):

first correction
             low  gamma   high  midtone
RED         0        1.5    255           -5
GREEN     0        1.4    255           -5
BLUE        0        1.2    255           -5

correction tweaked
             low  gamma   high  midtone
RED         0        1.4    255            0
GREEN     0        1.3    255            0
BLUE        0        1.1    255            0

                              LUT capture                                           35mm-guided color correction tweak

And, when blindly applied to the cartoon, it now looks less washed out:

This demonstrates that maybe a single correction can be used as a general correction across a single reel and, perhaps, multiple reels.

poita said:

I got the densitometer out to ascertain the best gamma and light balance settings for the scanner, so at least some colour has been recovered.

I'm finding that your custom LUTs have make this even easier to correct for the closest, original colors. In the test I'm now trying (your last reel3 LUT capture), my tweak has been reduce to strictly gamma (in the paint program) -- R=1.4 G=1.3 B=1.1 -- for an improved by-eye correction. That reflects just how close your capture has come to being it's own color correction. Well done!

Well, I'm glad the consensus is that the cel is not a production cel.  :D

ww12345 said:

I don't think that work you do on the coach scene will work because of the fade differences ...

Yeah, I was thinking that, too. Fortunately, there is the 35mm color-target of the girl (cropped to remove distractions):

Here's something quick to check the one setting approach for a master color correction. On the very beginning shot of the cartoon reel is shot of the girl. So, my color correction:

             low  gamma   high  midtone
RED         0        1.5    255           -5
GREEN     0        1.4    255           -5
BLUE        0        1.2    255           -5

                              LUT capture                                                  35mm-guided color correction

Of course, this may not be accurate for this shot, but it's close enough by eye.  This setting is then applied blind to the cartoon:

Not bad. It could be a little hot in the brighter areas (probably same with the girl for that matter). I'd really like to use 35mm sample shots to know for sure.  ;)

A Disney cell from ww12345's gallery http://originaltrilogy.com/forum/topic.cfm/Song-Of-The-South/post/639161/#TopicPost639161 was used as a same-scene color target:

To produce the color correction, I tried for the Olympic gold. But to assume that a cel is an accurate depiction of what was originally on film is a bad assumption. Animation was often made with over-reaching or just plain weird colors to counter particular film stock limitations to produce the desired colors. Unlike my first attempt some pages back, this attempt is on poita's new LUT-enhanced capture (a better & brighter starting point from the previous capture):

                              LUT capture                                                  cel-guided color correction

I did what I could to pull out the green of the fox's shirt, but that seemed to be to big a "spike" in the color curve that affected parts of his apron. Also, adding what green I could affected the blue of rabbit's trousers. All the while, I kept close watch on the rope and other fox & rabbit color. It ended up not much different from that previous adjustment's correction, but still short of the target (even if that target might be an unrealistic one). It would be great if Zip Doodah (hope you're follow the thread) could post a screenshot of this scene to see how it looks on film (with most of the original color intact).

Anyway, here are the numbers I used. Despite my desire for adjustment in R-G-B sections exclusively (in Histogram), I was forced to make additional tweaks using joined R-G-B adjustments, in Histogram and Hue/Saturation/Lightness, to increase the contrast and color saturation, respectively:

             low  gamma   high  midtone
RED        12        1.2   255        +25
GREEN     7         1.0   255        +20
BLUE        0         0.8   255        +20

                       low  gamma   high  midtone
LUMINANCE      0        1.0    255        -10

SATURATION   +10

I'll try this again on the live action part (the coach scene) to see if a good correction made there also produces a good result when applied blind to the cartoon part.

RE:"Avisynth script to sample a single frame inside each shot change"
AntcuFaalb said:

All you need to do is use the SCSelect() function.

Thanks for the reference. SCSelect() is not native to Avisynth but it is in the plug-in file removedirt.dll as per this Doom9 forum thread:

Real Life TV Scene Change Detection?
http://forum.doom9.org/showthread.php?p=1193539#post1193539

I was looking for something else I remember coming across, a native function -- ScriptClip() or another of Avisynth's Runtime Functions -- with a small script to do the sampling. (No need to load in additional plug-ins, for the novice user.) But whatever works.

ww12345 said:

Thanks for the input. ... because I'm learning as I go.

Glad to help. I'm learning, too (my "practical research").  :)

Macro-blocks are these artifacts (error-generated square boundaries within a picture):

Once you know what to look for, they practically jump out at you. They are generated when a video is rendered at too high compression (or if areas of the picture are excessively smooth). The render software works on squares of areas and flattens those of too-similar pixels (usually from noise reduction) into uniform-color blocks and/or edges.

One way to prevent this is to reduce the compression of the rendered video, which unfortunately increases it's file size. Another way is to trick the render software, to not flattening smooth areas, by adding mild pixel noise back into the video (!). This is interpreted as detail and is not inadvertently flattened. Clever, eh?

In the case of SOTS, use lower compression settings in your renders until macro-blocking no longer appears (also, look for sub-settings specifically for this issue that might be in your render software).

As for the sequence of fixes, I try to arrange them so that a first fix won't hinder or complicate the second fix. You must learn what the fixes do and how they do it to suspect if the order might be improved.  If not sure, experiment -- rearrange the sequence and see if the result is better.

For example, frame-movement stabilization uses tracking to see how "stationary" target pixels move around. Frame luminance fluctuations might mis-lead that tracking and cause positioning errors. On the other hand, luminance smoothing works on the full frame regardless of the internal picture position. Therefore, I would stabilize the frames luminance, first, to allow the best stabilization of their movement.