Song Of The South - many projects, much info & discussion thread (Released) — Page 24

Aha! (rimshot)

In all seriousness, what do people think of the scans so far? They seem less crisp than what I think all of us are used to (because they aren't taken from the o-neg, but from a print), but for me, I think the saving grace will be the fact that they don't have the stupid blended fields and the inherent motion blur that comes with that.

ww12345 said:

In all seriousness, what do people think of the scans so far?

They look GREAT!  And you might want to add some numbers ... such as:

histogram     low   gamma  high
RED             8      1.1        160
GREEN         8      1.8        208
BLUE           6      1.5        204

;)

Woah! That's killer! Too cool, definitely!!

Great work!

You know, as I'm thinking about this, I may want to do 2 different releases - the first raw, as it comes, and the second cleaned up and color corrected. That way, other people can do their own corrections and changes as they see fit.

You know, as I'm thinking about this, I may want to do 2 different releases - the first raw, as it comes, and the second cleaned up and color corrected. That way, other people can do their own corrections and changes as they see fit.

 

Good idea.

And don't worry too much about any softness. Once it is cleaned of frame-to-frame debris, de-noised, and sharpened, it'll look on-par to any same-time-frame Disney film. Compare the final Fantasia (full resolution but cropped) against the raw Song Of The South (full resolution but cropped) and you'll see we're pretty close thus far:

Here's what you can expect with the true, "non-halo" sharpening of LimitedSharpen() [strength=200] in Avisynth (note that the noise/grain is being sharpened, too, since it hasn't been dealt with yet):

That looks nice! I'm convinced now that this can be a really, really nice release. I was thinking that this would just come out OK because of print limitations, but I see now that what we have is so much better than what is already out there. Look at the detail in his collar, for example. That detail is just not there in other transfers.

Great work, Spaced Ranger. Thanks for the help so far! :D

Thanks -- glad to help!

BTW, is this eBay auction (6 days to go now, from a U.K. seller) for 35mm cell-strips any help as reference?
Disney's -Song of the south - 35mm Unmounted Film Cells
http://www.ebay.com/itm/Disneys-Song-of-the-south-35mm-Unmounted-Film-Cells-/130906644359

You mentioned you had assorted references and I was wondering if those were of this caliber.

Sure, I have some references like that, but I'd be glad to get some more, provided they don't go out of my price range. I'll see if I can pick them up...

In continuing to color correct by eye (not the best way to do it, BTW), I saw that the contrast was too great in the first, limited adjustment. When this happens, the picture loses it's color strength.

The double-cone representation of color (there are a variety of notations but we'll use HSL -- Hue Saturation Lightness) shows that colors changing lightness toward brightness or darkness automatically lose strength or saturation ("chroma" in this graphic):

By compressing more of the picture into the mid-range (while maintaining higher and lower lightness), it can become richer, more saturated.

The first color correction fixed only the low end and high end and raised brightness by gamma [picture on the left]. This time, the addition of midtones compression, while re-adjusting the other factors to maintain the total picture, allows for more richness [picture on the right]. Rule-of-thumb: when the skin-tones start looking better, you're going in the right direction:

These are the re-adjusted settings and how the original spectrum is affected [the red overlay]. Note the new midtone compress settings:

When doing such a process, mask-off extraneous elements of the capture so only the picture proper is "active" for both analysis and adjustment. Once the settings are established, remove the mask and apply the correction to entire frame.

This is about as far as one can go using a paint program's no-control-points curve in HISTOGRAM. To closely match a target picture, one must define the curves with greater flexibility using points in CURVES.

So from that point, how do you make the scene brighter and crisper (or do you)? When looking closely, I can see the shadows and detail in the picture on the right, but to a quick glance, the left looks better, due to the contrasty nature, I presume.

When you talk about using Curves, I assume that is a control available in most video editing programs? I am using PFClean, but have no idea where it might be in that; I only know where it is in Photoshop...

The problem is in the paint program's generated curve. Whether this is an operational bug or a wrong type of curve, I'm stuck with how they've implemented it -- acceptable for any off-the-cuff proof of concept. HISTOGRAM is an easy way to work, with it's few but critical controls, that instantaneously gives picture/graph feedback. It's great for a quick work-up (and gives one the feel of what's being done).

Note that, in this case, the curve should be a steeper on the high end, to allow highlights without overly washing out lesser bright areas. Shifting the mid-tones slightly upwards will also brighten the scene without losing saturation. (I could probably get it closer with some more tweaking, but I'm assuming you're wanting less by-eye and more by-references.)

When working on the THX 1138 "missing scene", I was looking for an Avisynth plugin to do the HISTOGRAM adjustments on the clip. Instead I found a VirtualDub CURVES plugin that could easily approximate the graphs generated from a test screenshot:
http://originaltrilogy.com/forum/topic.cfm/THX-1138-preservations-Italian-Cut-available-see-1st-post/post/589761/#TopicPost589761
With such a plug-in, one can attain pinpoint control over the resultant graph by specifying actual points for the graph. Of course, you will use whatever works best in your workflow.

I'm thinking, as a first step, the capture should be confirmed not to have crushed blacks or blown-out whites in each of the RGB elements (the combined luminance may distort this determination) across the entire film. Run a program/plug-in to produce a full-movie histogram. From that, I would recommend a second-pass capture, adjusted to fill up the full spectrum, without new crushes or blow-outs, of course. Run the program/plug-in again to verify.

Also, I suggest that all work be processed on the individual RGB elements (such as noise reduction, particle removal, etc.) -- you can continue to combine them for review as the project progresses. This way, adjacent elements will receive only the processing they require for fix-up. I demonstrated this approach in a Star Wars thread:
http://originaltrilogy.com/forum/topic.cfm/Star-Wars-OT-1997-Special-Edition-Blu-rays-Info-by-Team-Blu/post/630232/#TopicPost630232

One thing I discovered on the THX 1138 work was, with a faded print, fade is not consistent across any length of film. This means the print should be luminance stabilized, probably best done shot by shot (to RGB elements, of course).

Do you have a list of things to do? The order of doing them may affect the quality of the outcome.

Very interesting. As far as "by-eye vs. by-ref" we'll have to make do with what we have - I'm going to be working on making the references available, which actually leads to my next question - if I used a grey card, possibly scanned in with the film/cels/other material, would it make the colors somewhat accurate, or would we just be comparing one scanned-in, incorrect color source with another?

As far as a list, no, we do not have a list (as we don't know where to start). What would you suggest.

Also, thanks for contributing, especially on the open forum like this. I hope that this project will be out in the open so that others can duplicate the methods we come up with. 

Scanned items should have a scale included right next to each one. Something like this would be perfect:

Any correction then would be performed to match the scan to the actual scale.

I hope your scans are like this one (Song Of The South production cel with Walt Disney signature):

BTW, if I take the present raw scan:

normalize the RGB spectrums (expand to 0-255):

and match only 2 face colors (mouth & cheek) to the above production cel:

we'd have a targeted color correction.  :)

Here's some of my collection of color references. These are just grabs I got from the internet; nothing special.

http://s1238.photobucket.com/user/ww123-2008/library/Camera%20Awesome%20Photos/iPod%20touch%20uploads/Song%20of%20the%20South%20Color%20References

However, would you mind recoloring that frame to match this one: http://i1238.photobucket.com/albums/ff493/ww123-2008/Camera%20Awesome%20Photos/iPod%20touch%20uploads/Song%20of%20the%20South%20Color%20References/null_zpsd71c3fed.jpg

The only reason I ask is because that frame still seems really dark (although that could be because we haven't brightened it yet), but I wonder if they used different color timing for each of the sequences (ie: a brighter approach for the first segment, a darker for the third segment, and a more natural for the second segment).

A nice web collection and the best kind - cels! Add my find (above) to it. If you keep track of the sources, too, let me know and I'll find the link (again) for that one. If I come across others, I'll submit them to you. (eBay should be a good source of material, too.)

I added (the above) "normalization" adjustment sample -- a fully spread out luminance on each RGB element -- to simulate that scanning the film this way might give more quality when color correcting. I don't know if the scan machine allows for separate range settings on each RGB-element flash. If not, this would require 3 separate passes with each pass-setting weighted for the selected RGB element of that pass. Ultimately, these would be separated out and recombined with their complimentary RGB elements for the finished scan. It also would require an initial pass for making a full-film min-max-luminance histogram of the RGB elements to adjust for those RGB-element-settings. Naturally, it's more work for the scan process.

The reason for doing this is that when the spectrum is digitally expanded during computer color correction, it leaves gaps or steps in the resulting spectrum. That's a built-in digital anomaly. Film, on the other hand, is analog and capturing from it is always continuous. If the luminance range is max'ed in transfer, it effectively pre-expands the spectrum in continuous analog -- no gapping. There could be no expansion during digital color correction (because it's already max'ed) and compression (the only remaining operation during digital color correction) doesn't leave gaps. Hence, a better final product.

NOTE: Throughout, these are my best-approach suggestions; obviously, use those that you like, if any.  ;)

BTW, I should've noted that the above "targeted color correction" was a reference-approach example of how matching even only 2 colors positively affects the entire picture (although not enough for an accurate color correction).  Sorry for the scare.

Of course, your suggested cel (here cropped to remove distractions) is the superior choice for a film-scan-to-reference-colors test:

This time, I used CURVES, which requires getting RGB values for same  areas of source and target:

                   scan R:G:B          cel R:G:B
cheek         128 119 122  ->  140 102  79
mouth        152 159 157  ->  191 155 124
nose           122   60   67  -> 169  64  45
fur                85   43   47  ->  104  68  47
rope           116   57   49  ->  169  93  38
shirt           145 135 140  ->  179 122 112
teeth/eyes 156 168 161  ->  186 171 151

The numbers are manually entered into the curve graph with this result:

It looks nicely matched, but closer inspection shows broken-up and mis-colored areas. This is because of spikes in the spectrum to attain particular colors that are near different colors. Any slight variation in the color causes it to (wrongly) pick up the too-close-neighbor coloring. This means that point-to-point CURVES is too precarious to do the job in this manner. However, CURVES remains good to show the curves that could then be smoothly approximated in CURVES with curve-controls (the VirtualDub plug-in, for example) or in HISTOGRAM (this paint program):

The result is a very good approximation (some inevitable close-colors) but with a smooth color spectrum (no break-ups or wrong color pick-ups):

Wow, that is fantastic! That looks really, really good and really, really accurate!

OK, I've got another few questions now. Say we use 5-10 references per reel. Is that enough? Can they be merged or averaged? If we use those references, do we still go shot-by-shot or could we do a whole reel with those settings? Now my curiosity is piqued!

In the THX 1138 thread, I eye-balled the best middle-ground setting for a "missing scene" (with noticeable fade fluctuation). However, that single setting, used for the entire capture, produced a good result:
http://originaltrilogy.com/forum/topic.cfm/THX-1138-preservations-Italian-Cut-available-see-1st-post/post/589864/#TopicPost589864
and that (color correction) was good enough for the project. So a single setting could get things going.

Regarding reference material, care should be taken to use only that which can survive from the actual work. Packaged production cels are probably it, and the cels alone (not the backgrounds, which are later added for the sale). If dye based original prints are available, that would be a 2nd source. Beyond that, I don't know if anything is trustworthy. Many parts throughout the movie use similar locales -- light areas, dark areas, indoors, outdoors, toondoors. A variety of different-scenes references is key. Whatever is not available, a good guess then could be made from previous settings.

My suggested list of things to do (and probably to be done in the order shown) after reviewing the previously posted sample clips:

? capture
? de-spot (random debris)
? de-scratch (continuous debris)
? de-noise (grain)
? shot luminance stabilize
? frame motion stabilize
? sharpen (non-haloing)
? color correction
? mask/resize/frame (HD, DVD)

(Of course, working on RGB elements is my assumed best-way-to-work.) I don't necessarily know how well this can be done with what's out there. Depending what is used, these steps may be shuffled around. Avisynth has allot of filters and I've seen some that claim to do all of these. Here is their definitive list:
Avisynth Wiki - Internal filters
http://avisynth.org/mediawiki/Internal_filters
Avisynth Wiki - External filters
http://avisynth.org/mediawiki/External_filters
If you're using other software, see if it covers these steps.

For example, from the short clips, I tried a de-spotting filter called Spot Remover. I tested it with it's author's recommended settings but it's cryptic and I have no clue what's going on. Anyway, when it works, it works well:

Problem is, it doesn't work this well all the time (it's a toss of a coin). Maybe the every-fourth-frame-duplication is tricking it.

Also, I forgot to mention more about the VirtualDub plug-in, which was used for the THX 1138 color correction.  Gradation Curves (v.1.46b) by BBugBunny, A.K.A. Alexander Nagiller, is easy to use, has interactive adjustment just like in a paint program, but is more flexible. It's good:

:)

I came across someone's Song Of The South DVD project, coincidentally entitled ...
Song of the South DVD Project - Converting and preserving classic animation
http://atlas.kennesaw.edu/~dhirschl/song_of_the_south/

Lots of pictures and lots of detail (now that's my kind of webpage). Of particular note is the British TV broadcast entry:

This might be a useful reference source ... including his finished (?) project.

If that's the Hirsch web page, it's a lost cause. I was in contact with him for a few years, and after donating time and some resources to him, I got nothing. I don't like to talk badly about people, but he is very, very flaky.

The webpage link is now in my previous post -- it is the Hirschler page.

Too bad ... a nice sampling of video sources. That screen-cap looks good though.