Color matching and prediction: color correction tool v1.3 released! — Page 9

FrankT said:

Which would you say has the better colour?

 I think I generally prefer the WOWOW, because the bluray is very orange, but in this scene the bluray is looking pretty good.

The second situation, is when we are matching a print, in which case color inconsistencies within an aging the print may play a role. However, even for the Star Wars bluray, which probably has the most inconsistent color timing ever, accurate predictions can be done within shots.

First the model is calibrated on this frame:

Bluray:

Bluray matched directly to Tech IB:

Then I predicted the folowing frame:

Bluray matched directly to Tech IB:

Predicted correction with color correction model from the above reference frame:

The prediction is pretty much identical.

DrDre said:

FrankT said:

Which would you say has the better colour?

 I think I generally prefer the WOWOW, because the bluray is very orange, but in this scene the blury is looking pretty good.

 I think this looks fantastic. Your results are truly outstanding. I don't think the future of film preservations has ever looked so good! 

revgen said:

I decided to test it out last night on the Criterion & Arrow versions of Dress To Kill. Caps were grabbed from DVDBeaver.

Here's the Criterion Blu-Ray.

Here's the Arrow Blu-Ray.

Here's the Criterion Blu-Ray Color Matched to the Arrow Blu-Ray.

Other than a slightly pinker shade to Michael Caine's skin tone, the corrected image looks very accurate to the Arrow Blu-Ray.

 Take care to crop the frames in the same way, because it influences the accuracy of the color matching. I corrected the same frame, and the colors are an exact match:

RU.08 said:

DrDre said:

Hope you enjoy the tool. Of course if you use the tool for your projects, any acknowledgements will be appreciated. The same is true for any comments, critisism or suggestions you may have. In that case write a post in this thread or send me a PM. 

Can you use command-line processing? Example:

for %f in (*.tif) do regrade "%f" -ccm "correctionmodel.txt"

 Sadly not, MATLAB is a scripting language, so within MATLAB you can use command lines, but since this is a standalone application, you can not do this anymore.

Let's get back to the discussion of regrading a source to approach some kind of reference, with only limited resources aka reference frames. Take this frame from Raiders of the Lost Ark for example.

Bluray:

35 mm frame:

We can match the bluray to the 35 mm easily:

However, if I were to ask you to regrade these two frames from the same location to look like this 35 mm frame:

Would you be able to do it? How much time would it cost?

This is what you get from the prediction, in about 20 seconds:

First of all, I think they look stunning, but more importantly, they look like frames from a 35 mm print, consistent with the reference. 

NeverarGreat said:

Okay, I've done my first color test with the software and the results are quite accurate. FYI though, I had to reverse the order of steps 1 and 2 to get it to work. The biggest problem is that the Blu-ray has such compressed gradients that the color is flattened, and no algorithm can fix that.

Correct me if I'm wrong, but as of now this program will only accept separate images, so it's rather awkward to use this for an entire shot, much less a whole scene. So for me it's back to correcting entirely by hand, at least for now ;)

Theorizing here for a minute - Say you match the color of the Star Wars Blu-ray to a source like a 35mm scan or the GOUT, and then the program compares the final result for each frame to that reference source. Any colors that are matched 100% remain, but any colors that don't match, say, because of missing gradients, are then added to the Blu-ray through some sort of color blend mode (which I've done to the Blu-ray using GOUT color in certain places). This would require first registering each frame. Thoughts?

JEDIT - Another idea, instead of cropping the reference and test images, perhaps the program could employ a system of target points that the user can place on the frame and essentially tell the program to correct to those, rather than the entire frame. So details that are important, like C-3PO's gold color, or the skintones, will be weighted more heavily in the correction than other parts.

Regardless, and echoing what others have said, it's an exciting proof of concept and I look forward to seeing where you go with it!

 Actually, once you've build a model, you can select any number of frames you like for correction. Depending on the number of shots, you could accurately process 50-100 frames with one model. As I've shown it is very accurate within shots. 

The model doesn't directly compare pixel colors, it compares color distributions in many different color spaces. It then matches these distributions, such that the colors match. That's why you can match colors between two sources with a different resolution. It does require that the two frames represent the same ground truth, so to speak, therefore they should be cropped in the same way.

Also the information about the gold color is embedded in the other colors. Take the C3PO frame. I built a model using only a tiny bit of the frame, which includes R2D2:

I select the same part in the reference, create a model, and predict what the rest of the frame should look like:

It is an accurate representation of the corrected frame, predicted only by the color changes on R2D2, and a tiny bit of the surroundings. 

Swazzy said:

What would be the easiest way to implement this algorithm to an enitre film/shot, say through avisynth?

 Use Virtualdub to export the frames (or a selection of frames) to image files. Correct them using the GUI. Import the frames in Avisynth again (the names of the corrected files are the same, except in another directory, so this is easy). Add sound, and you're done. 

schorman13 said:

Dr. Dre, I wanted to say thanks for all this work.  This really is an amazing and useful tool you've shared.  

As far as whole films go, I would think the tool would have to be automated somehow.  It's one thing for it to take 20 seconds per frame, it's another to have to manually load and process the ~200,000 frames in a 2 hour film.  

Ideally you could just import 2 uncompressed avi files and output a third, rather than dealing with giant lists of png or bmp files.

I'm sure this would require much more development.  Frankly, I'm just happy with what this version can do.  

If there's a way to output a color correction matrix file for each frame, some other avisynth plugin (ColourLike?) might be able to do the color correction separately.

 Sadly, that is not possible, as far as I know. You still would have to know which references you would like to use, which frames to correct based on these references, etc. Color correction still requires a lot of work, and a human eye to check for anomalies, even with the help of software tools like this. 

It would be nice with the kind of shot change detection that exists in software like pfclean. Have the whole reel/film laid out into shots and choose the representative source shot/frame and apply it to as many target shots as you like until your whole sequence is covered.

towne32 said:

It would be nice with the kind of shot change detection that exists in software like pfclean. Have the whole reel/film laid out into shots and choose the representative source shot/frame and apply it to as many target shots as you like until your whole sequence is covered.

 Yes, that would certainly be ideal... 

DrDre said:

towne32 said:

It would be nice with the kind of shot change detection that exists in software like pfclean. Have the whole reel/film laid out into shots and choose the representative source shot/frame and apply it to as many target shots as you like until your whole sequence is covered.

 Yes, that would certainly be ideal... 

 Well maybe when Adobe or pixelfarm buys it from you we can have that. :P

(he said, having no clue about the licenses of any code involved)

I just created a correction model using the WOWOW referance frame that was posted earlier. (This one: http://screenshotcomparison.com/images/1442247198_4817509236.png )

I tried using the correction model on another frame from Raiders, but I can't say whether or not the corrected frame is better looking than the original bluray frame. The result is either way quite amazing.

http://screenshotcomparison.com/comparison/143209

Kurosawa10 said:

I just created a correction model using the WOWOW referance frame that was posted earlier. (This one: http://screenshotcomparison.com/images/1442247198_4817509236.png )

I tried using the correction model on another frame from Raiders, but I can't say whether or not the corrected frame is better looking than the original bluray frame. The result is either way quite amazing.

http://screenshotcomparison.com/comparison/143209

 As you can see the early parts of the film were color timed a little different than the later parts. Using a frame as a reference from the same shot, results in a much better prediction:

Bluray:

WOWOW:

Bluray matched directly to WOWOW:

Using this color correction model for the prediction leads to:

Bluray:

WOWOW:

Predicted color correction using color correction model calibrated on the above frame:

^^yeah, a little contrasty, but the colors are stellar! That method seems promising!

-G

Good grief, the WOWOW looks almost monochrome by comparison.

And for all of dvdmike's carping about the orange Nazi flags in the Blu color timing, it could well be that they kind of looked orange in the theatrical timing too.

nightstalkerpoet said:

:)

Here is that last shot (predictive from a different 35mm frame match) with slightly less contrast, compared to DrDre's Wowow to Blu match.

http://screenshotcomparison.com/comparison/143259

 Great results! Something similar actually can also be done in the GUI in principle, if you change one of the parameters of the model. This is based on color matching in a single color space. The matching isn't that good, but predictions in other scenes tend to be more robust. I have implemented this in an update of the GUI. There are two options for model selection: multi color space model (default), and single color space model. The latter is also much faster, but as I said if you're looking for exact matching, you need the first model. If anyone is interested, please send me a PM. 

Here's an example of a prediction for the single color space method using the 35 mm frame I posted earlier as a reference:

Bluray:

35 mm frame:

Prediction from single color space model:

Here are the predictions for some of the other frames:

So, in principle you could regrade the entire film, based on the single frame with this method, and it would look pretty awesome. I'm sure these predictions much closer to the 35 mm colors than either the bluray or WOWOW. Since this method is much faster, it would take about 2.5 days to do the entire film. Oh, and those nazi flags are definitely red in this regrade.