Color matching and prediction 2.0

So, many years ago I wrote an algorithm for matching the colors of two identical frames with different color grades. The algo has become quite popular among the community in the form of an aging color matching tool:

https://originaltrilogy.com/topic/Color-matching-and-prediction-color-correction-tool-v13-released/id/18128

Over the past year I have been working on a new algo, that does what the old algo does, and much, much more. Simply put the algo can transfer color grades between two frames that are not identical. Here’s an example, where I transferred the color grade of the cantina scene from the 2019 SE to the cantina scene in the Mandalorian season 1:

Target:

Reference:

Color match:

Here’s another example, where I transferred the technicolor color grade of the new restoration of The Ten Commandments (1956) to another classic Cleopatra (1963):

Target:

Reference:

Color match:

Finally, here’s an example, where I transferred the color grade of the Matrix (1999) to 13 Minutes (2015):

More examples are coming in the near future. So, let’s have some discussion on the subject of color matching and color grading in general…

Here’s another set of examples for the case, where you don’t have a proper color reference, but you do have production photos, that can serve as a starting point for a color grade.

The first example is the scene in ANH, where Luke first meets R2-D2 and C3-PO:

Target:

Reference:

Color match:

The second example is the scene following the torture of Han Solo in TESB:

Target:

Reference:

Color match:

The final example is our first introduction to Jabba the Hutt in ROTJ:

Target:

Reference:

Color match:

One important point to consider is, that not just any image is suitable as a reference. We’re looking for images with similar lighting, and similar content in terms of colors.

Final example for today, the algo can also handle references of let’s say questionable quality, where it will protect the shadow and highlight detail.

Here’s the scene of Tarkin refusing to evacuate matched to a photo of the same shot on a technicolor print:

Target:

Reference:

Color match:

This is phenomenal!

I really wanna see what this can do with the last few Harry Potters. There are naturally colored stills that can be sampled, along with the earlier films. I’d be so happy to have Half-Blood Prince without the brown and Deathly Hallows without the black.

I wonder what the limits of this technology is. Surely it can’t exceed human grading, only do it more precisely, right? Like if there’s no data there it’s not gonna magically enhance. Still, I’m suspect it will yield outstanding results.

Dang, this is incredible work Dre! Can’t wait to see progress on this. Would this new algorithm be able to handle HDR sources pre tone mapping? The previous one seemed to do fine when switching rec709 colors (put in a hdr container) to a rec 2020 source, just wondering if this would be able to work to create luts that are 12bit or 10bit friendly.