Sign In

DrDre

User Group
Trusted Members
Join date
16-Mar-2015
Last activity
26-Jun-2017
Posts
2528

Post History

Post
#1085856
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

NeverarGreat said:

Looks very promising. The Death Star walls still vary in color, but that is expected since my regrade didn’t go for strict adherence to a standard color, opting for a location-by-location color shift instead. The only other significant change I see is that the bot has less saturated skin tones, which makes sense since the bot probably doesn’t prioritize skin tones.

I should note that 5 or 6 of the test frames are from shots that I altered with color from another source, so the bot probably wouldn’t have been able to replicate that look anyway (see the R2 and 3PO shot and the trash compactor shot).

It’s not so much a case of not prioritizing skin tones, but adding more references to teach the algorithm how to deal with skin tones. Here’s another set of regrades, where I’ve also used the second set of 16 shots as references. Saturation of the skin tones is now much closer.

Bluray:

Bluray regraded by NeverarGreat:

Bluray regraded by the NeverarBot:

This post has been edited.

Post
#1085599
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

Here’s a test of the color grading algorithm using the preview of NeverarGreat’s regrade of reel 4. I selected 16 reference shots from the reel as reference shots to train the algorithm to approximate the look and feel of NeverarGreat’s regrading prowess on 16 different shots.

The reference frames I have chosen are:

The test frames from the bluray are the following:

Here are NeverarGreat’s regrades for these shots:

Here are the NeverarBot’s regrades for the same shots:

While there are slight differences here and there, the regrades of the NeverarBot closely approximate NeverarGreat’s manual regrades.

This post has been edited.

Post
#1085252
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

Kexikus said:

Just found this thread and I’m in awe. This looks really amazing and I’m definetly looking forward to its release.

One question though: Will the algorithm regrade the entire movie at once or do you give it individual scenes/shots that will then be regraded? I’m asking because in the latter case, scene wipes might be a problem (and also, just getting the entire movie regraded at once would be even more convenient^^).

The algorithm will regrade a frame for each individual shot generating a LUT, that will then have to be used to regrade the complete shot. Scene wipes will have to be fed to the algorithm individually, but this shouldn’t be a problem.

Post
#1085233
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

Darth Lucas said:

So forgive my technical ignorance, DrDre, but how exactly does this work? Do you have to manually attach a shot to a source frame? Or does it take the bulk of reference frames you have and build a universally applicable model to grade any shot with?
And how is the grading accomplished? Does it just take the color palette of your references and match the source to it as closely as possible?

A detailed explanation of what steps the software takes would go a long way for helping a non-technically minded person like myself to comprehend what’s going on here.

No, you don’t have to manually attach a shot to a source frame. The algorithm just needs a set of source frames and their references. These references are not the original reference frames, but the source frames matched to references by the color matching algorithm, such that the colors of a pixel in the source can be directly mapped to the same pixel in the reference. For any new shot that you want to regrade, the algorithm automatically assigns higher weights to the references, that are more most similar to the new shot. There are no manual steps involved. You just point it to the shots you want to regrade, and presto…

This post has been edited.

Post
#1085026
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

jedimasterobiwan said:

DrDre said:

jedimasterobiwan said:

You see the reason I have an issue with desert shot and some other shots is it seems too warm like they have completely yellow skin. Also some shots of obi wan he looks a little red still. Also on the death star there’s some color issues there as well.

Like Darth Lucas said, the desert shots are supposed to be warm, and I personally don’t see yellow skin tones. The Death Star color grading is Mike Verta’s so take it up with him 😉. That said, these are based on just 4 reference shots. In practise you don’t regrade an entire film on four reference shots, which in this case did not even contain any desert shots.

what about the scene in the lars home owen looks yellow and beru and luke look wrong as well.

What scene in the Lars home? There is no shot of Owen and Beru in this thread, apart from the shot of Luke and Owen on the first page, which is a regrade by NeverarGreat.

This post has been edited.

Post
#1085007
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

jedimasterobiwan said:

You see the reason I have an issue with desert shot and some other shots is it seems too warm like they have completely yellow skin. Also some shots of obi wan he looks a little red still. Also on the death star there’s some color issues there as well.

Like Darth Lucas said, the desert shots are supposed to be warm, and I personally don’t see yellow skin tones. The Death Star color grading is Mike Verta’s so take it up with him 😉. That said, these are based on just 4 reference shots. In practise you don’t regrade an entire film on four reference shots, which in this case did not even contain any desert shots.

Post
#1084805
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

^…^ said:

Dre, awesome job!

Idea: it’s possible to regrade a whole film using just a trailer for reference, right?

In theory yes, but the quality of the final result will depend on the consistency of the source, and the reference. If the color grading of the source you want to regrade varries wildly, in the worst case it’s possible you might need a reference for each shot. Garbage in is garbage out.

The algorithm recognizes the colors of a scene, and ensures the colors for each scene for which you have a reference accurately matches the reference. For the scenes for which it does not have a reference it estimates the colors based on the sources it has available. This is why this method works much better than color matching a shot, and using a LUT based on that shot to correct other scenes. Such a LUT would only contain information about the colors in that shot. If for example another person would walk into the frame, the colors could be off by quite a bit, especially if the colors are altered heavily. The self-learning algorithm would use the color information of that person from other shots, without adjusting the colors of the rest of the shot, as each pixel is graded individually. This also ensures color consistency between shots.

Post
#1084644
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

Currently the algorithm takes about 1 minute to regrade a shot and generate a LUT. So in batch mode all shots will be graded in about two days. My estimate is that you will need about 25 to 50 references to accurately reproduce the color grading of an experienced color grader depending on the consistency of the source and reference. For someone used to manual grading this means that you would need to grade at most 50 shots rather than a few thousand.

This post has been edited.

Post
#1084641
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

Stotchy said:

It’s so pleasing that there is potentially a solution to to appease everyone’s different color timing tastes without HUGE efforts of doing so manually shot-by-shot.

I for one am not a fan of the green technicolor look so Neverarbot and Vertabot alternatives are fantastic to have available!

RE your limited Verta references, can you use screen shots from Mike’s videos? Or will the Vimeo compression likely have messed with the colors?

If I understand correctly, the Verta videos are not graded yet. He first corrected Legacy to neutral colors, which he shows in his videos, and then does the color grading at the end. The four earlier shots all represent preliminary grades.

Post
#1084633
Topic
Automated color grading and color matching with a Machine Learning Algorithm
Time

Here are some results of the VertaBot, based on just four references.

Bluray:

Bluray regraded by the VertaBot:

Bluray:

Bluray regraded by the VertaBot:

Bluray:

Bluray regraded by the VertaBot:

Bluray:

Bluray regraded by the VertaBot:

Bluray:

Bluray regraded by the VertaBot:

Bluray:

Bluray regraded by the VertaBot:

Bluray:

Bluray regraded by the VertaBot:

This post has been edited.

To the top