I extended the method, where you match histograms in multiple color spaces. I did this in three ways:
1) Replace the histogram with an estimated probability distribution, using kernel fitting, which is easier to invert.
2) Invert the estimated cumulative probability distribution to obtain a direct mapping between the colors in the source frame and the reference in multiple color spaces. This can then be used to predict other frames.
3) Stabilize the solution.