I'm considering the idea of blending the layers together in either Premiere or AfterEffects instead of Photoshop. (Probably AfterEffects.) That way I can manage the color corrections to each layer and quickly see the results in motion.
You are right, Angel. I suspect that a bit of digital painting will be involved in order to reconstruct a few frames.
What do you mean by "creating patches as a global fixer"?