Ok been watching it over and over again. Try this idea: Just a quick flash of sith eyes right at, erm, penetration.
We are watching the eyes so much that it does seem very “obvious.” But what if the flash of color comes and goes very quickly, like a split second? like it goes full color right at the saber ignition and fades off pretty quickly?