If the smearing is only present in the Luma, wouldn't shots like this look a bit more weird, like the color not matching the double eyes of the stormtrooper?

You would think. That's why I found the results to be weird.

@AntcuFaalb: It doesn't only show as motion blur, it sometimes make moving things appear doubled or leaving trails of itself and things like that.

But it does sometimes show as motion blur, so maybe a de-motion-blur filter of some sort can fix things up a bit. Do you know of any we could try?