I don't recall the video moving around more than a few pixels. That being said, I've usually been more focused on what's in the frame. I recall looking over the video I had fixed up to that point, and seeing lots of white squigglies in the black letterboxing that I'd completely missed because I had the image zoomed in, cutting out most of those sections.
I would certainly like to get rid of the movement, and put new black mattes over the letterboxing.