Funny that you mention this, because I went down the same rabbit hole with Handbrake and ffmpeg and I still have the same problem too!
Really? *Exhale! People have ripped on me for not using ffmpeg like it’s gonna fix all the problems but this doesn’t sound promising. I posted this a while ago and still haven’t found a solution. 😦