I contacted TMPGEnc - they confirmed that their newer versions can't do the setting I want. So they sold me a license for the older version, and everything is working fine now on my new computer.
Finished encoding all three films. Now to start on the doc...