Could you not just slightly lower the bit rate for rotj to make room? And for mpg2 I highly, HIGHLY recommend hcenc. It’s free and gives fantastic results. And it has an option to where you just set max bitrate and desired file size and it will calculate the avg bit rate automatically and get very close to the filesize you desire. It works great and I even use it for blu ray encoding because it’s so easy and its encodes are as good as h264 at high bit rates. (Obviously at lower bit rates h264 will always win but at high rates I don’t see much difference)
What do you think might be a reasonable average bitrate for acceptable quality?
Well I primarily work with hd so I’m not sure. My best bet would be to pick a decent max bitrate then tell it how big you want the file to be and let it figure out the average for you.