Are you using actual 3840x2160, or are you using 4K loosely? (Meaning, like, a 4x upscale instead of a true 4K upscale) Assuming you mean it literally – have you compared the quality gain from jumping that high?
The reason I’ve stuck to 2560x1920 is because I don’t get any benefit from going higher. I do get a benefit from using Topaz in 4x mode, because the 2x modes seem to blend poorly, but I don’t see an improvement from anything more.
I have also found that using x265 with a CRF=19 is pretty solid. I can barely see the smallest difference between CRF=6 and CRF=20, while I can’t see a difference at all, really, at CRF=19.
Running H.265 in “Slow” or “Very Slow” will also improve your compression. Compression is much more improvable than I ever believed. But this is also why I ask about the benefits of 3840x2160 – because if you get the same quality at, say, 2560x1440 than you do at 3840x2160, you could save yourself a lot of bandwidth just by getting rid of pixels people don’t actually meaningfully benefit from.
If you do benefit from 4K, I recommend trying H.265 with Slow encode speed to see what kind of compression boost you get.

I am using Topaz 4K-mode but switch to custom settings to avoid black bars left/right, so I end up with an automatic display size of the values I had specified.

I am not enocoding with a CFR as you suggested, I use Adobe Premiere Pro with a CBR of 15 Mbps in x264; while I know x265 could increase compressibility, I ran into issues with that in the past, so I need to stick to x264 for now.

As for the display size itself, I will let the automatic setting do that, for it scales up while maintaining the aspect ratio.

By the way, since the rules here do not allow for link posting to complete material, please send me a private message to get the link for the current 4K-version of the pilot (Joel and FrankB).
It will be the same as in the release topic I am preparing right now, but this way you’ll get it sooner, since the topic will take a little bit.