Maybe try experimenting a little with CRF encoding values instead of targeting a bitrate? Quality will be more consistent that way, but file sizes will vary some.
I can’t recommend a CRF value for h265 though, that’s out of my area of expertise. I’d suggest 18 for h264 with slower/slowest encoding settings, but I honestly have no idea how h265 behaves with encoding as I never really use it.