As an encoder, I can empathize.
I use tuned x264 settings which produce markedly smaller files than just slapping a file into Handbrake on default. A lot of the files I see which are too large for their content are still bitrate starved because the default settings are hideously inefficient.