Whilst my PC is chugging away doing another encode, I was looking at my HCenc logs from the first go round and saw something that might potentially be the cause of my problems.
I noticed a line that said "Small source mismatch 764 (or whatever number it was) frames"