If I understand correctly, the audio was ripped by someone else? How did you receive it, is it one AC3 file per LD side?
Because it's a digital rip, I would not expect to see any audio drift, it should be locked to the video.
Firstly, I would check to see if it syncs to the raw video, to ensure that your IVTC hasn't changed the overall running time.