I feel like bringing the dialogue or whatever back in during the silent passages defeats the purpose of having an isolated score.
Having English subtitles (as an option?) wouldn't be a bad idea - I usually turn them on when I listen to isolated scores anyway.