Ian McDiarmid also indicated that many takes were done, and that he did it differently in each one to offer Lucas the choice of what he wanted in editing (since everyone knew 'fix it in editing' was his way of working). It is quite likely that the more hammy versions are what ended up getting used, even if they shouldn't have been.
Pretty much everything about that scene, from the way it was written, shot, and edited, is just plain bad. Even good actors can't salvage something so disastrously misconceived, so if it doesn't work it's hardly their fault.