I always assumed the Emperor was a reclusive leader that only a precious few high level Imperial dignitaries would ever come into contact with, bueraucracy would do the rest. I'm sure many would suspect he was pure evil, but I think he would have kept his dark powers well hidden operating only to subtley manipulate events on a large scale, Vader was his show-pony and his display of powers were enough to instill fear and obedience throughout the Galaxy.
unfortunately the EU only tends to muddy this issue by making his Dark Powers obvious to all.