I think Palpatine considers the lightsaber an inferior weapon. But it is also a good weapon to defend against other light sabers and to keep the extent of your powers hidden. Palpatine gets Anakin to act and then he has no need to hide because Anakin is his servant from that point on.
He still shouldn’t carry one of his own design.
To carry on with the sacrilege theme he could hold it by the blade and hit people with the metal handle 😃