If you believed you killed your wife/girlfriend, could you stand to look her family in the eye? Wouldn't you hide in shame?
It's all a moot point anyway. To me, Vader should have lost all his humanity after he was put in the suit and never thought twice about Padme again.