Alderaan said:
You can't pick up on the emotional changes in these fights?
I can... but anger is not one of them. Well at least not on Vader's side.
And losing patience in a sense of swiftly ending the fight has nothing to do with anger. He came to the point where Luke was physically cornered and he had move to the final step of his plan.