The Bespin duel can be divided into three parts.
1. Vader toying with Luke (he can kick his *** at any moment)
2. Vader turns the screws a bit by using the dark side
3. Vader finally gets annoyed and loses his patience
It's in the last stanza of the fight that he stops screwing around and just starts wailing away at Luke. He's clearly pissed off and mad and lost his cool. After he severs Luke's hand, you can hear the anger in James Earl Jones's voice.
The same thing happened in the ROTJ duel. They are just kind of fighting in front of the Emperor for awhile, then Vader threatens Leia, Luke uses the dark side and just starts physically wailing away at Vader with pure rage.
You can't pick up on the emotional changes in these fights?