And I would like to again stress me feeling that ROTJ doesn't need Fett at all. He can easily be excised and it in no way impacts the story. All that is lost character-wise is Han's bumbling revenge and defeat of Boba which is another bad attempt at humor and a poor end to the character.

Axe him. He did his job and got paid. Why would he hang around Jabba's palace macking on 'dancers' when there's more money to get in the universe? He can have sex with skanks any time, but that's a different movie and I bet there's a fan-fic somewhere out there about just that....