I agree that the original characters have more charm but in ROTJ most of that comes from carrying over what you learned about them from the previous two films.
As the PT didn't inspire any investment of attention in it's characters I can dismiss the whole thing.
I have already too much an investment in the characters of the OT to dismiss ROTJ which is why I find it a more painful viewing experience.
In a way ROTJ is the Star Wars sequel a lot of people may have expected in 1980 (a less fresh, slightly tatty rehash of the first film) by giving us ESB instead the audience was set up to expect a different kind of film series where instead of just recycling the first film (as most sequels did and still do) it would build on it.
Sadly ROTJ went back on that promise.
The PT films are more related to ROTJ than the first two films.