Star Wars is a classic, and if was still a standalone movie with zero sequels, I would still love it just as much. ESB is a rare instance where a sequel is as good as the original. There are many good sequels in movie series, but ESB is the greatest sequel ever, and that defined the Star Wars saga. Because ESB is so good, ROTJ gets a pass in some ways for me, because I have to finish the story, and i still really enjoy it. I don't love it, but I really like it as closes to the trilogy. For me personally, ROTJ does give closure for me in a positive way, so I will give my props to it for that reason.
Now lets say ESB was not as good as Star Wars, and it was an average sequel, but not the classic it is now. Then I probably wouldn't watch ROTJ, and in a sense I would just be a Star Wars '77 fan. It would be equivalent to me being a Jurassic Park fan, and thinking the sequels are inferior. It would be equivalent to me being a Matrix fan, and thinking the sequels are inferior.
I can always watch SW '77 and enjoy it, but ESB & ROTJ go together to complete the trilogy, even though ESB is light years better. I think many fans who say they don't love the PT, but still watch them now, are an extention of what I do with ROTJ. They are still watching movies that are inferior to its predecessors, but out of obligation to the story, feel the need to still watch all of them. In a sense the smartest thing Lucas did was using the number system, cause it is psychological mechanism that many feel they are not watching all the story. I know one friend who thinks TPM is awful, but feels he has to watch it because it is part of the story now.
Now was ESB a fluke? That is a great question, and sometimes I start to believe it. Think about it, since Raiders of the Lost Ark in 1981, what great movie has Lucas really been attached to? In that respect, he is way overrated.