In some ways I think people just weren’t prepared for the ST to actually be a thing.
Because the thing with the ST in relation to the OT, is I think (personally) they had no choice but to look inward and challenge Luke and the Jedi in someway. Because otherwise, what’s the conflict? You can’t have a story without a conflict (it can’t be the adventures of happy go lucky Luke, Leia and Han like the old EU books were).
You can’t really do an extragalatical threat like the Vong, because that’s not really Star Wars. Star Wars is about our world, and to do that would be to make an alien invasion movie, which would move SW from mythic but grounded to straight up sci-fi.
So what’s grounded? What would cause a trilogy long conflict? Well, as in real life and in myth, you get this idea of history repeating itself. But it’s not just Empire vs. Rebels v2, at least I don’t see it that way. You can’t just continue the simple good vs. bad that the OT did. You have to look deeper. By reintroducing a similar conflict, we get to actually look at the fight and ask what are they really fighting for? If more problems are arising, we need to ask, what are the Jedi really? What is their place in this? Especially after the PT (where the Jedi are quite shit, honestly), we have to ask, are the Jedi really helping? Even if the answer is yes (and it is), I think those questions need to be asked. And I love that it’s Luke asking them. He’s so idealistic in the OT he never gets the chance to stop and think it through. And so of course, when the dark side comes back, he finds himself in crisis.
But that’s the thing, to have an ST, we needed the dark side to come back. I don’t think there was any other way. I get why some people wanted the OT’s fairy tale ending to be the end of it, but the truth is, the moment the ST was announced that fairy tale ending disappeared. And yeah, I guess there were other ways to tell an ST story that didn’t so ruin the fairy tale. But I don’t think those kinds of stories would really be all that worth telling. They’d be fun in a light and breezy way, like a superhero sequel, but I think a Star Wars saga film needs to be more than that. It needs to really be worth it. And to be worth it, it had to face some hard truths about fairy tale endings, namely that they don’t really exist. Things don’t stay good forever. I think the heart of this whole trilogy goes back to what Maz said in TFA. Through the ages, there’s only one ever really been one fight: against the dark side. It doesn’t just go away when one threat goes away. You always have to face it, fight it.
And so that’s really what the ST is about. The OT was super optimistic and idealistic, good vs. bad, good wins. So it’s a little depressing when the bad comes back, sure. Fighting evil isn’t that simple. But learning that just helps you to look inward and double down on your resolve, as Luke ultimately does.