Yeah, I see that too, I just think it’s symptomatic of an unwieldy/messy script more than it is intentional malice or whatever for the series. That’s ridiculous to me, it’s at worst a guy who has different ideas [than you or someone else] about how this all works and who these characters are.
The most charitable interpretation I can come up with is Rian Johnson was going for something along the lines of a “Wizard of Oz” type message, where it turns out the Scarecrow and Cowardly Lion had all the brains or courage they needed all along, and just needed to believe in themselves to access it. Something like that. That is sort of compatible with what happens with Rey’s journey of self-discovery, where she sort of self-learns the Force. There’s nothing inherently wrong with a message like that, but it’s not a fit for Star Wars and what was established before, where the Force requires a mentor to learn and is already part of a pre-packaged, venerated mythology.
As for “intentional malice” - I’m not really sure what that would even mean in this case. I don’t believe that like, Rian Johnson sat down one day and started angrily writing the script, saying things like “I’ll show those stupid Star Wars fans… they want to see Luke do they? Oh I’ll give them Luke… I’ll give them Luke all right!!! Bwahaahaahaaa!!!1!!! *starts choking on iced latte*”
I think Rian Johnson just wanted to take Star Wars in a new direction he thought would be interesting, while avoiding accusations of just retreading Empire Strikes Back and working within the story parameters that carried over from Force Awakens, and he ended up writing a very misguided script. At the very least, I found it heartbreaking watching the Mark Hamill interviews about this.
I would compare it to the attitude from this iconic interaction between Blizzard and WoW fans. https://www.youtube.com/watch?v=0Wrw3c2NjeE
“You think that having a cool Jedi with a lightsaber in your movie would be good, but it wouldn’t. You think that’s what you want, but it isn’t.” That’s basically what Luke tells Rey at the beginning and it sets the tone for the whole thing. Every part of it is like that, even down to a visual gag where you think you’re looking at a cool space ship and it’s really a clothes iron or something. The “subverting expectations” meme. It even starts subverting itself within individual scenes, to the point where it kills off original ideas, like what if Rey actually joined Kylo.
Oh, you think you know what a good movie is? I’ll show you what a good movie is. You like this character? Well, maybe I’ll show you what the “realistic” version of that character is. It’s more like that attitude.
My conjecture is that Rian’s thought process was something like this: “So the audience is all psyched up to see Luke for the first time in like 40 years. Okay, so what are they most expecting to see? They’re probably expecting Luke to come in like Superman, kick some ass, and save the day. So I think it would be really cool if he did the exact opposite. Kind of like how, in Empire Strikes Back, your entire notion of who Vader is gets turned upside down. I want to do something similar with audience expectations for Luke.”
I also detect some meta-joky snarkiness in Rian’s script, perhaps taking a few light-hearted jabs at the audience for expecting such a cliche outcome for Luke. Of course, I’ll happily admit I would have preferred the cliche version of Luke that just straightforwardly kicks ass. But there has to be some conflict, obviously. Having Luke off soul searching after his Jedi school gets destroyed is a decent premise for a nice character arc. But Rian Johnson just took it WAY too far by making Luke nihilistic to the point of literally being suicidal, writing off the Jedi Order completely, and moping around waiting to die while his sister and best friends are in serious trouble. But to give Rian some credit, he actually does have Luke show up and kick ass at the end - just not in the way we would have expected. The “astral projection” thing could actually be a clever twist under different circumstances.
Also, the logic Luke uses to justify giving up on the Jedi order doesn’t really make sense in context. The Jedi failed to stop a coup one time back in like 20 BBY or something, therefore the entire Jedi Order is eternally condemned and the millennia of peace they presumably helped uphold doesn’t count for some reason. Rian just had this silly Zoroastrian-inspired idea of darkness rising to balance out the light, and vice-versa, perhaps the result of a corrupted interpretation of Lucas’ vague nonsense about balance in the Prequels. It sounds like some ad hoc idea Rian invented to justify Luke giving up on the Jedi.