Actually the Luke of TLJ is the one who has finally internalized the lessons of TESB. Rey says herself that Luke is purposefully ignoring his success in ROTJ (which repudiated Yoda and Obi-wan), but he has a reason for doing so - saving Anakin did not destroy the Death Star in the short term, nor the Empire in the long term. On the contrary, Luke sees his victory there and elsewhere as having a direct line to his hubris in training Ben.
And anyway, Luke in ROTJ is very pointedly not able to avoid the emotions affiliated with the dark side. He brings his weapon with him when he goes to see Vader. He gives in to his fear, anger, and hatred. It is only when he is on the verge of killing his father does his rationality come in, and he realizes what he has done - it is an exact mirror of the flashback in TLJ. Of course the argument then is that “he should have known better.” Well fine if you feel that way. But in my mind, the dark side is a constant temptation, and the factors leading to that moment in Ben’s hut were such that Luke was, in his arrogance, unaware of what he was getting into (it was a far more subversive challenge than the explicit manipulation of the Emperor on the Death Star). This arrogance is of course a mirror to the arrogance of the Jedi in the PT not realizing the fear and the anger they were giving into, which caused their downfall. Luke, seeing the cycle of things he’s found himself perpetuating, decides to end the Jedi for good. I don’t see any regression at all.
To me this presents a very narrow point of view, and just like much of Luke’s character development seems to be ignored, reducing him down to his mistakes at some moment in the past, so too the Jedi are reduced to their mistakes at the darkest time in their history. Like Luke there is much more to the Jedi than their mistakes at a specific moment in time. The Jedi guarded the peace in the galaxy for over a thousand generations. That to me is clear proof, that the Jedi code works, and that Luke in TLJ was turned into a fool, not being able to look beyond the flaws of a couple of individual Jedi, who by no means seem to be representative of the Jedi over their millenia long history.
I mean, yeah, Luke is looking at it the wrong way, and he’s ultimately proven to be wrong. But Luke is no fool. All we know about the Jedi is that they were the guardians of peace for a thousands generations, but even that doesn’t mean there were a thousand generations of peace. They were exerting their will of the Force over the galaxy. Luke only cites the rise of Darth Sidious and the creation of Vader (because these are things we can connect to as we’ve seen those films), but the whole point of him being on the island is to study the long history of the Jedi, and this is the conclusion he came to, that the Jedi must end. You can nitpick that they didn’t give you sufficient explanation, but in my opinion the implication is clear, that there are more flaws to the order than just the ones we’ve seen in the films.
Yet, Luke then changes his mind.
He doesn’t change his mind about the Jedi, he changes his mind about helping the Resistance. This is shown very well in the Yoda scene. Yoda tells Luke that the teachings of the Jedi were flawed, and tells Luke to pass on everything he learned, both success and failure. Luke still believes that the old Jedi were flawed, but he decides to create a new order of Jedi, learning from the failures of the old.
I didn’t hear anything about a new order of Jedi. All I saw was Yoda chastising Luke for not living up to his potential as a Jedi. Yoda didn’t treat that message as some new insight about the Jedi. It was Luke who chose to only see the Jedi through the lense of his own failure, and thus couldn’t see the bigger picture.