We’re all pretending the OT was all planned out at the beginning.
Honestly, TLJ going a completely different direction because it wasn’t planned out ahead of time is exactly what we got in Vader being Luke’s father, and Leia being his sister. The PT suffered from a lackluster continuity which was planned from the start - sure it was tweaked, but the story idea was there already.
Empire gave us a shocking ending, RotJ tried to match and failed. The prequels didn’t need to shock us the same way because we already knew the eventual outcome. Rogue One did great at offering a shocking ending, as well as a nice segway.
TFA attempted a different kind of shock ending, which was the cliffhanger with no payoff. While unexpected, it again failed to reach the originality level of ESB.
TLJ… succeeded in surprising us and telling a story in a different (very chaotic) way. As a single film, I can enjoy the frantic flow. Hopefully the next film won’t fall into the pitfall of RotJ and base itself on that same theme as a wow factor though, because i could never enjoy another film with this storytelling style.