He also kept his Jedi robes. And when he went to go burn the tree - he couldn’t burn it. Yoda had to do it for him. Luke didn’t really want the Jedi to end, (“I will not be the last Jedi” are some of the last words he ever speaks") and he couldn’t make himself do it. That he kept Leia’s lightsaber as a memento isn’t a “plot hole” because it doesn’t undo or mess up anything from a plotting perspective, much less a thematic or character one. It’s absolutely in keeping with the Luke from The Last Jedi. Luke was being dishonest to both himself and Rey about why he was really on that island, and it wasn’t some big noble aspiration. He was depressed and scared and ashamed.
Wish they made it clear within the film itself that he didn’t really want the Jedi to end.