But starting Jedi training young has important narrative function. They do that so the younglings don’t have a chance to form attachments to their parents (or anybody else) and be taught from the start to not be attached. The reason why they said Anakin was too old to be trained was because he was attached to his mother.
Plus Luke himself was regarded as too old for training by Yoda at first (though granted this exchange at the time didn’t imply Jedi spent literally all their life in the Order)