Yeah, I've always found the child indoctrination thing from Phantom Menace to be particularly unsettling.
To begin training at that age for such a harsh and dangerous lifestyle... a child is in no state to make that sort of decision; there's no way they could fully understand what it entails. And for their parents or the Jedi to decide for them is cruel and irresponsible.
In ESB, Yoda says Luke is too old to begin the training. At the time I took that to mean Jedi training should begin in a person's late teens as opposed to the mid-twenties or so that Luke seems to be, not that training begins as a toddler.