it was little more than a way to show he wasn’t going to be content as a farmer.
Much as we all like some mild Lucas bashing from time to time 😉 SW77 does say that Luke’s father was murdered by another Jedi (Regardless of any other information imparted in ESB and ROTJ). So I think it’s pretty logical to assume Owen/Lucas was refering to this when he grimly said/wrote “That’s what I’m affraid of” rather than just refrencing Luke’s dislike of farming.