I like that you made the comparison to Harry Potter, Bingowings. I actually think comparing it to wizards in Harry Potter perfectly explains it. Magic is passed down as a hereditary thing, but there are also the muggle-borns who can use magic, but have no magical bloodline. That's the way it is, but it's never explained in any more detail than that, and doesn't have to be. I think the same kind of principles could be applied to force-users in Star Wars. I would make some variations on it though. Basically, here's how I see it: Some individuals are born with a higher potential to manipulate the force. This cannot be measured by any scientific means, but can be sensed by those whose minds are in tune with the force, however, anybody has the ability to manipulate the force with training, some are just born stronger with the force than others, thus have an easier time learning and are more likely to be more powerful. This natural strength with the force is passed down through generations. Ben's line "I was amazed how strongly the force was with him." perfectly fits in with this ideology. The Skywalkers simply have a very high natural connection with the force.