Children can be taken from their parents due to neglect even if they don’t use drugs, so I don’t see the issue you think you’re raising.
You have to find neglect. There are already plenty of neglected kids that suffer with parents addicted to hard drugs. If hard drugs are legal I think the problem will be more widespread. And if hard drugs are legal, do you think that should or should not be a basis for deciding on child custody between divorced parents? Should evidence of hard drug use justify social services going into a home to at least temporarily take away children? Basically, are you going to pretend that hard drug use by parents doesn’t pose an inherent threat to children?