There are those that smoke a single joint and chill and have fun with their friends on weekends and who work and contribute to society and those who smoke 24/7 and only take away.
Of course Americans do nothing in moderation so legalizing it would probably make it too hard to control people who do it to extreme.
We are always worried of keeping up with Japan and other nations, giving our citizens another excuse to get drunk, high or lazy might be bad.