Sin really means to miss the mark or not to achieve an aimed goal (it was originally a term in archery and was chosen when the Bible was translated with that in mind).
Sin applying only to believer makes sense, you join a club called a religion, it has certain rules and if you don't agree with them you should leave.
It's when people apply those rules to people outside the club, promote ignorance or torture get people to join and stay in the club and use their influence to effect changes to the detriment of everyone that the concept of Sin enters the realm of the world beyond the walls of the temple.