Nice thread! Tyrphanax has pretty much got this, all his advice is quite sound. Bingo is on a good track too, but, a little style never hurt anyone. But he is right, don't be too self conscious about it. Whatever you're wearing, wear it with confidence. Regardless of your gender, confidence is sexy as hell.
I relate in a lot of ways, and not so much in others. From teenager on through college I always had a good number of female friends and female attention. But I totally lacked confidence. So much so, that I didn't even realize that I had more female friends than a lot of guys did or that I had others that were interested in me. Even if a girl did show interest in me and made it a point to talk to me, I'd write it off as just being friendly and not allow myself to try to pursue anything for fear of awkward rejection.
Flash forward to many years later and I reached kind of a low point in life where I didn't really care anymore. Apathy is generally not very attractive, but it cancelled out my self consciousness and during that time I discovered the whole "just take a plunge" thing, which in turn built up my confidence pretty well, which got me out of that funk in all sorts of ways.
Just go up and talk to people, that is what everyone else does. That group of nerds would probably be more than happy to have an new guy join the group. Just walk up to some guys playing magic and say, "You know, I've always thought that game looked really awesome, but sadly, none of my friends have ever been interested in it." That is probably a good enough conversation starter right there, if they don't engage you and just stay focused on their game, a simple "mind if I sit and watch?" will grant you permission to listen in on their conversations and decide your next infiltration method. They start talking about anything you know anything about, you can interject something.
With girls it is the same way, just find an excuse to start a conversation. If things go south, I do an out loud inner monologing thing (say something quirky humorous about how that didn't go over very well as if I am talking to myself, not her), it sometimes gets giggles, usually smiles, and from time to time completely turns things around, and at the very least makes me feel less awkward and kind of ends the exchange with a lighthearted tone. Just don't be afraid of rejection, not every girl you try to talk to is going to be interested in talking with you (most probably will, women like attention) and if they politely brush you off, it is no big deal. If you just let go and try, I think you'll find having enjoyable random conversations with girls isn't that hard.
Fear and self consciousness really hold a lot of guys back.
Also, there is no greater social crime for one to commit than to be boring. It is absolutely unforgivable. Everything is interesting to someone, and nothing is interesting to everyone; so this isn't yet another thing to be self conscious about, just find things you are interested in and keep educating yourself about them (for goodness sakes, not just frivolous things, things that actually matter). When you fill your head with knowledge, you'll always have something exciting to talk about. If it is exciting to you, but not to the people you are talking to, then it probably isn't a girl/person/group you are going to click with anyway, so don't sweat it.