Nanner, what have you heard so far? Their early stuff is much different, seeing as how they were two guys with keyboards. Later stuff incorporated an actual band...I think from Apollo 18 on, although John Henry felt like the first to really sound like a full band.

My thoughts:

They Might Be Giants (1986) - Actually my least favorite. A lot of filler.
Lincoln (1988) - Up and down. First half is great but second half loses a lot of steam.
Flood (1990) - Early masterpiece. You've probably heard Istanbul before.
Apollo 18 (1992) Solid but not as good as Flood. Make sure you listen to the entire album on shuffle so you get the full effect of Fingertips.
John Henry (1994) - Probably my favorite album.
Factory Showroom (1996) - Good follow-up.
Mink Car (2001) - Bit of a let down here. Several strong songs but several I've never listened too again after the first few times.
The Spine (2004) - Probably my 2nd least favorite. A few good songs but too many blah songs.
The Else (2007) - Nothing new sound-wise but a stronger album than The Spine.

Early period recommendation: Flood, maybe Apollo 18
Middle period recommendation: John Henry or Factory Showroom
Recent period recommendation: The Else, maybe Mink Car

Oh, I forgot to mention they've done several kids albums. I haven't heard any of those yet.