You don't need that much. Besides, some operating systems won't recognize more than 4GB of RAM (I have a dual-core 2.6 GHz processor with 4GB of RAM, and my Windows XP partition won't recognize more than 3GB of RAM). I wouldn't go more than 6 or 8GB - any more than that and you're just wasting money on something you can never use (at least right now).
I have this problem occasionally, but it usually happens when I'm watching very high-bitrate 1080p stuff. My solution is to re-encode at a lower bitrate, but not low enough that it degrades the quality too much. My Planet Earth 1080p H.264 mp4 files are all encoded at 10,000 Kbps (average bitrate), which brings them to around 4GB apiece. This doesn't stutter my system if I'm watching them on my computer with nothing else running, and they're all slightly less than 4GB which lets me throw them on my FAT32 hard drive to watch on my PS3. I don't have any 720p files to give you a bitrate estimate for that.
Just play around with the encoding. Pick a 5-10 minute section that exhibits this problem a lot (usually action-heavy scenes), and play around with re-encoding that section at lower bitrates until you find a bitrate that plays smoothly on your setup and isn't too heavily compressed.