If purchasing a PC now, simply get a conroe (intel core duo) based system, they are cheap and more than powerful enough for just about anything. Even the slowest one will be more than fine for your purposes. (they also seem happy to overclock quite a bit with standard cooling if that floats your boat) You can get boards that use standard DDR2 memory, make sure you have a few PCI slots free for add in cards. If it isn't a hardcore gaming rig, you may want to get a passively cooled graphics card (no fan) to keep the sytem quiet. (Personal choice, but video editing is time consuming, and the drone of PC fans drives me nuts after a while - silent is better)
Also, get a second hard drive (to capture to) and make sure it is 7200RPM. If money is an issue samsung drives are cheap and quiet and have 3yr warranties, otherwise the WD or Seagate drives at 7200RPM will do the job. I've had nothing but trouble with Maxtors, but YMMV.
As for software, most of the capture software is free or cheap. VirtualVCR, avisynth/virtualdub can all be downloaded.
You may want to get the cheap version of CCE (Cinema Craft Encoder) for encoding the DVD.
Most importantly, option up to a *quality* power supply, cheap power supplies tend to put electrical noise across the bus and it ends up as video noise/interference in your captures. A decent power supply will also make your system more stable for everyday use, and you can get ones that run quiet.