I'm one of those people who if I had to choose to keep either a 2004 SE or a 1997 SE DVD, I would take the 1997 SE DVD every time. The 1997 SE had far less problems with the colour timing, and there were aspects of the movies that still adhered to the originals.
The big thing that totally made up my mind about the 2004 SE being trash was how they screwed up the lightsabers - one of the most important features of the movies and they screw up the colours. Darth Vader with a pink lightsaber? I think I'll pass...