I saw the SE's in the theatre when they came out. I was only 12, and I thought they were really great at the time. I had seen the original trilogy on vhs many times before that, and was already very familiar with the films, so the changes were kind of startling, but they didn't bother me. I had no conception that they were ever meant to replace the originals--they were an interesting alternate, which for a while I actually preferred.
I didn't sour on the SE's until the '04 dvd's came out. Having a fairly keen ear for sound, and a good memory of the various nuances of the soundtracks, everything about the terrible ANH remix grated on me, and I don't just mean the obvious things like missing music. There's something "off" about the whole thing, which is hard to describe but definitely there. Much of the movie sounds as if it's been EQ'd towards the low end, giving it a muffled and distorted sound--you can especially hear this in the opening Star Destroyer flyby and the first stormtrooper shootout. Most of the Star Destroyer bass is supposed to be a 40 hz effect, but the '04 plays that down and emphasises a 60 hz tone in the mains to an unreasonable degree, while the strong higher frequencies that characterise the '97 mix in particular are practically unheard. Add that to missing laser blast sounds, background effects mixed far too loud, sometimes to the point of overpowering things that should be more prominently heard, wildly fluctuating quality of dialogue (announcements on the Death Star intercom actually being too clear, while people speaking onscreen are muffled), weird extra hissing and static added to the lightsaber duel, the music in general being too quiet--the result is an abomination, one that is practically unlistenable.
This actually bugs me far more than any of the picture defects, although the underexposure, crushed blacks, oversaturation, hideous blue cast, and wrong lightsaber colours make it look terrible as well. The '04 mixes of Empire and Jedi don't have these problems--they sound completely fine, except for the terrible redubbing of Boba Fett, and a few other little things. Some explosions are pumped up in volume, but not in a bad way.
Going back to the '97 SE versions instead of the '04, particularly for the first film, I find I can just watch it without getting too upset. Sure, I don't like the cgi inserts, Greedo should never shoot, and Luke should never scream as he falls (thank goodness they had the sense to later remove that!), but I'm tired of being upset about them, I guess. They're there, whatever. The original versions of the films are infinitely better, and I nearly always choose to watch them. But I can see the '97 once in a while and enjoy that, too. It has a very good sound mix that, while different, still sounds like the original for the most part. No idea why the dvd version is so butchered in comparison.
Aside from that one time in the theatres, I've never heard the '97 mixes in 5.1, only on stereo vhs. I didn't know anything about the technical aspects of sound then, surround or dynamics or LFE or any of those things, so I don't remember how the presentation would have differed from the stereo. I would dearly love to have the 5.1 versions, but since I never owned a laserdisc player, there has never been a chance of that. Darth Editous has mentioned that he's thinking about transferring them from the laserdiscs at some point (the first one at least), which would be very cool.