You find the SE resembling to much to the 2004 DVD release, but you also find the OT much different from the SE.
That sounds strange, because in the 2004 release, nearly all changes appllied in SE are furthermore altered.
Greedo shoots at the same time with Han now, Han doesen't step on Jabba's tail, Luke does not scream while falling etc...
Furthermore, the Emperor is a different person now (in SE and OT he is the same), and the ghost of Anakin takes the form of young Hayden.
These two last alterations would make all the difference in the word to me, as both characters are key factors to the whole plot, even if they appear rarely (imagine the Godfather with a "revised" Vito Corleone instead of Marlon Brando).
Actually, in some points, the 2004 release has a tendency to "return" to the original script, as changes made in SE that were clearly mistaken were cleared out.
EG the scene of Luke screaming while falling (OT: Does not scream SE: screams 2004: Does not scream).
After all being said, I actually believe that the SE should be handled like the OT is, that meaning that either refering to both version backups should be permited or it should be prohibited for both.