None of us know exactly why this has always happened; probably never will. Probably financial, in some way that doesn't make sense to us, shy of some lynch-pin piece of data. But the evidence for a "conspiracy," if you want to be so X-Files about it, is overwhelming. In the end, there are just too many of us, with too much material and expertise, too many inside contacts, and too many privileged sources to make the idea of a deliberately substandard release anything less than fact. In fact, even Lucasfilm is aware the images are substandard: they're deliberately posting compares on the StarWars.com site to show you how shitty the OT DVD's you just bought are.
Frankly, I don't care why Lucasfilm takes the position they do; I already have my own Star Wars vastly superior to anything they've put out. But in the interest of preservation, accuracy, and love for the trilogy, I'm not going to turn a blind eye to the deficiencies which are patently obvious in these transfers. I'm not going to pretend it's okay for Luke's lightsaber in ANH to be green, or Obi Wan's to be purple, or all the glows eaten off the laserbolts. I'm not going to pretend it's okay that the audio in the surrounds are fipped left-to-right. I'm not going to pretend it's okay that the 2006 DVD is grainier than a pre-93 laserdisc transfer I've had sitting on my hard drive for years. I'm not going to believe the source files for a proper restoration don't exist when I've seen them with my own eyes.
If you honestly can't conceive of any of the myriad reasons - financial, artistic, or personal - for why all of this has gone down, then I don't know what to tell you. It isn't particularly hard, nor uncommon a set of potential motivations. As far as conspiracy theories go, it's about as boring as it gets.
_Mike
P.S. Vigo: see my "mixed bag" comment earlier about the inconsistent nature of grain presence in the transfer. It seems I need to keep restating my position for those of you who can't read entire posts: There is a lot of evidence for deliberate image-quality reduction in the 2006 DVD. It's not a fact; it's an informed opinion. I can't teach you guys how to have an eye for the stuff I'm seeing - I can post compares, but a lot of it just comes from more than a decade of doing this stuff all day. You just learn to recognize stuff that's digital from stuff that's organic. I'll try and think of some really illustrative ways of showing the difference, but I'm not sure what the point is. The images are unnecessarily substandard, and that's a fact.