He probably hasn't helped himself by publicly dissing the prequels. And then he acts all bewildered that he wasn't asked to play Vader in ROTS?
Dave Prowse is Darth Vader? No, James Earl Jones is Darth Vader. In fact, James always said that Dave was Darth Vader - the kind of humility that Prowse could learn a lot from.