It's not entirely baseless though. Please, consider the following:
- The most aliens we see are on backwater worlds, or in the Alliance
- We never see Imperial aliens
- The guard in the Detention Block refers to Chewie as "that thing".
Is the Empire inherently speciest within OT-canon? It's not explicitly stated, but it sure is easy to draw that conclusion.