To add to what Scruffy said, I've posted a snippet from the Star Wars Original Radio Drama below (which, if I'm not mistaken, is considered canon). The quotation is from a promotional ad which Luke is listening to at the beginning of the episode, and is regarding "the Academy" which we hear him speak about later in the radio series (and in the movies):

* * *

"...so don't just dream about flying for the Academy, make it come true. You can find a career in space: exploration, starfleet, or merchant service. Choose from navigation, engineering, space medicine, contact liaison, and more. If you have the right stuff to take on the universe, and standardized examination scores that meet requirements, dispatch your application to:

Screening Office
C/O the Commandant
Imperial Space Academy, [your sector]

and join the ranks of the proud."

* * *

So it would appear the Academy is definitely affiliated with the Empire, however joining and graduating from it wouldn't necessarily mean you had to join the Empire and became a TIE pilot. The career choices, as you can see above, were numerous.