Puggo - Jar Jar's Yoda said:
So it may require building a mask externally. As ThrowgnCpr said, that's actually not particularly hard either. The hardest part may just be playing with it and verifying that it is good quality after encoding, that it doesn't get all jaggy.

If you do end up creating a mask externally, I don't think you would have too many problems with a jagged or stairstepped edge. How would you render from Vegas? You could save it as a lossless AVI, or frameserve directly to something like CCE for a good DVD render (I usually do not render to DVD straight from Vegas). Either way, I think that the result would be fine.