In my film THE FIXER, I had to upscale 4x3 letterboxed standard definition video to match the resolution of 16x9 video (long story as to why). That alone is about 134% upscale, and it immediately begins to show softness and grain since everything's been magnified. You can add a sharpen filter, but you're basically making a choice to either have soft image or a grainy one. To upscale 640x480 interlaced footage to 16x9 720p, that's gonna get ugly.
-NJM