Well, the biggest thing to remember is this: NEVER, under ANY circumstances, try to load the entire scene into Photoshop. Do it shot by shot, or as you've already experienced, it'll be too big a file to handle and Photoshop will crash.
As far as getting a sequence of images into a moving video again using VirtualDubMod, I have no idea.