Adding to this I must say that my denoising is possible the most lazy thing imaginable, as I simply use MC_Spuds at it's default setting (so no further parameters). Note that the filter needs a YV12 source, so you have to use "ConvertToYV12" before this filter, and "ConvertToRGB24" after it.

I just love how much detail it keeps, but as Harmy pointed out, despotting beforehand is highly recommended. Unfortunately this is very difficult do to when dealing with stuff like starfields, as those are bound to be reduced by a fair amount of stars in the process.

For shorter shots it's advisable to remove the biggest dirt pieces manually. After Effects' clone brush is very cool in that it allows the offset for the clone brush to not only be set as x and y postions, but also as frame offsets. So usually I just leave the position offset at 0,0 and the frame offset at -1 or +1 (depending on which end of the clip I'm working on). By that I can just stamp the same area from one frame earlier (or later) onto a piece of dirt, which is usually close enough in brightness and coloring to work.