logo Sign In

Film cleanup (in avisynth I guess)

Author
Time
 (Edited)

OK, so since it seems like I'll be working with an increasing number of film-based sources, I'd really like to learn how to do film cleanup and I don't even know where to begin. What I want is basic denoising for when I need to match grain-levels to other sources but mainly, I need a way, to automatically remove dirt and spots - it doesn't really matter, if it mistakenly erases other details, like stars and stuff like that, since I would then only utilize the cleaned up version for manual cleanup in After Effects.

Oh and some good image stabilization, which would only remove gate-weave would also be totally awesome.

Doing this kind of thing in After Effects is less than ideal, since it was never made for film-restoration and I also tried PF Clean and I just couldn't get it to work but I couldn't afford the full version of PF Clean anyway, so I'd really like to learn how to do by myself what Laserschwert has been doing for me so far, so that I don't have to bother him with it. So I guess my question really is how to do this sort of thing in Avisynth.

Author
Time

Harmy said:

Doing this kind of thing in After Effects is less than ideal, since it was never made for film-restoration and I also tried PF Clean and I just couldn't get it to work but I couldn't afford the full version of PF Clean anyway, so I'd really like to learn how to do by myself what Laserschwert has been doing for me so far, so that I don't have to bother him with it. So I guess my question really is how to do this sort of thing in Avisynth.

Before moving on to Avisynth suggestions, I'd like to ask: Do you know that The Pixel Farm is selling PFClean at a reduced "education discount" rate? It's $167 USD. I don't know if this will make it any more affordable for you, but I figure it's worth mentioning.

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time
 (Edited)

I still couldn't currently afford it for that price but even if I could, like I said, I tried it and couldn't get it to work - It got really frustrating after hours of unsuccessfully trying to get it to do anything at all.

Author
Time
 (Edited)

Yeah, I did a Google search before posting here and found that but that's way over my head - I'm really going to need a more basic step-by-step kind of thing.

Plus it seems that that guy's script doesn't do the main thing that I need and that is removing spots and dirt. And I don't need color correction, or sharpening or anything like that - I can do all that easier with AE, so the only thing of interest from that script is the stabilization but for me that script is far too complex to be able to extract just the stabilization part from it.

Author
Time

Harmy said:

the main thing that I need and that is removing spots and dirt.

This is rather difficult to automate. Even programs like PFClean mess up... a lot. I would know, as I've spent many, many hours dirt cleaning with PFClean.

With that being said, the best Avisynth plugin for this is the old RemoveDirt plugin: http://avisynth.org.ru/docs/english/externalfilters/removedirt.htm

If you want to do it right, then you'll have to do it shot-by-shot, using show=1 to make sure that it didn't clean a block it shouldn't have.

I suggest studying the hell out of the documentation. You really need to get a good sense for how the algorithm works in order to choose the right values.

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time

AntcuFaalb said:

Harmy said:

the main thing that I need and that is removing spots and dirt.

This is rather difficult to automate. Even programs like PFClean mess up... a lot. I would know, as I've spent many, many hours dirt cleaning with PFClean.

With that being said, the best Avisynth plugin for this is the old RemoveDirt plugin: http://avisynth.org.ru/docs/english/externalfilters/removedirt.htm

If you want to do it right, then you'll have to do it shot-by-shot, using show=1 to make sure that it didn't clean a block it shouldn't have.

I suggest studying the hell out of the documentation. You really need to get a good sense for how the algorithm works in order to choose the right values.

 

Harmy said:

...it doesn't really matter, if it mistakenly erases other details, like stars and stuff like that, since I would then only utilize the cleaned up version for manual cleanup in After Effects.

Author
Time

Harmy said:

I still couldn't currently afford it for that price but even if I could, like I said, I tried it and couldn't get it to work - It got really frustrating after hours of unsuccessfully trying to get it to do anything at all.

PM sent. :-)

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time

Harmy said:

...it doesn't really matter, if it mistakenly erases other details, like stars and stuff like that, since I would then only utilize the cleaned up version for manual cleanup in After Effects.

Ah, sorry. My reading comprehension is pretty bad at times. RemoveDirt might be the way-to-go then.

Try it with its default values on a dirty sample clip you have. Post here and I can more-or-less help you in real time.

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time

For stabilizing, I use the Virtualdub filter called Deshaker.  Actually I use it a LOT, on both old film and old video.  It's slow, but it really does a nice job. There are a lot of arcane settings, so if you have any questions on how to use it, or what settings I use for different situations, feel free to ask.

"Close the blast doors!"
Puggo’s website | Rescuing Star Wars

Author
Time

Warp Stabilizer effect in AE is a pretty decent stabilizer. It can even do content aware edge recreation to avoid cropping.

Author
Time

Yeah, it's good for shaky camera but it totally sucks for gate weave.

Author
Time

Just to be clear, you've tried position-only stabilization (no subspace warp), correct? Seems like that would be excellent for gate weave once tuned a bit.

Author
Time
 (Edited)

Yeah, I did. The problem is, it still gets thrown by the action in the shot, as tries to get the moving objects to stay stable, because it was designed for stabilizing camera moves. Plus there is actually very little tuning that you can do.

Author
Time
 (Edited)

In my opinion, a motion compensated dfttest is the best denoiser at the moment. It requires dfttest and mvtools2 if I recall correctly. Paste this into the bottom of any script and it should work as long as you have the dfttest and mvtools2 installed and autoloaded in the plugin directory.

function dfttestMC(clip input, clip "pp", int "mc", bool "mdg", bool "Y", bool "U", bool "V", float "sigma", int "sbsize", int "sosize", int "tbsize", int "dither",

\ string "dfttest_params", int "mdgSAD", int "thSAD", int "thSCD1", int "thSCD2", int "pel", int "blksize", int "search", int "searchparam", int "overlap", int "dct", bool "lsb")

{

# Set default options. Most external parameters are passed valueless.

mc = default(mc, 2).min(5)

mdg = default(mdg, false)

Y = default(Y, true)

U = default(U, true)

V = default(V, true)

tbsize = default(tbsize, mc * 2 + 1)

dfttest_params = default(dfttest_params, "")

overlap = default(overlap, 2)

lsb= default(lsb,false)

# Set chroma parameters.

chroma = U || V

plane = U && !Y && !V ? 1 : V && !Y && !U ? 2 : chroma && !Y ? 3 : Y && chroma ? 4 : 0

# Prepare supersampled clips.

pp_enabled = defined(pp)

pp_super = pp_enabled ? MSuper(pp, pel=pel, chroma=chroma) : MSuper(input, pel=pel, chroma=chroma)

super = pp_enabled ? input.MSuper(pel=pel, levels=1, chroma=chroma) : pp_super

# Motion vector search.

b5vec = MAnalyse(pp_super, delta=5, isb=true, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, dct=dct)

b4vec = MAnalyse(pp_super, delta=4, isb=true, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

b3vec = MAnalyse(pp_super, delta=3, isb=true, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

b2vec = MAnalyse(pp_super, delta=2, isb=true, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

b1vec = MAnalyse(pp_super, delta=1, isb=true, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

f1vec = MAnalyse(pp_super, delta=1, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

f2vec = MAnalyse(pp_super, delta=2, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

f3vec = MAnalyse(pp_super, delta=3, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

f4vec = MAnalyse(pp_super, delta=4, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

f5vec = MAnalyse(pp_super, delta=5, chroma=chroma, search=search, searchparam=searchparam, overlap=overlap, blksize=blksize, dct=dct)

# Motion Compensation.

b5clip = MCompensate(input, super, b5vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

b4clip = MCompensate(input, super, b4vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

b3clip = MCompensate(input, super, b3vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

b2clip = MCompensate(input, super, b2vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

b1clip = MCompensate(input, super, b1vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

f1clip = MCompensate(input, super, f1vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

f2clip = MCompensate(input, super, f2vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

f3clip = MCompensate(input, super, f3vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

f4clip = MCompensate(input, super, f4vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

f5clip = MCompensate(input, super, f5vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)

# Create compensated clip.

interleaved = mc >= 5 ? Interleave(f5clip, f4clip, f3clip, f2clip, f1clip, input, b1clip, b2clip, b3clip, b4clip, b5clip) :

\ mc == 4 ? Interleave(f4clip, f3clip, f2clip, f1clip, input, b1clip, b2clip, b3clip, b4clip) :

\ mc == 3 ? Interleave(f3clip, f2clip, f1clip, input, b1clip, b2clip, b3clip) :

\ mc == 2 ? Interleave(f2clip, f1clip, input, b1clip, b2clip) :

\ mc == 1 ? Interleave(f1clip, input, b1clip):

\ mc == 0 ? input:

\ input

# Perform dfttest.

filtered = Eval("dfttest(interleaved, Y=Y, U=U, V=V, sigma=sigma, sbsize=sbsize, sosize=sosize,tbsize=tbsize,dither=dither)")

return SelectEvery(filtered, mc * 2 + 1, mc)

}

I only ever use the sigma and mc (motion compensation) parameters. I don't go too high with the mc parameter because it sometimes introduces some artifacts with high motion scenes.

Something like this:

myfilmclip.dfttestmc(sigma=8,mc=1)

Fred's 8mm script is probably the best way to do automated dirt cleanup. You can disable almost any portion of the script and use only what you want.  He's got it all laid out in the appropriate order so you don't do any processing out of order. What method did Laserschwert use?  That seemed to work really well.

Luke threw twice…maybe.

Author
Time

Deshaker has a ton of tunable settings.  I sometimes set the maximum movement to a very small number, and I also limit the motion sensing area to a portion of the screen that I know doesn't have any action in it.

"Close the blast doors!"
Puggo’s website | Rescuing Star Wars

Author
Time

If we're just going for general denoising, then I'd recommend SMDegrain (a nice wrapper around MDegrain1/2/3) for Avisynth or Innobits Video Purifier if you want to go the commercial route.

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time

Puggo - Jar Jar's Yoda said:

Deshaker has a ton of tunable settings.  I sometimes set the maximum movement to a very small number, and I also limit the motion sensing area to a portion of the screen that I know doesn't have any action in it.

FWIW: I spoke with the author of deshaker sometime back regarding fixing the gate weave in the GOUT. His assessment (after I sent him a sample) was that another tool would probably be best to deal with the kind of slight subpixel motion that gate weave introduces.

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3

Author
Time
 (Edited)

Yeah, I learned the hard way, that the order in which you do things is very important - e.g. you definitely want to do any despotting before doing any denoising, because when I had Laserschwert do the denoising on the clips first and then I went in and manually painted out the spots, all the spots now stayed for 3-5 frames instead of one, which was a pain in the ass to paint out manually and I imagine, that it would totally throw any automatic despotter.

And I suppose, you'd want to do the stabilization first too.

Thanks for all the suggestions guys. I currently don't have any of the new clips yet, so I think I will start experimenting with this, once I get some - I just wanted to get ahead of it.

Author
Time
 (Edited)

Adding to this I must say that my denoising is possible the most lazy thing imaginable, as I simply use MC_Spuds at it's default setting (so no further parameters). Note that the filter needs a YV12 source, so you have to use "ConvertToYV12" before this filter, and "ConvertToRGB24" after it.

I just love how much detail it keeps, but as Harmy pointed out, despotting beforehand is highly recommended. Unfortunately this is very difficult do to when dealing with stuff like starfields, as those are bound to be reduced by a fair amount of stars in the process.

For shorter shots it's advisable to remove the biggest dirt pieces manually. After Effects' clone brush is very cool in that it allows the offset for the clone brush to not only be set as x and y postions, but also as frame offsets. So usually I just leave the position offset at 0,0 and the frame offset at -1 or +1 (depending on which end of the clip I'm working on). By that I can just stamp the same area from one frame earlier (or later) onto a piece of dirt, which is usually close enough in brightness and coloring to work.

Author
Time

Laserschwert said:

Unfortunately this is very difficult do to when dealing with stuff like starfields, as those are bound to be reduced by a fair amount of stars in the process.

Yep. This is why I prefer to clean manually in PFClean.

A picture is worth a thousand words. Post 102 is worth more.

I’m late to the party, but I think this is the best song. Enjoy!

—Teams Jetrell Fo 1, Jetrell Fo 2, and Jetrell Fo 3