I found some interesting solutions to compare image frames against a video using PSNR. I am going to play around with scripting this a bit and see if I can get something working.

I am hoping I can preselect all the key frames to search for and then do a single pass search against the video to get a lot of the correct frame matches. This would speed this up so much.

Edit: I got something working for a couple thousand frame videos. If the video is longer then that then the avisynth script crashes. It didn’t return an exact match but it got within ~15 frames in the tests i did. I am not sure how to track down the cause of the crash so back to just finding the cuts.