poita

That is correct, no NTSC playback on the modified unit.

I have a Crystalio II for sale if anyone wants it, I liked it better than the HDQ for LD sources, but I was watching on a projector, so that may have been part of it.
I converted my X9 to PAL after a lot of dicking around, it was noticeably better than the 2950, but replacing the power supply components with better ones in the 2950 improved its picture quite a bit.
I still have a mosquito unit kicking around too, I really need to ebay all my LD stuff.

ElectricTriangle said:

While the resolution is often worthless, the nice thing about the 4k Blu ray standard is the increased color depth. Less banding is always good.

Yes, massively increased colour gamut and dynamic range make it worthwhile, regardless of how you feel about resolution increases.

Yes, emulsion scratches are usually green, can be other colours depending on the depth of the scratch, base scratches are usually white if clean, or black once they get dirt in them.

Feel free to do so, I won’t be using any output created from the community for what I am working on, but I’m more than happy for people to try their own thing on it. I tend to find Photoshop fixes result in weird crawling generally, but it sounds like you have a good system, so by all means give it a shot.

You will need to PM me for access to the full files, but here is a short 1080P sample
https://infinit.io/_/VTNTxB6
I’m rendering it out fairly flat, with the gamma a bit high, to give people maximum latitude to correct it how they see fit.

Colson said:

Sorry guys. I haven’t found a good way to record HBO Now stuff. Any recommendations? Ideally, I’d be able to download the source feed without screen capping, but screen capping seems to be the most likely route.

A HDMI splitter and HDMI capture card will usually do the trick.

Reel 2 is done:
https://infinit.io/_/3n6LGiz

Anyone that donates, or has donated to something in the past, or can help in some way with feedback, or just has an interest, please drop me a PM and I’ll send you an invite to my file repository, I think I have already stretched the amount that the mods would be happy that I share via open links.
We are very close to having enough to get a third reel cleaned, so I am sending it off.

I am currently rendering out Reel1 at 1080P 10bit high bitrate for anyone that wants to take a crack at colour/repair/general fiddling. It will be a lot smaller than the 5TB raw file, but will still weigh in at around 26GB.

Thanks again to everyone helping out.

Swazzy said:

Would this do any good for laserdisc captures? (i.e. the neglected, unaltered version of Millennial Garbage Movie '98)

Yes, stacking frames can really help with laserdisc and VHS captures, it can get rid of player-induced noise, and if you have multiple discs, then it can help remove dropouts and pressing errors, while reducing the chroma noise etc.

Can’t see it on the Blu-ray at first glance, but their blacks are set a lot lower.

Jeez, we haven’t even seen the scan yet 😉

Oh shit, I mentioned Adywan…

If you read the text, the drizzling is only on the final images, I did that as an example of what drizzling was (or to use the proper term “Variable Pixel Linear Reconstruction”), as I thought many had probably not heard of it, and this was a discussion about image processing.

All the previous images are without the upscale/drizzle applied. The whole thing was an example of a technique, as I said, we don’t have enough full prints to do it to this level, you would need 20 or more prints to get this level, and not all scenes would work as well as the alignment gets tricky etc. and you can lose detail in the alignment process, so diminishing returns kick in.
It certainly isn’t what I am doing with the movies, I don’t have enough prints, and don’t have enough computational grunt to do the alignment and stacking, that single image used 63GB of RAM to process. Drizzling isn’t likely to be of much use for film captures, but I mentioned it as part of the discussion and applied it to see the results. I make no claims to be magic.

Normally, I wouldn’t have enough prints to do this, I don’t have 30 prints, but what I do have is a reel of dailies of this scene, I also have a reel of dailies from Empire, if I get time I will scan it and show it as well, if it is of interest to people here. The dailies reels have the same scene looped a dozen of more times, which is very handy for this kind of thing. I picked a scene with a static camera as (a) I already had it scanned, and (b) a scene with movement has motion blur which wouldn’t show the effect as clearly.
I used 3 adjacent frames from each dailies repeat, and an two IB prints and an Eastman, they were all colour matched and roughly aligned. I had all of that handy from a prior experiment, so could get an explanation up quickly.

Film does work well with this technique, due to the randomness of the film grain, the film weave means each print is slightly differently positioned, and the photons hit different areas of the grain etc. so if you have enough, you can get back much closer to only having neg grain involved. The film is, in my opinion, under sampled when scanned, and multiple prints and scans lets us recover a lot, and remove a lot of noise.

I’m not trying to peddle hope or anything else here, just joining a discussion of super-resolution and frame stacking with a freely admitted arbitrary example. If anyone has the misconception that we are doing this to the entire film, then it is just that, a misconception.

Where this does come in handy is helping to discover the correct colours without incorporating artefacts such as the yellowing blobs and grain structure, which gives a good colour target when grading, and working out what is dirt/micro-scratches and what is not, which is not always obvious, like those dark spots on the sand, they could easily be mistaken for dirt spots and incorrectly removed.
It is also handy for efforts like Harmy’s and Adywan’s, where they might want to recreate a background plate in far more detail than any single frame allows, like the sand crawler coming over the sand dune, the starry background and planets from the crawls, and many other shots, where they can create a much cleaner version and use that as a basis for the reconstruction.

Having a clean plate we also find really useful to get a more accurate grain analysis to use for de-graining algorithms.

Anyway, it is intended only as a discussion of image processing technique, not as anything specific that I am doing re the restoration of any particular film.

Spaced Ranger said:

poita said:

I’ve talked to Ben Burtt and those scenes never even had the score laid down, or proper sound effects done, … when I ask if another cut could have been made with extra scenes added or removed from Star Wars, they literally laugh out loud.

Okay, you can tell us then … Luke did throw his grappling hook … twice, right? First time he missed and stepped back to reloop as the Princess traded shots with the storm troopers. Second time he hooks it. Right?

Yep, and somehow they cut that scene a bit later without affecting the score.
Funnily enough, people really remember the second throw clearly, as in, they can picture it all, and their Uncle turning to them and saying ‘Whoopsie’ and laughing, so much detail, but can’t remember exactly how it went when asked, like did he haul it back in, was there another cutaway to stormtroopers and he just had the hook in his hand again, did Leia and him exchange worried glances, did he loop the rope or let it hang in a large loop etc.

The SSD has been replaced, the Shining scan should be arriving this week.
Joy!

Maybe they are trying to disguise the fact that the wear and scrapes on Luke’s canopy are, through an amazing co-incidence, exactly the same scrape and wear marks as everyone else’s speeder canopies. ;^)

Some interesting images have been spotted by sharp eyes amongst the OT crowd.

Look at the top corners of Luke’s canopy, there are what look like hand drawn squiggles in Permanent Texta/sharpie/marker dancing around for a few frames, and then again in another scene.

A failed attempt to address the transparency issue? A bored employee? Or…

camroncamera said:

poita said:

Oh yeah, normally you would do a ton of pre-processing before stacking, and then more cleanup after.
I literally knocked that out in about 15 minutes, 10 minutes of which was computation time, so it is a poor example, I just wanted to illustrate the technique.
Even doing it quite poorly gave enough of a result to show how it works.

Poita just open a film restoration school already, you know we’ll all enroll 😃

My knowledge is about 1/10th of what I would like it to be, there are some people out there that really know this stuff.

Often saturating the colours exposes the incorrect grade, if the shadows have a slight tint, you don’t see it except on the scopes, but boost the saturation and you can see what colour tints are lurking.

Dreamaster said:

poita said:

I’d be willing to bet my left nut that if we had the grindhouse print rescanned, with the light offset to deal with the fade, that the colour would be basically the same as the UK print.

LOL… I had a vasectomy two weeks ago; your choice of words have me both smiling and cringing at the same time.

As much as I hate to admit it, a lot of the evidence leads me to similar conclusions but my brain won’t let me believe. As if even if Dr. Who picked me up and took me back to 1980, and we see blue in the cave I’d accuse him off rigging the film before we got there! Because no matter how much I replay the memory fragments of me seeing that film at the drive in theater as a 5 year old I just can’t picture it that way.

I guess with that said, if a moderator feels the need to move this thread into the “Fan Edits” forum I won’t object. Heck… I woke up this morning thinking about your first response… I might just release it over-saturated as Empire: The Wonka Strikes Back edition. 😃

Yeah, our memories are funny things, and our memory for colour is even worse than our memories for events and facts. We remember colours based on what we think an objects inherent colour is. So we remember grass in a film being green, even when the colourist has shifted it to grey or nearly blue.

The amount of people that try to convince me the Tatooine scenes with Luke with the binoculars and Biggs at the cantina etc. were in the version they saw in 1977 is pretty amazing, and I’ve talked to Ben Burtt and those scenes never even had the score laid down, or proper sound effects done, they were only ever cobbled together for a screening for George and some pals after they complained that the film was too hard to follow without a bit more Tatooine background info.

There was simply no possible way that those scenes got into a release print, they were never finished, never scored etc. but people are convinced they saw them in the cinema.
People state their memory is so clear that they can remember comments they made to their friend at the time, what they were eating in the cinema and a slew of other details. But when I ask them what scene was directly before and after it, they suddenly get hazy, when I ask them what they or their friend was wearing, they can’t remember any of that either, asking anything else around their razor sharp memory of the event and oddly those details are also not there.

I’ve been told by multiple crew members that the scramble just to get the cut to the cinemas on time was intense, when I ask if another cut could have been made with extra scenes added or removed from Star Wars, they literally laugh out loud. “When the fuck would we have had the time to do that?” is pretty much the reply. And then the cost of getting another negative conformed and cut and printed and distributed?

Anyway, I’m off topic, the main thing I have discovered is that our memories are terribly unreliable, I have no memory of all the dirt and shit in the composite shots, I thought they looked fantastic at the time 😃

I’m more than happy for anyone to make their own version of empire, with the colours they like and with a level of cleaning that suits them. Personally I’d love my own personal version of Jedi with the Max Rebo band excised, but my main aim is to try and get the original films scanned and preserved, as close to the way they were at release as possible, and answer all those questions about matte lines, colours, errors etc. definitively, so that the information is not lost to time.
There is so much interesting stuff in there, like numbers written across frames, bits of black marker on a few frames, seemingly in an attempt to cover a compositing flaw, and lots of other little insights into the process of film-making at the time.

Sorry if I came across as harsh with my comments re colour, I’m all for people messing with this stuff, my entire career though has been in an environment where people critique the shit out of everything, with no malice or rancour, just to help get the job done to the best standard possible within the constraints, and I forget sometimes that outside production, criticism is taken differently 😃

Spaced Ranger said:

poita said:
I’ll whip up a quick example for a scene I have multiple prints of.

About that before/after demonstration … S T U N N I N G! I knew of this technique (from motion-compensated temporal “frame stacking” – but with the downside of noticeable temporal “grain crawl” at too-wide a range) but never thought to apply it in your manner (for multiple-source frame stacking). Absolutely stunning.

Even still, the blue layer always seems to be the worst hit …

Would any pre-processing (maybe some temporal stacking) help on the worse layer(s), while not negating the source stacking effectiveness?

.

NeverarGreat said:
In this example, there are only about 6 dye cloud centers, meaning that only 6 pixels are assigned values. With enough sources, the entire pixel grid can be filled in. … I’m the 1% inspiration guy …

That is excellent (and I love your visualizations)! The interesting thing is that this approach also might be useful in things like up-rez via vector reconstruction (must hit the math books for that one) or sharpening via vector calculus boundary redistribution (more math books).

Oh yeah, normally you would do a ton of pre-processing before stacking, and then more cleanup after.
I literally knocked that out in about 15 minutes, 10 minutes of which was computation time, so it is a poor example, I just wanted to illustrate the technique.
Even doing it quite poorly gave enough of a result to show how it works.

The blue channel is always the noisiest, on film or CCDs, and usually requires the most work. It also is the colour where detail matters the least for the way our eyes work, and can be much softer than the other channels without the softness being perceived by our eyes/minds.

I’ve had a look at the pre-scan of Reel2, there is some great colour in there, but unfortunately a bit of red-blob damage particularly in the second half of the reel. Nothing that can’t be fixed, but annoying that it is there when the film is otherwise in such nice shape.

NeverarGreat said:

I’d never heard of Drizzle before

Awww, and I thought you read my post 😦

😄

Yeah, if it is captured on digital, then a single image isn’t going to be helped, as every source will be from the digital master.

With prints scanned to film however, I think you end up with the same result, by stacking images correctly, you throw out the noise and are left with the information. With enough samples the chances of only that ‘single’ pixel being correct is basically nil, or at least, less likely to happen than the chances of, for example, incorrectly identifying the centre of a dye cloud.

I’d be interested to see this play out though, I’ll go through my siggraph library to see if anyone is already doing this, there was a lot of research into this kind of thing in the late 90s and early 2000s, there might be something in there. From a programming point of view, it wouldn’t be difficult to implement.

Dreamaster said:

poita said:
Your monitor is off because the cave is green, Luke is an Oompa Loompa, and Yoda is CRAZY green.

But the question is Poita… how do you REALLY feel about it? 😃

My monitor’s not too far off the snow is green there for me too. But it’s blue a few frames before! I purposefully cranked the saturation to see if that snow would change much, and it really didn’t. I have a conspiracy theory that there were two color timings made in film for ESB… which explains why the home release and the grindhouse are so different than your scans and the 16mm release (which was CRAZY blueberry blue!) but I got no real proof. But the other reason I cranked it so hot was I wanted to see just how much color was in that print. Quite a bit.

The colors are NOT consistent from shot to shot, but I don’t currently have the skills like you and Neverargreat to do a shot by shot adjustment, but I still feel like I can get a big enough improvement make it “worth it” overall. Honestly, with time it’s obvious some better releases are going to hit in the future, I am really looking forward to your new scans! Just playing while we wait for the best that’s yet to come.

No, definitely not two timings made on film.

However the home releases were done on a real time telecine, and that is colour corrected in realtime by the telecine operator. It is extremely likely that the telecine operator decided to try and ‘neutrally balance’ most scenes on the fly, it is what many would do, and that would leave you with something closer to greys in the snow etc.
The 16mm is a crapshoot, the blue on that is just a lab problem.

As for the grindhouse, it was from a very faded print, and colour corrected to their personal taste, doing the best they could with the faded material, and a scanner that doesn’t have as good a dynamic range. Their light source is also spectrally notched, and is missing complete wavelength bands, so they were up against it.
I’d be willing to bet my left nut that if we had the grindhouse print rescanned, with the light offset to deal with the fade, that the colour would be basically the same as the UK print.

The main problem is getting enough prints, to get really great results you would want 20 or more.

Though even having three can make a good inprovement.

Ultrasonic cleaning is very effective and safe for all types of film.

Filmguard does nothing to offset vinegar syndrome, it quite possibly makes it worse, not allowing the film to breathe. Filmguard is great for prints in good condition, it makes them run more quietly and smoothly through a projector, and minimises the risk of further scratching or damage during projection.

Here is an example of an ultrasonic film cleaner in action.

https://vimeo.com/133193190

“This is not possible with a digital video signal, since again, there is no way to tell which pixel is more likely than another to contain detail rather than noise.”

If you have enough samples, you can statistically work out which pixels contain noise and which contain information, that is the basis of image stacking. The signal to noise ratio is lowered by the root of the number of images.

Or do you mean images originally captured on a digital camera, not digital scans of film based capture?

I feel that scanning multiple prints at high resolution, and then stacking them achieves the same thing, but I’m probably missing something in my understanding here.

I’d love to see a proof of concept from your work, I’m happy to provide you with some higher resolution images, something like 6000x4000 pixels should be enough to work with?

I’m all for any technique that might improve the image.