Star Wars 1977 releases on 35mm — Page 26

Each scan is around the 20TB mark in its raw state. That is correct, at the moment I don’t have nough drives to keep backups of some of them.

20TB? What specifications are you scanning them at?

poita said:

Yes I have to drive between 2 and 6 hours each way, depending on the hospital I need to travel to, and it does usually mean extended stays away from the family. That and recuperation time can put a lot of stress on the family unit, when there are 3 kids to get ready and to and from school, one with special needs, and a boisterous 2 year old to contend with.

Yeah, wow.

I wasn’t meaning to pry, I’m very sorry to hear about that poita. I was just wanting to let you know that while I can’t imagine what you’re going through I’m somewhat aware of the challenges that are faced. : ( But I’m really pleased to hear that everything seems to be going well at the moment. And of course thanks for sharing! : )

FrankT said:

20TB? What specifications are you scanning them at?

It depends, and I’m not sure exactly what you are asking, but typically Full aperture 4K, 16bit log or 32bit linear depending on the scanner, and when possible with an IR pass for the damage matte, that is stored in the Alpha Channel.

So, somewhere around 4300 x 3200 resolution, in ACES colour space, saved in .DPX format or .EXR
Some are done in other colourspaces, particularly when I haven’t been the person running the scanner.

Ah, that’s understandable.

For those of you buying hard drives to donate, consider looking up some reliability stats. Of course backups are still important, but the data has to exist long enough to be backed up in the first place. Odd that the Seagate 5TB are so bad considering how they improved from 3TB to 4TB.

With the manufacturing consolidation that’s been going on since the flooding in 2011, it’s worth looking back now and then to see what’s changed in the industry.

I’ve only bought 2 seagate hdd’s in my life and they both failed in under 2 years with normal usage. I’ve never had a WD fail despite years of operation.

I’ve had a bunch of WD RED drives fail, the Seagate 4TB drives have all been good, and some are now more than 3 years old. The 5TB HDDs have been atrocious for me, but it may be that they are fine as long as they aren’t moved around. It could be shipping them that causes the problem.
Either way, the seagate 4TB HDDs are what I have settled on going forwards.
If anyone sees them for under USD100, let me know.

All HDD manufacturers have a percentage of drives that fail. People tend to go off a brand that they had a catastrophic failure (or 2) with, but the reality is, it can happen with any of them.

I concur on the checksums. With ~380TB of data, silent data corruption is something to be especially vigilant about. While it may be inconvenient, people should look into using filesystems like ZFS with end-to-end checksumming, error-correcting memory, proper shielding, as well as stress-testing their (non-overclocked) systems periodically to keep silent data corruption to a minimum. Hard drives can also be stress tested before holding actual data using utilities such as badblocks.
Film, tapes, laserdiscs, and players don’t last forever; eventually the digitized copy will be the better one in comparison and so data integrity is very important going into the future.

Once I get the scans done, they are transferred to RAID5, and once I have enough drives, onto an external RAID5 that is stored offsite.
This gets time consuming and expensive.
In all honesty, if I could afford it, I’d do a film-out back to 35mm, it is likely to outlast digital copies, especially with the new breed of archival film. In a hundred years, you can still utilise a well stored 35mm print, whereas it is getting difficult to access digital data from just 20 years ago.
Digital archives require constant maintenance and checking for corruption, and transferring to new media every few years, a good film archive just has to be kept cool and relatively dry.

At the moment LTO is a reasonable option.

I haven’t visited these forums in a long time, a long time.

This thread has made my day, I’m more excited to see the results of this project then I am for Episode VII!

Yeah, me too! 😃

Good to see you back btw 😃

I guess SSDs will be cheaper and bigger in capacity in a few years, I won’t be surprise if by 2015 we start seeing 2.5’’ 4TB SSD drives for $200 or less!

dclarkg said:

I guess SSDs will be cheaper and bigger in capacity in a few years, I won’t be surprise if by 2015 we start seeing 2.5’’ 4TB SSD drives for $200 or less!

With only 9 weeks of 2015 left, and the current 2TB SSDs clocking in at USD750 at the cheaper end, I can’t see 4TB of SSD for $200 happening in the next two months 😃
Would be nice though.

Yeah, SSD will always cost more anyway, it’ll remain cheaper to buy 2x disc-based HDDs than 1x SSD, which means the disc-based HDDs will remain better for archiving, although they might of course get considerably cheaper in the future. SSDs are better for running your system from, but probably not suited towards archiving, and I predict in the future that will remain so - at least for quite a few years from now. 😃

Well, SSDs are actually far less prone to mechanical damage, since they have no moving parts - they do get damaged by repeated re-writes but if you’re archiving, you wouldn’t do that, so if they come down in price considerably, they could be better for archiving.

Harmy said:

Well, SSDs are actually far less prone to mechanical damage, since they have no moving parts - they do get damaged by repeated re-writes but if you’re archiving, you wouldn’t do that, so if they come down in price considerably, they could be better for archiving.

They’re also much more difficult to recover data from following a failure. But of course, in the future if they come down in price very considerably they’d be better for archiving.

Unfortunately recent testing ahs shown that SSDs left on the shelf experience data corruption after a while, so they need powering up and refreshing, which is a pain.

http://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention
http://www.extremetech.com/computing/205382-ssds-can-lose-data-in-as-little-as-7-days-without-power

Well, that sucks.

poita said:

Unfortunately recent testing ahs shown that SSDs left on the shelf experience data corruption after a while, so they need powering up and refreshing, which is a pain.

http://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention
http://www.extremetech.com/computing/205382-ssds-can-lose-data-in-as-little-as-7-days-without-power

Alternatively, a company can make a giant SSD dock that keeps them powered on and not much else.

Personally I find that Samsung’s HDDs are good, but I only have one 500GB drive and two 1TB drives from them, so… it’s hard to say.