Long Term Storage of Astro Data

February 04, 2013  •  Leave a Comment

I'm sure many of you face the same challenge as I do - your Astro Photo data are starting to take up a lot of space on your various hard drives. Besides the storage space issue, developing an effective backup plan is difficult due to the extremely large volumes of data involved. Even though I clean out a lot of the intermediate processing files when I finish a project, just the raw LRGB frames, plus calibration frames, narrowband, etc, can easily add up to many gigabytes of data in a single project.

I back up everything to a Time Machine drive on my iMac, plus once in a while I copy everything to a portable hard drive that I take offsite. For a backup to be truly effective, you need multiple copies on multiple devices, with at least one of them offsite in case of a disaster like fire, earthquake, or meteor strike.

I've experimented with cloud storage, but Dropbox is fairly expensive for large data volumes, and until now I haven't been satisfied with any of the other cloud storage solutions. Recently, however, Amazon announced their Glacier storage product, which is specifically designed for long term, low cost, infrequent access data backup.

How low cost? How about $0.01 per GB per month- yes, a penny a month per GB. So far, I've uploaded about 100GB to Glacier, so that's costing me $1 per month. It's redundantly stored at Amazon, and I have no worries that Amazon won't still be around in a few years. There are a couple of downsides however:

Number 1 - Glacier has only an API access through Amazon Web Services (AWS). However, there are starting to be really good backup software apps that can access Glacier for you. I'm using a product called Arq from Haystack Software. It's $29, with a simple interface, and seems very robust. It's Mac only, but I'm sure there are equivalent Windows products around.

Number 2 - Retrieval access can take a day or so - it really is designed for remote long term storage that you won't need to access very often. In addition, there are actually penalties for uploading and then deleting data a few days later. The entire pricing structure is a bit strange, but if you're interested, check out the pricing page on Amazon. For my purposes, I probably won't ever need to retrieve any of the data except in unusual circumstances, so it fulfills my need for a different device/offsite backup.


Comments

No comments posted.
Loading...

Archive
January February March April May June July August (1) September (1) October November December
January February (2) March April (1) May June July (1) August (1) September October November December
January (1) February March April May June July August (1) September October November December
January February March April May (1) June July August September October November December
January February March April May June July August September October November December
January February March April May June July August September October November December