Category: Tech

I recently decided that I needed to begin backing up my multiple 3-4TB drives. I had recently copied files off the pile of smaller drives and discovered a few of them weren’t working very well any more. I’ve also begun archiving all of my DVDr and CDr backups.

I initially tried out Bluray BD-R XL 100GBdiscs.. while I like these because they are easy to mount and get files from it takes forever to backup to them and you have to split your file selections over many discs. Nero 10 will span your file selection over many discs but one problem I found is that when the number of discos you are

spanning is as many as 17 Nero becomes unstable. If it fails in the middle of the process you have to start over and it doesn’t remember what dis you were on.

To get around this I wrote out iso images to burn later. Unfortunately Nero does not automatically write out the incremented images and you have to babysit the process. Once I had all the iso’s it was merely a matter of burning all of them. All of this was way too much work considering the total amount of data I wish to back up.

Therefore I started looking into tape drives. For used drives LT05 seemed to be low priced and fairly capacious at 1.5 to 3TB(compressed). However considering that most of my files are already compressed I can really only expect to get 1.5 TB per tape and my individual drives had around 3TB on them. In hindsight I’m thinking maybe I should have just went for LTO 6 2.5/6.25TB. I managed to get an external Quantum LTO5 drive off ebay for around $400 .. checking now s LTO6 was at least a grand from Canada..

The next part of the saga was getting a PCIe SAS card.. after some trial and error (thank you Amazon Prime) I finally got a $90 LSI SAS card(LSI SAS 9207-4I4E) that was compatible with the Quantum drive. I had previously bought a LSI 9200-8E that was branded as a 10Gtek but as it turns out Broadcom has bought 10Gtek, LSI and several other companies and no longer offers the drivers for download.

The next issue was what backup software?? Well after much searching I finally found a relatively inexpensive option that will span the archive over multiple tapes. Much to my surprise this is not a standard feature in all backup packages capable of writing to LTO. Z-dbackup does work although my concern moving forward is that 5-10 years from now if I need to get files off these tapes that I may not beable to without  the same software.

Well one of the cool things about LTO is LTFS which makes it possible to mount the tape as if it were a regular harddrive on your desktop using free software.
So I’m considering this as a future proofed backup solution. However I’m back into the same search of back programs that will split the back up over several volumes and so far I have not found anything that does this.

I’m currently considering writing my own scripts using Python that will scan a drive store checksums and intelligently split the file selection over multiple volumes and simply prompting the user to point it to the new volume and inform it as to how much space it has available.

Some links..

LTFS Smart Copy Command Line Options

http://downloads.quantum.com/ltfs/3.0/6-67740-04_LTFS%20for%20Windows_UsersGuide_RevB.pdf

https://stackoverflow.com/questions/3431825/generating-an-md5-checksum-of-a-file

http://wculver.com/webdev/django-python-storing-a-sha256-hash-of-a-file-as-a-base64-encoded-string/

https://stackoverflow.com/questions/19714342/copytool-with-md5-sum-verification-python

https://github.com/clouserw/scripts/blob/master/md5verify.py

sha256 file checksum function in python
https://gist.github.com/eas604/b7e525fc135723028aed

http://pythoncentral.io/hashing-files-with-python/

 

One thing I didn’t realize until recently is that the LTO tape cartridge is just 1 reel but unlike an 8 track tape it is not looped (not surprisingly since 8 tracks are terrible) and the deck grabs the end of the tape and pulls it inside and attaches it to an internal reel.