Some time ago an HDD I had quite a while died. The worst part is, that it contained the backups. It worked for about two years in a fashion where it gets powered once in a while via a mosfet, receives a payload of backups and then goes offline again. Analyzing the corpse it looks like it was the spindel motor. Well done, WD!
This made me thing of another way to store critical backups. Consider me paranoid, but I do not trust cloud services. Then I saw an old cdrw disk wasting its remaining life as a stand for the coffee cup.
At first I wanted to make backups on a multisession CD/DVD, like puppy linux does. Write data to the back, and assemble everything with unionfs. This way had its drawbacks: You have to change disks quite often, it required quite a lot of scripting and etc. Luckily I remembered of a cool feature called packet writing. I failed to make it work 7 or 8 years ago under windows. (Yep, I used to be a windows junkie then).
Since setting up this stuff in Linux was not trivial, I decided to make a note about it here. So, we’ll need a drive that can handle packet writing (most of them do. Or nearly all.) And we’ll need udftools package. This stuff was created in 2002, and didn’t change much since then, so you won’t be able to compile it without a bunch of maintainer patches. ABUILD of udftools and the required patches are in my github repo, Agilia Linux users can grab the prebuilt package from stable.
Next we’ll need a medium. A cdrw or dvdrw will do. Since the cdrw used for the coffee stand was too old, I got myself 2 cd-rws and 2 dvd-rws for testing and fetched 3 DVD-RW drives from the attic. One even had a working laser.
So, what will we need?
udftools, bash and pktcddvd kernel module.
First we’ll need to set up the device. The main trick here – do NOT use /dev/sr0 directly. Since UDF will mount, even in rw fashion, the data written will be currupt and you’ll see tons of io errors in dmesg. As I got the theory – random writes should occur in packets (32k?), should be alligned. Otherwise – io error.
Now, the receipe:
First, we’ll need to format the cdrw. This will also call mkudffs and create an UDF filesystem.
cdrwtool -q -d /dev/sr0 |
For DVD+/-RW use dvd+rw-format. To do packet writing we need to switch the disk to Restricted Overwrite Mode instead of Sequential, which is done somewhat like that (They say that’s not needed for DVD+RW, only for DVD-RW):
dvd+rw-format -force=full /dev/sr0 |
And now goes the voodoo!
First, we need to setup the packet device:
pktsetup pkt0 /dev/sr0 |
And use /dev/pktcdvd/pkt0 for all operations (you can use any name instead of pkt0).
This way everything will work smoothly.
To unmount the disk you have to cast this spell:
umount /media/bck sync pktsetup -d pkt0 eject /dev/sr0 |
sync is not obligatory, but let it be so.
All this powerful magic was put into a small script I called pktcdtool, that would do all the dirty work for me (see the source at the very end)
All in all, packet writing reminded me much of how NAND worked. Ideally using jffs2 or any FS for MTD devices (like NAND, where sectors can fail). But to do so, we would need to dig into pktcddvd source code, make an MTD device of it via some translation layer and use jffs2. That would give really good results with ECC and wear leveling.
Now go the backups. Since I do not have much critical stuff to backup, 4 GiBs of a DVD+RW disk is more than enough (Whoever wants may use blueray). But the scenario of the backup is different. For git repos – do a gc, pull and clone for the new ones. For 1С bases – plain tar. Plus we need to make mysql and postgres dumps.
So, I didn’t find any ready to use solutions and threw up a small thing called pbck suite. Dumb as a brick, but hey, it works!
Cron fires up every now and then pbck_exec, which:
- Mounts an UDF DVD.
- Runs custom scripts from backup.d
- Unmounts the disk, syncs the cache etc.
Just as a joke I set my post card to display the stages of the backup using this utility.
To control the wear leveling 00-warmup saves the number of backups performed to .backup_count. Theoretically the disk can withstand up to 1000 rw cycles. Practically – you’ll scratch it faster. So, in some 800 backups it is time to put the DVD aside and take a new one. Even with daily backups that’s more than 2 years!
And, since the DVD+RW is taken out 3 times (insert into the driver, put on the shelf, in case of problems – restore the backup), we won’t have much scratches.
So what’s the profit of this?
- Backups won’t die if mechanics fail
- Dirt cheap, lasts long.
- A way to use old drives.
- Cheaper then memory cards
Finally, bash magic that does the tricks can be found right here on github. (GPLv3)