Tip #5: Minimizing the risk of Disk Failure

Mythtv recordings use a lot of disk space


One thing that us mythtv users use is a *lot* of disk space. Especially those of us who record HDTV, and what that means is (usually) a lot of disks. The more disks, the more often there are disk failures.

Some ways to keep your data safe


There are a couple ways to keep disk failure from being a major headache. 1) backups. nice idea, but when you have hundreds and hundreds of Gigabytes of videos and every time you record a high-def show, it drops another 8G on your disk, that option becomes costly, time-consuming, and unrealistic. 2) Raid. Raid configurations can come in several levels, but the one that favors data redundancy (Raid 1) means that you need to double the disk space for your existing data.

My method


Another methodology you can use is to separate your operating system and your videos (and other media) onto separate disks, and employ different strategies for each.
Here's what I do:
I have three systems, but this could work with one or two as well.

I have one disk dedicated to the following partitions:

/
/boot
/swap

This could be a relatively small disk. I find 80Gig works nicely, and since speed is not a big issue, I also go with IDE disks, and format them as ext3 filesystems. IDE is a bit cheaper, and ext3 is a bit more reliable as filesystems go.

I also grabbed a 250Gig IDE drive and mounted it to /mnt/backups, exported it, and mounted it on my other machines so that I could perform network (and local) backups of my root filesystems to that one disk.

In order to protect my 'root' filesystems, I do daily backups (full weekly, and incremental daily). I use Simple Linux Backup (which uses tar) and followed the instructions here to get it going. All three of my systems backup to that one drive, two over the network (via NFS mounts) and one locally. This way, I'm protecting all my data on all three machines, but not touching any of the mythtv media.


Now for my videos. Since my MythTV configuration is split into separate frontend/backend configurations, I put all of my video data in one Logical volume on my backend, and mount it to /video (currently, I'm sitting with 6 SATA drives totaling 2.2Terabytes). I then export that entire logical volume to my frontend, and mount it in the same location. Since it is just toooooo much data to backup to another disk, I choose the 'monitor my disks' method. I have mostly SATA disks, and until recent kernels, smartmontools did not support SATA, but now it does. I have smartd monitor all my drives with the following configuration file:

/dev/sda -a -d ata -H -m
/dev/sdb -a -d ata -H -m
/dev/sdc -a -d ata -H -m
/dev/sdd -a -d ata -H -m
/dev/sde -a -d ata -H -m
/dev/sdf -a -d ata -H -m

And every day, I get a nice report from smartmon that tells me if my disks are going wonky. If I start to get failures for a disk, I can run some more tests using smartctl, and if the disk needs replacing, I use pvmove to move data off of the failing disk, remove the disk from my logical volume, replace the disk and add it back in.

Final thoughts


It's not a perfect solution, but for me, keeping the solution as low cost as possible is what's important. I'm willing to fork over the money for a 250Gig drive to backup my root partitions, but everything else just isn't that important. Whatever method you choose. Make sure you choose *something*. Not backing up, or not monitoring your disk health will eventually hurt you.