Category Archives: hardware

My Drobo: A few months in…

Well, I've had my drobo for a while now. I really have no complaints on the hardware; it works as expected and works well. I do have issues, however.

My first issue is my own behavior. I sometimes fail to RTFM. In this case, it was the part of the manual on formatting. I simply formatted the drobo from Disk Utilities. If I had RTFM, I would have learned that the standard format was for 2 TB volumes – if you stick more available space in the system, you'd get multiple volumes. The drobo can be formatted to anticipate a volume of 16 TB, which would have made more sense to me. The drawback to this format is slower boot times, but I'm okay with that. In any case, it's simply a matter of reformatting the drobo and start over. Unfortunately, I have the older USB 2.0 drobo and offloading the data took at least a day (and some struggling to find the disk space). Now, the drobo is reformatted and I've started the process of moving all the data back! ugh! take my advice, if you get a drobo, RTFM.

The second issue I'm having is planning my capacity. Now, Data Robotics provides a nice tool to help you plan your space, called the Drobolator. My current drobo has 2 1 TB drives, 1 500 GB drive, and 1 320 GB drive. That gives me about 1.6 TB of usable disk space. However, I wanted to add space to the system so I started trying to figure out the most cost-effective approach. It's not so easy. In a pure cost-per-gig view, I should buy 1.5 TB seagates. However, if I bought only one, the cost per gig went up considerably because the drobo would only 1 TB of space, rather than 1.5 TB. This is because size of the largest drive has to be eliminated from the total capacity for data protection should it fail (e.g. if you have 3 1 TB and 1 1.5 TB and the system was full, you'd be protected even if the largest drive failed since the system would use only 3 TB of the 4.5 TB). Based on the fact a 1.5 won't get me anywhere, I went with a WD green 1 TB drive (cost $10 more than a similarly sized seagate, but supposedly uses less power). Now this set up leaves me with 1 500 gig drive. This drive, when it comes time to replace it, makes more sense to swap with a 1.5 TB+ drive. At this point, the costs go way up (depending on how long you wait). Any drive I swap into the drobo now must be the largest capacity of all the existing drives, not just equal to the largest. If I swapped the 500 gig for a 1 TB, I'd get an additional 500 gigs of capacity. If 3 total TB available is enough, that's not a problem. But what happens if I need another 500 gig later? If all 4 drives are 1 TB, I'd have to replace a minimum of two drives: the first would become the largest drive defining the space for protection, and the second would then provide the available new storage. Try that on the drobolator, 4 1 TB drives and swap one with a 1.5 or 2 TB drive. At current prices, then, to add 5oo GB to a drobo of 4 1 TB drives would cost about $260, a shift from 11 GB/$1.00 to 1.9 GB/$1.00. That doesn't include of the cost of pulling out the 2 1 TB drives.

I have to wonder whether at this point it's better just to buy a second drobo and use the working drives you pulled out of the old drobo when you upgraded. Maybe, maybe not. I do tend to use hard drives for backups that I keep in undisclosed locations. If they are full, then I don't have any drives available. Of course, I could use a drobo for that purpose instead of simple drives. I'll have to consider my options if and when my drobo starts running out of space again.

Revamping my data protection plan

Revamping might not be the right word, since I don't have a written plan, but I'm at least re-evaluating what I do to protect my data.

In the last week, I've been more seriously considering “cloud-based” data storage; that is, storing data on someone else's server out there on the “internets”. The advantage of this is that if my computer is stolen, house burns down, or a tornado hits (there have been 2 near misses over the last few years), data in the cloud would be preserved. Thus, it can be effective off-site storage solution.

The other off-site storage solution I already use: put your data on some media and physically store the media off site. This is a great solution, but if something goes wrong, it might take a while to access the media to recover the data. “Cloud” storage, on the other hand, offers potentially instantaneous access to your data.

A problem with cloud storage (aside from cost considerations) is that you are dependent on the hosting company to maintain security and solvency (i.e. you don't want them to go under). Another problem is you are limited by bandwidth, either your own connections or the bandwidth allowed by the hosting company. This limits the practicality of “cloud” storage for some solutions. For a small set of documents, such limitations are minor. For many gigabytes of storage, this becomes a problem.

An attractive service is Amazon's S3. It's reasonably priced but the costs are not consistent from month to month. If I put 4 Gb of data on the site for a year, and never access it after that, it would cost a minimum of $7.60 per month (or about $91 per year). Mozy, on the other hand, is cheaper and unlimited at the pro level, costing about$60 per year. However, it is more of a traditional backup solution approach rather than just file storage (I say this without testing, however).

I'm planning to try Mozy for their free membership, but it will have to used as a backup solution for a limited amount of my data. My climate model results will have to remain a physical media offsite storage plan since the cloud is out of reach at the moment.

The Drobo so far

With the Drobo up and running, I'm feeling a little safer. Right now, I have about 2 TB of disk space, which gives me just under 1 TB of available disk space. Of that space, I've used about half.

Overall, I'm very happy with the system. The largest drawback in this version of the Drobo is the USB 2. USB 2 is painfully slow compared to Firewire and SATA. However, in the way I'm using the system, the slow speed is tolerable.

The real test of the Drobo would be a hard drive failure. Sorry, I'm not going to even try to simulate that! Regardless, it will stay a part of the data protection plan for some time.

Another test of my backup strategy

It happened again! I had a hard drive crash. This time, the drive was a Seagate 1 TB drive about 60% full.

So what was my backup strategy and did it work?

I have three basic methods of backing up:

1. Time Machine – I use time machine to manage the backups of my laptop.
2. Backup to DVD – This method I was hoping to end. It's time consuming and hard to use with large groups of files. However, this has been relatively reliable.
3. Drive “mirroring” – Using tools such as rsync to maintain a second copy of a hard drive.

Method 3 is my latest approach and it's the system tested by my latest hard drive crash. Did it work?

Yes and no. I had a full copy of the drive where the data are safe. However, the backup itself was about 2 weeks old. As far as I can tell, however, about all I lost was a few downloads from the internet, including free iTunes songs and videos. So, I'd have to say the approach was reasonably successful.

One clear weakness of the approach was me. To make the system work, I have to do the mirroring far more frequently. Since I keep the mirrored drive offline when not backing up, this is not something I can do automatically (I have a LOT of data, so it's not practical for me to keep the drive online and active). So, I have to unmount another drive I'm using and add the mirror drive and then do the mirroring. This really can't change for me, so I need to make a regular schedule for now on.

The second weakness, which is far less obvious, is the lot of the hard drive. I worry that when I buy 2 hard drives of the same size and same brand, the drives come from the same manufacturing lot. Thus, if one is defective it's far more likely that the other is also defective (at least that's my fear). So, if one drive fails, as it just did for me, I worry that the other is also likely to fail putting my mirrored copy a high risk until I can get a replacement drive.

Given all these issues, I'm adding one more tier to my backup/data protection strategy. A Drobo. Drobo is essentially a low-maintenance RAID system (but not really a RAID). It holds up to 4 hard drives and allows you to mix and match drive sizes. If a drive fails, you pop it out and replace it. It is still likely to be vulnerable until replacement drives are installed, but the idea of not having to work about the timing of my mirroring would be a large improvement.

I've ordered a generation 1 drobo (since they're around $150 cheaper than the current model) from amazon and I'm already making plans to take advantage of the system.

However, this cannot replace the mirroring approach to backups. The Drobo will be used for more active projects and files, not long-term storage. Mirrored drives or DVDs will remain a part of my backup strategy for that work.

Right now, until I get the Drobo and replacement drives, I'm feeling vulnerable.

Disaster Recovery

My big fear is losing all my data in some sort of system crash. Well, I just had a system crash last week (bad hard drive). As it turned out, it was marginally painful. My backup system was designed for managing my data. Fortunately, my data were on the drive that survived. The dead drive housed the system OS.

After some false starts, I managed to find an old copy of the system and restored it onto a new disk. Within a few hours, the system was back up to normal, more or less.

Although I didn't backup as religiously as I should have, even a marginal backup scheme prevented a total loss. Any backup is better than none.

SATA vs. Firewire vs. USB 2.0

I’ve been needing a serious evaluation of my backup habits. One area is mixing my storage media for longer-term backup. Up until now, I’ve been using DVD’s, but given the hundreds of gigabytes of data I can create in a month or two, and questions regarding long-term storage of writable DVD media, I’ve begun delving into the possibility of using hard drives as part of an overall strategy. But the world of external drives has become exceedingly complicated. Firewire (IEEE-1394), USB 2.0, and more recently SATA/ESATA.

The performance rating of these interfaces are 400 mb/s for 1394a and 800 mb/s 1394b, 480 mb/s for USB 2.0, and 1500 mb/s for SATA 1 and 3000 mb/s for SATA 2. But what are their real-world performances like? The following is the first part of my exeriments on performance

The equipment:
1. The computer – Macbook pro – USB 2, 1394a, Express card for external SATA, internal SATA drive

2. External Firewire/USB 2.0 drive with a 250 GB IDE drive.

3. External SATA 1/USB 2.0 drive case with a 320 GB SATA drive.

Method:

I used a typical directory for me, 17.35 GBs, and copied from one drive to the other several times using a shell script. Although I’m mac based, none of these files required resource forks so no special handling was required.

Here are the results of copying back and forth to the mac’s internal drive. The figure below shows the copy time in seconds (best results are SHORTER).

Not surprisingly, the SATA interface performed the best. USB 2.0 performed the worst. Generally, the hardware limitations of the mac’s internal workings are a major factor here, but the results are inline with other results out there on the web. The other limitation is the external drives themselves. Keep in mind that most drives can only write about 30-80 MB/s and the best performer here is only about 230 mb/s, or about 30 MB/s (note that this speed is far far slower that each of the interface’s ratings).

The second experiment is copying external drive to external drive. Since I only have two cases and limited USB 2.0 ports, I was limited in the experiment. Here are the copy times in seconds (best results are SHORTER):

Again, USB 2.0 is the slower of the interfaces. Interestingly, the copy speed to the SATA drive from the Firewire drive is relatively fast: 293 mb/s, or about 37 MB/s; this is the fastest copy speed of the experiment, and it was approaching the 400 mb/s for the firewire interface. I consider that a very good result for Firewire along with SATA.

This first experiment show that my purchase of a SATA1/USB combo case was probably not the best solution – a case that had SATA1 or 2 and firewire would have been far better for my needs. USB 2.0 as a solution is clearly not good enough for me.