Post Feed
Comment Feed

About Me

My MP3 Player

Moby - Play: The B Sides Harry Potter and the Half-Blood Prince - (audio book) Kittie - Funeral for Yesterday Dream Theater - Live at Budokan

New Comments

2017 Archives

July All Archives...

Screwed by 3GB

Tuesday, June 27th, 2006

At work I have a server setup to store all the backup data from the other servers. Backing up to disk over the network is much faster than backing up all those servers to tape and is a lot cheaper. I have a RAID (Redundant Array of Independent Disks) system setup in this server to prevent any data loss if any of the disks should go bad. There are a few different flavors of RAID; the one I'm using is RAID 1, or mirroring. This involves two "identical" disks having the same data. When data is written to the array it gets put on both disks at the same time. In the event that one of the disks should fail, the other one will continue to function and no data will be lost. Obviously if both disks fail you're screwed.

When I was planning this server I ordered two 200GB disks for the backups. These disks were identical; they were the same manufacturer, make, model and size. So when I created the array I used the maximum amount of space on the disks for the partitions. It worked great for a year or so. Last week one of those disks failed. It was still under warranty, but I bought a new one instead because if I tried to get a replacement they would want the old disk back and it still has all sorts of private data on it that I can't remove.

I ordered a new 200GB disk last week (the day the old one failed) and it arrived yesterday morning. I transplanted the drives and run the "raidhotadd" command expecting to have the system recognize the new disk and add it the array. It didn't. After some hair pulling I found out that the new disk is actually smaller than the original. Both times I asked for 200GB disks and both times I received 200GB disks (according to the box and the labels on the disks), but the original pair were actually 203GB. You can't mirror two disks when one of them is 3GB larger than the other.

Luckily I was had enough room on another drive in that server copy all the data so I could recreate the partitions on the one old drive and the one new drive to around 185GB so they would both match and could mirror each other. I purposely left some room in case this happens again and I get a disk that is 199GB.

Comments (3) Subscribe

#1 - Jun 27, 2006 at 7:45 AM
So right after I posted I see this on the screen while the array was resyncing:

Jun 27 09:26:37 storage kernel: hde: dma_intr: status=0x51 { DriveReady SeekComplete Error }
Jun 27 09:26:37 storage kernel: hde: dma_intr: error=0x40 { UncorrectableError }, LBAsect=24746384, high=1, low=7969168, sector=24746320
Jun 27 09:26:37 storage kernel: end_request: I/O error, dev 21:01 (hde), sector 24746320
Jun 27 09:26:37 storage kernel: raid1: Disk failure on hde1, disabling device.
Jun 27 09:26:37 storage kernel: ^IOperation continuing on 1 devices
Jun 27 09:26:37 storage kernel: raid1: mirror resync was not fully finished, restarting next time.
Jun 27 09:26:37 storage kernel: md: updating md0 RAID superblock on device
Jun 27 09:26:37 storage kernel: md: hdg1 [events: 00000002]<6>(write) hdg1's sb offset: 184747392
Jun 27 09:26:37 storage kernel: md: recovery thread got woken up ...
Jun 27 09:26:37 storage kernel: md0: no spare disk to reconstruct array! -- continuing in degraded mode
Jun 27 09:26:37 storage kernel: md: (skipping faulty hde1 )
Jun 27 09:26:39 storage kernel: raid1: mirror resync was not fully finished, restarting next time.

So I guess the other disk of the orginal pair just failed. CRIPES!
#2 - Jun 27, 2006 at 11:14 AM
That sucks. Robert is getting a new Dell delivered today. The 98 gateway has got to be replaced. :) I emailed you for Steve's cell #.
#3 - Jan 17, 2007 at 7:51 PM
Just testing the new comment captcha

Make a comment!

Numbers from below


Email Address

Website URL

Remember me