gotdoug?

Today at work, we wanted to add a couple of 500 GB disks to a development server that we have. We already have two 500GB disks in the system, set up as raid 1. The server is currently running CentOS 5.1 without an X environment, so we need to do everything from the command line.

We were wanting to add two new disks as another raid 1 (mirror) setup. The biggest issue is that the system had to be running for development work the whole time. After doing some google searching, I came across a couple of pages that were quite helpful for us.

I followed the instructions on this page for most of the set up.

Obviously, before continuing, make sure that you have a backup of everything that you can’t easily recreate, or live without.

We are going to set up a software raid, to match what is already in the system. We are not going to do anything fancy, just a single partition on each new disk, to create one large 500 GB raid 1 (mirror) partition when we are done.

First, you have to do everything as root, so:

$ su -


To make sure that the drives were being seen by the system, ran fdisk

$ fdisk -l

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 60801 488384001 83 Linux

Disk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdd1 1 60801 488384001 83 Linux

The partition types were not correct for a raid setup, so those needed to be changed, but first we had to make sure they were not mounted.

# umount /dev/sdc1
# umount /dev/sdd1


Now to fix the partition types.


# fdisk /dev/sdc

The number of cylinders for this disk is set to 60801.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 60801 488384001 83 Linux

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L
17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fd Linux raid auto

Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): p

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdc1 1 60801 488384001 fd Linux raid autodetect

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Then I repeated the same steps for the second drive, /dev/sdd

Next, we needed to determine what the next available md devices would be. I ran

# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md1 48G 4.8G 41G 11% /
/dev/md0 401G 336G 44G 89% /home


This showed me that we had md0 and md1 used. We would use md3 for our next raid device. There wasn’t a reason for skipping md2, we just did.

Now to create the raid set. Continuing to follow this page, we ran this and got the following error:

# mdadm --create --verbose /dev/md3 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdd1
mdadm: error opening /dev/md3: No such file or directory


After doing some more Googling, we came across this blog post that suggested adding the –auto option to our command. So after some trial and error, this is the final command that we came up with.

# mdadm --verbose --create /dev/md3 --auto=md3 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdd1
mdadm: size set to 488383936K
mdadm: array /dev/md3 started.


The gotcha for us was where to put the –auto and the syntax. The above finally worked for us.

Next to confirm that the raid for properly initialized,

# cat /proc/mdstat
Personalities : [raid1]
md3 : active raid1 sdd1[1] sdc1[0]
488383936 blocks [2/2] [UU]
[>....................] resync = 0.4% (1964928/488383936) finish=111.3min speed=72775K/sec

We waited about 20 minutes and repeated the above command:

# cat /proc/mdstat
Personalities : [raid1]
md3 : active raid1 sdd1[1] sdc1[0]
488383936 blocks [2/2] [UU]
[==>..................] resync = 10.0% (48918592/488383936) finish=101.1min speed=72406K/sec

At this point, we decided to go ahead and try to format the device. Thinking that even if it hadn’t finished initialization, we could still format it, and it would catch up with the resyncing.

# mkfs.ext3 /dev/md3
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
61063168 inodes, 122095984 blocks
6104799 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
3727 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

Next we need to save the new raid configuration, so it will be available after the next reboot, but we need to make a backup copy of the existing configuration first

cp /etc/mdadm.conf /etc/mdadm.conf.BAK


Check out what mdadm thinks the configuration is:

# mdadm --detail --scan --verbose
ARRAY /dev/md1 level=raid1 num-devices=2 UUID=8892876d:67b738d2:59946b6a:8692b574
devices=/dev/sda1,/dev/sdb1
ARRAY /dev/md0 level=raid1 num-devices=2 UUID=c5d2ceee:7e7e98ad:2dcf58b6:a2976aba
devices=/dev/sda2,/dev/sdb2
ARRAY /dev/md3 level=raid1 num-devices=2 UUID=b0d54dd2:84093d6e:24c202ac:d5d97a5c
devices=/dev/sdc1,/dev/sdd1


Looks good, so put that into the conf file:

# mdadm --detail --scan --verbose > /etc/mdadm.conf


Create the new mount point, and add it to /etc/fstab

# mkdir /mnt/share/
# vi /etc/fstab

adding this line

/dev/md3 /mnt/share ext3 defaults 1 3


Now, to mount the new raid device.

mount /dev/md3 /mnt/share


There you go! If you ever want to check the status of the raid devices, you can

# cat /proc/mdstat

and that will show the status of all raid devices in the system.