Linux Software RAID

Introduction

This document contains information relating to using software RAID on Linux.

Make sure you've got good backups before attempting to make any changes to your RAID arrays.

Creating a Skeleton Configuration File

This is typically necessary if you're booting off a rescue disk, e.g. Knoppix and need to assemble one or more arrays. Firstly we need an appropriate configuration file, then we assemble the array.

To create a skeleton configuration file based on existing information:

# echo 'DEVICE /dev/hd*[0-9] /dev/sd*[0-9]' > mdadm.conf
# mdadm --examine --scan --config=mdadm.conf >> mdadm.conf

Examine and edit the config file as necessary.

See man mdadm for more information, especially the examples near the end.

Assemble the configured arrays:

# mdadm --assemble --scan --auto=yes --config=mdadm.conf

Creating a New RAID Array

The following example creates an array named 'md0' with /dev/hda1 and /dev/sda1 as the active devices, with /dev/sdb1 as a standby spare. The device partitions should be a similar size, and their partition type set to type 'fd', 'Linux raid autodetect'. The raid partition size will be the same size as the smallest active device.

# mdadm --create /dev/md0 --metadata=0.90 --level=1 --raid-devices=2 \
        --spare-devices=1 /dev/hda1 /dev/sda1 /dev/sdb1

Displaying information about the array:

# mdadm --detail /dev/md0

or

$ cat /proc/mdstat

Using a Rescue Disk with MDADM

If you've booted from a rescue disk such as Knoppix, you'll need to manually configure and start the RAID array. Much of this is described in the examples section of the man pages for mdadm, but here as a quick guide and reminder.

# echo "Creating a default config file"
# echo 'DEVICE /dev/hd*[0-9] /dev/sd*[0-9]' > mdadm.conf
# mdadm --detail --scan >> mdadm.conf
# mdadm --examine --scan --config=mdadm.conf >> mdadm.conf
# echo "Assembling the arrays and creating device nodes if necessary"
# mdadm --assemble --scan --auto=yes

Trouble Shooting

mdadm Reports device node does not exist

This can be caused if you're using a rescue disk, e.g. Knoppix and the RAID modules have not been loaded. Try loading them as follows:

# modproble md

Also see the 'auto' option in the man pages (man mdadm) which allows mdadm to create the device nodes.

SparesMissing Events

If mdadm is firing SparesMissing events, yet there are no spares missing, check the contents of /etc/mdadm/mdadm.conf. There may be more spares defined for the array than there actually are.

This can occure because the output of mdadm --examine --scan can include an extra spare if the array was still rebuilding. You could also re-run this command to confirm this is the case.

See 'mdadm Raid5 gives spares missing events'

Grub reports error reading stage2 file

This error may occur when both devices are active in a RAID 1 array. You may be able to install grub on one device but not the other. It seems grub fails to find ./grub/menu.lst on the target device, perhaps because of the way it is being mirrored. A workaround that has been known to work is to remove the other device so that the target device is now the only one in the (degraded) array.

Degrade the array, removing the other device, then install grub, then add the missing device back into the array.

Example installing grub on /dev/hdb (which has been failing) with an array of two devices, /dev/hda and /dev/hdb

   # mdadm /dev/md0 --fail /dev/hda
   # mdadm /dev/md0 --remove /dev/hda
   # grub-install /dev/hdb
   # mdadm /dev/md0 --add /dev/hda
   # mdadm --detail /dev/md0

Spare disk gets kicked out from array during reboot

This seems to occur where a disk has been added to an array as a spare, but the array was never mounted. Mounting and unmounting the array after adding the spare and before rebooting seems to cure it.

Example with an existing array (/dev/md0) of /dev/sda and /dev/sdb where we want to add /dev/sdc as a spare:

   # mdadm /dev/md0 --add /dev/sdc
   # mount /dev/md0 /mnt/tmp
   # umount /dev/md0
   # mdadm --detail /dev/md0

After reboot, spare should still be present.

   # mdadm --detail /dev/md0

Related Topics: LogicalVolumeManagement, CreatingRescuePartition

-- Frank Dean - 03 Jan 2009