RAID/NVRAID with dmraid

About the Install
The NVRAID BIOS is actually a software raid. The method used to install here will work with other operating systems such as Windows with no configuration changes needed with Windows.

These instructions will work with many BIOS RAID (Fake RAID) devices. Though, the actual device names will vary.

Make sure you follow the instructions in chapter 3 of the handbook for setting up your network, and in chapter 6 about copying over DNS information, as you may need to download files to complete this process.

In Linux 2.4 the Ataraid kernel framework provided support for software RAID assisted by the BIOS. For kernel 2.6 the device-mapper runtime can do the same kind of work, but it's configured in userspace however. Heinz Maulshagen introduced the dmraid tool to create RAID mappings using the kernel device-mapper subsystem. The controllers supported are (mostly cheap) Fake-RAID IDE / SATA controllers which have BIOS functions on it. Most common ones are: Promise Fasttrak controllers as well as HPT 37x, Intel and LSI. Also serial ata RAID controllers like Silicon Image Medley and Nvidia Nforce are supported by the program. Support for VIA Raid has been introduced in the latest version. While all of the controllers listed previously should work with this method, it was tested with an NVRAID on an nForce2 chipset-based board.

This howto does not cover everything in the installation process. It merely explains how to get a Gentoo installation started with the LiveCD, and then jumps in at the end to explain how to get GRUB installed and boot into Linux after the first restart. This howto has been tested with a stage1, stage3, and a stage 1/3 install, so I have no doubts that this howto will not have any incompatabilities with other installation methods.

Loading dmraid with the LiveCD
To install Gentoo in the first place, you must be able to have access to the raid. When you are at the Gentoo LiveCD autorun, make sure to add the "dodmraid" option.

This will ensure that the dmraid drivers are loaded, as they are not loaded by default as of this writing (2006.0).

Testing dmraid
To verify that dmraid will work with your configuration, check the contents of the directory.

If there is just a "control" file, then dmraid was either not loaded, or dmraid will not work with your configuration. If you are using a 2006.1 LiveCD, try using a more recent LiveCD instead. If there is still only a "control" file in, try to activate the RAID set with the  command.

With Adaptec Hostraid / ICH5 SATA RAID this error message was produced: ERROR: asr: Invalid RAID config table checksum (0xE11 vs. 0x805) on /dev/sdb

One workaround is to initialize the raid using windows setup cd. Format/Partition the array and then hit reset when it begins to copy files over and boot with the gentoo live cd, delete the windows partition and create/format the Linux partition(s).

Suggested Layout
This howto assumes that your boot partition is the first partition on the raid. If you set your partition as the first partition on the disk, you will likely save yourself the need of a floppy disk and a reboot later (Method 2 of installing GRUB mentioned later in this guide). This will not confuse Windows as long as you install Windows onto your second partition (or whichever partition you like). Here is a sample layout for dual-booting with Windows


 * Partition 1: /boot
 * Partition 2: NTFS (for Windows)
 * Partition 3: /swap
 * Partition 4: /

This partition layout is just an example. The partition does not need to be the first partition (but you will need to use a slightly more difficult method to install GRUB if it is not the first).

Using fdisk to Partition
When you partition your drives, do not use, etc. Although Gentoo knows that a raid is there, it also knows that there are two disks and is not afraid to treat them as such. Be sure to fdisk your raid as a whole, by referring to its mapped location found in.

Upon completing fdisk the partition table needs to be reloaded, so the kernel is aware of the changes.

Easy Method
Remember to include the dodmraid option when you start the LiveCD environment again.

"Fast" Method
Execute the following command:

Problem 1: the dmraid binary seems not to be on the gentoo live cd, so i copied it from another gentoo installation. Don't forget to run:

Problem 2: if there already was one or more partition on the raid, dmraid will not correctly reload the partition table; see easy method.

If you're not seeing any partitions after this, (/dev/mapper/nvidia_abiccada1, 2 3 etc) and you get errors like "Arguments out of bounds" when running, it might be caused by the fact that there are spaces in the array name. Use a tool like Intel Matrix Storage Manager to rename your array to something without spaces*

Mounting the RAID
After partitioning your raid, mount the partitions accordingly and use your favorite method to install Gentoo. Remember to mount the drives as the mapped drive and not as /dev/sda1, etc.

and can be mounted in the usual manner:

Building the Kernel
This guide prefers building the kernel yourself over using genkernel. When you are building your kernel, remember to add support for the RAID (and compile as static (*), not as a module (M)):

When you get to the point of installing GRUB, return to this guide.

Installing dmraid
Emerge dmraid (this may be masked or hard-masked, so be sure to unmask it).

If hard-masked:

If masked (change to your arch,  for example):

GRUB Setup and Configuration
This howto assumes that you are using the GRUB bootloader. Be sure that it is installed before continuing.

Grub by itself will not work with a raid0/raid1 configuration. You may be able to get your kernel to boot, but it will panic before you reach a prompt, preventing you from using Gentoo at all. The solution is to create a initial ram disk that will load the dmraid drivers and allow your system to boot.

Easy Solution (using genkernel)
You can just use genkernel to do all the work for you:

Run to make a dmraid kernel:

or (if kernel/initrd compiling crashes)

If you have trouble with the initrd compilation, you can resolve it with:

Compromise Solution (using genkernel only for dmraid and initrd)
If you do not want to go the hard way but still want a custom kernel, then emerge genkernel and execute it as follows.

or for just the initrd:

Hard Solution (using a self-built kernel)
The hard solution assumes that you are building your own kernel and not using genkernel. This solution has been tested on gentoo-sources 2.6.18-r3.

Gerte Hoogewerf created a script to create an image that will allow you to boot correctly. To start the process of creating the image, download the following files and mark the dmraidinitrd script as executable.

Before running the script, you will have to make some changes to it. These changes will update the script as some of the download links are borked. It will also force a download of the correct version of these utilities, as there are compatibility issues with certain versions. In the segment

Change the values to the following:

Note that using later versions of busybox will force you to use the patched script. They will also likely give you shell errors upon your first reboot.

In the segment

Change the values to the following:

In the segment

Change the values to the following:

Once you have the locations for the files done, go ahead and run the script. (Be sure that it has executable rights)

If the script fails for any reason, make sure the links are still active in the script. Sometimes the sites are rearranged (most notably: a file is moved into the /old/ folder). If you are building a SELinux box, make sure the correct libraries are installed:

If the script fails while complaining about dmraid compilation error then try adding "make \" after the line "./configure ..." in the "make_dmraid { ... }" segment.

This script will create a new ram drive image (initrd). Copy the initrd file and the linuxrc file to your /boot directory (or wherever the kernel images and grub reside)

Editing grub.conf
Now that the ram image has been created, you need to let GRUB know how to work with it. Use the following segment from my grub.conf and make the adjustments to yours accordingly (this grub.conf assumes that your /boot is located on the first partition, and that your root is on your last partition).

Note that the major changes are the root=/dev/ram0, the init=/linuxrc, and the real_root pointing to your root partition. Also note the initrd line at the bottom.

Boot-partition-is-first Method
This method of installing GRUB will work 99% of the time when your partition is the first partition on the disk. If you have on a partition other than the first, or if this method fails, proceed to Method 2. Do not run grub-install! It will not work properly, and will probably detect the sda1, etc. instead of your actual raid. Make also sure not to start grub inside of an chroot enviroment like you propably do if you install from the Gentoo LiveCD and do it like described in the installation manual of gentoo.org. It doesn't know about so you would get some errors with grub. In this case just jump into another terminal with ALT+F2. Start grub and make sure that it's pointing to as a device map.


 * note* grub is not on the gentoo-am64 live cd, so ALT+F2'ing to another shell will NOT solve your problem of using GRUB outside of a chrooted enviroment. (grub is located at /mnt/gentoo/sbin/grub)

Now that you're inside of the grub environment, set up your raid devices and install grub!

Filesystem is ext2fs, partition type 0x83

Checking if "/boot/grub/stage1" exists... yes Checking if "/boot/grub/stage2" exists... yes Checking if "/boot/grub/e2fs_stage1_5" exists... yes Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 16 sectors are embedded. succeeded Running "install /boot/grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"...succeeded Done.

Floppy Disk Method
If the previous method of installing GRUB failed, then you will need to create a GRUB boot disk and install GRUB natively. For this you will need a floppy disk. First you will need to install GRUB onto the floppy disk (note that this will erase all contents of the floppy disk!):

1+0 records in 1+0 records out

153+1 records in 153+1 records out

Now reboot your computer and boot from your new GRUB boot disk. Once started, GRUB will show the command-line interface. First, set the GRUB's root device to the partition containing the boot directory:

If you are not sure which partition actually holds this directory, use the find command:

This will search for the file name and show the devices which contain the file. Once you've set the root device correctly, run the setup command:

Checking if "/boot/grub/stage1" exists... yes Checking if "/boot/grub/stage2" exists... yes Checking if "/boot/grub/e2fs_stage1_5" exists... yes Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 16 sectors are embedded. succeeded Running "install /boot/grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/boot/grub/stage2 /boot/grub/menu.lst"...succeeded Done. You should be able to restart your computer now (or finish your installation and restart) and GRUB appear in its glory. Even more importantly, you should be able to boot into Linux and get a prompt.

Boot-partition-not-first Method
When your partition is not the first partition, try following the steps. In some cases, you will have to specify "drive" geometry. Run fdisk on the raid drive, and note C/H/S informations for later usage in grub.

We have to set the devices for grub manually, grub can't do this for us. We use the  option and input nothing:

Now we are in Grub shell mode, we tell grub which "linux device" becomes which "bios device" on boot. If your boot partition is /dev/mapper/your_raid_set1 then it becomes (hd0,0) on boot the last bios nr is the Linux partition nr -1.

First tell grub where your partitions are:

If you have problems with your partitions in grub, tell grub what your drive geometry is now. But don't forget to set the devices first or grub doesn't know what you are talking about. Replace C H S with cylinders, heads, and sectors.

Now Grub knows where the boot partition is, and where to write the MBR. Next step is Grub install, type  and press Tab. If you did nothing wrong grub should show you a list of possible partitions, if not you did something wrong.

Acknowledgements

 * Gerte Hoogewerf, for creating the initrd script
 * aus9 of Linux Questions Forums, for his constant and valuable help that made this possible
 * NeddySeagoon of the Gentoo Forums, for pointing me in the right direction.