Root on LVM or EVMS over dm-crypt/LUKS

Preface
This document is intended to be a comprehensive guide on how to set up a Gentoo system on LVM or EVMS, which itself is on top of a dm-crypt/LUKS encrypted disk.

Reasons
There are a few reasons that one might want to use such a setup.
 * No unencrypted volume information (metadata, partition sizes, etc.) is exposed besides necessary pre-encryption boot strap and the size of the whole large disks (and bootstrap).
 * Can manage everything (multiple volume objects, analogous to partitions) with LVM or EVMS (resize, move, copy, rename, add, delete volumes, etc.) within and across any number of large monolithic encrypted physical disk devices (without any unencrypted metadata).

Both of these reasons contrast sharply to the case in which they do not hold true, i.e., LUKSing each filesystem one by one, which seems to be the old default support for LUKS and those users who also used LVM/EVMS.

Kernel
In order to use encryption, you will need the following options. It is preferable that they be compiled in, but they can work as modules when using the (pretty much mandatory) initrd.

For LVM or EVMS, you need Device Mapper Support, but that was needed for encryption anyway.

Userland
Whether you use EVMS or LVM, you will need the following software: >=sys-fs/cryptsetup-1.0.5

EVMS
If you choose to use EVMS, you will need a patched version of sys-fs/evms.

Firstly, you need a local Portage overlay (or a local repository, in Paludis terms). Portage users should add the line PORTDIR_OVERLAY='/usr/local/portage' to their make.conf, while Paludis users should create a file called local.conf in /etc/paludis/repositories/ with the contents location = ${ROOT}/usr/local/portage sync = master_repository = gentoo format = ebuild names_cache = ${location}/.cache/names write_cache = /var/cache/paludis/metadata

No matter which you use, you will need to create the directory and add a repo_name entry. You will need to run (as root) mkdir -p /usr/local/portage/profiles echo local > /usr/local/portage/profiles/repo_name

Then, make the sys-fs category directory and copy over the EVMS ebuilds from the tree into it (again, as root). mkdir /usr/local/portage/sys-fs cp -a /usr/portage/sys-fs/evms /usr/local/portage/sys-fs/

Remove the Manifest file and all of the ebuilds except for the most recent one (evms-2.5.5-r10.ebuild at the time of writing), move it such that the number after '-r' is one greater (f.e. evms-2.5.5-r11.ebuild) and open that in your favorite editor. Add the flag 'evmsdminput' to the IUSE= line, and add the line use evmsdminput && epatch ${FILESDIR}/${PV}/2.5.5-dev-mapper-input.patch to the end of the src_unpack function.

You will also need to create that patch file, so here is its content (there was a link in the original page, but the link was down most of the time so I'm pasting it inline. If anyone knows a better way, please let me know.) diff -x'*~' -Nru /usr/src/evms/compare/evms-2.5.5/doc/evms.conf evms-2.5.5/doc/evms.conf --- /usr/src/evms/compare/evms-2.5.5/doc/evms.conf     2006-02-23 07:49:43.000000000 -0800 +++ evms-2.5.5/doc/evms.conf   2006-03-05 12:20:26.485719200 -0800 @@ -251,6 +251,25 @@                                                                              device_size_prompt = yes }                                                                                        +# User devmapper section +# This allows us to use a device-mapper device directly as a disk +# without the need for loopback +# Be careful not to specify any targets that are created by evms +# This can be useful for any devices created via dmraid or other means +                                                                                         +# In order for this to work "dm*" has to be in the list of includes +                                                                                         +devmapper { +      # This needs to be set to yes to enable +      #dm_user = yes +                                                                                         +       # List of targets to discover as disks +      # These are the target names as seen with "dmsetup ls" +      # only specific targets can be used +      # we need to avoid evms created targets +      #dm_user_targets = [ raid1_device raid0_device ] +}                                                                                        +                                                                                           md { diff -x'*~' -Nru /usr/src/evms/compare/evms-2.5.5/engine/dm-targets.c evms-2.5.5/engine/dm-targets.c --- /usr/src/evms/compare/evms-2.5.5/engine/dm-targets.c       2005-11-07 07:46:41.000000000 -0800 +++ evms-2.5.5/engine/dm-targets.c     2006-03-05 12:30:13.348119613 -0800 @@ -762,7 +762,7 @@                                                                                  * A mirror string has the form: *    []*  [ : ]{2,} *                                                                                                - * Currently, log_type will always be "core", num_log_params is 1, and + * Currently, log_type will always be "core", num_log_params is 1 or 2, and * log_params is chunk-size (in sectors). **/                                                                                               static int mirror_build_params(dm_target_t *target) @@ -807,21 +807,32 @@                                                                               {                                                                                                          dm_target_mirror_t *mirror = target->data.mirror; char *params = target->params; -      int i, rc; +      int i, rc, num_opts; LOG_PROC_ENTRY; -      /* Skip "core 1" at the start of the string. */                                            -       params = next_token(params); +      /* Skip "core" at the start of the string. */                                                      params = next_token(params); -      rc = sscanf(params, "%u %u", &mirror->chunk_size, &mirror->num_mirrors); +      /* get the number of options for core and the chunk size */ +      rc = sscanf(params, "%u %u", &num_opts, &mirror->chunk_size); if (rc != 2) { rc = EINVAL; goto out; }                                                                                          -                                                                                                           params = next_token(params); +                                                                                                  +       /* skip the options for core based on num_opts */ +      for (i = 0; i < num_opts; i++) { +              params = next_token(params); +      }                                                                                           +                                                                                                   +       /* get the number of mirrors */ +      rc = sscanf(params, "%u", &mirror->num_mirrors); +      if (rc != 1) { +              rc = EINVAL; +              goto out; +      }                                                                                                   params = next_token(params); for (i = 0; i < mirror->num_mirrors; i++) { @@ -855,13 +866,33 @@                                                                               static int mirror_pretranslate_params(char *params, u_int32_t *num_devs,                                                                 u_int32_t *num_groups) {                                                                                                 -       int rc; +      int rc, i, num_opts = 0; +      char *num_devs_point; LOG_PROC_ENTRY; -      rc = sscanf(params, "%*s %*d %*u %u", num_devs); +      /* The mirror target can have more than 1 option +      this affects the location of the number of member disks */ +                                                                                                  +       rc = sscanf(params, "%*s %u", &num_opts); +      if (rc != 1) { +              rc = EINVAL; +              goto out; +      }                                                                                           +                                                                                                   +       /* skip "core " */ +      num_devs_point = next_token(params); +      num_devs_point = next_token(params); +                                                                                                  +       /* skip the options for core based on num_opts */ +      for (i = 0; i < num_opts; i++) { +              num_devs_point = next_token(params); +      }                                                                                           +                                                                                                   +       rc = sscanf(num_devs_point, "%u", num_devs); rc = (rc != 1) ? EINVAL : 0; +out: LOG_PROC_EXIT_INT(rc); return rc; }                                                                                                 diff -x'*~' -Nru /usr/src/evms/compare/evms-2.5.5/plugins/disk/localdskmgr.c evms-2.5.5/plugins/disk/localdskmgr.c --- /usr/src/evms/compare/evms-2.5.5/plugins/disk/localdskmgr.c 2006-02-24 11:53:21.000000000 -0800 +++ evms-2.5.5/plugins/disk/localdskmgr.c      2006-03-05 12:36:27.728621485 -0800 @@ -1335,13 +1335,14 @@                                                                                                  rc = EngFncs->dm_get_targets(disk, &targets); if (rc) { LOG_ERROR("Error getting DM mapping for disk %s.\n", disk->name); +              rc=0; goto out; }                                                                                                                /* Reject all non-multipath devices. */                                                                          if (targets->type != DM_TARGET_MULTIPATH) { LOG_DEBUG("Disk %s is not a multipath device.\n", disk->name); -              rc = EINVAL; +              //rc = EINVAL; goto out; }                                                                                                        @@ -1541,6 +1542,98 @@                                                                                             }                                                                                                                 /**                                                                                                              + * check_user_devicemapper + *                                                                                                              + * Check if this disk is a User created DM device. + **/                                                                                                            +static int check_user_devicemapper(storage_object_t * disk) +{                                                                                                               +       dm_device_list_t * dm_list, * dm_entry; +      dm_target_t * targets = NULL; +      local_disk_t * ld = disk->private_data; +      int rc = 0, dm_user_targets_count = 0, i, dmup_len; +      boolean user_dm_enable; +      const char * const * dm_user_targets; +      char * dm_user_prefix = "dm/"; +                                                                                                                +       LOG_ENTRY; +                                                                                                                +       /* Get the list of active DM devices. */                                                                 +       dm_list = get_dm_device_list; +      if (!dm_list) { +              LOG_WARNING("Cannot get list of DM devices.\n"); +              goto out; +      }                                                                                                         +                                                                                                                 +       /* Search the DM list for an entry that matches this disk. */                                            +       dm_entry = find_disk_in_dm_devices(disk, dm_list); +      if (!dm_entry) { +              LOG_DEBUG("Disk %s is not a DM device.\n", disk->name); +              goto out; +      }                                                                                                         +                                                                                                                 +       user_dm_enable = FALSE; +      EngFncs->get_config_bool("devmapper.dm_user", &user_dm_enable); +                                                                                                                +       EngFncs->get_config_string_array("devmapper.dm_user_targets",                                             +                                        &dm_user_targets_count, &dm_user_targets); +                                                                                                                +       if (user_dm_enable == FALSE) { +              LOG_DEBUG("devmapper.dm_user not enabled.\n"); +              rc = EINVAL; +              goto out; +      }                                                                                                         +                                                                                                                 +       /* search the named targets in the config for the DM name*/ +      user_dm_enable = FALSE; +      for (i = 0; i < dm_user_targets_count; i++) { +              if (strncmp(dm_user_targets[i], dm_entry->name, EVMS_NAME_SIZE) == 0) { +                      user_dm_enable = TRUE; +              }                                                                                                 +       }                                                                                                         +                                                                                                                 +       if (user_dm_enable == FALSE) { +              LOG_DEBUG("%s not found in devmapper.dm_user_targets.\n", dm_entry->name); +              rc = EINVAL; +              goto out; +      }                                                                                                         +                                                                                                                 +       /* Get the DM mapping for this disk. */                                                                  +       /* we can use this in the future if we need to, and it's good to check */ +      strncpy(disk->name, dm_entry->name, EVMS_NAME_SIZE); +      rc = EngFncs->dm_get_targets(disk, &targets); +      if (rc) { +              LOG_ERROR("Error getting DM mapping for disk %s.\n", disk->name); +              goto out; +      }                                                                                                         +                                                                                                                 +       /* Copy the DM name to this disk. */                                                                     +       dmup_len = strlen(dm_user_prefix); +      LOG_DEBUG("Changing disk name from %s to %s%s.\n",                                                        +                 disk->name, dm_user_prefix ,dm_entry->name); +                                                                                                                +       strncpy(disk->name, dm_user_prefix, dmup_len); +      strncpy((disk->name)+dmup_len, dm_entry->name, EVMS_NAME_SIZE-dmup_len); +                                                                                                                +       /* Reject all multipath devices that +       * were created by other EVMS plugins. +       */                                                                                                       +       rc = check_multipath_name(disk); +      if (rc) { +              LOG_DEBUG("Multipath disk %s belongs to another EVMS plugin.\n", +                         disk->name); +              goto out; +      } + +       ld->flags |= LD_FLAG_USERDM; + +out: +      /*EngFncs->dm_deallocate_targets(targets); */ +      LOG_EXIT_INT(rc); +      return rc; +} + +/** * get_geometry * * First try to get the geometry from the partition table (if it exists). @@ -1834,6 +1927,13 @@               /* Get the disk's hard-sector-size. */               get_hardsector_size(&working_disk);
 * 1) MD plugin section

+              /* Check for User created DM devices. */ +              rc = check_user_devicemapper(&working_disk); +              if (rc) { +                      close_dev(&working_disk); +                      continue; +              } +                /* Get the disk's geometry. */               get_geometry(&working_disk);

diff -x'*~' -Nru /usr/src/evms/compare/evms-2.5.5/plugins/disk/localdskmgr.h evms-2.5.5/plugins/disk/localdskmgr.h --- /usr/src/evms/compare/evms-2.5.5/plugins/disk/localdskmgr.h 2006-02-24 09:00:44.000000000 -0800 +++ evms-2.5.5/plugins/disk/localdskmgr.h      2006-03-05 12:36:46.917967652 -0800 @@ -93,6 +93,7 @@ +#define LD_FLAG_USERDM        (1 << 3)
 * 1) define LD_FLAG_MULTIPATH     (1 << 0)
 * 2) define LD_FLAG_IDE           (1 << 1)
 * 3) define LD_FLAG_SCSI          (1 << 2)

extern engine_functions_t *EngFncs; extern plugin_record_t   *my_plugin_record;

Now you can install that version of EVMS, and it will do what is needed for this project.

LVM
If you choose to use LVM, you will need sys-fs/lvm2.

Setup
The first thing we will need to do is partition the disk we will be using. I will assume the disk is sda, and that the only partitions will be /boot and the encrypted partition. If you are using EVMS, this partitioning can be done with the EVMS utilities - just create two 'compatibility volumes'. Otherwise, just use fdisk, gparted, or whatever you prefer.

Once you have created the partitions (sda1 for /boot, presumed to be ext2, and sda2 for the encrypted partition, unformatted), then you need to create the encrypted volume and open it. The command I will show creates a Serpent-encrypted volume, using the XTS encryption mode and a 512-bit key (256 bits for Serpent, 256 bits as a tweak for XTS). You may use other options as you see fit. cryptsetup luksFormat -c serpent-xts-benbi -s 512 /dev/sda2 cryptsetup luksOpen /dev/sda2 root The 'root' argument in the second command is the name to assign to the volume. This is the name of the block device that will appear under /dev/mapper/ - you can make this anything you like, but genkernel-created initrd's will name the encrypted volume that / is on 'root' when booting.

EVMS
For EVMS, you will need to add 'root' in between the brackets in the devmapper section of /etc/evms.conf

Then, use either evmsgui or evmsn (or evms, if you have neither GTK nor ncurses) to do the following:
 * 1) Delete the 'dm/root' volume
 * 2) Create an LVM2 container on the dm/root logical disk
 * 3) Create the storage regions you want on that container - For instance, I have a Swap and a Root region.
 * 4) Create an EVMS volume, one on each storage region
 * 5) Create the appropriate filesystems on each volume (according to EVMS, swap is a filesystem)

LVM
Anyone want to write this? I have no experience with LVM, just EVMS --Eternaleye 09:06, 30 November 2008 (GMT)

Usage
No matter which you used, you are going to need an initrd. I recommend that you use genkernel to generate your initrd, as well as your kernel. It is a common misconception that using genkernel means you cannot manually 'tweak' your kernel options, but really you just need to add the --menuconfig option. One benefit of using genkernel is that it already supports EVMS or LVM over LUKS natively - if you pass the 'crypt_root=/dev/evms/sda2' and 'real_root=/dev/evms/Root' parameters to the kernel and initrd it'll do it automatically.

EVMS
Add the '--evms' parameter to the genkernel command, and 'doevms' to your kernel commandline.

LVM
Add the '--lvm' parameter to the genkernel command, and 'dolvm' to your kernel commandline