Welcome to TQ CMS knowledge base. Here you will find useful information for all sorts of things.

Adding software RAID to LVM configurations

Thinking about protection and disk drives, the first thing that comes to mind is RAID (Redundant Array of Inexpensive Disks). This technology provides redundancy by combining multiple disk drives and distributing data across. There are different ways of data distribution called “RAID levels” (1 to 6), as RAID 1 and RAID 5 being the most common.

In RAID 1 two or more drives keep mirrored copies of the data. No matter how many disk drives fail, if there is at least one operational drive, the whole information is accessible. This availability comes at a price of disk capacity.

With RAID 5 the data is distributed over three disk drives at least. This configuration uses block-level striping with parity data distributed across all member disks. If any of the drives fails, the data is still available (the parity data is used to recover the missing information). The advantage of using RAID 5 is that the total capacity is not reduced by half (as with RAID 1). It is reduced only by the capacity of a single drive. For example if you have four 1 TB disk drives you can configure two RAID 1 arrays, each with a capacity of 1 TB. This makes a total of 2 TB of available disk capacity. If you configure a RAID 5 array instead, you will loose only 1 TB of capacity totaling in 3 TB of total storage being available.

This increased capacity, however, comes at a price of reduced availability. RAID 5 configurations can tolerate only a single drive failure. If two or more disk drives become unavailable, the entire array fails.

To illustrate the usage of RAID within LVM we will change the LVM configuration that we built in the previous tutorial, adding RAID 1 protection to it.

We will put /dev/sdb1 and /dev/sdc1 in RAID 1 (mirroring without parity or striping) and mount it under /dev/md0. In a similar way we will combine /dev/sdd1 and /dev/sde1 and configure them to be accessible under /dev/md1.

To build this configuration we will first evict all data from /dev/sdc1 and /dev/sde1. After that we will remove the /dev/sdc1 and /dev/sde1 partitions from the volume group. We will next create /dev/md0, but we will make it use a single disk for the moment (/dev/sdc1). We will create /dev/md1 in a similar fashion (using only /dev/sde1). We will then move away all data from /dev/sdb1 and /dev/sde1. After these two partitions are freed we will add them as mirrored disks to /dev/md0 and /dev/md1. This will let us build our RAID configurations step by step, without losing any data during the process.

Let’s start by freeing /dev/sdc1. We use the pvmove command to evict its data.

 pvmove /dev/sdc1
  /dev/sdc1: Moved: 26.5%
  /dev/sdc1: Moved: 52.2%
  /dev/sdc1: Moved: 77.8%
  /dev/sdc1: Moved: 92.7%
  /dev/sdc1: Moved: 100.0%
 

Removing /dev/sdc1 from the database group is done via the vgreduce command.

 vgreduce database /dev/sdc1
  Removed "/dev/sdc1" from volume group "database"
 

Next we delete the physical volume by using pvremove.

 pvremove /dev/sdc1
  Labels on physical volume "/dev/sdc1" successfully wiped
 

Our next step should be to change the partition type for /dev/sdc1. Its type should be “fd” (Linux raid autodetect), if it is going to be used in a RAID configuration.

 fdisk /dev/sdc

The number of cylinders for this disk is set to 6527.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 

Building the RAID array is done via the mdadm command. The name is derived from Multiple Device (MD) and it is the main Linux command for managing RAID configurations.

 mdadm --create /dev/md0 --auto=yes -l 1 -n 2 /dev/sdc1 missing
mdadm: array /dev/md0 started.
 

After /dev/md0 is in place, we can build a physical volume on top of it.

 pvcreate /dev/md0 
  Physical volume "/dev/md0" successfully created
 

After the PV is created we can extend the database group using vgextend.

 vgextend database /dev/md0 
  Volume group "database" successfully extended
 

Our RAID 1 configuration is using a single disk for the time being and it does not provide any level of protection. Let’s move the data out of /dev/sdb1 and add it as a second device for /dev/md0.

Evicting the data is done by running the pvmove command.

 pvmove /dev/sdb1
  /dev/sdb1: Moved: 7.3%
  /dev/sdb1: Moved: 19.1%
  /dev/sdb1: Moved: 30.9%
  /dev/sdb1: Moved: 42.7%
  /dev/sdb1: Moved: 54.5%
  /dev/sdb1: Moved: 66.3%
  /dev/sdb1: Moved: 78.1%
  /dev/sdb1: Moved: 89.9%
  /dev/sdb1: Moved: 100.0%
 

Before changing the partition’s type we have to take it out of the database group and remove the PV that is built on top of it.

 vgreduce database /dev/sdb1
  Removed "/dev/sdb1" from volume group "database"
 pvremove /dev/sdb1
  Labels on physical volume "/dev/sdb1" successfully wiped
 

We can now safely change the partition type to “fd”.

 fdisk /dev/sdb

The number of cylinders for this disk is set to 6527.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 

Adding /dev/sdb1 to /dev/md0 is done by running mdadm.

 mdadm --manage /dev/md0 --add /dev/sdb1
mdadm: added /dev/sdb1
 

Executing the mdam command starts a synchronization process. Its task is to mirror the data from /dev/sdc1 to /dev/sdb1. We can check the process’s progress by looking at /proc/mdstat.

 cat /proc/mdstat
Personalities : [raid1] 
md0 : active raid1 sdb1[2] sdc1[0]
      52428032 blocks [2/1] [U_]
      [>....................]  recovery =  3.0% (1600000/52428032) finish=3.7min speed=228571K/sec
      
unused devices: 

The synchronization process mirrored about 3% of the data so far. When it completes, the output from /proc/mdstat will look like this.

 cat /proc/mdstat
Personalities : [raid1] 
md0 : active raid1 sdb1[1] sdc1[0]
      52428032 blocks [2/2] [UU]
      
unused devices: 
 

Our first RAID 1 configuration /dev/md0 is now operational. Let’s move on with building the second one – /dev/md1.

We start by evicting the data from /dev/sde1

 pvmove /dev/sde1
  /dev/sde1: M	oved: 6.0%
  /dev/sde1: Moved: 17.8%
  /dev/sde1: Moved: 29.6%
  /dev/sde1: Moved: 41.4%
  /dev/sde1: Moved: 53.2%
  /dev/sde1: Moved: 65.0%
  /dev/sde1: Moved: 76.8%
  /dev/sde1: Moved: 88.6%
  /dev/sde1: Moved: 100.0%
 

Next we take /dev/sde1 out of the group and remove the PV.

 vgreduce database /dev/sde1
  Removed "/dev/sde1" from volume group "database"
 pvremove /dev/sde1
  Labels on physical volume "/dev/sde1" successfully wiped
 

Changing the partition type to “fd” is done via fdisk.

 fdisk /dev/sde

The number of cylinders for this disk is set to 6527.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 

As with /dev/md0 we build the /dev/md1 RAID configuration with /dev/sde1 and a missing second disk.

 mdadm --create /dev/md1 --auto=yes -l 1 -n 2 /dev/sde1 missing
mdadm: array /dev/md1 started.
 

We create a PV on top of /dev/md1 and extend the database group with it.

 pvcreate /dev/md1
  Physical volume "/dev/md1" successfully created
 vgextend database /dev/md1
  Volume group "database" successfully extended
 

Let’s inspect the PV’s, just to be sure that everything is correct.

 pvdisplay
  --- Physical volume ---
  PV Name               /dev/sdd1
  VG Name               database
  PV Size               50.00 GB / not usable 3.31 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              12799
  Free PE               5119
  Allocated PE          7680
  PV UUID               RF37f3-SfBR-aozU-7l01-Rqh9-4YV2-1o7GdK
   
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               database
  PV Size               50.00 GB / not usable 3.25 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              12799
  Free PE               7679
  Allocated PE          5120
  PV UUID               PAEniI-bCth-w430-3gsV-ToeS-JARL-wsJ2g6
   
  --- Physical volume ---
  PV Name               /dev/md1
  VG Name               database
  PV Size               50.00 GB / not usable 3.25 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              12799
  Free PE               12799
  Allocated PE          0
  PV UUID               I9Hrbo-ehjq-vn9b-PgNa-PhAY-57tj-j8f8Vx
   
 

Our final tasks is to move the data out of /dev/sdd1 and add it as a second disk for /dev/md1.

 pvmove /dev/sdd1
  /dev/sde1: M	oved: 6.0%
  /dev/sde1: Moved: 17.8%
  /dev/sde1: Moved: 29.6%
  /dev/sde1: Moved: 41.4%
  /dev/sde1: Moved: 53.2%
  /dev/sde1: Moved: 65.0%
  /dev/sde1: Moved: 76.8%
  /dev/sde1: Moved: 88.6%
  /dev/sde1: Moved: 100.0%
 vgreduce database /dev/sdd1
  Removed "/dev/sdd1" from volume group "database"
 pvremove /dev/sdd1
  Labels on physical volume "/dev/sdd1" successfully wiped
 

We change the partition type to Linux raid autodetect (fd).

 fdisk /dev/sdd

The number of cylinders for this disk is set to 6527.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 

Adding /dev/sdd1 to /dev/md1 is done via mdadm.

 mdadm --manage /dev/md1 --add /dev/sdd1
mdadm: added /dev/sdd1

This starts the mirroring process that we already saw for /dev/md0.

 cat /proc/mdstat 
Personalities : [raid1] 
md1 : active raid1 sdd1[2] sde1[0]
      52428032 blocks [2/1] [U_]
      [=>...................]  recovery =  5.3% (2800000/52428032) finish=3.8min speed=215384K/sec
      
md0 : active raid1 sdb1[1] sdc1[0]
      52428032 blocks [2/2] [UU]
      
unused devices: 
 

When the process completes we will notice that /proc/mdstat shows two RAID 1 groups (md0 and md1).

 cat /proc/mdstat 
Personalities : [raid1] 
md1 : active raid1 sdd1[1] sde1[0]
      52428032 blocks [2/2] [UU]
      
md0 : active raid1 sdb1[1] sdc1[0]
      52428032 blocks [2/2] [UU]
      
unused devices: 
 

Looking at the information about our physical volumes we can notice that from LVM’s perspective there are only two PV’s – /dev/md0 and /dev/md1.

 pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               database
  PV Size               50.00 GB / not usable 3.25 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              12799
  Free PE               7679
  Allocated PE          5120
  PV UUID               PAEniI-bCth-w430-3gsV-ToeS-JARL-wsJ2g6
   
  --- Physical volume ---
  PV Name               /dev/md1
  VG Name               database
  PV Size               50.00 GB / not usable 3.25 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              12799
  Free PE               5119
  Allocated PE          7680
  PV UUID               I9Hrbo-ehjq-vn9b-PgNa-PhAY-57tj-j8f8Vx
   
 

Let’s take a look at the database group as well.

 vgdisplay
  --- Volume group ---
  VG Name               database
  System ID             
  Format                lvm2
  Metadata Areas        2
  Metadata Sequence No  31
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                2
  Act PV                2
  VG Size               99.99 GB
  PE Size               4.00 MB
  Total PE              25598
  Alloc PE / Size       12800 / 50.00 GB
  Free  PE / Size       12798 / 49.99 GB
  VG UUID               DcuO3X-HFRV-ZVyx-MVlD-gY3c-jyo0-FwR3uO
   
 

The group consists of two PV’s and its capacity is reduced to 100 GB. This is the price we have to pay for having mirrored copies of each of the volumes.

You probably noticed that all changes that we did for configuring the RAID groups were accomplished with mounted devices. This is one of the key advantages provided by LVM – the ability to manipulate physical disks without disrupting the system’s availability.

Increasing the RAID’s capacity

The available free space always depletes, sooner or later. There are two ways for increasing the capacity for a RAID configuration like the one we built. The first approach is to add more physical disks of the same size. The md0 group for instance consists of two drives, 50 GB each, and provides a total of 50 GB of disk space. If add two more 50 GB disks, we will increase the group’s capacity to 100 GB (4×50 GB divided by two, because of the mirroring).

Another approach would be to take out a single disk from md0 and md1 and replace them with disks, having bigger capacity. We can then use these new disks and built a third, bigger RAID 1 group (md2). We can then move all data from md0 and md1 to the newly created md2 group. After no data is left in the old RAID 1 groups, we can destroy them, replace the remaining two disks with bigger ones and built another RAID 1 group. At the end we will have two RAID 1 groups again, but they will have much bigger capacity, as they will use bigger disk drives.

We are already familiar with the process of adding new disks for increasing the capacity, so there is nothing new for us in the first approach. It will be more interesting to see how we can implement the second approach, changing the current drives with bigger ones.

Let’s start by inspecting the PV’s that are in use (we have two RAID 1 groups at the moment).

 pvdisplay
  --- Physical volume ---
  PV Name               /dev/md0
  VG Name               database
  PV Size               50.00 GB / not usable 3.06 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              12799
  Free PE               7679
  Allocated PE          5120
  PV UUID               PAEniI-bCth-w430-3gsV-ToeS-JARL-wsJ2g6
   
  --- Physical volume ---
  PV Name               /dev/md1
  VG Name               database
  PV Size               50.00 GB / not usable 3.06 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              12799
  Free PE               5119
  Allocated PE          7680
  PV UUID               I9Hrbo-ehjq-vn9b-PgNa-PhAY-57tj-j8f8Vx
   
 

Let’s remove the second and the fourth disks of the machine. We have to mark /dev/sdb1 and /dev/sdd1 as failed and remove them from their RAID groups (md0 and md1).

 mdadm --manage /dev/md0 --fail /dev/sdb1
mdadm: set /dev/sdb1 faulty in /dev/md0
 mdadm --manage /dev/md0 --remove /dev/sdb1
mdadm: hot removed /dev/sdb1
 mdadm --manage /dev/md1 --fail /dev/sdd1
mdadm: set /dev/sdd1 faulty in /dev/md1
 mdadm --manage /dev/md1 --remove /dev/sdd1
mdadm: hot removed /dev/sdd1
 

We turn of the virtual machine, remove the disks and replace them with two new drives, 200 GB each. After the change is completed, the VM’s configuration looks like this:

Our first task after the system comes up is to build “fd” partitions on the two new disks (/dev/sdb and /dev/sdd).

 fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 26108.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-26108, default 1): 
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-26108, default 26108): 
Using default value 26108

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 fdisk /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 26108.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-26108, default 1): 
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-26108, default 26108): 
Using default value 26108

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 

Next, we use mdadm to build the third RAID 1 group (md2). This time there will be no missing disks in the initial setup.

 mdadm --create /dev/md2 --auto=yes -l 1 -n 2 /dev/sdb1 /dev/sdd1
mdadm: array /dev/md2 started.
 

Looking at /proc/mdstat will show us the three RAID groups.

 cat /proc/mdstat
Personalities : [raid1] 
md2 : active raid1 sdd1[1] sdb1[0]
      209712384 blocks [2/2] [UU]
      
md1 : active raid1 sde1[0]
      52428032 blocks [2/1] [U_]
      
md0 : active raid1 sdc1[0]
      52428032 blocks [2/1] [U_]
      
unused devices: 
 

Moving the data from md0 to md2 is done by running pvmove.

 pvmove /dev/md0 /dev/md2
  /dev/md0: Moved: 10.3%
  /dev/md0: Moved: 20.6%
  /dev/md0: Moved: 30.9%
  /dev/md0: Moved: 41.5%
  /dev/md0: Moved: 51.9%
  /dev/md0: Moved: 61.7%
  /dev/md0: Moved: 72.2%
  /dev/md0: Moved: 82.7%
  /dev/md0: Moved: 93.1%
  /dev/md0: Moved: 100.0%
 

We do the same for md1.

 pvmove /dev/md1 /dev/md2
  /dev/md1: Moved: 3.5%
  /dev/md1: Moved: 21.0%
  /dev/md1: Moved: 35.0%
  /dev/md1: Moved: 45.5%
  /dev/md1: Moved: 59.4%
  /dev/md1: Moved: 73.3%
  /dev/md1: Moved: 87.2%
  /dev/md1: Moved: 100.0%
 

After md0 and md1 are freed, we can safely remove them from the database group. This is done by running vgreduce.

 vgreduce database /dev/md0
  Removed "/dev/md0" from volume group "database"
 vgreduce database /dev/md1
  Removed "/dev/md1" from volume group "database"
 

Let’s remove the PV’s as well.

 pvremove /dev/md0
  Labels on physical volume "/dev/md0" successfully wiped
 pvremove /dev/md1
  Labels on physical volume "/dev/md1" successfully wiped
 

If we run pvdisplay, we will see that the only available physical volume is the newly created /dev/md2 and that its capacity totals to 200 GB.

 pvdisplay
  --- Physical volume ---
  PV Name               /dev/md2
  VG Name               database
  PV Size               200.00 GB / not usable 1.25 MB
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              51199
  Free PE               38399
  Allocated PE          12800
  PV UUID               hmTRBJ-gjMJ-Dqnm-QhFL-6187-Zhby-vHL9ko
   
 

This is also the only physical volume used by the database group.

 vgdisplay
  --- Volume group ---
  VG Name               database
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  44
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               200.00 GB
  PE Size               4.00 MB
  Total PE              51199
  Alloc PE / Size       12800 / 50.00 GB
  Free  PE / Size       38399 / 150.00 GB
  VG UUID               DcuO3X-HFRV-ZVyx-MVlD-gY3c-jyo0-FwR3uO
   
 

We just have to remove /dev/md0 and /dev/md1. After that we should replace the two remaining disks with 200 GB ones. We can then build a new RAID 1 group and extend database with it. Performing these steps should already be trivial for you, so let’s finish this tutorial here.

There is 56 guides in the database.

More news

Bootstrap
25 February, 2014

Bootstrap

I have implemented bootstrap into the project and changed all code in the administration to use this new framework and I really love the result that I see after a couple of hours of coding. To make things look so good has never been so easy as with
Guides
20 February, 2014

Guides

I have added some of my guides for different things when it comes to the different areas in the computer, I will continue to add guides and especially when it comes to things that I am using every now and then in my own work.
First phase completed
17 February, 2014

First phase completed

Then I had time to finish the dirty work with the page and now it only remains little touches here and there and I need to upload information about what I'm doing and my progress on the project as well.
Time for a change
16 February, 2014

Time for a change

Today I've decided to update my own personal website with the latest version of my project called TQ CMS and with a completely new design that is responsive. My goal with this project is to make it as easy as possible for the end user while

Spotlight

The next step

The next step

I have now resumed my programming after a couple of months of vacation which was very much needed. I got a call one day from a friend that needed a system which can handle peoples reports on various problems like kitchen problems, water problems etc and I decided to make a system for it. Little did I know that it would totally revolutionise my own project and that it would teach me much needed jquery skills to further enhance the CMS. I also