Back Up LVM XEN Guest Containing LVs

In my day-job all our Linux boxes (bar 3) are Xen VMs. I wanted a way to take a backup of these with out the risk of the files changing underneath. For performance reasons I am running all of them on Logical Volumes.

Within these VMs the DomU OS is once again using LVM for various reasons. This does create some headaches for taking the backup.

The basic idea is:

1. Create a snapshot Logical Volume
2. Allow the Dom0 to see the Logical Volumes within the Logical Volume (phew).
3. Mount the newly discovered Logical Volumes.
4. Take backup
5. Tidy up

There are couple of assumptions that I have made. I have assumed that you have used custom names for the Volume Groups in either (or both) the Dom0 (host) or DomU (guest). Personally, I always use a kickstart file to create my DomUs, so I have the main VG set to XenVG00. All my DomU’s are all in a VG on the Dom0 called vg_xen - this is not essential though.

The second assumption is that you have some spare space in the Volume Group holding the DomU images. It does not need to be much, as we will be removing the snapshot volume when we have finished with it.

First off, we need to create our snapshot:

[root@dom0 ~]# lvcreate -L 1G -s -n lv_snapshot/dev/vg_xen/lv_xen_snaptest

Logical volume “lv_snapshot” created

This creates a snapshot volume called lv_snapshot of the a DomU which is using a Dom0 LV called lv_xen_snaptest. This is useful in itself as you could now take an image of this LV. I do exactly this for Disaster Recovery (dd if=/dev/vg_xen/lv_snapshot | bzip2 | dd of=/path/to/dr_image.bz2). In a worst case scenario, this image could be dd’ed back into a new Logical Volume, on a new system, for a new instance of Xen to run.

What we want to do though is get to the files in this snapshot. As the DomU is using LVM, this is unfortunately non-trivial. First we have to use kpartx to extract the LV data:

[root@dom0 ~]# kpartx -av /dev/vg_xen/lv_snapshot

add map lv_snapshot1 : 0 208782 linear /dev/vg_xen/lv_snapshot 63
add map lv_snapshot2 : 0 20755980 linear /dev/vg_xen/lv_snapshot 208845

You can see 2 LVs have been found. As lv_snapshot1 is smaller, we know that is the swap file and lv_snapshot2 is the / file system. We now need to get LVM to see these LVs and activate them.

[root@dom0 ~]# vgscan

Reading all physical volumes. This may take a while…
Found volume group “vg_xen” using metadata type lvm2
Found volume group “XenVG00″ using metadata type lvm2
Found volume group “VolGroup00″ using metadata type lvm2

[root@dom0 ~]# vgchange -ay XenVG00

2 logical volume(s) in volume group “XenVG00″ now active

Now the LVs are active we can mount them as if they were any other disk.

[root@dom0 ~]# mount /dev/XenVG00/LVroot /mnt/

And run any command we like

[root@dom0 ~]# rsync -avhp /mnt/ /backup/xen_test/

Obviously we now have to clean after ourselves:

[root@dom0 ~]# umount /mnt
[root@dom0 ~]# vgchange -an XenVG00

0 logical volume(s) in volume group “XenVG00″ now active

[root@dom0 ~]# kpartx -d /dev/vg_xen/lv_snapshot
[root@dom0 ~]# lvremove /dev/vg_xen/lv_snapshot

Do you really want to remove active logical volume “lv_snapshot”? [y/n]: y
Logical volume “lv_snapshot” successfully removed

The important thing here is that we have not at any point paused our VM, but we have been able to make a full backup of it.

The size of the snapshot volume is the hard bit here. I have used a 1G snapshot LV. To give an idea of how big it needs to be: by the time a DR image of my wiki has been taken, bzipped and sent over SSH to another box for storage, the snapshot usage has reached nearly 7%.