Xen On Debian Wheezy With LVM
Author: Jason Norwood-Young jason [at] freespeechpub dot co dot za
So you've got some serious server needs. You want a bunch of servers, with a couple of operating systems, running their own crazy sites doing righteous stuff, serving the world and making it a better place. There's only one problem - you're poor. Too poor to fire up AWS servers and Rackspace servers without counting the hours they're up, and weeping every time you recieve the bill.
Don't fret - you can have what you're looking for, for way less than those cloud servers cost ya, especially if you need them up 24/7/365. With virtualization, you can use one beeeg server to host lots of little servers. It's almost like having your own cloud!
This tutorial covers installing Xen and your first virtual machine on top of Debian Wheezy. If you're using Squeeze, check out Falko Timme's tutorial, which I used as inspiration for mine (even though we do things a little differently).
0. Picking a server to run this on
So I use Hetzner Germany. Why? Because they're bloody cheap. With a virtualised environment, what you're really looking for is lots of memory. The more memory, the more virtual systems you can cram on there, and the bigger they can be. I have two 32GB servers, which lets me run about 8 servers between 1GB and 4GB each with no hassle. This isn't a recommendation of any sort and I have no relationship with Hetzner apart from forking over my hard-earned cash to them every month.
What you're looking for is a provider that will give you a root server nice and cheap, with lots and lots of memory, and the ability to remotely reboot and rescue in case you really screw up.
1. Installing Xen
I assume you've installed Wheezy, you're logged in as root, and you've done all the apt-get updates and apt-get upgrades you need to do. So let's get cracking!
apt-get install xen-linux-system xen-tools
This installs Xen, some high-mem stuff, and the very useful xen-tools which we'll use to create our virtual images later.
Next we want to tell Xen how to handle networking. I've got a small confession to make: I'm not a networking guy. In fact I detest networking. It's the plumbing of the internet, and I refuse to bend over and show my plumber's crack. And Xen networking is complicated, with weird routes and bridges and god-knows what else. So I just do the easiest, quickest, dirtiest solution - I stick it in NAT mode. (No, I don't really know what NAT is, but it works, okay? Stop asking questions.)
pico /etc/xen/xend-config.sxp
Uncomment the following two lines:
(network-script network-nat)
(vif-script vif-nat)
Even though we've installed Xen, it's not actually running. Xen runs on the kernel level, which means you actually have to boot into Xen.
First, we need to tell Debian to boot Xen instead of its usual kernel:
dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen update-grub
Then we need to reboot:
reboot
I know you can't wait to ssh into your new, awesome Xen system, but give it a second to reboot.
Okay, try it now.
How about now?
You in? We good? Good. Let's carry on.
2. Controlling Xen
So now you want to make sure that Xen is actually running. On Squeeze, you could just run uname -r, and you'd see something like: 2.6.32-5-xen-amd64. For some mysterious reason unknown to mankind, this doesn't work on Wheezy, so instead do this:
xen dmesg
You should see something like:
(XEN) Xen version 4.1.4 (Debian 4.1.4-3+deb7u1) ([email protected]) (gcc version 4.7.2 (Debian 4.7.2-5) ) Sun May 5 14:44:49 UTC 2013 (XEN) Bootloader: GRUB 1.99-27+deb7u2 (XEN) Command line: placeholder (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds (XEN) Disc information: (XEN) Found 2 MBR signatures (XEN) Found 2 EDD information structures ...
If you don't see that, you've done something wrong. Go back and try again. For everyone else, SUCCESS! Congratulations! You now have an awesome Dom0 instance!
Dom-what? Who you calling dom?
Relax chap. Dom0, or Domain 0, is your host server, the thing you're on now, the bottom of the pile. The guest servers are referred to as DomU, possibly because it also sounds slightly insulting.
xm list
Name ID Mem VCPUs State Time(s) Domain-0 0 31089 8 r—— 8.8
That command lists all our virtual servers, including Dom0. At the moment that's the only server we have running.
We can do a lot with xm, including xm create which brings a server up, xm destroy which kills a server, xm reboot which reboots a server, and xm top which monitors all the running servers.
3. Configuring LVM
You can create Xen servers either as block devices or as LVM instances. I like LVM, because it's more manly, and it also makes it waaay easier to resize the partitions later.
First we need an empty, unmounted drive to put LVM onto. I don't have an empty, unmounted drive, so I'm just going to steal one away from my server.
df -h
Filesystem Size Used Avail Use% Mounted on rootfs 1008G 915M 956G 1% / udev 10M 0 10M 0% /dev tmpfs 3.0G 292K 3.0G 1% /run /dev/disk/by-uuid/5642a4c4-1bcb-49e2-b89c-7d754e6a521a 1008G 915M 956G 1% / tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 9.2G 0 9.2G 0% /run/shm /dev/md1 496M 34M 437M 8% /boot /dev/md3 1.7T 196M 1.7T 1% /home
1.7TB for /home? Are you serious? That seems totally unneccessary.
pico /etc/fstab
Comment out the offending drive
#/dev/md/3 /home ext4 defaults 0 0
Unmount the drive
umount /dev/md3
Right, now we can create our LVM partition thingy.
pvcreate /dev/md3
Writing physical volume data to disk "/dev/md3" Physical volume "/dev/md3" successfully created
vgcreate xen-vol /dev/md3
vgdisplay
--- Volume group --- VG Name xen-vol System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 1.71 TiB PE Size 4.00 MiB Total PE 448996 Alloc PE / Size 0 / 0 Free PE / Size 448996 / 1.71 TiB VG UUID wa0QCN-lBc1-545B-zMwf-vIuS-VXXI-GZAfbW
Righteous!
4. Configuring Xen Tools
Xen Tools makes our Xen life much easier, especially when it comes to creating new servers. But first we need to configure it a bit.
pico /etc/xen-tools/xen-tools.conf
lvm = xen-vol size = 20Gb # Disk image size. memory = 4Gb # Memory size swap = 512Mb # Swap size gateway = 192.168.1.2 netmask = 255.255.0.0 broadcast = 192.168.1.255 mirror = http://ftp.de.debian.org/debian/ mirror_precise = http://archive.ubuntu.com/ubuntu # (and any others you want) serial_device = hvc0 disk_device = xvda #default copyhosts = 1
5. Creating a guest server
Time for the main event! This is the process you'll repeat every time you want to create a new Xen guest server.
For our first test server, I'm going to make something a bit smaller than the defaults we've set up in xen-tools.conf. You can override any of the defaults on the command line, depending on what you want. Since we're not using a dhcp server, you'll need to set "hostname" and "ip" every time. Everything else is optional.
xen-create-image --hostname=test --ip=192.168.1.99 --memory=512Mb --size=5G --swap=256Kb
General Information
--------------------
Hostname : test
Distribution : wheezy
Mirror : http://ftp.de.debian.org/debian/
Partitions : swap 256Kb (swap)
/ 5G (ext3)
Image type : full
Memory size : 512Mb
Kernel path : /boot/vmlinuz-3.2.0-4-amd64
Initrd path : /boot/initrd.img-3.2.0-4-amd64
Networking Information
----------------------
IP Address 1 : 192.168.1.99 [MAC: 00:16:3E:27:33:2D]
Netmask : 255.255.0.0
Broadcast : 192.168.1.255
Gateway : 192.168.1.2
Creating swap on /dev/xen-vol/test-swap
Done
Creating ext3 filesystem on /dev/xen-vol/test-disk
Done
Installation method: debootstrap
Running hooks
Done
No role scripts were specified. Skipping
Creating Xen configuration file
Done
No role scripts were specified. Skipping
Setting up root password
Generating a password for the new guest.
All done
Logfile produced at:
/var/log/xen-tools/test.log
Installation Summary
---------------------
Hostname : test
Distribution : wheezy
IP-Address(es) : 192.168.1.99
RSA Fingerprint : de:92:b7:63:55:20:74:46:d3:de:cc:ff:55:22:69:24
Root Password : MlkaIn8
This creates the config file and the partition for us, but doesn't boot the server for us. The config file is located at /etc/xen/hostname.cfg. You can edit this file directly if you need to change stuff like memory later.
Let's go ahead and start the server
xm create /etc/xen/test.cfg
Using config file "/etc/xen/test.cfg". Started domain test (id=1)
What's pretty cool is that "test" will be added to our /etc/hosts file, which means we can just do this:
ssh test
Use the password that you received at the end of the xen-create-image output, and you're in!
Make sure you can see the interwebs:
ping google.com
PING google.com (173.194.70.113) 56(84) bytes of data. 64 bytes from fa-in-f113.1e100.net (173.194.70.113): icmp_req=1 ttl=48 time=6.05 ms 64 bytes from fa-in-f113.1e100.net (173.194.70.113): icmp_req=2 ttl=48 time=6.03 ms ^C --- google.com ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 6.030/6.044/6.058/0.014 ms
6. Have a beer
You deserve it!
7. Pointing IP addresses at your virtual servers
You'll need a bunch of IP addresses from your ISP (called a subnet). I pay a little more a month for an extra eight IP addresses. But getting them to go straight through to your virtual server takes a bit of work.
pico /etc/network/interfaces
auto br0 iface br0 inet static address 192.168.1.2 netmask 255.255.0.0 pre-up brctl addbr $IFACE post-down brctl delbr $IFACE
/etc/init.d/networking restart
ifconfig br0
br0 Link encap:Ethernet HWaddr 82:18:d1:b8:37:c9
inet addr:192.168.1.2 Bcast:192.168.255.255 Mask:255.255.0.0
inet6 addr: fe80::9118:d1ff:feb8:37c9/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:172 (172.0 B)
ip addr add 80.1.2.41 dev eth0
iptables -A PREROUTING -t nat -p all -i eth0 -d 80.1.2.41 -j DNAT --to 192.168.1.99
Use --bridge=br0 when using xen-create. For existing /etc/xen/*.cfg files, edit this line:
vif = [ 'ip=192.168.1.99,mac=00:15:3F:B4:AC:34' ]
to look like this:
vif = [ 'ip=192.168.1.99,mac=00:15:3F:B4:AC:34','bridge=br0' ]
Destroy and create the virtual server if necessary.
