Warning: I recently went through this process, and took notes, but a few pieces here are from memory. If things don’t work as expected, please let me know and I’d be happy to help.
I’ve been running Debian for a long time – probably steadily about 6 years now. Before that I was a BSD fan, so many things about Debian appealed to me.
Over time my needs have changed. Much of my work involves using CentOS, and I quite like the ease of managing it. Debian is still nice, but it always had a few things I wasn’t a fan of, such as split (
conf.d style) configurations. No disrespect to Debian, it’s a solid distro, the style of distribution I’m drifting to is purely personal preference.
My existing systems are all Debian VMs which live in a Debian Xen Dom0.
Given our Debian Dom0, we want to install a CentOS 6 DomU.
The DomU must have XFS on the filesystem, and load it’s own recent kernel rather than the one supplied by the Debian Dom0.
Problems and solutions
- Installing a CentOS (or RedHat, or Fedora) client is non-trivial. Debian has
debootstrapwhich bootstraps the initial OS on the filesystem. We will use
- We need a newer kernel. CentOS 6 doesn’t come with Xen enabled kernels since RedHat pulled Xen support from their kernels, so we need a 3rd party kernel, or to build our own.
- We need to load the new kernel. In order to do this we need to use
pygrubwhich comes with Xen.
pygrubdoesn’t understand reading XFS filesystems, so we need a separate
/bootpartition which it can read the grub menu.
The right tools for the job
To make this work, we need the following tools:
pygrub(comes with Xen)
apt-get install mock)
Xen and LVM and filesystems
This particular setup uses LVM to create filesystems for each virtual machine. Many moons ago I found this to be more performant and cause fewer issues than file-backed filesystems. However that isn’t the case these days. My examples may show LVM partitions being used but this isn’t necessary.
Additionally, I wanted to start using XFS on the new system. After 6 hours of trying to make it work, I realised that
pygrub doesn’t yet support reading XFS filesystems.
If you want to use XFS on your root filesystem, you’ll need an
/boot partition which
pygrub will boot from.
Basic VM initialisation
Create two filesystems which will be used by the new VM. My LVM volume group is called
xen-vol. Yours will probably be called something else. If you want to use file-backed filesystems, you can do that instead at this point. Here’s the LVM way:
$ lvcreate -n newvm-boot -L 200M xen-vol
$ lvcreate -n newvm-root -L 100G xen-vol
mock is the tool we use to create the bare VM. Normally it is used to create a chroot environment for tasks like building packages. However it works really well for creating a basic OS layout which can then be booted.
By default it doesn’t come with a configuration file for CentOS 6, so create the file
/etc/mock/centos-6-i386.cfg and populate it with this:
Populating the VM
Now we’ll mount and populate the VM using
mkdir -p /mnt/newvm/boot
mount /dev/xen-vol/newvm-root /mnt/newvm
mount /dev/xen-vol/newvm-boot /mnt/newvm/boot
mock --init -r centos-6-i386 /mnt/newvm
mock will now set up a mostly complete chroot.
Configure the VM
For each of the following steps, you will need to be chrooted into
chroot /mnt/newvm /bin/bash
Install and configure Grub
Grub is the bootloader we use, so install that:
yum install -y grub
Now you’ll need a
/boot/grub/grub.conf file. Create it and make it look like this:
default=0 timeout=2 title CentOS 6.3 i686 root (hd1,0) kernel /vmlinuz-3.6.7-1.el6xen.i686 ro root=/dev/xvdb crashkernel=auto noquiet rdshell initrd /initramfs-3.6.7-1.el6xen.i686.img
Now make a new
/boot/grub/device.map file which looks like this:
(hd0) /dev/xvda (hd1) /dev/xvdb
The astute readers will notice that we just told Grub to boot from a kernel which doesn’t exist! Let’s do that next.
Install the new kernel
One of the beautiful things about Linux is the way people love to help each other. In our case, someone has built the kernel we need to run CentOS 6 as a Xen DomU client. To install it do:
yum install http://au1.mirror.crc.id.au/repo/kernel-xen-release-6-4.noarch.rpm
yum install kernel-xen
You’ll need an
/etc/fstab file, like this:
/dev/xvda /boot ext3 noatime 1 1 /dev/xvdb / xfs noatime 1 1 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0
And set the root password:
$ passwd root
Leave the chroot and return to the Dom0 shell.
You will need a Xen configuration file for this VM, which should look something like this:
bootloader = '/usr/bin/pygrub' extra = 'console=hvc0 xencons=tty' memory = '256' vcpus = 2 name = "newvm.example.com" hostname = "newvm.example.com" vif = [ 'ip=<ip address>' ] netmask = '<netmask>' gateway = '<gateway address>' ip = '<ip address>' broadcast = '<broadcast address>' disk = [ 'phy:xen-vol/newvm-boot,xvda,w', 'phy:xen-vol/newvm,xvdb,w' ]
Boot the VM
Umount the partitions andstart the new VM:
xm create -c