Friday, November 17, 2006
8:35 AM

Installing Xen 3.0.3 from source on Ubuntu Edgy

A step by step tutirial for people getting the most out of their PC with Xen virtualization
Getting the latest version of Xen

mkdir /home/xen
wget http://bits.xensource.com/oss-xen/release/3.0.3-0/src.tgz/xen-3.0.3_0-src.tgz


I used xen-3.0.3_0-src.tar from xen source. It contains the folders
patches\
There are a couple of generic linux patches in patches/ and these can be added to over the course of time.
linux-2.6-xen-sparse
This is the xen source tree

Adding Packages to system prior to install
apt-get update

apt-get install iproute bridge-utils python-twisted gcc-3.3 binutils make libcurl3-dev zlib1g-dev python-dev transfig bzip2 screen ssh debootstrap libcurl3-dev x-dev xorg-x11-devel libncurses5-dev tetex-base

These are the important packages I downloaded after many frustrating attempts at failed makes.


gunzip xen-3.0.3_0-src.tgz

tar -xvf xen-3.0.3_0-src.tar

Compiling Xen
Making changes to Makefile and Config.mk prior to install

Add the following options to CFLAGS -fno-stack-protector and -mno-tls-direct-seg-refs

make world

make install
Folders Created by make world followed by make install

pristine-linux-2.6.16.29 :- We will refer to original "untainted" kernel source directory that comes with the distribution as the pristine kernel sources. Pristine is unmodified reference and has patches applied from the patches/ directory.

linux-2.6.16.29-xen[0U]:- A xenized Linux

dist :-
Inside dist the boot folder contains files to be copied to /boot, the lib/modules/2.6.16.29-xen0 to be copied to /lib/modules and usr/include/xen to be copied to /usr/include

Files Created by install.sh

The above copying is performed by this script. The important files are

1) xen-3.0.3-0.gz is the Xen hypervisor. It is the one that boots the machine in grub config file with label "kernel".It's the lowest layer of software in the system. You need it but it's not much use on its own - you also need a kernel for domain 0 (usually Linux)

2) vmlinuz-2.6.16.29-xen is the kernel for domain 0, and should be called by xen-3.0.3-0.gz for starting domain0, with label "module" in the grub config.

Other than these two there are configuration files and syms (debug) versions of these also created in the boot directory.

Compiling Dom0
make linux-2.6-xen0-config CONFIGMODE=menuconfig KERNELS="linux-2.6-xen0"

In the kernel comfiguration menu that shows up we enable quota, iptables, dummy network driver as modules.

make linux-2.6-xen0-build
make linux-2.6-xen0-install
depmod 2.6.16.29-xen0
./install.sh

This will copy the Dom0 kernel and the Xen VMM to the boot directory. Now we create the initrd

mkinitramfs -o initrd.img-2.6.16.29-xen0 2.6.16.29-xen0


Move all xen related files to a separate folder inside boot
cd /boot
mkdir xen
mv *xen* xen

Setting up Boot Parameters

I added the following entry to my /boot/grub/menu.lst
title Xen 3.0, kernel 2.6.16-xen <-- Label for boot screen selection
root (hd0,1)
<--Harddrive and partition where your MBR resides
kernel /boot/xen/xen-3.gz root=/dev/hda1 dom0_mem=400000
<--Xen VMM module /boot/vmlinuz-2.6.16.29-xen ro console=tty0<--Dom0 ker
module /boot/initrd.img-2.6.16.29-xen
<--initrd image. vmlinuz-2.6.11-9-xenU is the kernel that should be used to boot the other domains, the guests.


Usually, it is in the /etc/xen/domU config file as "kernel" parameter. This kernel is a bit smaller than the xen0 kernel but can only run in domUs. You may want to use it in your guests, or you can just use the xen0 kernel everywhere.

Final Checks
To improve performance disable TLS
mv /lib/tls /lib/tls.disabled

Make sure that hotplug udevs shortcuts have been created. There should be a softlink to xen-backend.rules in /etc/udev/rules.d. Make sure it starts with number and not a character.

Reboot Machine into Xen
Edit /etc/apt/sources.list
Uncomment the following lines
deb http://us.archive.ubuntu.com/ubuntu/ edgy universe
deb-src http://us.archive.ubuntu.com/ubuntu/ edgy universe
deb http://security.ubuntu.com/ubuntu edgy-security main restricted
deb-src http://security.ubuntu.com/ubuntu edgy-security main restricted
deb http://security.ubuntu.com/ubuntu edgy-security universe
deb-src http://security.ubuntu.com/ubuntu edgy-security universe


Compile DomU Kernel
cd /home/xen/xen-3.0.3_0-src/
make linux-2.6-xenU-config CONFIGMODE=menuconfig KERNELS="linux-2.6-xenU"


make linux-2.6-xenU-build
make linux-2.6-xenU-install
depmod 2.6.16.29-xenU


Download, Install and Configure xen-tools
mkdir /home/xen
apt-get update
apt-get install xen-tools
vim /etc/xen-tools/xen-tools.conf
dir = /home/xen
debootstrap = 1
size = 6Gb # Disk image size.
memory = 128Mb # Memory size
swap = 256Mb # Swap size
fs = ext3 # use the EXT3 filesystem for the disk image.
dist = edgy # Default distribution to install.
image = sparse # Specify sparse vs. full disk images.
gateway = 192.168.1.1
netmask = 255.255.255.0
passwd = 1
kernel = /boot/vmlinuz-2.6.16.29-xenU
initrd = /boot/initrd.img-2.6.16.29-xen0
mirror = http://ftp.us.debian.org/debian/
mirror = http://gb.archive.ubuntu.com/ubuntu/


Make sure Xend is running
xend start

Create a new virtual Machine
xen-create-image --hostname=vmubuntu1
xm create vmubuntu1.cfg -c


Network Setup
The above xen-tools configuration assigned a static IP address but if you are using dhcp for dom0 you might just want to stick to that for the DomUs.

iface eth0 inet dhcp <-- stick this into your /etc/network/interface file in DomU Make sure that your Dom0's /etc/network/interfaces file is configured correctly because the Xen network scripts pick up the settings from there. I also had to completely get rid of NetworkManager that I was using to manage by Wireless interfaces.
The VM image config file vmubuntu1.cfg should only have the following networking options.

dhcp = 'dhcp'
vif = [ ' ' ]

Make sure the bridge options are configured in /etc/xen/xend-config.sxp.


(network-script network-bridge)
(vif-script vif-bridge)


An ifconfig on Dom0 should show a eth, vif, peth and a xenbr interface. If you don't see these then something broke along the way and you need to try running the networking scripts in /etc/xen/scripts manually and examine the errors. What are all these new interfaces you ask? Well here is how the default bridged networking is supposed to work.

      Domain0 network stack
^^
||
||
eth0 [local virtual interface]
^^
||
|| local traffic intended for domain0
||
||
xen-br0[bridge]<===> vif0.0 [virtual dom0] <===> eth0 [domU virtual nic]
||
||
|| external traffic
||
VV
peth0 [real, physical interface]
||
--XX ------------ NETWORK -----------------

If you are not too keen on the default network configuration scripts, it is also easy to set up a bridge yourself

brctl addbr xenbr0
brctl stp xenbr0 off
brctl sethello xenbr0 0
brctl setfd xenbr0 0
ifconfig xenbr0 192.168.1.1 netmask 255.255.255.0 up


Copy over the library files to the new virtual disk
mkdir /mnt/vmdisk
mount -o loop /home/xen/domains/disk.img /mnt/vmdisk
cd /mnt/vmdisk/lib/modules
cp -a /lib/modules/2.6.16.29-xenU .
umount /mnt/vmdisk/


Check that the domains running
xm list

0 comments:

Post a Comment