Monday, 9 April 2012

Xen Part 12: Windows Guest Installation


Create a logical volume for Windows.

# lvcreate -L 50G -n winxp-disk xendomu

Create a Xen configuration file, specifying the location of your Windows installation media.

# vim /etc/xen/ace2x2

kernel = '/usr/lib/xen/boot/hvmloader'
device_model = '/usr/lib/xen/bin/qemu-dm'
builder = 'hvm'
memory  = '2048'
cdrom = 'file:/os/Windows XP/windowsxp.iso'
disk    = [ 'phy:/dev/xendomu/winxp-disk,hdc,w', 'file:/os/Windows XP/windowsxp.iso,hdb:cdrom,r' ]
name    = 'ace2x2'
vif  = [ 'bridge=guestbr' ]
boot='dc'  #CDROM=d, Disk=c.
vnc=1 # Can remove this after installation
vncviewer=1
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'

Start the domain, which should boot straight into the installation media.

# xen create /etc/xen/ace2x2
# xen vncviewer ace2x2

The Cursor Doesn't Track Mouse Movement Properly 

You'll probably find that the doesn't align with your mouse movement within the VNC window. You just need to add usbdevice = 'tablet' to your xen config file.

Next > VGA Passthrough: Another failed attempt

Xen Part 11: ATi Graphics drivers on the domU

If you're looking to run a raft of monitors on your domU, and require fast 3D rendering and more, you're probably looking to run a proprietary driver. I have a HD 6970 which I'm using for passthrough, so the following discusses the AMD/ATI driver only.

First things first: follow my instructions for installing the proprietary ATI driver.

When you try to start it, you'll more than likely find it fails with the following error.

# modprobe -v fglrx

insmod /lib/modules/3.2.13/kernel/drivers/acpi/button.ko 
WARNING: Error inserting button (/lib/modules/3.2.13/kernel/drivers/acpi/button.ko): No such device
FATAL: Error inserting fglrx (/lib/modules/3.2.13/updates/dkms/fglrx.ko): No such device

Unfortunately, this isn't awfully easy to fix. You'll find the AMD fglrx module has a single dependency; the same module that gave the WARNING above:

# modinfo fglrx | grep depends
depends:        button

Predictably, if you try loading this dependency, you get the same fatal error: "No such device".


# modprobe -v button

insmod /lib/modules/3.2.13/kernel/drivers/acpi/button.ko 
FATAL: Error inserting button (/lib/modules/3.2.13/kernel/drivers/acpi/button.ko): No such device

Further investigation doesn't yield many results. There are no errors reported in dmesg. What can the problem be?

# modinfo button | grep description
description:    ACPI Button Driver
# ls /proc/acpi
ls: cannot access /proc/acpi: No such file or directory
# dmesg | grep ACPI | head -n 1

[    0.000000] ACPI in unprivileged domain disabled

It's an ACPI driver, but ACPI isn't loaded. So we found the problem. If we look in arch/x86/xen/setup.c we find what's responsible for disabling ACPI:

#ifdef CONFIG_ACPI
        if (!(xen_start_info->flags & SIF_INITDOMAIN)) {
                printk(KERN_INFO "ACPI in unprivileged domain disabled\n");
                disable_acpi();
        }
#endif

In the domU we won't have ACPI, and we won't have /proc/acpi/*. Which means that modprobe button isn't ever going to work, and by extension nor is modprobe fglrx. So how can we install the proprietary ATI driver if we can't satisfy its dependency?

I'm sure brighter people than me have stumbled across this little irritation and figured out a sensible solution. After an hour or so I still didn't have any bright ideas, which left me with only one terrible option: make a stub ACPI Button driver. After all, I don't think I care whether the graphics driver knows when the metaphorical power button ("xen shutdown") is pressed.

All this really entails is stripping the ACPI includes and much of the code from drivers/acpi/button.c, and finally playing around to get the thing compiling. The resulting "stub" driver (based on kernel 3.2.13) is here:


Instructions: First, look over the file above to check you're not downloading nefarious code. (Obviously it's clean, but it's important to get into that habit. Nobody else is looking over your shoulder checking these things for you). Second, take a backup of the original drivers/acpi/button.c file in your kernel source tree. Third, replace the original with this bastardised stub copy. Finally, recompile the kernel and copy the module over to domU. Or, in code:

$ cp drivers/acpi/button.c drivers/acpi/button.c.original
$ wget http://content.nixnotes.co.uk/xen/acpi/button.c -O drivers/acpi/button.c
$ make clean
$ make -j 10 modules
# xen shutdown ace2x1
# mount /dev/xendomu/ace2x1-disk /mnt
# mv /mnt/lib/modules/3.2.13/kernel/drivers/acpi/button.ko /mnt/lib/modules/3.2.13/kernel/drivers/acpi/button.ko.original
# cp ./drivers/acpi/button.ko /mnt/lib/modules/3.2.13/kernel/drivers/acpi/button.ko
# umount /mnt
# xen create /etc/xen/ace2x1

Connect to the domU

root@ace2x1:~# lsmod | grep -E "(button|fglrx)"
fglrx                3150992  3 
button                 12535  1 fglrx
root@ace2x1:~# dmesg | grep fglrx
[    4.011571] fglrx: module license 'Proprietary. (C) 2002 - ATI Technologies, Starnberg, GERMANY' taints kernel.
[    4.047300] [fglrx] Maximum main memory to use for locked dma buffers: 1876 MBytes.
[    4.047320] [fglrx:firegl_init_device_list] *ERROR* No supported display adapters were found
[    4.047331] [fglrx:firegl_init_module] *ERROR* firegl_init_devices failed
[    5.966928] [fglrx] Maximum main memory to use for locked dma buffers: 1876 MBytes.
[    5.966974] [fglrx]   vendor: 1002 device: 6718 count: 1
[    5.967189] [fglrx] ioport: bar 4, base 0xe000, size: 0x100
[    5.967503] [fglrx] Kernel PAT support is enabled
[    5.967527] [fglrx] module loaded - fglrx 8.95.3 [Mar  8 2012] with 1 minors
[   11.422975] [fglrx] ACPI is disabled on this system
[   11.877991] [fglrx] Firegl kernel thread PID: 921
[   11.878106] [fglrx] Firegl kernel thread PID: 922
[   11.878223] [fglrx] Firegl kernel thread PID: 923
[   11.878359] [fglrx] IRQ 58 Enabled
root@ace2x1:~# aticonfig --initial

root@ace2x1:~# grep fglrx /etc/X11/xorg.conf
Driver      "fglrx"


Finally, we have to ensure the xen-pcifront driver loads before fglrx, otherwise we'll get errors about fglrx being unable to find any display adapters. The ordering of the following is important.


root@ace2x1:~# vim /etc/modules
xen_pcifront
fglrx


Next > Windows Guest Installation

Sunday, 1 April 2012

Xen Part 10: Compiling a Custom DomU Kernel

I upgraded the kernel on the Debian dom0, and switched domU to the newer kernel. Suddenly things started to break, including PCI passthrough.

Xen uses PV kernels residing on the dom0, but the modules obviously need to reside on the domU. In this case, it looks like module compatibility broke when I upgraded the kernel for domU without replacing its modules.

In my opinion, this separation between the kernel on dom0 and the modules on domU is rather odd and inconvenient. It seems like one should mount the domU partition in ro mode and load the kernel from there; in that way, the domU would be in near-complete control of its own kernel.

However, I've never seen a method like that mentioned. The other obvious workaround is to compile a domU kernel without any modules, but that sounded like a terrible idea. So I figured I'd try a compile-on-dom0 solution first.

Remember, do not use root for kernel compilation. The example below is for 3.2.13.

Get the kernel sources for the version you want from kernel.org.

$ VERSION=3.2.13
$ wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-$VERSION.tar.bz2
$ tar xjf linux-$VERSION.tar.bz2
$ cd linux-$VERSION

Here I'm basing the Ubuntu domU kernel config on my Debian Wheezy dom0 kernel config; whether that's wise is another question.

$ make oldconfig
Unless you have special requirements, just hold down enter to accept the defaults
$ make menuconfig
Make any customisations you require
$ vim .config
Further customisations; e.g., search for XEN and check everything you require is enabled ('y' or 'm')
$ make -j 10 
$ make -j 10 modules
# VERSION=3.2.13
# cp arch/x86_64/boot/bzImage /boot/bzImage-$VERSION
# make modules_install
# update-initramfs -c -k $VERSION 

Mount your domU partition/image (you'd be well advised to stop your domU first, otherwise... well, you can figure it out)

# xen shutdown ace2x1
# mount /dev/xendomu/ace2x1-disk /mnt

Copy the modules and source to domU. The second command in particular may take a while.

# make INSTALL_MOD_PATH=/mnt modules_install
# cp -ru . /mnt/usr/src/linux-$VERSION
# cp .config /mnt/boot/config-$VERSION
# umount /mnt

Use the compiled kernel for the domU

# vim /etc/xen/ace2x1
Set the kernel and ramdisk variables to /boot/bzImage-3.2.13 and /boot/initrd.img-3.2.13 respectively 

Back on the domU, just finish tidying up after the hacks.

# xen create /etc/xen/ace2x1
# xen console ace2x1 / $ ssh root@ace2x1
root@ace2x1# cd /lib/modules/3.2.13
root@ace2x1# rm build source
root@ace2x1# ln -s /usr/src/linux-3.2.13 build
root@ace2x1# ln -s /usr/src/linux-3.2.13 source


Next > ATi Graphics drivers on the domU