The Book of Xen

Chapter2.GETTING STARTED

Fascinating though the theoretical underpinnings and implementation details of Xen are, we should probably move on to working directly with Xen for a bit of practice. There is, after all, no subst.i.tute for experience.

So! Welcome to Xen. This chapter is an easy quick start aimed at gently introducing Xen to one of your machines. We will hold your hand and not let it go.

Because this is a detailed walk-through, we"re going to give focused, specific instructions, deviating from our normal policy of being vague and distro-agnostic. For the purposes of this chapter, we"ll a.s.sume you"re installing CentOS 5.x with the server server defaults and using its built-in Xen support. defaults and using its built-in Xen support.

If you"re using something else, this chapter will probably still be useful, but you might have to improvise a bit-the goals will be the same but the steps will probably be different.

RED HAT VS. CENTOS VS. FEDORASo what is this CentOS, and why are we using it? The short answer is that CentOS, the Community ENTerprise OS Community ENTerprise OS, is an RPM-based distro derived from Red Hat Enterprise Linux with all of Red Hat"s trademarks stripped out. We"re focusing on it here because it"s well supported by Xen, reasonably generic in structure, and fairly popular. Additionally, it"s more stable than Fedora and a lot cheaper than Red Hat"s official product, Red Hat Enterprise Linux (RHEL for short). for short).We have difficulty recommending Fedora for production use simply because the release cycle is unrealistically fast. While Red Hat handpicks updates for stability, Fedora operates as Red Hat"s pressure cooker. Fedora applies the same sort of philosophy as Red Hat to picking kernel patches and such, but they release far more often. (If you"ve ever run a stock 2.6 kernel, you know that someone someone needs to handpick updates, be it you or your distro overlords, to get anything like enterprise performance. needs to handpick updates, be it you or your distro overlords, to get anything like enterprise performance.[14]) Fedora releases could be considered alpha versions of the next RHEL, and therefore, like any alpha software, we hesitate to rely on it.The reason we suggest CentOS rather than Red Hat Enterprise Linux is simply that RHEL is quite expensive. (Especially by Linux standards.) It may well be worth it if you want support-but we"ll stick to CentOS. It"s a good product, benefiting from Red Hat"s years of work with Linux, and it"s stable and easy to manage.Red Hat"s effort shows to particular advantage in the work they"ve done integrating Xen. Red Hat has taken Xen and done a fair amount of work to make it a product. Thankfully, because Xen is open source, everyone benefits.

In general, the goals for this walk-through are as follows: Make sure your hardware can run Xen.

Install a basic OS.

Install Xen.

Familiarize yourself with the Xen environment.

Install a domU.

Log in to your domU and configure it to make sure everything works.

Rest.

Hardware Compatibility First, make sure that your hardware"s up to the task of running Xen. (It almost certainly is.) All you need is a Pentium Pro or better, 512MiB of memory,[15] and a few hundred MiB of hard drive s.p.a.ce. If you can"t manage that, dig some change out of your couch cushions and buy a machine. The PPro came out in, what, 1996? Haven"t you heard? This Is The Future. and a few hundred MiB of hard drive s.p.a.ce. If you can"t manage that, dig some change out of your couch cushions and buy a machine. The PPro came out in, what, 1996? Haven"t you heard? This Is The Future.

At present, Xen runs only on x86 (that is, Intel and AMD) processors and IBM"s PowerPC. X86_64-the 64-bit extension to the x86 instruction set-is supported on both AMD and Intel processors. Xen also supports Intel"s Itanium. For the sake of this walk-through, we"ll a.s.sume that you"re using an x86 or x86_64 machine.

Our test box, for example-chosen to be as common as possible-was a three-year-old Dell, with a Pentium 4, 1GB of RAM, and a quant.i.ty of hard drive s.p.a.ce beyond my wildest imaginings. Personally, I think it"s all so fast it makes me sick.

Anyway, so you"ve got a machine capable of running Xen. Congratulations. First, it needs a basic operating system that Xen can run on top of.

NoteRun on top of is probably not the best way of characterizing Xen"s interaction with the dom0 OS, in view of the fact that the dom0 kernel runs on the hypervisor, but it is a convenient and frequently used phrase. We beg your patience, Constant Reader is probably not the best way of characterizing Xen"s interaction with the dom0 OS, in view of the fact that the dom0 kernel runs on the hypervisor, but it is a convenient and frequently used phrase. We beg your patience, Constant Reader.

[14] We recognize that some people may disagree with this opinion. We recognize that some people may disagree with this opinion.

[15] The absolute minimum would probably be 128MiB, but CentOS itself requires 256, and each domU will also require a significant amount of memory. The absolute minimum would probably be 128MiB, but CentOS itself requires 256, and each domU will also require a significant amount of memory.

Installing CentOS First, we"ll install CentOS in a completely ordinary way. Put the install medium in the drive, boot from it, and install it according to your preference. We opted to accept the default part.i.tioning, which creates a small /boot /boot part.i.tion and devotes the rest of the drive to an LVM group, with a logical volume for swap and a volume for root. We also accepted the default configuration for the GRUB boot loader and the default network config. part.i.tion and devotes the rest of the drive to an LVM group, with a logical volume for swap and a volume for root. We also accepted the default configuration for the GRUB boot loader and the default network config.

NoteNow would also be a good time to make sure you have some sort of Internet access.

Set your time zone and enter a root pa.s.sword. We"re just running through the standard CentOS install process at this point-it"s probably familiar territory. Follow the prompts as usual.

Next comes package selection. We chose the virtualization server package group, since that"s the one that includes the Xen hypervisor and supporting tools, and left the rest blank. If you"d like to, you can also choose other package groups to install, like the GNOME desktop or server-gui set of packages, without modifying any of the steps in this section.

Select Next. Now the machine will install the packages you"ve selected. This may take a while, varying with package selection and install medium. It took us about 15 minutes to install from a DVD. When that"s done, the machine will reboot and give you the chance to do postinstall configuration-firewall, services, SELinux, and so on.

At this point you may wish to do other system configuration related stuff, not directly related to Xen. Feel free.

Now, we"re ready to create a virtual machine. But first let"s look at Xen"s boot messages and get familiar with the Xen environment.

THE LIVECDSo, you downloaded the LiveCD? May as well try it. It"s pretty slick, but when it"s booted, it"s just another operating system. This captures what makes Xen so fascinating-its sheer ba.n.a.lity. After all this work, you wind up sitting in front of a Linux box. Yes, it will pretend to be multiple Linux boxes on demand, but there"s not really much "there" there, to borrow a well-turned phrase from Gertrude Stein.The LiveCD does showcase some useful features, though. For example, it uses a copy-on-write filesystem to give each Xen domain persistent writable storage across VM reboots. The LiveCD also has some neat scripts that use the XenBus to pop up a VNC console automatically on domU startup. (See Chapter14 Chapter14 for more information on that.) for more information on that.)It"s a nice demo, but it"s not really useful for production, and it doesn"t give you the sort of hands-on experience that comes with setting up a machine. The other problem is that the LiveCD hasn"t been updated in a while. The last version (as of this writing) is two years old and is based on Xen 3.0.3. There have since been some changes to Xen.

Booting with Xen features extra Xen-specific boot output, as shown in Figure2-1 Figure2-1.

Figure2-1.Success! The Xen kernel wants to have a conversation with us.

After GRUB loads, it loads the Xen hypervisor, which then takes control of the hardware and outputs its initialization lines (starting with (XEN) (XEN)). Then it loads the dom0 kernel, which takes over and spews the familiar Linux boot messages that we know and tolerate. (Note that you may not see these messages if you"re using the VGA console, since they go by rather quickly. You can type xm dmesg xm dmesg to see them at any time.) to see them at any time.) After the system boots, you should be looking at the normal login prompt, with nary a sign that you"re running in a Xen virtual machine. (Albeit a specially privileged virtual machine.) Now would be a great time to log in, if you haven"t already.

Getting Familiar with Your Xen System Before we start creating virtual machines, let"s take a brief look at the Xen configuration files in the dom0. We"ll be referring to these a lot in future chapters, so now might be a good time to take a quick tour.

First, there"s the Xen configuration directory, /etc/xen /etc/xen. Most, though not all, of the Xen configuration happens via files here or in the scripts scripts subdirectory. subdirectory.

The main Xen config file is xend-config.sxp xend-config.sxp. This where you"d perform tasks like enabling migration or specifying the network backend. For now, we"ll be content with the defaults.

NoteIf you"re planning on using this Xen installation for anything besides this walk-through, now is a good time to set the (dom-min-mem) (dom-min-mem) option in option in xend-config.sxp xend-config.sxp to something sensible. We use to something sensible. We use (dom-min-mem 1024) (dom-min-mem 1024). See See Chapter14 Chapter14 for more details for more details.

The /etc/xen/scripts /etc/xen/scripts directory contains scripts to handle tasks like setting up virtual devices. directory contains scripts to handle tasks like setting up virtual devices.

Finally, domain configuration files live in /etc/xen /etc/xen. For example, you can take a look at xmexample1 xmexample1 to see a liberally commented sample config. to see a liberally commented sample config.

The /etc/init.d /etc/init.d directory contains scripts to start and stop Xen-related services. The Xen control daemon, directory contains scripts to start and stop Xen-related services. The Xen control daemon, xend xend, runs as a standard service from the /etc/init.d/xend /etc/init.d/xend script. Although you probably won"t need to modify it, this can be a handy place to change script. Although you probably won"t need to modify it, this can be a handy place to change xend xend "s parameters. It"s also the easiest way to restart "s parameters. It"s also the easiest way to restart xend xend, by running /etc/init.d/xend restart /etc/init.d/xend restart. The xendomains xendomains script may also be of interest-it automatically saves domains when the system shuts down and restores them when it boots up. script may also be of interest-it automatically saves domains when the system shuts down and restores them when it boots up.

There"s also /boot/grub/menu.lst /boot/grub/menu.lst. This file tells the bootloader, GRUB, to boot the Xen kernel, relegating the dom0 Linux kernel to a "module" line. Here you"d change the boot parameters for both Xen and Linux. For example, you might want to specify a fixed memory allocation for dom0 using the dom0_mem dom0_mem hypervisor option, or increase the number of network loopback devices via the Linux hypervisor option, or increase the number of network loopback devices via the Linux nloopbacks nloopbacks option. option.

domU data itself, if you"re using file-backed virtual disks under CentOS and following the default prompts for virt-install virt-install, resides in /var/lib/xen/images /var/lib/xen/images. Other distros and frontends are likely to have different defaults.

Management with xm The main command that you"ll use to interact with Xen is xm xm. This tool has a wide variety of subcommands. First, since we"re still looking around the environment, try xm list xm list: #xmlist NameIDMem(MiB)VCPUsStateTime(s) Domain-009342r-----37.6 The output of xm list xm list shows the running domains and their attributes. Here we see that only one domain is running, Domain-0 (abbreviated shows the running domains and their attributes. Here we see that only one domain is running, Domain-0 (abbreviated dom0 dom0 throughout the book) with ID 0, 934MiB of memory, and two VCPUS. It"s in the "running" state and has used 37.6 seconds of CPU time since boot. throughout the book) with ID 0, 934MiB of memory, and two VCPUS. It"s in the "running" state and has used 37.6 seconds of CPU time since boot.

NoteRed Hat doesn"t officially support xm xm, although they unofficially expect it to continue working through RHEL 5 although they unofficially expect it to continue working through RHEL 5.x. For this reason For this reason, xm xm " "s doc.u.mentation may advertise capabilities that don"t work on RHEL or CentOS. The supported management tool on RHEL and friends is virsh virsh, for for virtualization sh.e.l.l. virtualization sh.e.l.l.

You might also try xm info xm info for more information on the hypervisor. We"ll introduce more for more information on the hypervisor. We"ll introduce more xm xm subcommands in later chapters, and there"s a complete list in subcommands in later chapters, and there"s a complete list in AppendixA AppendixA.

Making a DomU For the moment, since we want to create a domU, the xm xm subcommand we"re most interested in is subcommand we"re most interested in is create create. However, before we can create a domain, we need to create an OS image for it to boot from and use as storage.

Because this is an initial walk-through, we"re going to install our Xen image using Red Hat"s virt-install virt-install tool. For information on building your own domU images, take a look at tool. For information on building your own domU images, take a look at Chapter3 Chapter3.

Begin by starting virt-install virt-install. It"ll start in interactive mode, and have a bit of a conversation with you, as shown. Our inputs are shown in bold. (If you decided to install the GUI, you can also use the graphical virt-manager virt-manager tool. The prompts will look very similar.) tool. The prompts will look very similar.) #virt-install

Whatisthenameofyourvirtualmachine?prospero HowmuchRAMshouldbeallocated(inmegabytes)?256 Whatwouldyouliketouseasthedisk(filepath)?/var/lib/xen/images/prospero.img Howlargewouldyoulikethedisk(/var/lib/xen/images/prospero.img)tobe(ingigabytes)?4 Wouldyouliketoenablegraphicssupport?(yesorno)no Whatistheinstalllocation? The machine then begins an interactive network install of CentOS. We won"t go into the details of its operation for now-suffice it to say that great pains have been taken to preserve the appearance of installing an OS on a physical machine. As such, the install process should eerily resemble the one we performed at the start of this chapter. Follow its prompts. (For the curious, we discuss virt-install virt-install and its accompanying tools more thoroughly in Chapters and its accompanying tools more thoroughly in Chapters Chapter3 Chapter3 and and Chapter6 Chapter6.) Once you"ve made your selections and gone through the install, the machine will reboot. Log in, and then shut the machine down via shutdown -h now shutdown -h now (remember, it"s an ordinary Linux box) so that we can look a bit more at things from the dom0 end. (remember, it"s an ordinary Linux box) so that we can look a bit more at things from the dom0 end.

Anatomy of a Domain Configuration File Let"s take a moment to examine the config file that virt-install virt-install generated for us. As we"ve mentioned already, the config file is generated for us. As we"ve mentioned already, the config file is /etc/xen/ /etc/xen/ by convention. by convention.

#cat/etc/xen/prospero name="prospero"

uuid="9f5b38cd-143d-77ce-6dd9-28541d89c02f"

maxmem=256 memory=256 vcpus=1 bootloader="/usr/bin/pygrub"

on_poweroff="destroy"

on_reboot="restart"

on_crash="restart"

vfb=[]

disk=["tap:aio:/opt/xen/images/prospero.img,xvda,w"]

vif=["mac=00:16:3e:63:b7:a0,bridge=xenbr0"]

As you see, the file consists of simple name=value pairs, with Python-style lists in square brackets. Note the values that we specified in the virt-install virt-install session, plugged into appropriate places-the name, the memory amount, and the disk image. session, plugged into appropriate places-the name, the memory amount, and the disk image. virt-install virt-install also fills in some network configuration, specifying a MAC address and dom0-level bridge device. also fills in some network configuration, specifying a MAC address and dom0-level bridge device.

We"ll examine many of the config file parameters more deeply in subsequent chapters. For now, let"s just move on to seeing the effects of these values on our domain.

Configuring the DomU Finally, start the image! We"ll run xm xm with the with the create create subcommand, which expects a config file name as an argument. We can omit the path, since it defaults to looking in subcommand, which expects a config file name as an argument. We can omit the path, since it defaults to looking in /etc/xen /etc/xen.

#xmcreate-cprospero Since we pa.s.sed the -c -c option to option to xm create xm create, it"ll immediately connect us to the domain"s console, so that we can interact with the bootloader. Hit ENTER to boot with default options, and watch it go.

Once it boots, you should be looking at the console of a shiny new Xen domU, as ill.u.s.trated in Figure2-2 Figure2-2. Log in as root and frolic.

Start by looking at the output of the dmesg dmesg command within the domU. Note that the disk and network devices are Xen"s special paravirtualized devices. command within the domU. Note that the disk and network devices are Xen"s special paravirtualized devices.

Figure2-2.We a.s.sure you that this is the domU console.

You can also take a look at the domU"s networking, which is essentially indistinguishable from that of a normal Linux system: #ifconfigeth0 eth0Linkencap:EthernetHWaddr00:16:3E:63:B7:A0 inetaddr:216.218.223.74Bcast:216.218.223.127Mask:255.255.255.192 inet6addr:2001:470:1:41:a800:ff:fe53:314a/64Scope:Global inet6addr:fe80::a800:ff:fe53:314a/64Scope:Link UPBROADCASTRUNNINGMULTICASTMTU:1500Metric:1

RXpackets:73650errors:0dropped:0overruns:0frame:0 TXpackets:49731errors:0dropped:0overruns:0carrier:0 collisions:0txqueuelen:1000 TXbytes:106033983(101.1MiB)RXbytes:2847950(2.7MiB) Note that we"re using standard commands-one of the major features of Xen is that most of the management takes place via familiar Linux commands. This makes it easy to customize your Xen environment, usually by making changes to the support scripts. Furthermore, standard commands will generally behave in expected ways-for example, you can give yourself a new IP address via ifconfig ifconfig, and it"ll work just as it would on a physical machine.[16]

Let"s return to the dom0 for a moment, just to take a look at the running domain from outside. To break out of the domU"s console, type CTRL-]. You can reconnect at any time by running xm console xm console from the dom0. from the dom0.

Now that we"re back in the dom0, we can note that our new domain shows up in xm list xm list, consuming memory and CPU time: #xmlist NameIDMem(MiB)VCPUsStateTime(s) Domain-007392r-----136.7 prospero12551-b----116.1 And that it"s got a visible network device: #ifconfigvif1.0 vif1.0Linkencap:EthernetHWaddrFE:FF:FF:FF:FF:FF inet6addr:fe80::fcff:ffff:feff:ffff/64Scope:Link UPBROADCASTRUNNINGNOARPMTU:1500Metric:1 RXpackets:49731errors:0dropped:0overruns:0frame:0 TXpackets:73650errors:0dropped:0overruns:0carrier:0 collisions:0txqueuelen:32 RXbytes:2847950(2.7MiB)TXbytes:106033983(101.1MiB) Some points to mention about the network device: First, it"s got a dummy MAC address. You can see the actual MAC address of the virtual Ethernet device from within the domU. Second, the counters are reversed-the domain"s actually downloaded 100MiB, and transmitted 2.7. Third, both IPv4 and IPv6 "just work" with the default setup. We go into further detail in Chapter5 Chapter5.

From here you can treat the domain just like any other Linux box. You can set up users on it, SSH into it, or access its console via xm console xm console. You can reboot it via the xm reboot xm reboot command, and shut it down using command, and shut it down using xm shutdown xm shutdown.

[16] The administrator can disable this, however. You"ll still be able to change the IP from the domU, but the dom0 will block traffic from the new IP. See The administrator can disable this, however. You"ll still be able to change the IP from the domU, but the dom0 will block traffic from the new IP. See Chapter5 Chapter5 for details. for details.

You"re Finished. Have a Cookie.

Now that you"ve got a domain, read the next chapter.

If it didn"t work ... what a fabulous opportunity to examine Chapter15 Chapter15. We are sorry. Please email us and mention that our directions need work.

Chapter3.PROVISIONING DOMUS You can suck Linux right out of the air, as it were, by downloading the right files and putting them in the right places, but there probably are not more than a few hundred people in the world who could create a functioning Linux system in that way.-Neal Stephenson, In the Beginning Was the Command Line In the Beginning Was the Command Line

Up until now, we"ve focused on administering the dom0, leaving the specifics of domU creation up to the virt-install virt-install tool. However, you"ll probably need to build a domU image from scratch on occasion. There are plenty of good reasons for this-perhaps you want an absolutely minimal Linux environment to use as a base for virtual private server (VPS) hosting setups. Maybe you"re deploying some custom application-a tool. However, you"ll probably need to build a domU image from scratch on occasion. There are plenty of good reasons for this-perhaps you want an absolutely minimal Linux environment to use as a base for virtual private server (VPS) hosting setups. Maybe you"re deploying some custom application-a server appliance server appliance-using Xen. It might just seem like a good way to keep systems patched. Possibly you need to create Xen instances without the benefit of a network connection.

Just as there are many reasons to want custom filesystem images, there are many ways to make the images. We"ll give detailed instructions for some that we use frequently, and briefly mention some others, but it would be impossible to provide an exhaustive list (and very boring besides). The goal of this chapter is to give you an idea of the range of options you have in provisioning domU filesystems, a working knowledge of the principles, and just enough step-by-step instruction to get familiar with the processes.

A Basic DomU Configuration All of the examples that we"re presenting here should work with a basic-in fact, downright skeletal-domU config file. Something along the lines of this should work: kernel=/boot/vmlinuz-2.6-xen.gz vif=[""]

disk=["phy:/dev/targetvg/lv,sda,w"]

This specifies a kernel, a network interface, and a disk, and lets Xen use defaults for everything else. Tailor the variables, such as volume group and kernel name, to your site. As we mention elsewhere, we recommend including other variables, such as a MAC and IP address, but we"ll omit them during this chapter for clarity so we can focus on creating domU images.

NoteThis doesn"t include a ramdisk. Either add a ramdisk= ramdisk= line or include line or include xenblk xenblk (and (and xennet xennet if you plan on accessing the network before modules are available) in your kernel. When we compile our own kernels, we usually include the if you plan on accessing the network before modules are available) in your kernel. When we compile our own kernels, we usually include the xenblk xenblk and and xennet xennet drivers directly in the kernel. We only use a ramdisk to satisfy the requirements of the distro kernels drivers directly in the kernel. We only use a ramdisk to satisfy the requirements of the distro kernels.

If you"re using a modular kernel, which is very likely, you"ll also need to ensure that the kernel has a matching set of modules that it can load from the domU filesystem. If you"re booting the domU using the same kernel as the dom0, you can copy over the modules like this (if the domU image is mounted on /mnt /mnt): #mkdir-p/mnt/lib/modules #cp-a/lib/modules/"uname-r"/mnt Note that this command only works if the domU kernel is the same as the dom0 kernel! Some install procedures will install the correct modules automatically; others won"t. No matter how you create the domU, remember that modules need to be accessible from the domU, even if the kernel lives in the dom0. If you have trouble, make sure that the kernel and module versions match, either by booting from a different kernel or copying in different modules.