Anatomy of the libvirt virtualization library

An API for easy Linux virtualization

M. Tim Jones 05 January 2010

Independent author

The libvirt library is a Linux API over the virtualization capabilities of Linux that supports
a variety of hypervisors, including Xen and KVM, as well as QEMU and some virtualization
products for other operating systems. This article explores libvirt, its use, and its architecture.
Share your expertise: How do you manage guest OSes in your virtual environment? Would
libvirt help? Add your comments below.

Connect with Tim

Tim is one of our most popular and prolific authors. Browse all of Tim's articles on
developerWorks. Check out Tim's profile and connect with him, other authors, and fellow
readers in My developerWorks.

When it comes to scale-out computing (such as cloud computing), libvirt may be one of the most
important libraries you've never heard of. Libvirt provides a hypervisor-agnostic API to securely
manage guest operating systems running on a host. Libvirt isn't a tool per se but an API to build
tools to manage guest operating systems. Libvirt itself is built on the idea of abstraction. It provides
a common API for common functionality that the supported hypervisors implement. Libvirt was
originally designed as a management API for Xen, but it has since been extended to support a
number of hypervisors.

Basic architecture
Join the green groups on My developerWorks
Discuss topics and share resources about energy, efficiency, and the environment on the
GReen IT Report space and the Green computing group on My developerWorks.

Let's start our discussion of libvirt with a view of the use model, then dig into its architecture and
use. Libvirt exists as a set of APIs designed to be used by a management application (see Figure
1). Libvirt, through a hypervisor-specific mechanism, communicates with each available hypervisor
to perform the API requests. I explore how this is done with QEMU later in the article.

Copyright IBM Corporation 2010 Trademarks

Anatomy of the libvirt virtualization library Page 1 of 11
developerWorks ibm.com/developerWorks/

Figure 1. Comparison and use model of libvirt

Also shown is a comparison of the terminology that libvirt uses. This terminology is important,
as these terms are used in API naming. The two fundamental differences are that libvirt calls the
physical host a node, and the guest operating system is called a domain. Note here that libvirt (and
its application) runs in the domain of the host Linux operating system (domain 0).

Means of control
With libvirt, you have two distinct means of control. The first is demonstrated in Figure 1, where
the management application and domains exist on the same node. In this case, the management
application works through libvirt to control the local domains. The other means of control exist
when the management application and the domains are on separate nodes. In this case, remote
communication is required (see Figure 2). This mode uses a special daemon called libvirtd that
runs on remote nodes. This daemon is started automatically when libvirt is installed on a new node
and can automatically determine the local hypervisors and set up drivers for them (to be discussed
shortly). The management application communicates through the local libvirt to the remote libvirtd
through a custom protocol. For QEMU, the protocol ends at the QEMU monitor. QEMU includes
a monitor console that allows you to inspect a running guest operating system as well as control
various aspects of the virtual machine (VM).

Figure 2. Control of remote hypervisors with libvirtd

Hypervisor support
To support extensibility over a wide variety of hypervisors, libvirt implements a driver-based
architecture, which allows a common API to service a large number of underlying hypervisors in
a common fashion. This means that certain specialized functionality of some hypervisors is not

Anatomy of the libvirt virtualization library Page 2 of 11

ibm.com/developerWorks/ developerWorks

visible through the API. Additionally, some hypervisors may not implement all API functions, which
are then defined as unsupported within the specific driver. Figure 3 illustrates the layering of the
libvirt API and associated drivers. Note also here that libvirtd provides the means to access local
domains from remote applications.

Figure 3. Driver-based architecture of libvirt

As of this writing, libvirt implements drivers for the hypervisors listed in Table 1. Other drivers will
no doubt be available as new hypervisors emerge from the open source communities.

Table 1. Hypervisors that libvirt supports

Hypervisor Description

Xen Hypervisor for IA-32, IA-64, and PowerPC 970 architectures

QEMU Platform emulator for various architectures

Kernel-based Virtual Machine (KVM) Linux platform emulator

Linux Containers (LXC) Linux (lightweight) containers for operating system virtualization

OpenVZ Operating system-level virtualization based on the Linux kernel

VirtualBox Hypervisor for x86 virtualization

User Mode Linux Linux platform emulator for various architectures

Test Test driver for a fake hypervisor

Storage Storage pool drivers (local disk, network disk, iSCSI volume)

Libvirt and the virtualization shell

Now that I've covered some of the architecture of libvirt, let's look at some examples of the use of
the libvirt virtualization API. I start by using an application called virsh (virtualization shell), which is
built on top of libvirt. This shell permits use of much of the libvirt functionality but in an interactive
(shell-based) fashion. In this section, I demonstrate some of the aspects of VM manipulation using
virsh.

The first step is to define the domain configuration file (shown in Listing 1, below). This code
specifies all the necessary options for defining a domainfrom the hypervisor (emulator) to the

Anatomy of the libvirt virtualization library Page 3 of 11

developerWorks ibm.com/developerWorks/

resources that the domain uses and peripheral configuration (such as the network). Note that this
is a very simple configuration: the actual attributes that libvirt supports are much more diverse. For
example, you can specify a BIOS and host bootloader, resources to be used by the domain, and
devices to be usedfrom floppy disks and CD-ROMs to USB and PCI devices.

The domain configuration file defines some of the basic metadata to be used for this QEMU
domain, including the domain name, maximum memory, and initially available memory (current)
as well as the number of virtual processors to be made available to this domain. You don't
assign a Universally Unique Identifier (UUID); instead, you allow libvirt to assign one. You define
the type of machine to emulate for this platformin this case, a 686 processor that is fully
virtualized (hvm). You define the location of the emulator (in case you need to support multiple of
the same type) and the virtual disk for the domain. Note here that you indicate the VM, which is a
ReactOS operating system in Virtual Machine Disk (VMDK) format. Finally, you specify the default
networking configuration, and you use Virtual Network Computing (VNC) for graphics.

Listing 1. Domain configuration file

<xml version="1.0"?>
<domain type='qemu'>
<name>ReactOS-on-QEMU<name>
<uuid<uuid>
<memory>131072<memory>
<currentMemory>131072<currentMemory>
<vcpu>1<vcpu>
<os>
<type arch='i686' machine='pc'>hvm<type>
<os>
<devices>
<emulator>usr/bin/qemu<emulator>
<disk type='file' device='disk'>
<source file='/home/mtj/libvtest/ReactOS.vmdk'/>
<target dev='hda'/>
<disk>
<interface type='network'>
<source network='default'/>
<interface>
<graphics type='vnc' port='-1'/>
<devices>
<domain>

Now, with the domain configuration file complete, let's start a domain with the virsh tool. The virsh
tool takes a command argument for the particular action to be taken. In the case of starting a new
domain, you use the create command and the domain configuration file:

Listing 2. Starting a new domain

mtj@mtj-desktop:~/libvtest$ virsh create react-qemu.xml
Connecting to uri: qemu:///system
Domain ReactOS-on-QEMU created from react-qemu.xml

mtj@mtj-desktop:~/libvtest$

Note here the Universal Resource Indicator (URI) used to attach to the domain (qemu:///system).
This local URI attaches to the system mode daemon for the local QEMU driver. To attach to a

Anatomy of the libvirt virtualization library Page 4 of 11

ibm.com/developerWorks/ developerWorks

remote QEMU hypervisor over the Secure Shell (SSH) protocol on host shinchan, you could use
the URI qemu+ssh://shinchan/.

Next, you can list the active domains on a given host using the list command within virsh. Doing
so lists the active domains, their domain IDs, and their state, as shown below:

Listing 3. Listing active domains

mtj@mtj-desktop:~/libvtest$ virsh list
Connecting to uri: qemu:///system
Id Name State
----------------------------------
1 ReactOS-on-QEMU running

mtj@mtj-desktop:~/libvtest$

Note that the name defined here is the name you defined in your domain configuration file
metadata. You can see that this domain has a domain ID of 1 and is currently running.

You can also suspend a domain with the suspend command. This command stops the domain
from being scheduled, but the domain continues to reside in memory and can be quickly resumed.
The following example illustrates suspending the domain, performing a list to see status, and then
restarting the domain:

Listing 4. Suspending a domain, checking status, and restarting

mtj@mtj-desktop:~/libvtest$ virsh suspend 1
Connecting to uri: qemu:///system
Domain 1 suspended

mtj@mtj-desktop:~/libvtest$ virsh list

Connecting to uri: qemu:///system
Id Name State
----------------------------------
1 ReactOS-on-QEMU paused

mtj@mtj-desktop:~/libvtest$ virsh resume 1

Connecting to uri: qemu:///system
Domain 1 resumed

mtj@mtj-desktop:~/libvtest$

The virsh utility also supports a number of other commands, such as saving a domain (save),
restoring a saved domain (restore), rebooting a domain (reboot), and many others. You can also
create a domain configuration file from a running domain (dumpxml).

So far, you've started and manipulated a domain. But what about attaching to it so that you
can see the domain in action. You can do this using VNC. To create a window representing the
graphical desktop of the particular domain, you can use VNC as:

Listing 5. Attaching to a domain

mtj@mtj-desktop:~/libvtest$ xvnc4viewer 127.0.0.1 0

Anatomy of the libvirt virtualization library Page 5 of 11

developerWorks ibm.com/developerWorks/

Libvirt and Python

The previous example illustrated the control of domains using the command-line utility virsh. Let's
now look an example of domain control using Python. Python was the libvirt-supported scripting
language and provides a clean, object-oriented interface to the libvirt API.

In this example, I explore some of the same operations that I demonstrated with the virsh
utility (list, suspend, resume, and so on). The Python example script is provided in Listing 6. In
this example, you begin by importing the libvirt module. You then connect to the local QEMU
hypervisor. From here, you iterate through the domain IDs that are available; for each one, you
create a domain object, and then suspend, resume, and finally destroy the domain.

Listing 6. Sample Python script for domain control (libvtest.py)

import libvirt

conn = libvirt.open('qemu:///system')

for id in conn.listDomainsID():

dom = conn.lookupByID(id)

print "Dom %s State %s" % ( dom.name(), dom.info()[0] )

dom.suspend()
print "Dom %s State %s (after suspend)" % ( dom.name(), dom.info()[0] )

dom.resume()
print "Dom %s State %s (after resume)" % ( dom.name(), dom.info()[0] )

dom.destroy()

Although this is a simple example, you can see the power that libvirt provides through Python.
Through a simple script, you are able to iterate through all of the local QEMU domains, emit some
information about the domain, and then control the domain. The output of this script is shown in
Listing 7.

Listing 7. Output from the Python script in Listing 6

mtj@mtj-desktop:~/libvtest$ python libvtest.py
Dom ReactOS-on-QEMU State 1
Dom ReactOS-on-QEMU State 3 (after suspend)
Dom ReactOS-on-QEMU State 1 (after resume)
mtj@mtj-desktop:~/libvtest$

API overview
At a high level, the libvirt API can be divided into five API sections: the hypervisor connection API,
the domain API, the network API, the storage volume API, and finally the storage pool API.

All libvirt communication occurs after a connection is created for a given hypervisor (for example,
as shown with the open call in Listing 6). The connection provides a path for all other APIs to work
through. In the C API, this behavior is provided through the virConnectOpen call (as well as others
for authentication). The response of these functions is a virConnectPtr object, which represents a

Anatomy of the libvirt virtualization library Page 6 of 11

ibm.com/developerWorks/ developerWorks

connection to a hypervisor. This object serves as the basis for all other management functionality
and is therefore a required argument for subsequent API calls to a given hypervisor. Important
subsequent calls are virConnectGetCapabilities, which returns the capabilities of the hypervisor
and driver, and virNodeGetInfo, which retrieves information about the node. This information is
returned as an XML document that can be parsed to understand which behaviors are possible.

Now, having access to a hypervisor, you can iterate through the various resources on that
hypervisor with a set of API calls. The virConnectListDomains API call returns a list of domain
identifiers representing the active domains on that hypervisor.

The API implements a large number of functions targeted toward domains. To explore or
manage a domain, you first need a virDomainPtr object. You can get this handle in a number
of ways (using either the ID, UUID, or domain name). Continuing with the example of iterating
domains, you can use the index list that this function returns and call virDomainLookupByID
to get the domain handle. With the domain handle in hand, you can now perform a large
number of operations, from exploring the domain (virDomainGetUUID, virDomainGetInfo,
virDomainGetXMLDesc, virDomainMemoryPeek) to controlling the domain (virDomainCreate,
virDomainSuspend, virDomainResume, virDomainDestroy, and virDomainMigrate).

You can also use the API to manage and inspect virtual networks and storage resources.
Following the model of the API, a virNetworkPtr object is necessary to manage and inspect
virtual networks, and a virStoragePoolPtr (storage pool) or virStorageVolPtr (volume) object is
necessary to manage these resources.

The API also supports an event mechanism with which you can register to be notified of particular
events (such as a domain being booted, suspended, resumed, or stopped).

Language bindings
The libvirt library was implemented in C (supporting C++) and includes direct support for Python.
But it also supports a number of language bindings. Bindings have been implemented for Ruby,
the Java language, Perl, and OCaml. Work has also been done for calling libvirt from C#. Libvirt
supports the most popular system programming languages (C and C++), a variety of scripting
languages, and even a unified functional language (Objective caml). So whatever your language
focus, libvirt provides a path to control your domains.

Applications using libvirt

From just the small amount of capabilities that I've demonstrated in this article, you can see the
power that libvirt provides. And as you can expect, there are a number of applications that are
being successfully built on libvirt. One of the interesting applications is virsh (demonstrated here),
which is a virtualization shell. There's also virt-install, which can be used to provision new domains
from operating system distributions. The utility virt-clone can be used to clone a VM from another
VM (covering both operating system and disk replication). Some of the higher-level applications
include virt-manager, which is a general-purpose desktop-management tool, and virt-viewer, which
is a lightweight tool for securely attaching to the graphical console of VMs.

Anatomy of the libvirt virtualization library Page 7 of 11

developerWorks ibm.com/developerWorks/

One of the most important tools built on libvirt is called oVirt. The oVirt VM management
application was designed to manage a single VM on a single node or thousands of VMs over
hundreds of hosts. In addition to simplifying management of large numbers of hosts and VMs,
it can be used to automate clustering and load balancing and works across platforms and
architectures.

Going further
As you can see from this short article, libvirt is a great library for building applications that manage
domains in many different hypervisor environments over large networks of systems. Given
the growing popularity of cloud computing, libvirt will no doubt grow along with it, finding new
applications and users. As of this writing, libvirt is only just over four years old, so it's relatively new
in the massively scalable computing space. More is most certainly to come.

Anatomy of the libvirt virtualization library Page 8 of 11

ibm.com/developerWorks/ developerWorks

Resources
Learn

Check out the libvirt Web site for the latest news about libvirt and to download the latest
version. You'll also find a complete API reference manual that covers the core interfaces and
error-handling interfaces.
Libvirt supports a large number of hypervisors, including:
Xen
QEMU
KVM
LXC
OpenVZ
VirtualBox
User-Mode Linux
In "Virtual Linux: An overview of virtualization methods, architectures, and
implementations" (developerWorks, December 2006), learn more about the various types
of virtualization. Cloud computing relies on virtualization for optimal use of server-available
resources. With virtualization, servers can be used to host multiple operating systems and
application sets.
In "LXC: Linux Container Tools" (developerWorks, February 2009), you can learn more about
the management tools built specifically for LXC. You'll notice some parallels here with the
management methods for libvirt.
Virtual Network Computing, or VNC, is a method for sharing graphical desktops over a
network. This abstraction is ideal for massive scale-out computing, as it means a single
station can manage many distributed clients.
The Linux subsystem for managing file systems is large and complex. You can
learn more about the larger file system subsystem in "Anatomy of the Linux file
system" (developerWorks, October 2007).
In the developerWorks Linux zone, find more resources for Linux developers, and scan our
most popular articles and tutorials.
See all Linux articles and Linux tutorials on developerWorks.
Stay current with developerWorks technical events and webcasts.

Get products and technologies

Red Hat's oVirt open VM management platform is a user of libvirt and demonstrates what
kinds of applications can be built. You can use oVirt to manage large numbers of hosts, and
the platform easily scales to support thousands of VMs.
In this article, you used the ReactOS as a means to demonstrate a domain on QEMU. This
article provides more information on ReactOS (a free Windows clone), which is easily
executed on the QEMU platform.
With IBM trial software, available for download directly from developerWorks, build your next
development project on Linux.

Discuss

Anatomy of the libvirt virtualization library Page 9 of 11

developerWorks ibm.com/developerWorks/

Get involved in the My developerWorks community. Connect with other developerWorks

users while exploring the developer-driven blogs, forums, groups, and wikis.

Anatomy of the libvirt virtualization library Page 10 of 11

ibm.com/developerWorks/ developerWorks

About the author

M. Tim Jones

M. Tim Jones is an embedded firmware architect and the author of Artificial

Intelligence: A Systems Approach, GNU/Linux Application Programming (now in
its second edition), AI Application Programming (in its second edition), and BSD
Sockets Programming from a Multilanguage Perspective. His engineering background
ranges from the development of kernels for geosynchronous spacecraft to embedded
systems architecture and networking protocols development. Tim is a Consultant
Engineer for Emulex Corp. in Longmont, Colorado.

Copyright IBM Corporation 2010

(www.ibm.com/legal/copytrade.shtml)
Trademarks
(www.ibm.com/developerworks/ibm/trademarks/)

Anatomy of the libvirt virtualization library Page 11 of 11