An experiment-driven energy consumption model for

virtual machine management systems

Mar Callau-Zori, Lavinia Samoila, Anne-Cécile Orgerie, Guillaume Pierre

To cite this version:

Mar Callau-Zori, Lavinia Samoila, Anne-Cécile Orgerie, Guillaume Pierre. An experiment-driven
energy consumption model for virtual machine management systems. [Research Report] RR-8844,
IRISA; Université de Rennes 1; CNRS. 2016. <hal-01258766>

HAL Id: hal-01258766

https://hal.inria.fr/hal-01258766
Submitted on 19 Jan 2016

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est

archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non,
lished or not. The documents may come from émanant des établissements d’enseignement et de
teaching and research institutions in France or recherche français ou étrangers, des laboratoires
abroad, or from public or private research centers. publics ou privés.
An experiment-driven
energy consumption
model for virtual machine
management systems
Mar Callau-Zori
Lavinia Samoila
Anne-Cécile Orgerie
Guillaume Pierre
ISRN INRIA/RR--8844--FR+ENG

RESEARCH
REPORT
ISSN 0249-6399

N° 8844
January 2016
Project-Team Myriads
 An experiment-driven energy consumption
model for virtual machine management
systems

∗
Mar Callau-Zori
∗
Lavinia Samoila
†
Anne-Cécile Orgerie
∗
Guillaume Pierre

Project-Team Myriads

Research Report n° 8844  January 2016  26 pages

Abstract:
As energy consumption is becoming critical in Cloud data centers, Cloud providers are adopting
energy-ecient virtual machines management system. These systems essentially rely on what-if 
analysis to determine what the consequence of their actions would be  and to choose the best one
according to a number of metrics. However, modeling energy consumption of simple operations such
as starting a new VM or live-migrating it is complicated by the fact that multiple factors takes part.
It is therefore important to identify which factors inuence energy consumption before proposing
any new model. We claim in this paper that one critical parameter is the host conguration,
characterized by the number of VMs it is currently executing. Based on this observation, we
present an energy model that provides energy estimation associated to VM management operation,
such as VMs placement, VM start up and VM migration. The average relative estimation error is
lower than 10% using the transactional web benchmark TPC-W, making it a good candidate for
driving the actions of future energy-aware cloud management systems.

Key-words: Green computing, Cloud computing, virtual machine management.

Acknowledgments: This work was partially supported by the EcoPaaS project of the Brittany
Region (project number 8269).

∗ IRISA/Université de Rennes 1, mar.callau-zori@irisa.fr, lavinia.samoila@irisa.fr, guillaume.pierre@irisa.fr.

† IRISA/CNRS, anne-cecile.orgerie@irisa.fr.

RESEARCH CENTRE
RENNES – BRETAGNE ATLANTIQUE

Campus universitaire de Beaulieu

35042 Rennes Cedex
 An experiment-driven energy consumption model for
virtual machine management systems

Résumé : La consommation d'énergie devient critique dans les centres de données en nuage.
Les fournisseurs cloud adoptent donc des systèmes de gestion des machines virtuelles avec des
critères pour l'ecace sur la consomation energétique. Ces systèmes reposent essentiellement sur
des analyses de type what-if  pour déterminer les conséquences des actions  et pour choisir
la meilleure action selon plusieurs metriques. Cependant, la modélisation de la consommation
d'énergie d'opérations simples, tel que le démarrage d'une nouvelle machine virtuelle ou la migra-
tion en direct, est une tâche complexe parce que de multiples facteurs participent aux modèles.
Nous proposons un modèle simple pour l'estimation des consommations énergétiques liées aux
diérentes opérations de gestion des machines virtuelles.

Mots-clés : Informatique en nuage, gestion de machines virtuelles, économies d'energie.

An experiment-driven energy consumption model for virtual machine management systems 3

Table of contents

1 Introduction 4
2 Related work 6
2.1 VM power models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Live-migration power models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Experimental facts 8
3.1 Impact of the number of VMs allocated in a host . . . . . . . . . . . . . . . . . . 9
3.2 Impact of VM start up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.3 Impact of VM migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4 Impact of workload prole . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.5 Impact of GPU accelerator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4 Modeling energy 16
5 Experimental results about model validation 17
5.1 First use-case: VM allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.2 Second use-case: starts and migrations together . . . . . . . . . . . . . . . . . . . 18

6 Learning model parameters in a real system 19

7 Discussion 21
8 Conclusions 22

RR n° 8844
4 An experiment-driven energy consumption model for virtual machine management systems

1 Introduction

Context Advances in distributed systems have historically been related to improving their
performance, scalability and quality of service. However, the energy consumption of these systems
is becoming more and more concerning. Although the emergence of Cloud Computing has led
to a massive virtualization of the resources, their energy cost is still real and rapidly increasing
due to a growing demand for Cloud services. As an example, for 2010, Google used 900,000
servers which consumed 1.9 billion kWh of electricity [20]. This ever-increasing electricity bill
puts a strain on operating expanses, and has thus led researchers to look for more energy-ecient
frameworks.
The most common approach to save energy in Cloud infrastructures consists in consolidating
the load on the smallest possible number of physical resources [29], thus making it possible to
switch idle machines to sleep modes [36]. To this end, scheduling heuristics are used for allocating
new virtual machines (VMs) or for re-allocating running VMs in conjunction with live-migration
techniques for dynamic consolidation, with a close-to-zero downtime.
In this paper, we propose and evaluate an experiment-driven model to estimate the energy
consumption associated to VM management operations, such as VM placement, VM start up
and VM migration. This model provides reliable and accurate energy consumption values, which
are required by energy-aware VM management frameworks in order to take ecient decisions.

Motivating example Cloud providers oers on-demand computational resources in the form
of virtual machines, disks and network functions. These resources can be used freely by the ten-
ants. Meanwhile, the provider may perform VM management operations, such as create/delete,
suspend/resume, shut down/restart and migrate to reduce the energy consumption of the plat-
form or simply for maintenance purposes or to redistribute the load. These operations can be
performed in dierent ways leading to dierent energy consumption.
A scenario based on consolidation illustrates the variability in energy consumption of two
typical algorithms in Figure 1. The consolidation goal is to reduce the number of physical hosts
where VMs are allocated. The example starts with four identical hosts (same hardware) and
eleven identical VMs uniformly distributed among the hosts (left-side of Figure 1). The nal
conguration in both cases has only two provisioned hosts  allowing to switch o the two others.
Figure 1 on the bottom shows the consumption for the two algorithms: the rst-t consolida-
tion where all VMs are moved to host 1, and the balancing consolidation where VMs are uniformly
distributed between host 1 and host 2. Our experimental results provide the energy overhead due
to the migrations and the power consumption of the hosts after the re-conguration. Despite
the numerous approaches based on the rst-t algorithm in the literature [29], the balancing
consolidation performs better on this example. Indeed, it saves 12% of the energy with respects
to the rst-t consolidation for the migration operations, and the nal consumption of the two
hosts is less by 4.5% with this conguration. In particular, this example clearly highlights that
identical VMs are not equal in terms of energy consumption.
This paper argues that VM management systems must be aware of the energy consumption of
the dierent operations (such as VM allocation, VM start up and VM migration) and must have
an accurate energy model of their physical hosts depending on their conguration (i.e. number
of hosted VMs) in order to be truly energy-ecient. More details about these experiments are
provided in Section 6.

Energy-ecient VM management techniques On the one hand, heuristic-based methods

for VM allocation take practical decisions to satisfy immediate goals, such as maximizing the
number of idle hosts [4, 5, 8], minimizing the number of VM migrations [23], or determining

Inria
An experiment-driven energy consumption model for virtual machine management systems 5

Figure 1: Energy overhead depends on the consolidation algorithm.

the most energy-ecient available host for each new VM [2, 31]. However, solutions derived
from heuristics are not guaranteed to be optimal in the long term, especially in heterogeneous
environments [30]. Therefore, it is unclear which heuristic will guarantee the most energy-ecient
system. On the other hand, although live-migration leads to almost no performance degradation,
it is not energy-free. While for the hosts, the migration cost mainly depend on the VM size
and network bandwidth [42, 45], the sequential aspect of multiple VMs migration has also an
impact [47], and widely-used Cloud software platforms, like OpenStack [35] for instance, perform
multiple migrations in a sequential way. These points enlighten the necessity of accurate energy
models of VM management operations for applying the adequate energy-ecient technique.
However, dening an accurate energy consumption model for VMs is still an open chal-
lenge [30]. This is mainly because the energy consumption depends on multiple factors: the
workload, the hardware, the host conguration, the VM characteristics, etc. Furthermore, deter-
mining the suitable parameters which describe the inherent properties is crucial to enforce the
model. Previous work [7, 6, 12, 14, 19, 21] has already proposed models based on factors, such
as the server hardware, the IaaS-software (such as the hypervisor and the management software)
or the VMs resources (such as CPU, memory, disc and cache). However, the impact of the host
conguration has been slightly studied. Related energy models are further detailed in Section 2.

Energy model for VM management operations The paper presents directions for a com-
prehensive energy-estimation tool assuming that the VM management system performs actions
sequentially, as it is often the case in current Cloud software platforms [35]. This means that if
the system receives two requests (e.g. a VM creation and a VM migration), it waits for the rst
one to nish before executing the other one. The model is built over a wide set of experiments
aiming at observing variations according to VMs management operations, host conguration and
workload application. This experimental study (Section 3) provides helpful information claiming
for an energy model based on VMs management operations and hosts conguration. The pro-
posed model (Section 4) takes into account the host conguration, and in particular the number
of already running VMs on this host.
In [10], the authors study the energy consumption according to the number of VMs allocated
in a single host and to the number of virtual CPU. Similarly to our motivating example, they
show that the energy consumption depends on the previous conguration of the host. However, in
their experiments, the number of virtual cores is never greater that the number of physical cores.
So, they do not explore overcommit situations, which are frequent in Cloud platforms [3, 9],
although not studied in literature from an energy point of view. In this paper, we explore host
congurations with CPU overcommit, and we show that the proposed model works for these
overcommit situations.

RR n° 8844
6 An experiment-driven energy consumption model for virtual machine management systems

Evaluation This paper targets online applications hosted by web servers on Cloud infrastruc-
tures and represented here by the TPC-W benchmark [44] for evaluation purposes (Section 5).
TPC-W is a transactional web benchmark simulating a Web-based shop for searching, brows-
ing and ordering books. We have validated the proposed energy model through two use-case
scenarios using OpenStack as the Infrastructure-as-a-Service Cloud layer. In the rst scenario,
the system has a workload peak which produces an increasing number of VMs allocated on the
physical host. In the second scenario, migrations and VMs starts-up occur in a system with
several hosts. In both cases, the model is able to predict the involved energy consumption with
an average relative error lower than 10%, and thus to adapt the system accordingly.
The applicability of the model is studied over a realistic scenario (Section 6). The model is
trained over a simple scenario and then applied to a more complex scenario on which it is used
to evaluate the energy-eciency of three consolidation algorithms. We hereby demonstrate the
ability of the model to provide valuable results for a what-if analysis. Experiments on a real
platform exhibit an average relative error lower than 10%. The advantages and drawbacks of our
model are further discussed in Section 7.

Contributions In summary, in this paper: (1) we provide an experimental study of the en-
ergy consumption of VM management operations under realistic workload conditions including
overcommit situations, (2) we propose an energy model for physical host consumption, (3) we
evaluate the accuracy of this model via experimentation on a real system deploying web appli-
cations, and (4) we show its applicability and how it can signicantly help Cloud management
systems to take truly energy-ecient decisions.

2 Related work

Energy models have mainly been studied in the context of a single host, a single virtual machine
or an entire cluster [33]. The virtualization proposes to improve resource utilization through
dividing one physical server into multiple isolated virtual machines. Resources sharing makes
complex to leverage experiences from previous eorts on energy estimation in single host [18, 26,
32, 37, 38, 39] and on energy estimation in cluster models for data centers [15, 24].
In virtualized environments, we distinguish previous models to estimate power consumption
of a single VM and previous models to estimate power overhead during live-migration. In the
rest of this section, we discuss research in both directions.

2.1 VM power models

In [19], the authors propose a linear power model in terms of CPU utilization, memory (last level
cache misses) and disk, with a linear regression to learn model parameters. Authors achieve a
total power error on the host of 1.6 Watts to 4.8 Watts.
Another linear model is presented in [7] considering CPU, memory, disk and cache. The
principal component analysis obtains a high correlation between CPU and cache; and between
disk and DRAM. This fact motivates the utilization of two regression models to estimate model
parameters. The results show that the model is able to predict energy power with an average
mean error of 7%.
In [21], the authors present a linear model considering CPU and memory. They indicate
the importance in energy consumption of the utilization of the cache levels and memory level
parallelism. The evaluation shows that VM power is estimated within 7% of the measured value.
Gaussian mixture models are proposed in [14] to estimate power consumption based on ar-
chitecture metrics (such as IPC, cache, memory usage, among other performance monitored

Inria
An experiment-driven energy consumption model for virtual machine management systems 7

counters). In the training phase, a set of Gaussian mixture models that represent dierent ar-
chitecture interactions and power levels are obtained. An average prediction error of less than
10% is achieved.
Another linear model based on CPU utilization, memory and disk is proposed in [6]. This
model is learned according to the dynamic voltage frequency (using the DVFS technique). For
the evaluation, up to 10 virtual CPUs are allocated in servers with 2 cores. The results exhibit
an average error lower than 5%.
Based on the major power consumer which is the CPU for a server, a process-level power
estimation is presented in [12]. A polynomial regression is applied to estimate the power con-
sumed by the CPU in terms of the frequency and the unhalted-cycles for each active process
and core. The average estimation error achieved is below 10%.
In the previous work, we observe three main characteristics. Firstly, presented models assumes
that a VM always consumes the same amount of energy if the VM performs similar work (e.g.
similar utilization of CPU, memory and cache). Thus, these contributions assume that energy
consumption of a VM is independent of the number of hosted VMs in the hosts. Secondly,
the total number of virtual cores used the running VMs is at most the number of physical
cores (except in [6]). However, in real Cloud platforms, over-provisioning is common for users,
leading Cloud providers to resort to over-commitment [3, 9]. Hosting more virtual cores than
the number of physical ones can consequently aect the assumption that energy consumption of
a VM is independent of the number of hosted VMs on the server. Thirdly, most of the previous
work is evaluated through benchmarks specically designed to stress CPU, memory and/or I/O.
However, in real environments, resources do not reach such high utilization levels, mainly to
guarantee performance and quality of service. Only work presented in [12] employs for their
evaluation SPEC JBB 2013 [13], a benchmark which emulates a supermarket company.

2.2 Live-migration power models

Previous analyzes about energy overhead during live-migration show dierences in the energy
consumption for source and destination hosts [16, 37, 40].
In [25], a model based on the VM memory size, the memory transmission rate and the memory
dirtying rate is proposed. The authors not only estimate the energy overhead during migration,
but also the network trac overhead and the downtime. However, they assume that energy
consumption is similar in source and destination hosts. The evaluation using Xen hypervisor [46]
shows an average error below 10%.
It is shown in [42] that energy overhead during migration depends on RAM and available
bandwidth. Based on this work, the authors propose in [41] a power model for estimating the
energy overhead of live-migration for an idle VM. The average error using KVM hypervisor [22]
is lower than 10%.
In [27], the authors present a model depending on the CPU utilization (considering both hosts
and the migrated VM), the VM memory utilization, the available bandwidth and the memory
dirtying rate. The evaluation is conducted through live-migrating a VM in a host where there
are already 8 running VMs. Results using Xen hypervisor with a CPU and memory intensive
workload shows a normalized root mean square error lower than 18%. Authors expended this
work in [28] by proposing a model for network transfers achieving a normalized root mean square
error lower than 9%.
One can notice that, in the previous work, only one VM is migrated for the validation of the
various models. However, the consolidation usually imply to deallocate all the VMs of a host.
Therefore, migrations are produced consecutively and energy overhead can be aected due to
these successive migrations.

RR n° 8844
8 An experiment-driven energy consumption model for virtual machine management systems

Table 1: TPC-W conguration parameters.

Number of browsers 100

Transaction Mix Factory
Ramp-up time 600 sec
Ramp-down time 600 sec
Measurement time 1800 sec
Slow-down factor 1.0 (default)
Nb. customers 144000
Nb. items 10000
Think time multiplication 0.25
Maximum errors allowed unbounded
Maximum number of images downloaded at once 4

3 Experimental facts

We conducted an in-depth experimental analysis of VM management operations in order to

understand their impact on energy consumption. The experiments have been executed on
Grid'5000 [1], a French test-bed designed for experiment-driven research. We base experiments
on four servers equipped with two 6-cores Xeon E5-2630@2.3GHz processors (12 cores in total),
32 GB of RAM, 10 Gbps Ethernet and embedding with a Nvidia Tesla M2075 GPU accelerator.
The power consumption is collected externally at each second for each server using adapted
watt-meters provided by the SME Omegawatt. We use OpenStack Icehouse [35] to dynamically
create VMs through the KVM hypervisor [22]. VM deployment is relying on an NFS server
(Network File System) for the storage. This solution is well suited for web servers, and has a
facilitating impact on VM live-migration. The VMs are congured with 4096 MB memory, 80 GB
of virtual disk and 4 virtual CPUs. This conguration is similar to Amazon EC2's `m4.xlarge'
instance type, although with less memory [17]. As the hosts considered in this experimental
study comprise 12 cores each, without overcommit, they can only run 3 of such VMs each.

We create realistic workload by running the MySQL+Java version of TPC-W in the VMs.
TPC-W is a transactional web benchmark which simulates the activities of a business oriented
transactional web server [44]. TPC-W allows three proles of web trac (called ordering, brows-
ing and shopping ). The main trac prole used is ordering, where the number of book's pur-
chases is large  this means, large number of write operations into disk. Other TPC-W param-
eters are summarized in Table 1. TPC-W measures the throughput in number of WIPS (Web
Interactions Per Second) that the server manages to sustain.

Summarizing, each VM has the virtual hardware described above and runs a TPC-W server.
We therefore measure two metrics: the power consumption of each physical server, and the
throughput measured in WIPS of each TPC-W server, each one running in a single VM. Exper-
iments ran once on the four dierent servers.

We study three kinds of VM management operations: the VM allocation in a single host

(Section 3.1), the VM start up (Section 3.2) and the VM migration (Section 3.3) Then, we study
the impact of the workload prole in Section 3.4 and the inuence of the GPU accelerator in
Section 3.5.

Inria
An experiment-driven energy consumption model for virtual machine management systems 9

Figure 2: Power consumption and throughput when varying the number of VMs in the host.

3.1 Impact of the number of VMs allocated in a host

Our rst experiment evaluates how many VMs a single host can manage in an energy-ecient
way. The scenario consists in increasing the number of VMs put on a single host. Figure 2 shows
the power consumption and throughput depending on the number of VMs running on the host.
The X-axis shows the number of VMs in the host, whereas the Y-axes present three metrics: the
power consumption in Watts (blue circles, left side), the total throughput over all VMs in WIPS
(green triangles, right side), and the power/performance ratio in Joules/WI (Joules over number
of web interactions, red squares).
We see that the power consumption and VM throughput increase up to 11 VMs per host, but
increasing is linear up to 8 VM. If we keep increasing the number of VMs, then the consumed
power starts decreasing from 11 VMs and throughput also drops at 11 VMs. The optimal
power/throughput ratio is obtained using 9-10 VMs per host.
If focusing only on throughput, the most ecient conguration is the one ensuring linear
scalability: so 8 VMs per host at maximum. However, maintaining a maximum of 8 VMs per
host is not the most power-ecient case. For example, a system with 1 host with 10 VMs
spends less power (219.84 W) than a system with 2 hosts: one with 8 VMs and another with
2 VMs, (335.76 W) saving 52.73% of power. This relationship between power and throughput
corresponds to the red line on Figure 2, and shows that placing either 9 or 10 VMs per host is
more power-ecient than staying with 8.
Regarding power consumption in details, Figure 3 presents a synthetic view of the power
consumption distribution using box-plot. Classically, the box-plot has a box and a whisker plot
for each case. The box extends from the 25th to 75th percentile with a red line at the median. The
whiskers extends from the box, and points outside the whiskers are considered outliers. Distances
from whiskers to box are a factor of 1.5 of the interquartile range (box height). Finally, outliers
are plotted individually.
In the power distribution, Pearson correlation coecient (0.97) with the number of VMs in
[1, 10] shows a strong linear correlation between the number of VMs and the power consumption.
A special case occurs for 6 VMs: where the 75th percentile (top on the box) is lower than the
regression line.
As we said, previous work assumes that the power consumption is a linear function in the
number of VMs.
P Concretely, the assumption on the power of the host is P host
= P idle
+
v∈VMs Pv , where Pidle is the idle power of the host and Pv is the power of the VM v . In

RR n° 8844
10 An experiment-driven energy consumption model for virtual machine management systems

350 Configuration type-B

1 8
300 2 9
energy (W)

250 4 10
6
200
150
100
0 100 200 300 400 500 600 700
total thr. (WIPS)
Figure 3: Details on power consumption by increasing the number of VMs in the host.

our case, the idle power is 95.92W (measured independently) and all the VMs perform the same
work having the same consumption before saturation (less than 9 VMs). However, in Table 2, we
show the average power value for one VM, Pv , and hereby evidencing that the power consumed
in one VM depends on the number of running VMs on the host.
From this rst experiment, we can draw several conclusions. Firstly, in order to achieve
energy-eciency, allowing small degradation of the throughput can generate important power
savings, as it has already been noticed in [30]. Secondly, power consumption is signicantly
impacted by the host conguration (i.e. the number of running VMs).

3.2 Impact of VM start up

We now investigate the power consumption overhead and throughput degradation during a VM
creation. Figure 4 shows an example of VM creation on a host already running one VM. After
1200 seconds, a new VM is created. VM start-up nishes when TPC-W starts to run (in this

Inria
An experiment-driven energy consumption model for virtual machine management systems 11

Table 2: Average power consumption per VM.

Nb. of VMs 1 2 4 6 8 9 10
Power per VM (W) 67.5 40.3 26.1 21.1 20.9 19.8 18.9

Figure 4: Example of VM start-up: power consumption and host throughput.

example it takes 28 seconds). As expected, the throughput does not increase during the creation.
However, starting a new VM causes a spike in power consumption (with consumption peak greater
than 180W).
After this example, we provide a general study in Figure 5 representing average power con-
sumption and throughput variation during a VM creation. The left side (blue colors) corre-
sponds to the total power consumption, while the right side (green colors) represents the average
throughput (WIPS) across all the VMs. Power consumption bars (resp. throughput bars) are
split in two: rst, the power consumption (resp. throughput) before VM start up; and second,
the power consumption (resp. throughput) observed during VM start-up (averaged over the
start-up duration).
While the power consumption is always greater during start-up, the throughput does not
suer degradation except for 9 VMs. One can notice that degradation happens passing from 9
to 10 VMs, which conrms the previous conclusion asserting that the 10 VMs case breaks the
linear scalability.
Details about power consumption during start-up process are presented in Figure 6. The gray
line represents the linear regression over data obtained from 1 to 8 VMs before the VM start-up.
We see that power increases during start-up from 1 to 8 VMs. Moreover, power consumption
overhead is smaller for 6 VMs than for the other cases.
We have explored this phenomenon more deeply in Table 3 representing the power overhead
percentage of VM start-up. We observe a decreasing trend in the power overhead of 1-5 and 7-9.
We clearly see that starting up a VM is not power-neutral. Hence an energy-ecient system
should guarantee a minimum lifetime after VM creation. Furthermore, this minimal lifetime
depends on the previous conguration on the host.

3.3 Impact of VM migration

We now evaluate the impact on power consumption and throughput of VM live-migration. The
scenario consists of migrating a VM between two hosts which already contain some running

Table 3: Average power consumption overheads upon VM start-up.

Nb. of VMs 1 3 5 7 8 9
Power overhead 5.1% 3.2% 3.1% 6.0% 3.3% 1.9%

RR n° 8844
12 An experiment-driven energy consumption model for virtual machine management systems

Figure 5: Throughput and power consumption overheads upon VM start-up.

Figure 6: Details in power consumption overhead upon VM start-up.

VMs. Under the live-migration with pre-copy approach [11], the VM continuously runs while its
memory is copied from source host to destination host in several runs (pre-copy phase). Requests
are processed by the VM in the source host, until the system is notied of the pre-copy phase
termination. At this point, the VM is suspended to perform the last copy (stop-and-copy phase).
Finally, requests start to be processed by the VM in the new location.
Figure 7 presents the experimental results of a VM migration in terms of power (top) and
throughput (bottom). Throughput is equal to zero during few seconds, waiting for the system
to be informed of the new location. Power consumption overhead is investigated in the source
and in the destination hosts separately.
During migration, the power consumption of source host starts to decrease, while in the
destination host, it increases. Dotted lines represent the average power consumption in source
and destination hots, before and after migration. Interestingly, it takes about 20 seconds after
the end of the migration for both hosts to reach power consumption at levels corresponding to
the number of allocated VMs. Hence, the experiment show that live-migration produces power
consumption overhead beyond the migration time.
For a deeper understanding of the impact of migration, we present a general study where
a VM is migrated after 1200 seconds. Source and destination hosts already have a number of
static (i.e. non-migrated) VMs. This means that we distinguish three types of VMs: migrated
VM ( mig ), src )
static VMs in the source host ( and static VMs in the destination host ( dst ).

Inria
An experiment-driven energy consumption model for virtual machine management systems 13

Figure 7: Example of power consumption and throughput upon VM migration.

Figure 8: Overheads upon VM migration.

In the following, for the sake of clarity, we take the same number of static VMs in source and
destination host for a given experiment ( st-src = st-dst ).
Results are summarized in Figure 8. The X-axis represents the number of static VMs for each
host (source and destination). Fixing the number of static VMs, we have the power consumption
of the source and destination host: in lighter blue the power consumed before migration and in
darker blue the power consumed during migration. On the throughput side, we consider three
metrics: one for the migrated VM, one for the static VMs in the source host, and, respectively,
in the destination host.
During migration, the power consumption of the destination host increases (1.35%  7.63%),
and in the source host, it does not have a stable behavior either (−2.96%  +0.92%).
Details about the total power consumption overhead are provided in Figure 9. The total power
is the accumulated power consumed in the source and destination hosts during migration. The
gray line represents the regression line over the total power consumption before the migration.
The power increment is more signicant in extreme cases (VMs per host = 1, 2, 8 and 9), while
for 4 to 6 VMs per host, power does not signicantly increase during migration. As expected,
the total power consumption increases during migration (0.55%  2.69%).
As explained in the example from Figure 7, it takes time for the power consumption in the
destination host after migration to stabilize. To explore this issue, we determine the minimum
time after a migration where the average power consumption in the destination host, on a window
of 10 seconds, is less or equal than the percentile 60% of the power consumption in source host
before the migration. We are exploiting the fact that after migration, the destination host
allocates the same number of VMs than the source host before the migration. These times are
summarized in Table 4. We observe a non-negligible impact on the power consumption: the time
to recover expected power consumption varies between 1 and 271 seconds depending on the host
conguration.
The throughput of non-migrated VMs is not aected by the migration, as shown in Figure 8.
However, the migrated VM has an important degradation reducing its throughput by 21% to 29%.
Although migration takes only a few seconds, the impact of the resulting throughput degradation

RR n° 8844
14 An experiment-driven energy consumption model for virtual machine management systems

Figure 9: Details in power consumption overhead upon VM migration.

Table 4: Time to recover expected power consumption after a VM migration.

Nb. of VMs 1 2 4 6 8 9
Delay (sec) 1.3 14.8 36.5 97.0 61.8 271.5

will largely depend on the moment it happens. The average down-time varies from 2.73 to
3.30 seconds being similar to that obtained in the literature [43]. We expected the downtime
to be independent on the hosts conguration because hosts are not saturated. However, our
experimental study shows that it is not the case.
As other studies [37] have already pointed, live-migration produces energy overhead occurring
mainly in the destination host. Moreover, we have also observed that migration produces power
consumption overhead also after the migration time. This fact is a sensitive issue in consolidation,
because migrations are often produced consecutively. Finally, we see that the power consumption
overhead depends on the number of running VMs in the hosts.

3.4 Impact of workload prole

We now evaluate the impact of the client behavior, thus inuencing the workload on the servers.
As previously stated, TPC-W allows to simulate three dierent proles by varying the ratio of
browse to buy transactions: shopping, browsing and ordering.
Figure 10 shows a scenario where the number of VMs put on the same host increases for the
TPC-W proles ordering and browsing. While the X-axis represents the number of VMs in the
same host, the Y-axes represent two metrics: the average power consumption in Watts (blue
colors, left side) and the average throughput in WIPS (green colors, right side).
We observe that throughput is equal for the two proles between 1 and 9 VMs. From 10
VMs however, ordering workload starts to degrade, reaching saturation at 11 VMs. On the
other hand, placing less that 13 VMs with browsing workload has no negative impact on the
throughput. This behavior  faster host saturation with ordering workload than with browsing
workload  was expected, because for ordering, more writes to the disk are performed.
In terms of power consumption, it is larger under ordering than under browsing, while both
have the same throughout. The dierence is greater as the number of VMs is increasing. More-
over, under ordering the power consumption slope changes at 6 VMs, while under browsing the
slope break happens at 8 VMs.

Inria
An experiment-driven energy consumption model for virtual machine management systems 15

Figure 10: Power consumption and throughput according to workload proles.

Figure 11: Power consumption and throughput according to GPU accelerator.

3.5 Impact of GPU accelerator

In the previous experiment, all the servers are equipped with an Nvidia Tesla M2075 GPU
accelerator. GPU accelerator is designed to use a graphics processing unit (GPU) with a CPU in
order to accelerate scientic, analytics, engineering, consumer, and enterprise applications [34].

We investigate the impact of having a GPU accelerator embedded on servers  although we

are not using it. Figure 10 presents the results increasing the number of VM in the host for
servers with the GPU accelerator and servers without it (same hardware conguration). As in
the previous gures, the X-axis represents the number of VMs in the same host, the Y-axes
represent two metrics: the average power consumption in Watts (blue, left side) and the average
throughput in WIPS (green, right side).

As expected, having a useless GPU accelerator does not improve performance. However, even
if the GPU accelerator is not used, the power consumption of servers with GPU accelerator is
22% larger than without it. Power is wasted if unsuitable servers are used to allocate VMs that
do not exploit all the hardware capabilities.

RR n° 8844
16 An experiment-driven energy consumption model for virtual machine management systems

Table 5: Model parameters in a host h running m VMs.

VM Model parameter
operation Variable Description
- P (m) Power consumption of a host with m VMs (W)

Estart (m) Energy consumption (J)

start up
Tstart (m) Time duration (sec)

Esrc (m) Energy consumption in source (J)

migration Edst (m) Energy consumption in destination (J)
Tmig (m) Time duration (sec)

4 Modeling energy

Next to the experimental facts studied in the previous section, we formalize an energy model for
the VMs management operations. The model is additive, this means the energy consumption of
each VM operation (start up and migration) is added to achieve the total energy consumption
in a time period.
At every time t, the system can start a VM management operation. The energy consumption
of the hosts depends on this choice. Lets E(t, h) the reconguration energy cost in the host h, that
is the energy overhead during the whole VM operation concerning host h; and t the reconguration
time meaning the time to apply the changes. Noting that while start up and migration imply
energy overhead in a interval time; if the systems keeps the same VM placement, the energy
consumption corresponds just to one second (assuming we need to provide energy consumption
each second).
From results of Section 3, we know that the reconguration energy cost depends on the
previous state of the host. Therefore, in Table 5, we dene the model parameters according to
the number of running VMs and the VM operation associated.
Given m the number of VMs in a host h previous to the reconguration, the reconguration
energy cost takes the following values:


 Neither VM start up nor migration begins at time t in h:
E(t, h) = 0




A VM start up begins at time t in host h:




E(t, h) = Estart (m)



A VM migration begins at time t implying host h:
if h is the source host and has


E(t, h) = Esrc (m)


m + 1 VMs at t




if h is the destination host and


E(t, h) = Edst (m)


has m VMs at t


An illustrative example is provided on Figure 12 for the VM start up case. The model has
to learn the dierent parameters, namely P (m), P (m + 1), Estart (m) and Tstart (m).
Finally, the energy consumption of a host during a time period I (during which the number
of VM does not change) is the sum of reconguration energy costs during this period plus its
regular energy consumed by its m VMs:

X
E(t, h) + P (m) × I
t∈I

Inria
An experiment-driven energy consumption model for virtual machine management systems 17

Figure 12: Power consumption and throughput according to GPU accelerator.

Parameters described in Table 5 are learned from a set of experiences by linear regression.
In the following two sections, we present two realistic scenarios, and we validate the model in
both cases. In Section 5, model parameters are learned from the experimental study previously
presented. However, in a real environment, it is rarely feasible to perform the experimental
scenarios of Section 3 to this end. So, in Section 6, we learn the model parameters from a single
scenario on a running system.

5 Experimental results about model validation

In this section, we validate the model by a set of experiments. These experiments are also
designed to show potential use cases of the model. As we have seen in the motivation exam-
ple(Section 1), the proposed model aims at easing the working of energy-ecient VM manage-
ment systems. The estimation of the energy consumption for the dierent options oered by the
system gives information to conduct a what-if  analysis.

Experimental setup. We conduct experiments with the settings already described in Sec-
tion 3 with the ordering prole workload.

5.1 First use-case: VM allocation

An ecient VM allocation depends on the energy consumed by the host and on the throughput
produced by the VMs. We have seen in Section 3.1 that a small throughput degradation can
achieve a high energy savings. In this case, we use the model to estimate the energy consumption
of hosts in order to decide for the most suitable VM allocation.

We mimic a system having a workload peak which requires a variable number of VMs. Fig-
ure 13a shows the number of VMs required and the throughput produced in a real execution.

The 10 VMs required by the workload can be allocated in one host or can use two hosts. We
exploit results from Section 3.1, to evaluate the four VMs placements described in Figure 13b.
We have observed that throughput scales lineal until 7 VMs in the same host. Hence, to maintain
linear throughput, the next 3 VMs are placed in other host (in the top of the gure). In an other
hand, placing 10 VMs in the same host has the largest energy savings with some throughput
degradation (in the bottom of the gure). As intermediate placement between having linear
throughput and having the largest energy savings, we propose two hybrid placements (in the
middle of the gure).

RR n° 8844
18 An experiment-driven energy consumption model for virtual machine management systems

(a) Throughput pattern for the rst use-case

(b) Dierent VMs placements for the rst use-case

Figure 13: Number of VMs required and throughput produced in a real execution.

We assume that the exact moments to start up and delete the VMs are known. This is not a
major constraint in model validation, because the model's goal is to predict energy consumption
of VM management operations and not real workload.

Figure 14 presents the results over 10 repetitions. For each possible placement, the gure
shows the box plot of the relative error between the value estimated by the model and the
real value. The model estimates the energy with a relative error lower than 4% (with average
1.50%). This experiment demonstrates the ability of the proposed model to accurately estimate
the energy consumption of systems with peak loads.

5.2 Second use-case: starts and migrations together

VMs start-ups and migrations can happen together in the system and the model must be able
to t well also during concurrent operations.

For each host, every 300 seconds one VM management operation starts according to the
scenario displayed on Figure 15. For example, for the host 1 allocating 1 initial VM: rstly, a
VM starts at 300 sec (it has 2 VMs); then a VM is received from server 2 at 600 sec (it has 3
VMs); and nally, another VM migrated from server 3 arrives at 900 seconds (it has 4 VMs).
The experiment nishes at 1200 sec.

Inria
An experiment-driven energy consumption model for virtual machine management systems 19

4.0%
3.5%
3.0%
Relative error

2.5%
2.0%
1.5%
1.0%
0.5%
0.0% Linear- Hybrid-1 Hybrid-2 Energy-
throughput efficiency

Figure 14: Relative error in the rst use-case is lower than 4%.

Figure 15: Start-up and migration events together in the second use-case.

Figure 16 presents the results over 10 repetitions. For each host, the gure shows the box plot
of the relative error between the value estimated by the model and the real value. The model
estimates the energy with a relative error lower than 7% (2.46% on average). This experiment
shows that the model correctly grasps the energy consumption of systems where VM start-up
and migrations happen together.
From the achieved results, we show that the proposed model performs well and is able to
accurately estimate the energy consumption of VM management operations, even when dierent
operations occur at the same time. In all cases, the obtained relative error between estimated
and real values are below the 10% line, which is considered as a reasonable limit from literature
(Section 2).

6 Learning model parameters in a real system

In a real running Cloud platform, it is not realistic to run the toy scenarios presented in Section 3
to set model parameters. This means the estimation of model parameters must be done by
observing the system. In this section, we study how to learn VM consolidations from previous
information. We present how to learn the model parameters from a realistic training scenario.
We use another test scenario to evaluate the model accuracy.

RR n° 8844
20 An experiment-driven energy consumption model for virtual machine management systems

Figure 16: Relative error in the second use-case is lower than 7%.

Figure 17: Training scenario for consolidation.

Training scenario. To learn the model, we start with three hosts allocating eight VMs as
shown in Figure 17. Consolidation consists in migrating VMs running on hosts 2 and 3 to host
1. This means, after consolidation hosts 2 and 3 will be completely free. From this scenario, we
infer the average duration and power consumption of migration in source and destination hosts.

Test scenario. In the training scenario, there are not much more choices about how to per-
form consolidations. However, in a system with more hosts there are dierent algorithms to
perform consolidation  especially if hosts are heterogeneous. As we have seen in the motivation
example (Section 1), each algorithm takes dierent migration choices implying dierent power
consumption.
In the test scenario, we have four hosts with twelve VMs overall as shown in Figure 18 on
the top. In order to have two provisioned hosts after consolidation, algorithms move VMs from
hosts 3 and 4 to hosts 1 and 2. Figure 18 on the bottom shows the nal congurations for three
consolidation algorithms called rst-t, balancing and hybrid.

Results. The training set runs 5 times and the test set 15 times. Figure 19 shows the relative
error in the energy estimation over the test set. For each consolidation algorithm, the gure
shows the box plot of the relative error between the energy estimated by the model and the real
consumption.
The average error is always lower than 10%. However, some error points are greater than
10%. Table 6 shows the percent of experiments having an error greater than 10%. This case
occurs when the time actually spent in migration is larger than the estimated time. Although

Inria
An experiment-driven energy consumption model for virtual machine management systems 21

Figure 18: Test scenario for consolidation.

35%
30%
25%
Relative error

20%
15%
10%
5%
0%
First-fit Balancing Hybrid
consolidation consolidation consolidation

Figure 19: Average relative error is lower than 10% in test scenario.

the power consumption of the migration is correctly estimated, increasing the time spent in
migration produces a bigger error.

7 Discussion

We have proposed an experiment-driven model to estimate the energy consumption associated

to VM management operations. This model provides reliable and accurate energy consumption
estimates required by energy-ecient VM management frameworks.
The proposed model is intended for online applications hosted by web servers where levels of
resources utilization are fairly stable. For this reason, our model do not consider performance
monitoring counters. In addition, the model does not cover the energy consumption on the
network during migration, as done in [28] for instance. However, in order to target dierent
workloads and network consumption, a complex model must be designed based on the principles
settled in our model. On top of this solid basis, the next renement of our model will consist in
taking into account VM sizes.
In the experimental facts, energy consumption is confronted to throughput to better under-
stand the energy-performance trade-o. Nevertheless, the presented model only address energy
aspects. After the experimental investigations presented here, we believe an analogous model
can be designed for performance, and it will be the target of a future work.

RR n° 8844
22 An experiment-driven energy consumption model for virtual machine management systems

Table 6: Percent of runs having a relative error greater than 10% in test scenario.

First-t Balancing Hybrid

consolidation consolidation consolidation

17.24% 2.74% 14.71%

In addition to the what-if  analysis enabled by the proposed model, the energy consumption
dependence on the VM management operations shows a potential to increase energy savings
in Cloud platforms. Energy savings could come from the opportunity to exchange resource-
level and application-level information. While Infrastructure-as-a-Service (IaaS) energy-savings
approaches actually provide energy savings, they can only base their decision on low-level metrics
such as CPU usage, memory, I/O etc. However, these infrastructure-level metrics are insucient
to give enough insight in the behavior of applications which generate the load. Instead of treating
applications as black boxes, we argue that signicant energy savings could result from allowing
the consolidation algorithms to access application-level knowledge present in the Platform-as-
a-Service (PaaS) level of the Cloud stack. For example, only the PaaS layer can know if the
workload is expected to decrease in the next few minutes producing the stop of some existing
VMs. Making this information available in the consolidation system would allow to avoid costly
and unnecessary actions such as migrating a VM while the PaaS plans to stop it soon. Such
energy-related decisions could be taken by the IaaS using our model to perform the what-if 
analysis including PaaS information, and thus leading to cooperative energy saving strategies for
Cloud platforms.

8 Conclusions

In this paper, we investigated the energy consumption associated to VM management operations

such as VM placement, VM start up and VM migration. Firstly, we studied, through an exten-
sive set of experiments, the performance and energy consumption, the impact of the dierent
operations and the role of the number of running VMs on the considered hosts. Secondly, we
propose a model for estimating the energy consumption with an average error lower than 10%
for transactional web applications allowing overcommit situations.

As future work we will extend the evaluation to a wide range of applications, such as data
intensive applications. Furthermore, we will introduce the throughput behavior in the model to
estimate the throughput-energy trade-o providing a whole picture of the system.

References

[1] Daniel Balouek, Alexandra Carpen Amarie, Ghislain Charrier, Frédéric Desprez, Em-
manuel Jeannot, Emmanuel Jeanvoine, Adrien Lèbre, David Margery, Nicolas Niclausse,
Lucas Nussbaum, Olivier Richard, Christian Pérez, Flavien Quesnel, Cyril Rohr, and Luc
Sarzyniec. Adding virtualization capabilities to the Grid'5000 testbed. In IvanI. Ivanov,
Marten Sinderen, Frank Leymann, and Tony Shan, editors, Cloud Computing and Services
Science, volume 367 of Communications in Computer and Information Science, pages 320.
Springer International Publishing, 2013. www.grid5000.fr.

Inria
An experiment-driven energy consumption model for virtual machine management systems 23

[2] Donato Barbagallo, Elisabetta Di Nitto, Daniel J. Dubois, and Raaela Mirandola. A Bio-
inspired Algorithm for Energy Optimization in a Self-organizing Data Center. In First
International Workshop on Self-Organizing Architectures, SOAR '09, pages 127151, 2009.
[3] Salman A. Baset, Long Wang, and Chunqiang Tang. Towards an Understanding of Over-
subscription in Cloud. InUSENIX Workshop on Hot Topics in Management of Internet,
Cloud, and Enterprise Networks and Services (Hot-ICE). USENIX, 2012.
[4] Anton Beloglazov, Jemal H. Abawajy, and Rajkumar Buyya. Energy-aware resource al-
location heuristics for ecient management of data centers for Cloud computing. Future
Generation Computer Systems, 28(5):755768, 2012.
[5] Anton Beloglazov and Rajkumar Buyya. Energy Ecient Allocation of Virtual Machines in
10th IEEE/ACM International Conference on Cluster, Cloud and
Cloud Data Centers. In
Grid Computing, CCGrid '10, pages 577578, 2010.
[6] Ramon Bertran, Yolanda Becerra, David Carrera, Vicenç Beltran, Marc González, Xavier
Martorell, Nacho Navarro, Jordi Torres, and Eduard Ayguadé. Energy accounting for shared
virtualized environments under DVFS using PMC-based power models. Future Generation
Computer Systems, 28(2):457468, 2012.
[7] Ata E. H. Bohra and Vipin Chaudhary. VMeter: Power modelling for virtualized clouds.
In24th IEEE International Symposium on Parallel and Distributed Processing - Workshop
Proceedings, IPDPS '10, pages 18, 2010.
[8] Damien Borgetto, Henri Casanova, Georges Da Costa, and Jean-Marc Pierson. Energy-
aware service allocation. Future Generation Computer Systems, 28(5):769779, 2012.
[9] David Breitgand, Zvi Dubitzky, Amir Epstein, Alex Glikson, and Inbar Shapira. SLA-
aware Resource Over-commit in an IaaS Cloud. In International Conference on Network
and Service Management (CNSM), pages 7381. International Federation for Information
Processing, 2013.

[10] Alexandra Carpen-Amarie, Anne-Cécile Orgerie, and Christine Morin. Experimental Study
on the Energy Consumption in IaaS Cloud Environments. In IEEE/ACM 6th International
Conference on Utility and Cloud Computing, UCC '13, pages 4249, 2013.
[11] Christopher Clark, Keir Fraser, Steven Hand, Jacob Gorm Hansen, Eric Jul, Christian
Proceed-
Limpach, Ian Pratt, and Andrew Wareld. Live Migration of Virtual Machines. In
ings of the 2nd Conference on Symposium on Networked Systems Design & Implementation,
NSDI'05, pages 273286, 2005.

[12] Maxime Colmant, Mascha Kurpicz, Pascal Felber, Loïc Huertas, Romain Rouvoy, and Anita
Sobe. Process-level power estimation in VM-based systems. In Proceedings of the Tenth
European Conference on Computer Systems, EuroSys '15, page 14, 2015.
[13] SPECjbb2013 Benchmark. Standard Performance Evaluation Corporation. www.spec.org/
jbb2013/.

[14] Gaurav Dhiman, Kresimir Mihic, and Tajana Rosing. A system for online power prediction
in virtualized environments using Gaussian mixture models. In Proceedings of the 47th
Design Automation Conference, DAC '10, pages 807812, 2010.

RR n° 8844
24 An experiment-driven energy consumption model for virtual machine management systems

[15] Anshul Gandhi, Varun Gupta, Mor Harchol-Balter, and Michael A. Kozuch. Optimality
analysis of energy-performance trade-o for server farm management. Performance Evalu-
ation, 67(11):11551171, 2010.
[16] Qiang Huang, Fengqian Gao, Rui Wang, and Zhengwei Qi. Power Consumption of Virtual
Third International Conference on Communications
Machine Live Migration in Clouds. In
and Mobile Computing, CMC '11, pages 122125, 2011.
[17] Amazon EC2 Instances. https://aws.amazon.com/ec2/instance-types/?nc1=h_ls.

[18] Abhishek Jaiantilal, Yifei Jiang, and Shivakant Mishra. Modeling CPU Energy Consumption
for Energy Ecient Scheduling. In Proceedings of the 1st Workshop on Green Computing,
GCM '10, pages 1015, 2010.

[19] Aman Kansal, Feng Zhao, Jie Liu, Nupur Kothari, and Arka A. Bhattacharya. Virtual
Machine Power Metering and Provisioning. In Proceedings of the 1st ACM Symposium on
Cloud Computing, SoCC '10, pages 3950, 2010.
[20] Jonathan Koomey. Growth in data center electricity use 2005 to 2010. Analytics Press,
August 2011.

[21] Bhavani Krishnan, Hrishikesh Amur, Ada Gavrilovska, and Karsten Schwan. VM power me-
tering: feasibility and challenges. SIGMETRICS Performance Evaluation Review, 38(3):56
60, 2010.

[22] KVM: Kernel-based Virtual Machine. http://www.linux-kvm.org.

[23] Hongjian Li, Guofeng Zhu, Chengyuan Cui, Hong Tang, Yusheng Dou, and Chen He.
Energy-ecient migration and consolidation algorithm of virtual machines in data centers
for cloud computing. Computing, pages 115, 2015.
[24] Dan Liao, Ke Li, Gang Sun, V. Anand, Yu Gong, and Zhi Tan. Energy and performance
International Confer-
management in large data centers: A queuing theory perspective. In
ence on Computing, Networking and Communications, ICNC '15, pages 287291, 2015.
[25] Haikun Liu, Cheng-Zhong Xu, Hai Jin, Jiayu Gong, and Xiaofei Liao. Performance and
energy modeling for live migration of virtual machines. Proceedings of the 20th ACM
In
International Symposium on High Performance Distributed Computing, HPDC '11, pages
171182, 2011.

[26] Martina Maggio, Henry Homann, Alessandro V. Papadopoulos, Jacopo Panerati, Marco D.
Santambrogio, Anant Agarwal, and Alberto Leva. Comparison of Decision-Making Strate-
gies for Self-Optimization in Autonomic Computing Systems. ACM Transactions on Au-
tonomous and Adaptive Systems, 7(4):36:136:32, 2012.
[27] Vincenzo De Maio, Gabor Kecskemeti, and Radu Prodan. Workload-Aware Energy Model
for Virtual Machine Migration. In IEEE International Conference on Cluster Computing,
CLUSTER '15, 2015.

[28] Vincenzo De Maio, Radu Prodan, Shajulin Benedict, and Gabor Kecskemeti. Modelling
energy consumption of network transfers and virtual machine migration. Future Generation
Computer Systems, 2015. In press, Accepted Manuscript.

Inria
An experiment-driven energy consumption model for virtual machine management systems 25

[29] Zoltán A. Mann. Rigorous results on the eectiveness of some heuristics for the consolidation
of virtual machines in a cloud data center. Future Generation Computer Systems, 51:1  6,
2015.

[30] Toni Mastelic, Ariel Oleksiak, Holger Claussen, Ivona Brandic, Jean-Marc Pierson, and
Athanasios V. Vasilakos. Cloud Computing: Survey on Energy Eciency. ACM Computer
Surveys, 47(2):33:133:36, 2014.
[31] Michele Mazzucco, Dmytro Dyachuk, and Ralph Deters. Maximizing cloud providers' rev-
enues via energy aware allocation policies. In IEEE International Conference on Cloud
Computing, CLOUD '10, pages 131138, 2010.
[32] Nikita Mishra, Huazhe Zhang, John D. Laerty, and Henry Homann. A Probabilistic
Graphical Model-based Approach for Minimizing Energy Under Performance Constraints. In
Proceedings of the 20th International Conference on Architectural Support for Programming
Languages and Operating Systems, ASPLOS '15, pages 267281, 2015.
[33] Christoph Mobius, Waltenegus Dargie, and Alexander Schill. Power Consumption Estima-
tion Models for Processors, Virtual Machines, and Servers. IEEE Transactions on Parallel
and Distributed Systems, 25(6):16001614, 2014.
[34] Nvidia Corporation. www.nvidia.com/object/what-is-gpu-computing.html#sthash.
TDnNYEDE.dpuf.

[35] OpenStack Project. OpenStack: The open source cloud operating system. openstack.org.

[36] Anne-Cecile Orgerie, Marcos Dias de Assuncao, and Laurent Lefèvre. A survey on techniques
for improving the energy eciency of large-scale distributed systems. ACM Computing
Surveys, 46(4), 2014.
[37] Anne-Cécile Orgerie, Laurent Lefèvre, and Jean-Patrick Gelas. Demystifying energy con-
sumption in Grids and Clouds. In International Green Computing Conference 2010, IGCC
'10, pages 335342, 2010.

[38] Paula Petrica, Adam M. Izraelevitz, David H. Albonesi, and Christine A. Shoemaker.
Flicker: A Dynamically Adaptive Architecture for Power Limited Multicore Systems. ACM
SIGARCH Computer Architecture News, 41(3):1323, 2013.
[39] Srinath Sridharan, Gagan Gupta, and Gurindar S. Sohi. Holistic Run-time Parallelism
Management for Time and Energy Eciency. In Proceedings of the 27th International ACM
Conference on International Conference on Supercomputing, ICS '13, pages 337348, 2013.
[40] Anja Strunk. Costs of Virtual Machine Live Migration: A Survey. In Eighth IEEE World
Congress on Services, SERVICES '12, pages 323329, 2012.
[41] Anja Strunk. A lightweight model for estimating energy cost of live migration of virtual
machines. In 2013 IEEE Sixth International Conference on Cloud Computing, CLOUD '13,
pages 510517, 2013.

[42] Anja Strunk and Waltenegus Dargie. Does Live Migration of Virtual Machines Cost En-
ergy? In IEEE 27th International Conference on Advanced Information Networking and
Applications, AINA '13, pages 514521, 2013.

RR n° 8844
26 An experiment-driven energy consumption model for virtual machine management systems

[43] Petter Svärd, Benoit Hudzia, Steve Walsh, Johan Tordsson, and Erik Elmroth. Principles
and Performance Characteristics of Algorithms for Live VM Migration. SIGOPS Operating
Systems Review, 49(1):142155, 2015.
[44] TPC Benchmark W (Web Commerce) Specication. version 1.0.1, 2000. www.tpc.org/tpcw.
[45] Akshat Verma, G. Kumar, R. Koller, and A. Sen. CosMig: Modeling the Impact of Re-
International Symposium on Modeling, Analysis Simulation of
conguration in a Cloud. In
Computer and Telecommunication Systems (MASCOTS), pages 311, July 2011.
[46] Xen. http://www.xenproject.org.
[47] Kejiang Ye, Xiaohong Jiang, Dawei Huang, Jianhai Chen, and Bei Wang. Live Migration of
Multiple Virtual Machines with Resource Reservation in Cloud Computing Environments.
In Cloud Computing (CLOUD), 2011 IEEE International Conference on, pages 267274,
July 2011.

Inria
RESEARCH CENTRE Publisher
RENNES – BRETAGNE ATLANTIQUE Inria
Domaine de Voluceau - Rocquencourt
Campus universitaire de Beaulieu BP 105 - 78153 Le Chesnay Cedex
35042 Rennes Cedex inria.fr

ISSN 0249-6399