Analysis of Docker Security - 1501.02967
Analysis of Docker Security - 1501.02967
Analysis of Docker Security - 1501.02967
Thanh Bui
Aalto University School of Science
thanh.bui@aalto.fi
virtualization are two main types of virtualization technolo- security features of the Linux kernel, such as SELinux and
gies that have emerged to the market. Of these two classes, AppArmor, in order to harden the host system. The analysis
container-based virtualization is able to provide a more examined the internal security of Docker based on the level
lightweight and efficient virtual environment, but not without of isolation Docker can provide to its virtual environments.
security concerns. In this paper, we analyze the security level The interaction between Docker and the security features of
of Docker, a well-known representative of container-based the kernel was estimated based on how the features are sup-
approaches. The analysis considers two areas: (1) the inter- ported by Docker. To the best of our knowledge, Docker is a
nal security of Docker, and (2) how Docker interacts with the relatively new technology, and this is one of the first analyses
security features of the Linux kernel, such as SELinux and of this kind that focus on its security aspects.
AppArmor, in order to harden the host system. Furthermore, The paper is structured as follows: Section 2 provides a
the paper also discusses and identifies what could be done high-level view of the two classes of virtualization solutions.
when using Docker to increase its level of security. Section 3 gives an overview of Docker and its underlying
technologies. Section 4 presents our analysis of Docker se-
KEYWORDS: Containers, Docker, Security curity, and then in Section 5, we discuss the security level of
Docker and what could be done to raise its level of security.
The paper concludes with a summary in Section 6.
1 Introduction
The last decade has seen an explosion of development in the 2 Virtualization Approaches
area of virtualization technologies, which allow the partition-
ing of a computer system into multiple isolated virtual en- Most of the virtualization technologies can be classified into
vironments. The technologies offer substantial benefits that two major approaches: container-based virtualization and
have been driving their development rapidly. One of the most hypervisor-based virtualization. The former provides virtu-
common reasons for adopting virtualization technologies is alization at the operating system level, while the latter pro-
server virtualization in data centers. With server virtualiza- vides virtualization at the hardware level. Each of the ap-
tion, an administrator can create one or more virtual system proaches has its own advantages and disadvantages, which
instances on a single server. These virtual systems operate as are described in this section.
real physical servers and can be rented out on a subscription Container-based virtualization is a lightweight virtual-
basis. Amazon EC2, Rackspace, and DreamHost are some ization approach using the host kernel to run multiple vir-
popular instances of such data center service providers. An- tual environments. These virtual environments are often re-
other common use is for desktop virtualization, where one ferred to as containers. Linux-VServer [31], OpenVZ [11],
computer can run several OS instances. Desktop virtualiza- and Linux Container (LXC) [10] are the three main repre-
tion provides support for applications that can run only on a sentatives of this approach. The general architecture of a
specific OS. container-based virtualization solution is illustrated in Fig. 1.
The growth in the use of virtualization technologies pro- Container-based virtualization virtualizes at the operating
motes the demand for a virtualization solution which can system level, thus allowing multiple applications to operate
provide dense, scalable, and secure user environments. A without redundantly running other operating system kernels
large number of virtualization solutions have emerged to on the host. Its containers look like normal processes from
the market. They can be classified into two major classes: outside, which run on top of the kernel shared with the host
container-based virtualization and hypervisor-based virtual- machine. They provide isolated environments with neces-
ization. Of these two classes, container-based virtualization sary resources to execute applications. These resources can
is able to provide a more lightweight and efficient virtual en- be either shared with the host or installed separately inside
vironment. It allows ten times more virtual environments to the container.
run on a physical server compared to hypervisor-based virtu- Hypervisor-based virtualization solutions provide virtu-
Aalto University T-110.5291 Seminar on Network Security Autumn 2014
3 Docker Overview
Docker is an open source container technology with the abil-
ity "to build, ship, and run distributed applications" [17]. It
has been used in some popular applications, such as Spotify,
Yelp, and Ebay.
Although container technologies have been around for
more than a decade, Docker - a relatively new candidate -
is currently one of the most successful technologies since
Figure 2: Architecture of Hypervisor-based Virtualization it comes with new abilities that earlier technologies did not
possess. First, it provides interfaces to simply and safely cre-
ate and control containers. Secondly, developers can pack
applications into lightweight Docker containers which can
alization at the hardware level. In contrast to container-based operate on almost anywhere without modification. Further-
virtualization, a hypervisor establishes complete virtual ma- more, Docker can deploy more virtual environments than
chines (VMs) on top of the host operating system (Fig. 2). other technologies can on the same hardware [19]. Last
Each virtual machine comprises of not only an application but not least, Docker cooperates well with third-party tools,
and its dependencies, but also an entire guest OS along with which simplify the management and deployment process of
a separate kernel. There are two classes of hypervisors: the Docker containers. DevOps tools, such as Puppet [13], An-
Type 1 hypervisor, also known as the bare metal hypervisor, sible [1], and Vagrant [16] can integrate with Docker, thus
which works directly on top of the underlying hardware of making Docker containers to be easily deployed to a cloud.
the host, and the Type 2 hypervisor, also known as the hosted Moreover, many orchestration tools, such as Mesos [22],
hypervisor, which works on top of the host operating system Shipyard [15], and Kubernetes [7], also support Docker con-
[26]. Xen [18] is an example of the former, while KVM [25] tainers. These tools provide an abstract layer of resources
is of the latter. Since the Type 1 hypervisor does not include management and scheduling over Docker.
an extra layer of the host OS, it provides better performance
than the Type 2 hypervisor. Docker consists of two major components: Docker engine
and Docker Hub. The former is an open source virtualization
The differences in the architecture bring some benefits to
solution, while the latter is a Software-as-a-Service platform
container-based virtualization over hypervisor-based virtu-
for sharing Docker images. The following sections describe
alization. First, container-based virtualization can provide
in details these two components.
higherer density of virtual environments. Since a container
does not include an entire OS, the size and the required re-
sources to run an application in a container are less than that
of a VM running the same application. As a result, more con- 3.1 Docker Engine
tainers than traditional virtual machines can be deployed on
the same host. Secondly, container-based virtualization also Docker engine is a lightweight and portable packaging tool
offers better performance. This has been demonstrated by [17] which relies on container-based virtualization. There-
experiments in some studies [32, 28, 27, 21]. These studies fore, the architecture of the Docker engine (Fig. 3) is simi-
show that the performance of container-based virtualization lar to that of container-based virtualization in general. The
is better than with hypervisor-based virtualization in most Docker containers run on top of the Docker daemon which
cases, and it is almost as good as native applications. is in charge of executing and managing all of the Docker
However, despite all of the mentioned advantages, containers. The Docker client, which provides an user inter-
container-based virtualization is unable to support a variety face for interacting with containers to Docker users, accepts
of environments in the way hypervisor-based virtualization commands from the users and then sends it to the Docker
does since all the environments of the containers must be of daemon through RESTful APIs. Using this method of com-
the same type as that of the host. For example, Windows munication enables the Docker client to run on the same host
containers cannot be run on top of a Linux host. as the containers, or even on different hosts.
Aalto University T-110.5291 Seminar on Network Security Autumn 2014
ages. Users can also search for published images and down-
load them with the Docker client. Furthermore, users can
verify the authenticity and integrity of the downloaded im-
ages since Docker signed and verified the images when their
owner submitted them to the hub.
This mechanism operates with the support of the PID Thus, it is crucial to limit the set of device nodes that a con-
namespaces, which isolate the process ID number space of tainer can access.
a container from that of the host. Since PID namespaces are The Device Whitelist Controller feature [3] of cgroups
hierarchical [12], a process can only see the other processes provides means to limit the set of devices that Docker al-
in its own namespace or in its "children" namespaces. As a lows a container to access. It also prevents the processes
consequence, once a new namespace is created and assigned in containers from creating new device nodes. Furthermore,
to a container, the host can observe and affect the processes Docker mounts container images with nodev, meaning that
inside the new PID namespace of the container, but the pro- even if a device node was pre-created inside the image, the
cesses inside the container cannot observe or do anything to processes in the container using the image cannot use it to
the other processes running in the host or in other containers. communicate with the kernel. By default, Docker does not
If the attacker cannot observe other processes, it is harder to give extended privileges to its containers. Therefore, they
attack them. cannot access any devices. However, if the operator exe-
The PID namespaces also allow each container to have its cutes a container as "privileged", Docker grants access to all
own init-like process (PID 1), which causes all the processes devices to the container.
in a namespace to be terminated if it is terminated. This
process assists the administrator in completely shutting down
a container when something suspicious is detected. IPC Isolation
all of the access controls over objects, not their owners. This it does not provide any filter on the network traffic passing
provides a secure separation for containers as it can prevent through the bridge. However, this problem can be solved if
processes, even with root privileges, within a container to the administrator manually adds filtering, such as ebtables
illegitimately access objects outside the containers. [6], to the bridge, or changes the networking connectivity to
Docker uses two classes of policy enforcement: Type en- a more secure one, such as virtual network.
forcement and MCS enforcement [24]. The Type enforce- It is also worth highlighting that if the operator runs a con-
ment protects the host from the processes in containers, and tainer as "privileged", Docker grants full access permissions
the MCS enforcement protects a container from another con- to the container, which is nearly the same as that of processes
tainer. running natively on the host. Therefore, it is more secure to
With Type enforcement, Docker labels all container pro- operate containers as "non-privileged".
cesses with svirt_lxc_net_t type and all content within a Furthermore, even though containers can provide higher
container with svirt_sandbox_f ile_t type. The processes density of virtual environments and better performance, they
running with svirt_lxc_net_t type can only access/write to have a bigger attack surface than virtual machines since con-
the content labeled with svirt_sandbox_f ile_t type, but tainers can directly communicate with the host kernel. How-
not to any other label on the system. Therefore, the pro- ever, it is possible to reduce the attack surface while main-
cesses running within containers can only use the content in- taining these advantages. For example, this can be achieved
side containers. However, only with this policy enforcement, by placing containers inside virtual machines.
Docker allows the processes in one container to have access
to the content of other containers. MCS enforcement is nec-
essary to solve this issue. When a container is launched, the 6 Conclusion and Future work
Docker daemon picks a random MCS label and then puts this
label on all of the processes and content of the container. The Container-based virtualization can provide higher den-
kernel only allow processes to access content with the same sity virtual environments and better performance than
MCS label, thus preventing a compromised process in one hypervisor-based virtualization. However, the latter is ar-
container from attacking other containers. gued to be more secure than the former. In this paper,
we conducted an analysis on Docker, which is one of the
AppArmor most popular container-based virtualization technologies, to
discover how safe its containers are. Our analysis shows
AppArmor is also a security enhancement model to Linux that Docker containers are fairly secure, even with the de-
based on Mandatory Access Control like SELinux, but re- fault configuration. The security level of Docker contain-
stricting its scope to individual programs. It permits the ers could also be increased if the operator runs them as
administrator to load a security profile into each program, "non-privileged" and enables additional hardening solutions
which limits the capabilities of the program. AppArmor sup- in Linux kernel, such as AppArmor or SELinux.
ports two modes: enforcement mode and complain/learning The future work after this paper could be to compare the
mode. The enforcement mode enforces the policies defined security of Docker containers with that of other container-
in the profile. However, in the complain/learning mode, the ization systems or with virtual machines. Such studies could
violations of profile policies are permitted, but also logged. lead to e.g. a detailed static analysis Docker or a broader
This log can be useful for developing new profiles later. view of security in containers in general.
On systems that support AppArmor, Docker provides an
interface for loading a pre-defined AppArmor profile when
launching a new container. This profile is loaded into Acknowledgement
the container in enforcement mode in order to ensure that
the processes in the container are restricted according to This research paper is made possible through the help and
the profile. If the administrator does not specify a profile support of Miika Komu, Roberto Morabito, Jimmy Kjäll-
when launching a container, the Docker daemon automat- man, and Tero Kauppinen from Nomadiclab.
ically loads a default profile to the container, which de-
nies access to important filesystems on the host, such as
/sys/f s/cgroups/ and /sys/kernel/security/. References
[1] Ansible. http://www.ansible.com/home/.
5 Discussion [Accessed 25 October 2014].
The analysis shows that Docker provides a high level of iso- [2] Containers & docker: How secure are they?
lation and resource limiting for its containers using names- https://blog.docker.com/2013/08/
paces, cgroups, and its copy-on-write file system, even with containers-docker-how-secure-are-they.
the default configuration. It also supports several kernel se- [Accessed 25 October 2014].
curity features, which help to hardening the security of the
host. The only problem we found with Docker was related [3] Device whitelist controller. https://www.
to its default networking model. The virtual ethernet bridge kernel.org/doc/Documentation/
which Docker uses as its default networking model, is vul- cgroups/devices.txt. [Accessed 12 Octo-
nerable to ARP spoofing and MAC flooding attacks since ber 2014].
Aalto University T-110.5291 Seminar on Network Security Autumn 2014
[4] Docker hub. https://hub.docker.com/. [Ac- [21] W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. An
cessed 30 September 2014]. updated performance comparison of virtual machines
and linux containers. Technical Report RC25482
[5] Docker: Network configuration. https://docs. (AUS1407-001), IBM Research Division, July 2014.
docker.com/articles/networking/. [Ac-
cessed 24 September 2014]. [22] B. Hindman, A. Konwinski, M. Zaharia, A. Ghodsi,
A. D. Joseph, R. Katz, S. Shenker, and I. Stoica. Mesos:
[6] Ebtables. http://ebtables.netfilter. A platform for fine-grained resource sharing in the data
org/. [Accessed 25 October 2014]. center. In Proceedings of the 8th USENIX Confer-
[7] Kubernetes project. https://github.com/ ence on Networked Systems Design and Implementa-
googlecloudplatform/kubernetes. [Ac- tion, NSDI’11, pages 295–308, Berkeley, CA, USA,
cessed 10 November 2014]. 2011. USENIX Association.
[8] Libcontainer project. https://github.com/ [23] D. J. Walsh. Are docker containers really se-
docker/libcontainer. [Accessed 25 October cure? http://opensource.com/business/
2014]. 14/7/docker-security-selinux. [Accessed
25 October 2014].
[9] Linux capabilities. http://linux.die.net/
man/7/capabilities. [Accessed 12 October [24] D. J. Walsh. Bringing new security features to docker.
2014]. https://opensource.com/business/
14/9/security-for-docker. Available at:
[10] LXC. https://linuxcontainers.org/. [Ac- [Accessed 25 October 2014].
cessed 30 September 2014].
[25] A. Kivity, Y. Kamay, D. Laor, and U. Lublin. KVM:
[11] OpenVZ. http://openvz.org/. [Accessed 30 the linux virtual machine monitor. In Proceedings of
September 2014]. the Linux Symposium, volume 1, pages 225–230. 2007.
[12] PID namespaces in the 2.6.24 kernel. http://lwn. [26] D. Merkel. Docker: Lightweight linux containers for
net/Articles/259217/. [Accessed 30 Septem- consistent development and deployment. Linux J.,
ber 2014]. 2014(239), Mar. 2014.
[13] Puppet. http://puppetlabs.com/. [Accessed [27] P. Padala, X. Zhu, Z. Wang, S. Singhal, and K. G. Shin.
18 October 2014]. Performance evaluation of virtualization technologies
for server consolidation. HP Laboratories, 2007.
[14] SECure COMPuting with filters. https:
//www.kernel.org/doc/Documentation/ [28] N. Regola and J.-C. Ducom. Recommendations for vir-
prctl/seccomp_filter.txt. Available at: tualization technologies in high performance comput-
[Accessed 02 November 2014]. ing. In 2010 IEEE Second International Conference
on Cloud Computing Technology and Science (Cloud-
[15] Shipyard project. https://github.com/ Com), pages 409–416, Nov. 2010.
shipyard/shipyard. [Accessed 12 November
2014]. [29] E. Reshetova, J. Karhunen, T. Nyman, and N. Asokan.
Security of OS-level virtualization technologies. In
[16] Vagrant. https://www.vagrantup.com/. [Ac- Proceedings of the 2014 NordSec Conference, pages
cessed 15 November 2014]. 77–93, Norway, 2014.
[17] What is docker? https://docker.com/ [30] S. Smalley, C. Vance, and W. Salamon. Implementing
whatisdocker/. [Accessed 15 November 2014]. SELinux as a linux security module. NAI Labs Report
#01-043, NAI Labs, Dec. 2001. Revised May 2002.
[18] B. R. Anderson, A. K. Joines, and T. E. Daniels. Xen
worlds: Leveraging virtualization in distance educa- [31] S. Soltesz, H. Potzl, M. E. Fiuczynski, A. Bavier,
tion. In Proceedings of the 14th Annual ACM SIGCSE and L. Peterson. Container-based operating system
Conference on Innovation and Technology in Computer virtualization: A scalable, high-performance alterna-
Science Education, ITiCSE ’09, pages 293–297, New tive to hypervisors. In Proceedings of the 2Nd ACM
York, NY, USA, 2009. ACM. SIGOPS/EuroSys European Conference on Computer
Systems 2007, pages 275–287, USA, 2007. ACM.
[19] C. Burniske. Containers: The next generation of virtu-
alization? http://ark-invest.com/webx0/ [32] M. G. Xavier, M. V. Neves, F. D. Rossi, T. C. Ferreto,
containers-next-generation-virtualization. T. Lange, and C. A. F. De Rose. Performance eval-
[Accessed 22 November 2014]. uation of container-based virtualization for high per-
formance computing environments. In Proceedings of
[20] C. Cowan, S. Beattie, G. Kroah-Hartman, C. Pu, P. Wa- the 21st Euromicro International Conference on Paral-
gle, and V. Gligor. SubDomain: Parsimonious server lel, Distributed, and Network-Based Processing, pages
security. In Proceedings of the 14th USENIX Confer- 233–240, Washington, DC, USA, 2013. IEEE Com-
ence on System Administration, LISA ’00, pages 355– puter Society.
368, Berkeley, CA, USA, 2000. USENIX Association.