Exploring The Performance of ROS2
Exploring The Performance of ROS2
Exploring The Performance of ROS2
net/publication/309128426
CITATIONS READS
114 21,597
3 authors, including:
Takuya Azumi
Saitama University
79 PUBLICATIONS 608 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Takuya Azumi on 19 March 2019.
ABSTRACT OpenCV and the Point Cloud Library (PCL) [3]), and tools
Middleware for robotics development must meet demand- to help software developers create robotics applications.
ing requirements in real-time distributed embedded systems. However, the ROS does not satisfy real-time run require-
The Robot Operating System (ROS), open-source middle- ments and only runs on a few OSs. In addition, the ROS
ware, has been widely used for robotics applications. How- cannot guarantee fault-tolerance, deadlines, or process syn-
ever, the ROS is not suitable for real-time embedded sys- chronization. Moreover, the ROS requires significant re-
tems because it does not satisfy real-time requirements and sources (e.g, CPU, memory, network bandwidth, threads,
only runs on a few OSs. To address this problem, ROS1 and cores) and can not manage these resources to meet time
will undergo a significant upgrade to ROS2 by utilizing the constraints. Thus, the ROS is not suitable for real-time em-
Data Distribution Service (DDS). DDS is suitable for real- bedded systems. This critical problem has been considered
time distributed embedded systems due to its various trans- by many research communities, including ROS developers,
port configurations (e.g., deadline and fault-tolerance) and and various solutions have been proposed and evaluated [13],
scalability. ROS2 must convert data for DDS and abstract [19], [36]. However, these solutions are insufficient1 to ad-
DDS from its users; however, this incurs additional over- dress the ROS’s limitations for real-time embedded systems.
head, which is examined in this study. Transport latencies To satisfy the needs of the now-broader ROS community,
between ROS2 nodes vary depending on the use cases, data the ROS will undergo a significant upgrade to ROS2 [23].
size, configurations, and DDS vendors. We conduct proof of ROS2 will consider the following new use cases: real-time
concept for DDS approach to ROS and arrange DDS char- systems, small embedded platforms (e.g., sensor nodes), non-
acteristic and guidelines from various evaluations. By high- ideal networks, and cross-platform (e.g., Linux, Windows,
lighting the DDS capabilities, we explore and evaluate the Mac, Real-Time OS (RTOS), and no OS). To satisfy the
potential and constraints of DDS and ROS2. requirements of these new use cases, the existing version
of ROS (hereinafter ROS1) will be reconstructed to im-
Keywords prove user-interface APIs and incorporate new technologies,
robot operating system; data distribution service; quality of such as Data Distribution Service (DDS) [24], [30], Zero-
service; real-time; embedded; publish/subscribe conf, Protocol Buffers, ZeroMQ, Redis, and WebSockets.2
The ROS1 transport system will be replaced by DDS, an
1. INTRODUCTION industry-standard real-time communication system and end-
In recent years, real-time distributed embedded systems, to-end middleware. The DDS can provide reliable pub-
such as autonomous driving vehicles, have become increas- lish/subscribe transport similar to that of ROS1.
ingly complicated and diverse. Autonomous driving has at- DDS is suitable for real-time embedded systems because of
tracted attention since the November 3, 2007 DARPA Urban its various transport configurations (e.g., deadline, reliabil-
Challenge [34]. The Robot Operating System (ROS) [28] is ity, and durability) and scalability. DDS meets the require-
open-source middleware that has undergone rapid develop- ments of distributed systems for safety, resilience, scalability,
ment [11] and has been widely used for robotics applications fault-tolerance and security. DDS can provide solutions for
(e.g., autonomous driving systems). The ROS is built almost some real-time environments and some small/embedded sys-
entirely from scratch and has been maintained by Willow tems by reducing library sizes and memory footprints. De-
Garage [7] and Open Source Robotics Foundation (OSRF) veloped by different DDS vendors, several implementations
[2] since 2007. The ROS enhances productivity [12], pro- of this communication system have been used in mission-
viding publish/subscribe transport, multiple libraries (e.g., critical environments (e.g., trains, aircrafts, ships, dams, and
financial systems) and have been verified by NASA and the
United States Department of Defense. Several DDS imple-
mentations have been evaluated and validated by researchers
ACM acknowledges that this contribution was authored or co-authored by an em-
[37], [32] and DDS vendors. These evaluations indicate that
ployee, contractor or affiliate of a national government. As such, the Government DDS is both reliable and flexible.
retains a nonexclusive, royalty-free right to publish or reproduce this article, or to al- Contribution: In this paper, we provide proof of concept
low others to do so, for Government purposes only.
1
EMSOFT’16, October 01-07, 2016, Pittsburgh, PA, USA Reasons why prior work is insufficient are discussed in Sec-
c 2016 ACM. ISBN 978-1-4503-4485-2/16/10. . . $15.00
tion 4.
2
Currently ROS2 only supports some DDS implementations.
DOI: http://dx.doi.org/10.1145/2968478.2968502
ZK^ϭ WRSLF
ƉƉůŝĐĂƚŝŽŶ ƉƉůŝĐĂƚŝŽŶ
$SSOLFDWLRQ
/D\HU DĂƐƚĞƌ EŽĚĞ EŽĚĞ EŽĚĞ EŽĚĞ EŽĚĞ ,PDJHV
3XEOLVK 6XEVFULEH
ZK^Ϯ
YƵĂůŝƚLJŽĨ^ĞƌǀŝĐĞWŽůŝĐLJ 5(/,$%,/,7<
ZĞĂůͲdŝŵĞWƵďůŝƐŚͬ^ƵďƐĐƌŝďĞWƌŽƚŽĐŽů
ĚĂƚĂϰ ĚĂƚĂϯ ĚĂƚĂϮ ĚĂƚĂϭ
ŝƐƚƌŝďƵƚĞĚEĞƚǁŽƌŬ
'($'/,1(SHULRG
Threads 4 4
D526UHPRWH E526ORFDO F526LQWUDSURFHVV
Memory 16 GB 8 GB
ZLWKRXW''6
Network 100 Mbps Ethernet / Full-Duplex
ROS1 Indigo
ROS2 Cement (alpha3) 526 526
Connext1/ OpenSplice2/ FastRTPS
526 526
526
DDS implementations 526
support reliable transport of initial messages. This influ- we cannot collect sufficient samples and conduct evaluations.
ences ROS2 when using a ros_bridge. In contrast, ROS2 Several evaluations fail in (3-b) and remote cases, as shown
does not lose initial messages, even when using large data in Table 4. Currently, the above results indicate that ROS2
such as 4 MB. This proves the reliability of DDS. In best- is not suitable for handling large messages.
effort policy, a subscriber-node must be launched before 3.3 Latency Characteristics of ROS1 and ROS2
a publisher-node begins to send messages for no “Initial
loss”. On the other hand, with ROS2 reliable policy, As shown in Figures 7, 8, 9, 10, a tendency of end-to-end
a subscriber-node does not have to be launched before a latencies characteristics is clarified in each situation shown
publisher-node starts sending messages. This is attributed in Figure 5. In (2-a) and (2-b), ROS2 uses OpenSplice with
to TRANSIENT_LOCAL in DURABILITY of the QoS Policy. the reliable policy because ROS1 uses TCPROS, i.e., re-
The reliable policy is tuned to provide resilience against liable communication. In (3-a) and (3-b), to evaluate la-
late-joining subscriber-nodes. In ROS1, published messages tencies with large data (e.g., 512 KB and 1 MB), Connext
are lost and never recovered. This QoS Policy accelerates with the best-effort policy is used. First, we analyze
fault-tolerance. ROS2 performance compared to ROS1. We then evaluate
Another interesting observation from Table 4 is that ROS2 ROS2 with different DDS implementations and configura-
has many problems when transporting large data. Many tions, such as the QoS Policy.
experiments fail in various situations with ROS2; however, 3.3.1 Comparison between remote and local cases
we can observe differences in performance between Connext ROS1 and ROS2 is much less than the difference between
and OpenSplice. These constraints on large data originate remote and local cases. Figures 7 and 8 show the medians
from the fact that the maximum payload of Connext and of the latencies for the remote and local cases. Since the
OpenSplice is 64 KB. This is the maximum packet size of IP conversion influences from ROS1 to ROS2 and from ROS2
protocol. It is hard to maintain divided packets with QoS to ROS1 are similar, Figures 7 and 8 contain one-way data.
Policy by default API. Therefore, we consider that DDS In Figure 7, the behavior of all latencies is constant up to
is not designed to handle large data. This is important 4 KB. In contrast, the latencies in the remote cases grow
for the analysis of ROS2 performance. For example, Fas- sharply from 16 KB, as shown in Figures 7 and 8. This
tRTPS does not support large data because it is designed as is because ROS1 and ROS2 divide a message into 15 KB
a lightweight implementation for embedded systems. Even a packets to transmit data through Ethernet. This difference
string of 256 B exceeds the maximum length in FastRTPS. between the remote and local cases corresponds to the data
Many DDS vendors do not support publishing large data transmission time between Machine1 and Machine2, which
with reliable connections and common APIs. To send and was measured in a preliminary experiment. The prelimi-
manage divided packets, such DDS vendors provide an al- nary experiment measured transmission time for each data
ternate API such as an asynchronous publisher and flow size using ftp or http. This correspondence indicates that
controller, which has not been abstracted from ROS2. In the RTPS protocol and data about the QoS Policy have lit-
our experiments, Connext with reliable policy yields er- tle influence on data transmission time in the network. In
rors when data are greater than 64 KB. Some failures with addition, all latencies are predictable by measuring the data
the best-effort policy are due to frequent message losses transmission time.
caused by non-reliable communication. When a publisher-
node fails to transfer data to a subscriber-node frequently,
3.3.2 Comparison among local, nodelet, and intra-
process cases
1.5
8 (1−a) ROS1 remote (1−c) ROS1 nodelet
(1−b) ROS1 local (1−b) ROS1 local
12
(2−a) ROS2 remote [OpenSplice reliable] (2−c) ROS2 intra
(2−b) ROS2 local [OpenSplice reliable] (2−b) ROS2 local [OpenSplice reliable] DDS
6
(3−a) ROS1 to ROS2 remote [Connext best−effort] (3−b) ROS1 to ROS2 local [Connext best−effort] convert: DDS to ROS2
10
1.0
convert: ROS2 to DDS
Latency [ms]
Latency [ms]
(3−b) ROS1 to ROS2 local [Connext best−effort]
others
Latency [ms]
(1−b) ROS1 local
8
4
6
0.5
4
2
2
0.0
0
0
256 512 1K 2K 4K 8K 16K 32K 64K 256 512 1K 2K 4K 8K 16K 32K 64K 16K 32K 64K 128K 256K 512K 1M 2M 4M
Data Size [byte] Data Size [byte] Data Size [byte]
Figure 7: Medians of end-to-end Figure 9: Medians of end-to-end Figure 11: (2-b) reliable policy
latencies with small data in remote latencies with small data in local, breakdown of ROS2 latencies with
and local cases. nodelet, and intra-process cases. the OpenSplice.
100
10
(1−a) ROS1 remote (1−c) ROS1 nodelet
(1−b) ROS1 local (1−b) ROS1 local
12
(2−a) ROS2 remote [OpenSplice reliable] (2−c) ROS2 intra
80
8
(2−b) ROS2 local [OpenSplice reliable] (2−b) ROS2 local [OpenSplice reliable] DDS
(3−a) ROS1 to ROS2 remote [Connext best−effort] (3−b) ROS1 to ROS2 local [Connext best−effort] convert: DDS to ROS2
10
convert: ROS2 to DDS
Latency [ms]
Latency [ms]
(3−b) ROS1 to ROS2 local [Connext best−effort]
others
60
Latency [ms]
(1−b) ROS1 local
8
40
6
4
4
20
2
0
0
64K 128K 256K 512K 1M 64K 128K 256K 512K 1M 2M 4M 16K 32K 64K 128K 256K 512K 1M 2M 4M
Data Size [byte] Data Size [byte] Data Size [byte]
Figure 8: Medians of end-to-end Figure 10: Medians of end-to-end Figure 12: (2-b) best-effort policy
latencies with large data in remote latencies with large data in local, breakdown of ROS2 latencies with
and local cases. nodelet, and intra-process cases. the OpenSplice.
The latenciy characteristics differ in the cases of small and table differences can be observed. However, the influence
large data. For discussion, we divide the graph into Figures 9 appears small in Figure 9 because small data hides the im-
and 10, which show the medians of the end-to-end latencies pact of shared memory.
for local loopback and shared memory transport. This is Another interesting observation is that the latencies in the
because whether a message is divided into several packets (2-c) intra-process are greater than the latencies in (1-b) de-
or not is an import issue to consider end-to-end latencies. spite using shared memory. This result is not due to conver-
For data size less than 64 KB, a constant overhead with sions for DDS and processing of DDS, because intra-process
ROS2 is observed, as shown in Figure 9, because DDS re- communication does not route through DDS. As ROS2 is in
quires marshaling various configurations and decisions for development, that gaps will be closed. Intra-process com-
the QoS Policy. We observe a trade-off between latencies munication needs to be improved.
and the QoS Policy regardless of data size. Although the
QoS Policy produces inevitable overhead, the latencies are 3.3.3 Comparison within ROS2
predictable and small. (3-b) has significant overhead due to End-to-end latencies with data less than 16 KB exhibit
the ros_bridge transaction. In the (3-b) case, a ros_bridge similar performance in (2-b). We discuss performance for
incurs more overhead to communicate with ROS1 and ROS2. data of 16 KB to 4 MB.
With large data, ROS2 has significant overhead depend- A comparison of different DDS implementations in (2-b) is
ing on the size of data, as shown in Figure 10. The over- shown in Figure 13. We evaluate OpenSplice and Connext
head of ROS2 in (2-b) is attributed to two factors, i.e., data with and without shared memory in (2-b) with the best-
conversion for DDS and processing DDS. Note that ROS2 effort policy. Despite shared memory, the performance
in (2-a) and (2-b) must convert messages between ROS2 is not significantly better than that of local loopback. This
and DDS twice. One conversion is from ROS2 to DDS, and is caused by marshaling of various tools (e.g., logger and ob-
the other conversion is from DDS to ROS2. Between these server), even when using shared memory transport. More-
conversions, ROS2 calls DDS APIs and passes messages to over, OpenSplice is superior to Connext in terms of latency,
DDS. Figures 11 and 12 show a breakdown of the end-to- as shown in Figure 13, because we use Connext DDS Profes-
end latencies in the (2-b) OpenSplice reliable policy and sional, which has much richer features than the OpenSplice
best-effort policy. We observe that ROS2 requires only DDS Community Edition. We assume that the performance
conversions and processing of DDS. As shown in Figures 11 of Vortex OpenSplice is similar to that of OpenSplice DDS
and 12, there are nearly no transactions for “others”. In ad- Community Edition. However, Vortex OpenSplice needs a
dition, note that data size influences both conversions and commercial license and is not supported by ROS2.
the DDS processing. Compared to ROS1, the DDS overhead In addition, the influence of the QoS Policy on end-to-end
is not constant, and the impact of DDS is notable with large latencies is evaluated in (2-b) OpenSplice with the reli-
data. As a result, ROS2 has significant overhead with large able policy, best-effort policy, and *-depth policy.
data, while the impact of DDS depends on the QoS Policy. *-depth policy is prepared for this evaluation and con-
Furthermore, the influence of shared memory with large figured by depth as shown in Table 5. Figure 14 shows
data is observed in Figure 10. As data becomes large, no- differences in latencies depending on the reliable policy
12 Connext: local loopback depth = 1
12
best−effort policy
10
Connext: shared memory depth = 10
10
reliable policy
OpenSplice: local loopback depth = 100
10
Latency [ms]
Latency [ms]
8
Latency [ms]
8
6
6
4
4
2
2
2
0
0
16K 32K 64K 128K 256K 512K 1M 2M 4M 16K 32K 64K 128K 256K 512K 1M 2M 4M 16K 32K 64K 128K 256K 512K 1M 2M 4M
Data Size [byte] Data Size [byte] Data Size [byte]
Figure 13: (2-b) Different DDS in Figure 14: (2-b) Two QoS policies Figure 15: (2-b) Configured *-depth
ROS2 with best-effort policy. in ROS2 with OpenSplice. policy in ROS2 with OpenSplice.
80
fragmentsize 64KB subscriber 1 subscriber 1
2.0
subscriber 2 subscriber 2
15
fragmentsize 16KB
fragmentsize 4KB subscriber 3 subscriber 3
60
subscriber 4 subscriber 4
Latency [ms]
Latency [ms]
Latency [ms]
1.5
subscriber 5 subscriber 5
10
40
1.0
5
20
0.5
0.0
0
0
16K 32K 64K 128K 256K 512K 1M 2M 4M 16K 32K 64K 128K 256K 512K 1M 16K 32K 64K 128K 256K 512K 1M
Figure 16: (2-b) Different frag- Figure 17: (1-b) ROS1 multiple Figure 18: (2-b) ROS2 multiple
ment sizes in ROS2 with Open- destinations publisher. destinations with OpenSplice reli-
Splice best-effort policy. able policy.
and best-effort policy. The impact of the QoS Policy as much as subscriber 1. In contrast, ROS2 has small dif-
is shown in Figures 11 and 12. In this evaluation, the net- ferences as shown in Figure 18. All subscribers’ behavior is
work is ideal, i.e., publisher-nodes resend messages very in- fair in ROS2. However, ROS2 latencies significantly depend
frequently. If the network is not ideal, latencies with the on the number of packets. This is same characteristic we
reliable policy increase. The differences in RELIABIL- learned from Figure 10. Figure 19 also indicates fair laten-
ITY and DURABILITY in the QoS Policy lead to overhead cies and dependency of packets. Although we cannot say
at the cost of reliable communication and resilience against that latency variance of ROS1 is larger than one of ROS2
late-joining Subscribers. Figure 15 shows no differences de- due to the difference of the scale, Figures 17, 18, and 19
pending on the depth of *-depth policy. These QoS poli- prove ROS2 message publication is more fair to multiple
cies are different in the number nodes save messages. Al- subscriber-nodes than ROS1 one.
though this number influences resources, this does not affect
latencies because archiving messages is conducted in every
3.4 Throughput of ROS1 and ROS2
publication. We also measure each throughput of ROS1 and ROS2 in
Finally, fragment overhead is measured using OpenSplice the remote case. In our one-way message transport experi-
in (2-b) by changing the fragment size to the maximum UDP ment, maximum bandwidth of the network is 12.5 MB/sec
datagram size of 64 KB. A maximum payload for Connext because we use 100 Mbps Ethernet (100BASE-TX) and Full-
and OpenSplice originates from this UDP datagram size, Duplex as shown in Table 2. Publisher-nodes repeatedly
because dividing large data into several datagrams has sig- transport each message with 10Hz.
nificant impact on many implementations of the QoS Policy. In small data from 256 B to 2 KB, we can observe constant
As shown in Figure 16, the end-to-end latencies are reduced, gaps among ROS1, ROS2 with OpenSplice, and ROS2 with
as fragment data size increases. With a large fragment size, Connext from Figure 20. These additional data correspond
DDS does not need to split large data into many datagrams, with RTPS packets for QoS Policy and heartbeat. Hence,
which means fewer system calls and less overhead. In terms these gaps do not depend on data size. Moreover, Connext
of end-to-end latencies, we should preset the fragment size throughput is lower than OpenSplice one. This becomes a
to 64 KB when using large data. big impact when users handle many kinds of small data with
high Hz and/or network bandwidth is limited.
3.3.4 Multiple Destinations Publisher in local cases In large data from 2 KB to 4MB, curves of Figure 21
In this section, we prepare five subscriber-nodes and mea- demonstrate sustainable theoretical throughput. ROS1 and
sure latencies of each node. Much of information shared ROS2 are able to utilize all of available bandwidth and sim-
in real applications is destinated to multiple destinations. ilarly behave in this situation. Throughput is limited by the
Hence, this evaluation is practical for users. Figure 17 shows network and not by DDS.
latencies of ROS1. We can observe significant differences
between subscriber-nodes. This means ROS1 schedules mes- 3.5 Thread of ROS1 and ROS2
sage publication in order and is not suitable for real-time In this section, we measure the number of threads on each
systems. For example, in 1 MB, subscriber 5 is about twice node. Table 6 shows the result of measurements. Note that
30
1.4
IDEAL
12
subscriber 1 ROS1 MAX
25
Throughput [MB/sec]
Throughput [KB/sec]
1.2
subscriber 2 ROS2 OpenSplice reliable IDEAL
10
subscriber 3 ROS2 OpenSplice best−effort ROS1
1.0
subscriber 4
20
Latency [ms]
8
ROS2 OpenSplice best−effort
0.8
15
ROS2 Connext best−effort
6
0.6
10
4
0.4
2
0.2
0.0
0
16K 32K 64K 128K 256 512 1K 2K 2K 8K 32K 128K 512K 2M
Data Size [byte] Data Size [byte] Data Size [byte]
Figure 19: (2-b) ROS2 multiple Figure 20: (1-a) and (2-b) remote Figure 21: (1-a) and (2-b) remote
destinations with OpenSplice reli- cases throughput with small data. cases throughput with large data.
able policy.
3.7 Lessons Learned
Table 5: Depth Configurable QoS Policies
*-depth policy So far, we have clarified characteristic of DDS implemen-
tations through ROS2 from several standpoints: ROS2 ca-
DEADLINE 100 ms pability, latencies, throughput, the number of threads and
HISTORY LAST memory consumption. We can get insight and guidelines for
depth 1, 10, or 100 DDS through ROS2 from experimental results. They will
RELIABILITY RELIABLE be meaningful for DDS and ROS users.
DURABILITY TRANSIENT_LOCAL DDS supports QoS Policy but there is trade-off between
Table 6: The Number of Thread on ROS1 or ROS2 end-to-end latencies and throughput. In the local case,
ROS1 Connext OpenSplice FastRTPS overhead latencies of ROS2 are not trivial. From Section
node 5 8 49 5 3.3, the latencies are caused by two data conversions for
master-node 3 - - - DDS and DDS transaction. DDS end-to-end latencies are
constant until message data size is lower than maximum
Table 7: Memory of .so Files for ROS1 and ROS packet size (64 KB) as shown in Figure 9. On the other
DDS [KB] Abstraction [KB] Total [MB] hand, as one large message is divided into several packets,
ROS1 2,206 2.26 the latencies sharply increase as show in Figures 10 and
Connext 11,535 9,645 21.18 18. Whether message data size is over 64 KB or not is
ROS2 OpenSplice 3,837 14,117 17.95 important issue especially in DDS because management of
FastRTPS 1,324 3,953 5.28 divided packets with QoS Policy needs significant process-
ing time and alternative APIs provided by some vendors.
the number described in Table 6 depends on DDS configu- We should understand influence of divided packets and keep
rations including QoS Policy. The number does not be fixed in mind this issue when using DDS. While DDS and ROS2
by vendors. abstraction have overhead latencies, OpenSplice utilizes a
First of all, we can observe that a ROS2 node with Open- lot of threads and processes faster than Connext as shown
Splice has a lot of threads. This may cause parallelized in Figure 13. This is a reason why we currently should use
processing and the fact that OpenSplice is much faster than OpenSplice in the underlying implementation of DDS in the
Connext as shown in Figure 13. local case. In the remote case, although overhead latencies
Another interesting point is FastRTPS threads. A ROS2 are trivial, we must consider throughput for bandwidth. As
node with FastRTPS realizes discovery and serialization, and shown in Figure 20, Connext is superior to OpenSplice in
pub/sub data transport with the same number of ROS1 terms of throughput. This constant overhead throughput is
node threads. This result proves improvement of fault tol- predictable and exists no matter how small message data
erance without additional resources because FastRTPS does size is. It influences especially when many kinds of topic
not need a master-node. are used with high Hz. We recommend Connext to consider
3.6 Memory consumption of ROS1 and ROS2 minimum necessary throughput in the remote case.
DDS brings supports of real-time embedded systems to
We also measure memory size of shared library object ROS2. We believe ROS2 outweigh its cost for using DDS.
(.so) in ROS1 and ROS2. Shared libraries are libraries that Fault tolerance of DDS is superior because it is able to save
are dynamically loaded by nodes when they start. They are past data with QoS Policy and does not need a master-node.
not linked to executable files but they will be vital guidelines DDS guarantees fair latencies as shown in Figures 18 and
for estimation of memory size. We arrange the result in Ta- 19. In addition, DDS is able to run on multiple platforms
ble 7. In this table, we add up library data size for pub/sub include RTOS and switch DDS implementation as needed.
transport. In ROS2, shared libraries are classified into the Under RTPS protocol, any ROS2 nodes communicate with
DDS library and the ROS2 abstraction library. While DDS each other without relation to its platform. FastRTPS is
libraries are provided by each vendor, ROS2 libraries ab- currently the best DDS implementation for embedded sys-
stract DDS APIs and convert messages for DDS. In Table 7, tems in thread and memory as Table 6 indicates, but it is
DDS and ROS2 libraries vary depending on vendors. These not suitable for small embedded system.
library data size tends to increase because its QoS capabil- Since ROS2 is under development, we have clarified room
ity and abstraction. For small embedded systems, we need for improvement of ROS2 performance and capability to
a minimal DDS implementation and light abstraction layer.
Table 8: Comparison of ROS2 to Related Work
Small Real-Time Publish/ Frequent Open Library RTOS Mac/ QoS
Embedded Subscribe Update Source and Tools Windows
RTM [8]
Extended RTC [10]
RT-Middleware for VxWorks [17]
RTM-TECS [16]
rosc [13]
μROS [19]
ROS Industrial [4]
RT-ROS [36]
ROS1 [28]
ROS2 [23]
maximize DDS potential. First, current QoS Policies sup- tems and any OS. rosc was motivated by a bare-metal, low-
ported by ROS2 provide fault tolerance but they are insuf- memory reference scenario, which ROS2 also targets. While
ficient for real-time processing. ROS2 has to expand the rosc is available as an alpha release, it is in development and
scope of supported QoS Policies. Second, for small embed- has not been updated since 2014.
ded system, ROS2 needs a minimum DDS implementation μROS: μROS [19] is a lightweight ROS client that can
and minimum abstraction layer. For example, we need C run on modern 32-bit micro-controllers. Targeting embed-
API library for ROS2 and a small DDS implementation. ded systems, it is implemented in ANSI C and runs on an
ROS2 easily supports them because of its abstraction layer. RTOS, such as ChibiOS. μROS supports some of the fea-
FreeRTPS [22] [27] is a good candidate for this issue but it tures of ROS and can coexist with ROS1. However, as of
is under development. Third, we also clarify a need of al- 2013, development has ceased.
ternative API for large message to manage divided packets. ROS Industrial: ROS-Industrial [4] is an open-source
This is critical to handle large message. Abstraction of this project that extends the advanced capabilities of ROS soft-
will shorten DDS end-to-end latencies and fulfill deficiency ware to manufacturing. This library provides industrial de-
of Table 4. Finally, we must tune DDS configurations for velopers with the capabilities of ROS for economical robotics
ROS2 because there are numerous vendor specific configu- research under the business-friendly BSD and Apache 2.0 li-
rations options. censes.
RT-ROS: RT-ROS [36] provides an integrated real-time/non-
4. RELATED WORK real-time task execution environment. It is constructed us-
In addition to the ROS, the Robot Technology Middle- ing Linux and the Nuttx Kernel. Using the ROS in an
ware (RTM) [8] is well known and widely used for robotics RTOS, applications can benefit from some features of the
development. In this section, we discuss research related to RTOS; however, this does not mean that the ROS provides
the ROS and RTM. options for real-time constrains. To use RT-ROS, it is neces-
RTM: RTM applications consist of Robotic Technology sary to modify legacy ROS libraries and nodes. In addition,
Component (RTC), whose specifications are managed by the RT-ROS is not open-source software; therefore, it is devel-
OMG [1]. RTM cannot handle hard real-time and embedded oped more slowly than open-source software.
systems because it generally uses not real-time CORBA [31] Table 8 briefly summarizes the characteristics of several
but CORBA [35]. CORBA is an architecture for distributed related methods and compares them to ROS2. ROS1 has
object computing standardized by the OMG. CORBA man- more libraries and tools for robotics development than RTM.
ages packets in a FIFO manager and requires significant re- At present, ROS2 has only a few libraries and packages be-
sources. CORBA lacks key quality of service features and cause it is currently in development. However, by using
performance optimizations for real-time constraints. multiple DDS implementations, ROS2 can run on embed-
Extended RTC: [10] extends RTC for real-time require- ded systems. In addition, by utilizing the capabilities of
ments using GIOP packets rather than CORBA packets. DDS and RTOSs, ROS2 is designed to overcome real-time
The interface of the Extended RTC provides additional op- constraints and has been developed to be cross-platform.
tions such as priority management and multiple periodic ROS2 inherits and improves the capabilities of ROS1.
tasks. However, it is difficult to implement Extended RTC in
embedded systems because it is based on only an advanced 5. CONCLUSION
real-time Linux kernel. This paper has conducted proof of concept for DDS ap-
RT-Middleware for VxWorks: Using lightweight CORBA proach to ROS and arranged DDS characteristic, guidelines,
and libraries, [17] enables RTM to run on VxWorks, which and room for improvement. From various experiments, we
is an RTOS, and embedded systems. Nonetheless, [17] did have clarified the capabilities of the currently available ROS2
not consider real-time requirements. Furthermore, it uses and evaluated the performance characteristics of ROS1 and
global variables and cannot run on distributed systems. DDS through ROS2 in various situations from several as-
RTM-TECS: RTM-TOPPERS Embedded Component pects: latencies, throughput, the number of threads, and
Systems (TECS) [16] proposes a collaboration framework of memory consumption. Furthermore, we have measured the
two component technologies, i.e., RTM and TECS. TECS [9] influence of switching DDS implementations and the QoS
has been added to RTM to satisfy real-time processing re- Policies in ROS2. Understanding each DDS characteristic,
quirements. [16] adapted RPC and one-way data transport we should use a different DDS implementation for different
between TECS components and RTC. RTM-TECS enhances situations. DDS gives ROS2 fault tolerance and flexibility
the capability for real-time embedded systems. for various platforms. Utilization of DDS is not limited in
rosc: rosc [13] is a portable and dependency-free ROS ROS because ROS2 is one of systems using DDS. Above
client library in pure C that supports small embedded sys- contributions are valuable for many people.
In future work, we will evaluate real-time applications Conference, 2013.
such as an autonomous driving vehicle [18] as case studies us- [20] Object Management Group (OMG). the Real-time
ing ROS2. Moreover, we have to breakdown DDS processing Publish-Subscribe Protocol (RTPS) DDS
time and execute ROS2 on RTOS. We also are interested in Interoperability Wire Protocol v2.2.
ROS2 behavior on embedded devices. Since ROS2 is under http://www.omg.org/spec/DDSI-RTPS/2.2/, 2014.
development, we must maximize DDS potential by tuning [21] Object Management Group (OMG). Data
and abstracting more QoS Policies for real-time processing Distribution Services (DDS) v1.4.
and DDS configurations. http://www.omg.org/spec/DDS/1.4/, 2015.
[22] Open Source Robotics Foundation (OSRF).
6. REFERENCES FreeRTPS. https://github.com/ros2/freertps.
[1] Object Management Group (OMG). [23] Open Source Robotics Foundation (OSRF). ROS2.
http://www.omg.org/. https://github.com/ros2.
[2] Open Source Robotics Foundation (OSRF). [24] G. Pardo-Castellote. OMG Data-Distribution Service:
http://www.osrfoundation.org/. Architectural Overview. In Proc. of IEEE
[3] Point Cloud Library (PCL). http://pointclouds.org/. International Conference on Distributed Computing
[4] ROS Industrial. http://rosindustrial.org/. Systems Workshops, pages 200–206, 2003.
[5] Source code using ROS1 evaluations. [25] PRISMTECH. OpenSplice DDS Community Edition.
https://github.com/m-yuya/ros1 evaluation. http://www.prismtech.com/dds-community.
[6] Source code using ROS2 evaluations. [26] PRISMTECH. Vortex OpenSplice.
https://github.com/m-yuya/ros2 evaluation. http://www.prismtech.com/vortex/vortex-opensplice.
[7] Willow Garage. https://www.willowgarage.com/. [27] M. Quigley. ROS2 on small embedded systems. In
ROS Developers Conference, 2015.
[8] N. Ando, T. Suehiro, K. Kitagaki, T. Kotoku, and
W.-K. Yoon. RT-middleware: distributed component [28] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote,
middleware for RT (robot technology). In Proc. of J. Leibs, R. Wheeler, and A. Y. Ng. ROS: an
IEEE/RSJ International Conference on Intelligent open-source Robot Operating System. In Proc. of
Robots and Systems, pages 3933–3938, 2005. IEEE International Conference on Robotics and
Automation Workshop on Open Source Software,
[9] T. Azumi, Y. Nagahara, H. Oyama, and N. Nishio.
volume 3, page 5, 2009.
mruby on TECS: Component-Based Framework for
Running Script Program. In Proc. of IEEE [29] Real-Time Innovations. RTI Connext DDS
International Symposium on Real-Time Distributed Professional.
Computing, pages 252–259, 2015. http://www.rti.com/products/dds/index.html.
[10] H. Chishiro, Y. Fujita, A. Takeda, Y. Kojima, [30] J. M. Schlesselman, G. Pardo-Castellote, and
K. Funaoka, S. Kato, and N. Yamasaki. Extended B. Farabaugh. OMG data-distribution service (DDS):
RT-component framework for RT-middleware. In Aarchitectural Update. In Proc. of IEEE Military
Proc. of IEEE International Symposium on Communications Conference, volume 2, pages
Object/Component/Service-Oriented Real-Time 961–967, 2004.
Distributed Computing, pages 161–168, 2009. [31] D. C. Schmidt, D. L. Levine, and S. Mungee. The
[11] S. Cousins. Exponential growth of ROS [ROS Topics]. design of the TAO real-time object request broker.
IEEE Robotics & Automation Magazine, 1(18):19–20, Computer Communications, 21(4):294–324, 1998.
2011. [32] S. Sierla, J. Peltola, and K. Koskinen. Evaluation of a
[12] S. Cousins, B. Gerkey, K. Conley, and W. Garage. real-time distribution service. In Proc. of the 3rd
Sharing software with ROS [ROS topics]. IEEE International Symposium on Open Control System,
Robotics & Automation Magazine, 17(2):12–14, 2010. 2003.
[13] N. Ensslen. Introducing rosc. In ROS Developers [33] D. Thomas, E. Fernandez, and W. Woodall. State of
Conference, 2013. ROS2: Demos and the technology behind. In ROS
Developers Conference, 2015.
[14] eProsima. FastRTPS.
http://www.eprosima.com/index.php/products- [34] C. Urmson, J. Anhalt, D. Bagnell, C. Baker,
all/eprosima-fast-rtps. R. Bittner, M. Clark, J. Dolan, D. Duggins,
T. Galatali, C. Geyer, et al. Autonomous driving in
[15] A. Garg. Real-time Linux kernel scheduler. Linux
urban environments: Boss and the Urban Challenge.
Journal, 2009(184):2, 2009.
Journal of Field Robotics, 25(8):425–466, 2008.
[16] R. Hasegawa, H. Oyama, and T. Azumi. RTM-TECS:
[35] S. Vinoski. CORBA: integrating diverse applications
Collabolation Framework for Robot Technology
within distributed heterogeneous environments. IEEE
Middleware and Embedded Component System. In
Communications Magazine, 35(2):46–55, 1997.
Proc. of IEEE International Symposium on Real-Time
Computing, 2016. [36] H. Wei, Z. Shao, Z. Huang, R. Chen, Y. Guan, J. Tan,
and Z. Shao. RT-ROS: A real-time ROS architecture
[17] A. Ikezoe, H. Nakamoto, and M. Nagase. OpenRT
on multi-core processors. Future Generation Computer
Platform/RT-Middleware for VxWorks.
Systems, 56:171–178, 2015.
ROBOMECH2010, pages 2A1–F19, 2010.
[37] M. Xiong, J. Parsons, J. Edmondson, H. Nguyen, and
[18] S. Kato, E. Takeuchi, Y. Ishiguro, Y. Ninomiya,
D. C. Schmidt. Evaluating the performance of
K. Takeda, and T. Hamada. An Open Approach to
publish/subscribe platforms for information
Autonomous Vehicles. IEEE Micro, 35(6):60–68, 2015.
management in distributed real-time and embedded
[19] M. Migliavacca and A. Zoppi. μROSnode: running
systems. omgwiki. org/dds, 2009.
ROS on microcontrollers. In ROS Developers