SDN in Wide-Area Networks: A Survey

Oliver Michel, Eric Keller

Abstract—Over the past several years, Software Defined Net- API provides an abstraction of the data plane state, as well as
working (SDN) has emerged as a new and promising paradigm possible operations on this state (independently from the exact
for the management of computer networks. While we have seen type of network device or its manufacturer).
many use-cases and deployments of SDN in data center networks,
wide-area networks still heavily rely on legacy routing and traffic This separation between the control and data plane allows
engineering technologies. Rapidly increasing traffic demands for data plane devices to be designed in a generic and
(mainly due to increasing usage of video streaming and voice over simplified way. In other words, network devices can become
LTE deployments), however, motivate the development of novel simple, unintelligent boxes that have the sole responsibility
routing and more efficient traffic engineering mechanisms. New
approaches leveraging an SDN paradigm in wide-area networks
of handling packets based on rules as dictated by the control
promise to mitigate many of today’s limitations, inefficiencies, plane. SDN-enabled forwarding elements commonly expose
and scalability issues. In this paper, we give an overview of the a match-action abstraction. This means that network packets
current state of the art in Software Defined wide-area networking passing through such a device are assigned to an action, such
research and technologies and give directions and discuss ideas as forward or drop, by matching a subset of the packet header
for future work.
to a header space defined in a rule.
I. I NTRODUCTION While most research in SDN applications has been focused
on data center settings, recently there has been an increased in-
Software Defined Networking (SDN) continues to attract terest in SDN for wide-area network (WAN) scenarios. WANs
both industry and research communities as a modern paradigm are typically expensive and difficult to manage. They consist of
for the management and operation of computer networks. specialized and expensive high-performance routers increasing
Large-scale computer networks typically consist of a variety capital expenditure (CAPEX), as well as complex configura-
of devices that perform tasks ranging from switching and tion requiring extensive management efforts increasing OPEX
routing to advanced network functions, such as packet filtering and failure probability. Additionally, WANs almost always
or Network Address Translation (NAT). These boxes come rely on a failure-prone underlying (often optical) transport
from different vendors, are incrementally deployed, have dis- network. In order to mask such router or link failure and
tinctive management interfaces, and are usually configured on deal with demand spikes, WANs are often over-provisioned
a per-device basis. Operating a network in this manner hinders [9]. Centralized control and load planning through SDN can
innovation and does not only make the operation unnecessarily help distributing load more wisely, adapting to varying demand
complicated, but also comes with high operational expenses more dynamically, and coping with failure more quickly. This
(OPEX) [3], [4]. can help reduce the degree of required over-provisioning and
While there are centralized management systems that can overall complexity dramatically.
configure systems of different types, these solutions are often
In this paper, we provide an overview of the state of
vendor-specific. Software Defined Networking promises to
research in the field of Software Defined Networking for wide-
solve many of these management headaches. It is vastly
area networks. We first give an introduction to the evolution
changing how computer networks are managed by providing
of programmable networks and SDN, then present current
a centralized vantage point and management interface to the
research in the field of Software Defined WANs (SD-WAN),
entire network. In this way, network operators can configure
before concluding with directions for future research in this
the network from a central location using a standardized
area.
application programming interface (API) across devices and
vendors.
This is mainly enabled by SDN’s two main defining char- II. H ISTORY OF S OFTWARE D EFINED N ETWORKING
acteristics. First, SDN separates the network’s control plane
(which decides where and how a packet is forwarded) from Underlying concepts of Software Defined Networking and
the data plane (which then handles the packet at the device programmable networks date back over 20 years. This is long
according to rules defined in the control plane). Furthermore, before the term SDN was coined and became widespread. In
SDN consolidates the control planes of different devices particular, the programmability aspect of networks was already
providing a unified, centralized view of the network. a hot research topic in the mid-1990s. We now give a brief
In this architecture, the centralized controller runs programs overview of the three main steps that eventually led to the
that control data plane elements using a well-defined API. This SDN technology and architecture that we have today.
A. Active Networks iBGP sessions together with the vastly improved scalability of
With the rapid growth of the Internet beginning in the last a route reflector setup.
decade of the 20th century, there was a need for new, advanced Manageability. With higher demands in network pre-
protocols as the primary use case of the Internet moved away dictability, reliability, and performance, better management
from email and file transfer for the academic community abstractions and more fine-grained control over the actual path
to more generic and consumer-oriented use cases. This was a packet takes through the network (i.e., traffic engineering)
mainly driven by the rise of the World Wide Web; however, the became a desired functionality. As computer networks are
slow standardization process through the Internet Engineering large, complex, distributed systems with countless dependen-
Task Force (IETF) was a source of frustration for researchers. cies between modules and configurations, small, local events
As a result, researchers believed a programmable network and mistakes can have significant, cascading impact across
would accelerate innovation like it has been the case with high- the entire network resulting in service deterioration, security
level, general-purpose programming languages for computers. vulnerabilities, or even complete outages [3], [4]. In order to
In this context, the Active Network architecture [17] was the mitigate these issues, in [4], Greenberg proposes a complete
first effort toward a fully programmable network. The idea refactoring of network functionality based on configuration of
was to inject customized programs into network nodes that network-level objectives using a network-wide view from a
are executed on the respective node when a packet traversed central vantage point with direct control over network devices
it. These programs were either installed on the network device resulting in a separation between the control and data planes.
and invoked by the presence and setting of a certain header The main argument for this approach is that networking
field or integrated into the packet itself. This allowed third logic should be separated from distributed systems issues
parties to develop and deploy custom functionality within that commonly lead to failures. Casado et al. [3] further
the network enabling faster network innovation. While Active elaborates on this concept and presents a deployment of such
Networks certainly enabled network programmability, the ar- an architecture with Ethane, the first instantiation of what
chitecture never became widely deployed. We believe that the eventually became SDN. Ethane is a management framework
Active Networking approach was too radical at the time, as it for enterprise networks using a single network-wide, fine-
broke many conventional primitives widely used in networked grained security policy that is centrally configured and then
systems (such as the concept of a protocol stack). Still, Active enforced throughout the network.
Networking was a very important step towards generalized
network programmability. Later trends and technologies that C. Control Protocols
eventually formed SDN, resemble many of the ideas of Active
Networks in a less radical fashion. With the emergence of various architectures separating the
network control and data planes, the need for more generalized
B. Control- and Data Plane Separation control protocols became apparent. In order to enable real-
With the Internet rapidly growing even further in the begin- world deployment, a control protocol for hardware vendors
ning of the 2000s, Internet Service Providers (ISPs) and other to support was required. OpenFlow [15] was the first (and
network operators were looking for new ways to manage their is still the most prominent) effort to balance the idea of a
in size and complexity increasing networks. In particular, two programmable network and some pragmatism (i.e., standard-
issues needed to be addressed: ization) that would enable deployment on actual networking
Scalability. The Internet’s inter-AS routing protocol is the devices. OpenFlow brought two major intellectual insights to
Border Gateway Protocol (BGP). While AS Border Routers the world of Software Defined Networking. First, it aimed at
(ASBR) maintain a TCP session with an adjacent ASBR inside generalizing network devices and functions by a simple packet
the other AS the network is peering with, all routers inside an based match-action pattern. Secondly, OpenFlow introduced
AS must be connected in a full mesh using the AS-internal the notion of a Network Operating System. A software that
version of BGP (iBGP) to propagate routes learned at the exposes a simplified API and programming abstractions (as
border routers into the network. Clearly, the required full mesh compared to the plain OpenFlow wire protocol) to applications
leads to an undesirable quadratic scaling behavior of iBGP that desire to control the network. Such applications can now
sessions as a function of the number of routers within the be written in any language that the network operating system
AS. Route Reflectors mitigate this problem up to some degree supports or use some sort of standardized API that the network
but are prone to route oscillations and forwarding loops. This operating system exposes.
problem led to the first systems advocating a full separation As mentioned earlier, OpenFlow is not the only protocol
of the control and data planes; the Routing Control Platform that can be used for communication between the control
(RCP) [2] is such a system. RCP is a centralized platform and data planes. For example, in carrier networks, a Path
mimicking a full mesh of iBGP sessions using a central Computation Element (PCE) architecture is often used to
peering platform. It performs route selection on behalf of obtain a global view at the network topology and to centrally
routers and communicates a single best (selected) route using control traffic engineering. The Path Computation Element
standard iBGP sessions to the routers. The main advantage is Protocol (PCEP) is then used in a similar fashion to OpenFlow
that RCP provides the intrinsic correctness of a full mesh of to install forwarding state on routers. Other examples for such
communication protocols are NETCONF, RESTCONF, and must be handled gracefully without disrupting network op-
the Open vSwitch Database Protocol (OVSDB). eration or causing state corruption. ONOS (Open Network
Operating System) is a controller architecture complying with
III. C HALLENGES IN W IDE -A REA SDN these requirements. It offers two main properties: (1) a global
network view and abstraction sharing network state among all
Wide-area networks pose special challenges to designers of
ONOS instances, (2) extensive scale-out capabilities for both
SDN systems. This is mainly due to the reason that link failure
performance and fault-tolerance [1]. In a network controlled
and disrupted connectivity between the control and data planes
by ONOS, each switch has a primary controller that programs
in a WAN are somewhat more common than in a data center
its forwarding plane; however, the switches’ state is shared
setting. Data centers often have dedicated control networks,
among all controller instances and persisted in the distributed,
encompass a high degree of parallel links, are scarcely exposed
eventually consistent Apache Cassandra key-value store mak-
to physical threats (such as cable cuts), and are deployed
ing ONOS’s network view eventually consistent as well. In
in a safe and controlled environment. Thus, imposing strict
the case of a switch or link failure, a consensus protocol
failure resiliency requirements on SD-WANs (as opposed to
provided by Apache Zookeeper automatically selects a new
data center networks) makes the use of a distributed control
primary controller.
plane almost inevitable. Furthermore, the varying propagation
delays through a WAN are typically in great excess of those in B. Placing Controller Instances
a data center. This is prolonging the time to reach a consistent
As alluded in the previous section, placing multiple con-
updated network configuration.
trollers in a WAN deployment, can greatly benefit control
We now highlight three main challenges we identified that
plane latency and fault tolerance. Still, the question is how
need to be tackled when designing a SD-WAN system. Failure
many controllers to deploy and where exactly to place them.
resiliency and scale-out behavior can be achieved with a
Heller et al. [7] provide a first definition the controller place-
decentralized controller architecture, however the question
ment problem and quantify the impact (particularly in terms of
remains how to keep the control plane logically centralized.
latency) of different placement options and strategies on real
That is how to maintain SDN’s global network view requiring
WAN topologies. The authors reduce the placement problem to
consistent, centralized state among all controller instances?
three underlying, fundamental problems based on the desired
Furthermore, when using multiple controllers, where should
metric: (1) when optimizing for average-case latency, the
these controllers be placed in regard to the different data
problem reduces to the minimum k-median problem, (2) when
plane elements and how many controllers are necessary at
optimizing for worst-case latency, it reduces to the minimum
all? Finally, in the presence of high latencies between the
k-center problem, (3) when nodes must not exceed a specific
control and data planes, how can the data plane configuration
latency bound to the next controller, the problem can be stated
be updated such that every packet traversing the network
as the maximum cover problem. While all of these problems
is handled based on a single, consistent policy across all
are computationally intractable (N P-hard), the authors show
forwarding devices?
that for medium-sized networks, approximations can vastly
improve latency metrics over random placement (1.4 to 1.7
A. Distributing SDN Controller State
times higher latency). Surprisingly, the paper also shows that in
The separated control and data planes together with a the majority of cases for the (real) wide-area networks studied,
control program and a network operating system define the a single controller is sufficient to provide protection and
overall architecture that most SDN deployments today follow. restoration guarantees comparable to optical 1 : 1 protection.
However, in this simplified picture, the centralized controller
does not only often come with poor scaling behavior as it C. Updating SDN Switches in a Consistent Manner
represents a single choke point, it also is a single point of While Software Defined Networks enable dynamic and cus-
failure. Control platforms with undesirable properties com- tomized data plane configurations, updating the data plane in a
ing with a centralized architecture include NOX, Floodlight, large network (i.e., the packet handling rules on the switches)
and Ryu. In order to overcome these two issues, distributed can cause severe inconsistencies in how packets are handled
controllers have been proposed [1], [12]. These controllers among devices. This is mainly due to the infeasibility to update
provide scale-out performance and fault tolerance. While in the entire network atomically while maintaining full network
a data center setting, an SDN controller mainly has high- operation. As a result, forwarding devices are typically updated
throughput requirements (i.e., operations per second), we argue in steps. Even if the initial and final configuration states are
that distribution capabilities are more important for WAN SDN correct, network devices may be in inconsistent states during
controllers. the update process. This can lead to major instabilities such as
Nevertheless, distributing controller instances across the interrupted connectivity, forwarding loops, or access control
network naturally also requires distributing the controller state violations. Moreover, intermediate update states can violate
(i.e., configured match/action rules, topology information, and bandwidth constraints on a given path potentially causing
statistics) posing the challenge that this state must be kept major congestion. This happens when a link is oversubscribed
consistent among instances. Furthermore, controller failure between the initial and final update steps. Several solutions
have been proposed to cope with such inconsistencies. Most The first system, Software-driven WAN (SWAN) [8], from
of them, however, rely on planning an exact update order Microsoft, is a centralized traffic engineering (TE) solution
statically before actually rolling out updates to the switches that uses fine-grained policy rules and a centralized scheduler
in this pre-determined order [8], [11], [14]. These solutions to carry significantly more high-priority traffic (e.g., traffic
do not adapt to runtime variabilities within the control plane for interactive applications) while maintaining fairness among
(such as varying CPU load or varying RPC delays), which can services of the same class. SWAN addresses shortcomings of
still cause inconsistencies while rolling out an update. today’s primarily used decentralized TE practice (i.e., MPLS-
Reitblatt et. al. [16] address these challenges by providing TE in conjunction with ECMP routing) and makes a strong
a theoretical foundation and proposing a novel abstraction for point for using a global network view and SDN in order to
network updates that guarantees consistent handling on a per- solve the global and network-wide problem of TE. SDN’s
packet basis. The authors introduce a per-packet consistency global network view is used here to find globally optimized
model where each packet flowing through a network is guar- bandwidth to path assignments through the network. Fine-
anteed to be processed according to a single configuration. grained control in SDN is used to make and enforce bandwidth
Additionally, a per-flow consistency abstraction model, where reservations on a meticulous, per-application basis. By doing
each packet belonging to a flow (e.g., a TCP connection) will so, the need for over-subscription is drastically reduced, and
be processed consistently across the network, is presented. resources are used more effectively. As a result, SWAN
Jin et. al. [10] propose a runtime scheduling system that can carry about 98% of the theoretically possible network
uses heuristics to pick a path through a dependency graph traffic while MPLS-based technologies carry only around 60%.
containing the required update steps. Using this approach, a Furthermore, the authors identify the need for frequent data
scheduler can select correct paths through this graph and the plane updates in order to maintain utilization at a high level
ordering of updates on the fly based on network behavior. and address this problem by presenting a consistent update
The system predetermines valid orderings of updates and then algorithm.
heuristically applies them based on realtime behavior of the The second deployment, B4, is Google’s globally-deployed
switches and network. Dionysus takes a two-step approach software defined inter-DC WAN solution [9]. The system
where first a dependency graph of the steps to reach a con- exploits three unique properties of their network to deploy
sistent final state is computed. Then, at runtime, a scheduler a centralized TE solution achieving high link utilization and
selects a path through this graph that becomes the order of up- a two times to three times efficiency improvement relative to
dates maintaining consistency and correctness in the network. standard (decentralized MPLS-based) practice. First, Google
Dionysus significantly reduces overall update times by 53% to controls all IP-layer network equipment all the way to the
88% depending on the type of update and topology compared edge of the network. Second, most bandwidth is used for
to other solutions. Dionysus also reduces link oversubscription low-priority, planned, large replication tasks between sites.
and thus congestion during updates significantly (41% less Finally, the network only consists of a few dozen sites making
than comparable approaches like [8]). a centralized control-plane approach feasible. B4 employs
custom switches controlled by a slightly modified version
IV. B ENEFITS OF W IDE -A REA SDN of OpenFlow. The network controller dynamically reallocates
Before continuing our discussion, we present two SD- bandwidth for shifting application demands and also provides
WAN deployments from major cloud providers to illustrate dynamic rerouting in the case of link or switch failures. The
the benefits that this technology offers for network efficiency SDN-based control plane also integrates with standard legacy
and resiliency. Cloud providers like Apple, Microsoft, and routing protocols that make it possible to completely disable
Google run their services from data centers distributed around centralized TE while keeping the network fully operational.
the planet. While those data centers primarily serve content Many B4 links run at 100% utilization and all links average
to customers, there is an increasing demand for inter-data at 70% utilization over extended time periods.
center traffic exchange. This is mainly due to the specialized While, both B4 and SWAN use SDN in the context of
and distributed character of services and subsystems requiring inter-DC wide-area networks and present similar high-level
communication among each other, as well as extensive backup, architectures focused on traffic engineering for optimized
synchronization and migration tasks across physical locations. resource usage of expensive wide-area links, their secondary
Thus, inter-data center (wide-area) networks are a critical focuses are vastly different. Hong et al. with SWAN make
infrastructure for such providers and are commonly over- significant contributions in the field of update consistency
provisioned and consequently underutilized by around 30% to (cp. section III-C) and leverage frequent data plane updates
60% in order to cope with failure [8]. The global network together with correctness guarantees to maintain high uti-
view and fine-grained control over routing decisions that lization over extended periods of time. The authors of B4
SDN promises can be leveraged to achieve centralized traffic put an emphasis on integrating a (TE-centered) SDN control
engineering optimizing for more efficient use of the expensive plane with existing legacy routing protocols (in this case BGP
network resources. Microsoft and Google both employ SDN and IS-IS) as a fallback mechanism. The Routing Application
technologies for their inter-data center networks and have Proxy (RAP) of B4 interfaces with standard Quagga software
published technical details of their systems. switches and receives BGP and IS-IS control messages directly
from the SDN-enabled switches through the software slow- (ASes) over a common layer 2 fabric to which all networks are
path. connected. IXPs present a natural starting point for wide-area,
inter-domain SDN deployments because SDN deployment at
V. E XPANDING B EYOND A S INGLE D OMAIN a single IXP immediately affects a large number of providers.
Intra-domain SDN solutions mainly tackle problems around SDX (Software Defined Internet Exchange Point) [6] is an
shortcomings of legacy interior gateway routing protocols or IXP that consists of an SDN controller instead of a route server
traffic engineering practices (such as MPLS-TE) to make better and an SDN-enabled switch instead of the plain L2 fabric.
use of expensive wide-area links (as explained in section IV). Each AS connected to this SDN switch can now write its
The problems addressed with inter-domain deployments are custom policies that are being enforced on its private virtual
somewhat different. Most of these problems stem from the de- switch. The SDX controller composes these rules (which are
facto standard inter-domain routing protocol BGP, in particular expressed in the Pyretic programming language) behind the
from its exterior version eBGP. BGP, as the standard protocol scenes and installs the actual forwarding rules on the SDN-
for almost all inter-domain peering in the Internet, keeps the enabled IXP switch. Furthermore, SDX integrates with legacy
Internet running despite extremely complex operational and BGP routing through its integrated route server. This also
financial interests from the various network operators. This means that not every ISP at an SDX must use the SDN
means that BGP does not only come with technical challenges capabilities of the SDX system but can also exchange its
such as intra-domain scalability issues [2], slow convergence routes using standard BGP, like in a traditional IXP. The SDX
times, route flap damping, or route oscillation; it also imposes controller runs integrity checks on all requested policies to
operational challenges for network operators because they make sure that they do not create conflicts or forward traffic
have to use BGP’s embedded mechanisms like local preference to prefixes that have not been advertised by the receiving
attributes, Multi-Exit Discriminators (MED), or AS-PATH AS. SDX however, explicitly enables overwriting BGP default
prepending to implement complex policies representing their routes for use cases like application-specific peering, which
business objectives. Those instruments need to be manually can be realized by not only using the destination IP prefix for
configured and are error-prone, which commonly leads to route selection, but also other match fields that are available
(sometimes global) routing instability or divergence. in OpenFlow. The policy composition of SDX can generate
Similar to the intra-domain case, SDN can offer new significant numbers of rules to be installed that are well beyond
opportunities to overcome these problems. Again, a global the capabilities of modern OpenFlow-enabled switches (in the
view of the network can be leveraged to make network-wide order of tens of millions). The authors further improved their
decisions over how packets are forwarded. Since autonomous system to an industrial-scale SDX (iSDX) [5] that leverages
systems are operated by different parties with different net- various rule transformations, outsourcing of rules to ASBRs,
work equipment, policies, and business objectives that cannot and recent advances in switch hardware (L2 bit mask match-
be shared with other networks, applying logically centralized ing) to encode reachability information in a packet’s L2 header
control across multiple administrative domains is somewhat further reducing required centralized data plane state.
more complex and poses different challenges compared to the
intra-domain case. VI. R ESEARCH C HALLENGES AND THE F UTURE OF
One of the first papers on this issue was published by SD-WAN
Kotronis et al. [13]. The authors propose a rather drastic After giving an overview of the history of SDN and
approach that is to completely outsource inter-domain routing presenting the state of research in SD-WAN, we identified
logic to a third party (i.e., a trusted contractor). This contractor areas with ongoing challenges in the context of wide-area
can specialize in the practices of Internet routing, freeing networking that we believe can greatly benefit from an SD-
network operators from complex BGP-related administration WAN approach. While most of these challenges have been
tasks to comply with its policies. This approach has three studied in the context of a traditional networking paradigm, we
main advantages: (1) as the contractor performs routing con- believe leveraging the main tenets of SDN can have significant
trol for multiple ASes, it can optimize global routing using impact on the way we handle common issues in wide-area
its birds-eye view, (2) it can detect policy conflicts and networks. We now briefly present some of these potential
help with collaborative troubleshooting between ASes, (3) as research areas:
inter-domain routing would be centrally controlled, existing Distributed Control Planes. While [7] discusses the place-
mechanisms (i.e., eBGP) could be improved among networks ment problem and [1] presents a fully distributed controller
having the same contractor without disrupting connectivity, architecture, it remains unexplored how distributed control
thus accelerating innovation in inter-domain routing. planes behave in real-world wide-area topologies that are
A more recently presented system uses some of these ideas significantly harder to study compared to standard data center
but applies them in a more natural way that keeps network topologies such as CLOS or FatTree.
operators entirely in control of their peering policies. Their Data Plane Load Balancing. Wide-area networks heavily
approach is applied at Internet Exchange Points (IXP). An IXP use parallel links based on constraints in the underlying trans-
is a physical infrastructure where network operators exchange port network (e.g., wavelength availability). SDN’s global view
traffic and BGP route information between their networks of the network and more fine-grained forwarding capabilities
offer the opportunity for better abstractions and load balancing have significant impact on the future operation of wide-area
schemes in comparison to ECMP routing. networks.
Traffic Engineering and Network Monitoring. The SDN As WANs typically consist of specialized, expensive, high-
control plane has a global, centralized view of the network performance hardware and are built incrementally, we, how-
and thus also can access network-wide statistics and proper- ever, project that the adoption of radical new SDN tech-
ties. With centralized traffic engineering solutions, this data nologies and deployments happen at longer time scales than
can be leveraged to find globally optimal path assignments. we have seen it in data center networks, where completely
Traditional traffic engineering mechanisms such as RSVP-TE new deployments are needed more often. Although, through
or LDP for MPLS rely on the local, limited view of the technologies like PCEP and OpenFlow, a lot of equipment is
network from the ingress router. We must find ways to use already SDN-enabled potentially shortening adoption times.
such network-wide properties with application-specific traffic
engineering solutions effectively.
Data Plane Fault Tolerance and Low-Latency Routing.
R EFERENCES
Programmability in SDN offers possibilities to implement
custom, fast, and efficient adaptive routing schemes that op- [1] P. Berde, M. Gerola, J. Hart, Y. Higuchi, M. Kobayashi, T. Koide,
B. Lantz, B. O’Connor, P. Radoslavov, W. Snow, and G. Parulkar.
timize for low end-to-end latency, as well as failure recovery ONOS: Towards an Open, Distributed SDN OS. In Proc. of the Third
mechanisms that can be achieved with low communication Workshop on Hot Topics in Software Defined Networking, 2014.
overhead and little controller interaction. Nevertheless, it re- [2] M. Caesar, D. Caldwell, N. Feamster, J. Rexford, A. Shaikh, and
J. van der Merwe. Design and Implementation of a Routing Control
mains unsolved how to pick better paths in real time in the Platform. In Proc. of the 2nd Conference on Symposium on Networked
presence of link failure or congestion, which are scenarios that Systems Design & Implementation. USENIX Association, 2005.
are common in wide-area networks. [3] M. Casado, M. Freedman, and J. Pettit. Ethane: Taking control of the
enterprise. ACM SIGCOMM CCR, 37(4), 2007.
Packet-Optical Convergence. Multilayer coordination be- [4] A. Greenberg, G. Hjalmtysson, D. A. Maltz, A. Myers, J. Rexford,
tween the transport and IP/MPLS packet layers of a WAN G. Xie, H. Yan, J. Zhan, and H. Zhang. A clean slate 4D approach
is a promising approach towards more optimized and easier to network control and management. ACM SIGCOMM CCR, 35, 2005.
[5] A. Gupta, R. MacDavid, R. Birkner, M. Canini, N. Feamster, J. Rex-
to manage carrier networks. Programmability in software ford, and L. Vanbever. An Industrial-Scale Software Defined Internet
defined networks lowers the burden for operators to integrate Exchange Point. In 13th USENIX Symposium on Networked Systems
the network layers. Reachability, performance, and protection Design and Implementation, mar 2016.
[6] A. Gupta, L. Vanbever, M. Shahbaz, S. P. Donovan, B. Schlinker,
properties from both planes can then be used to optimize traffic N. Feamster, J. Rexford, S. Shenker, R. Clark, and E. Katz-Bassett.
engineering and fault tolerance. However, integrating these two SDX: A Software Defined Internet Exchange. In Proc. of the ACM
vastly different systems with their custom control software and SIGCOMM 2014 Conference, 2014.
[7] B. Heller, R. Sherwood, and N. McKeown. The Controller Placement
very different experience and expertise of their operators poses Problem. In Proc. of the 1st Workshop on Hot Topics in Software Defined
various challenges. Networks, 2012.
Internet-Scale Attacks. Large-scale, Distributed Denial of [8] C.-Y. Hong, S. Kandula, R. Mahajan, M. Zhang, V. Gill, M. Nanduri,
and R. Wattenhofer. Achieving High Utilization with Software-driven
Service (DDoS), attacks have become widespread in today’s WAN. In Proc. of the ACM SIGCOMM 2013 Conference, 2013.
Internet and are one of the major threats for network and [9] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh,
service providers as they may cause severe outages coming S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla, U. Hölzle, S. Stuart,
and A. Vahdat. B4: Experience with a Globally-deployed Software
with significant business consequences. A global network van- Defined Wan. In Proc. of the ACM SIGCOMM 2013 Conference, 2013.
tage point and centralized data collection (especially in inter- [10] X. Jin, H. H. Liu, R. Gandhi, S. Kandula, R. Mahajan, M. Zhang,
domain scenarios) can be valuable to identify optimal locations J. Rexford, and R. Wattenhofer. Dynamic Scheduling of Network
Updates. In Proc. of the ACM SIGCOMM 2014 Conference, 2014.
to monitor and block large-scale attacks much closer to the [11] N. P. Katta, J. Rexford, and D. Walker. Incremental consistent updates.
attacker (as opposed to the target) than normally possible. In Proc. of the Second ACM SIGCOMM Workshop on Hot Topics in
Software Defined Networking, HotSDN ’13. ACM, 2013.
VII. C ONCLUSION [12] T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu,
R. Ramanathan, Y. Iwata, H. Inoue, T. Hama, and S. Shenker. Onix:
In this paper we gave an overview of the current state of A distributed control platform for large-scale production networks. In
research in the field of Software Defined Wide-Area Network- Proc. of the 9th USENIX Conference on Operating Systems Design and
Implementation. USENIX Association, 2010.
ing along with a discussion of potential future research topics. [13] V. Kotronis, X. Dimitropoulos, and B. Ager. Outsourcing the Routing
We believe, that, especially with growing industry attention Control Logic: Better Internet Routing Based on SDN Principles. In
due to ever-growing traffic demands, SD-WAN will remain an Proc. of the 11th ACM Workshop on Hot Topics in Networks, 2012.
[14] H. H. Liu, X. Wu, M. Zhang, L. Yuan, R. Wattenhofer, and D. Maltz.
area of high interest over the upcoming years. zupdate: Updating data center networks with zero loss. In Proc. of the
While we are still in the beginning phases of research in ACM SIGCOMM 2013 Conference, SIGCOMM ’13. ACM, 2013.
this area, we see considerable space for improvement of wide- [15] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,
J. Rexford, S. Shenker, and J. Turner. OpenFlow: enabling innovation
area network performance, reliability, and scalability through in campus networks. SIGCOMM CCR, 38(2), mar 2008.
Software Defined Networking technologies. In particular, aug- [16] M. Reitblatt, N. Foster, J. Rexford, C. Schlesinger, and D. Walker.
menting visibility in WANs through monitoring and telemetry Abstractions for Network Update. In Proc. of the ACM SIGCOMM
2012 Conference, 2012.
technologies as well as the integration of the control planes [17] D. L. Tennenhouse and D. J. Wetherall. Towards an Active Network
of the packet and optical (transport) layers of WANs may Architecture. ACM SIGCOMM CCR, 37(5), oct 2007.