Significant efforts have been devoted to creating large scale compute and network testbeds for st... more Significant efforts have been devoted to creating large scale compute and network testbeds for studying future Internet challenges. Besides large geographic span, the common emphasis is programmability, allowing researchers to reserve or create, via software, flexible sets of compute and network resources over specified topologies to execute research prototypes of new protocols, processes, and applications. Also emphasized are virtualization, instrumentation, and software defined networking (SDN) capabilities of the infrastructure. SDN in particular stimulated significant interests in academia, industry, and public sectors to re-imagine the future computing and networking infrastructure landscape and roadmap while it becomes increasingly utilized in production environments. Amidst these interests, one can start to capture desirable characteristics to glimpse the potential architecture of the future Internet. In this paper, we discuss the significance of compute-network interaction across complex, highly customized federated architecture in the future Internet. Infrastructure federation has been happening across multiple dimensions. Federation expands the scope of infrastructure, geographically and administratively, for use by members of different organizations. For example, federation initiatives are underway among: 1) US Global Environment for Network Innovations (GENI), Europe Future Internet Research and Experimentation (FIRE), and future Internet testbeds in Asia, South America, and Canada, 2) university production infrastructure, 3) US cities, 4) US public research institutes, and 5) commercial infrastructure. While requirements and objectives differ, they must all address a common set of issues. Such federation suggests the fundamental needs of applications to interact with compute and network resources across a generic, federated, future Internet environment.
A complete system for distributing quantum entangled signals over fiber based on commercially ava... more A complete system for distributing quantum entangled signals over fiber based on commercially available equipment will be demonstrated. Measurements are collected and controlled from a single location using an embedded optical data link.
The increase in real time ultra-high definition video services presents a challenging issue to cu... more The increase in real time ultra-high definition video services presents a challenging issue to current network infrastructures, because of its high bandwidth usage, which saturate network links. The required bandwidth is related to strict QoS requirements for digital media. There are systems in place currently to help reduce these problems, such as transcoders and application layer multicasting. However, these approaches are limited because they are usually implemented as static resources. In contrast, by using the OpenFlow based system presented in this paper, it is possible to provide a more effective approach using dynamic resources-by both optimally placing transcoders in the network, as well as by migrating them to different locations while the streaming is taking place. This migration mechanism provides a near seamless switchover with minimal interruption to the clients.
As many of you know, one of the outcomes of the Monterey Conference this past fall was the creati... more As many of you know, one of the outcomes of the Monterey Conference this past fall was the creation of an informal group which later called itself the "MFuG" or the Monterey Future's Group. This group has met several times since Monterey and developed a paper which discusses the networking needs and supporting technical requirements of the university community as it is now evolving. Doug Gale and a great many in MFuG contributed to this effort. In addition, there was work by a small group of folks (Stuart Lynn, David Wasley, Russ Hobby, Mike Staman) to write a concept paper describing a National Higher Education Networking Organization. Finally, at the request of the NTTF Steering Committee, Bill Graves wrote a background piece in preparation for the joint NTTF/FARNET meeting on April 15th and 16th.
Communication services, architecture, and technologies are rapidly evolving in response to applic... more Communication services, architecture, and technologies are rapidly evolving in response to application demand and research innovation. These changes are motivating a fundamentally new approach to the design and provisioning of services, facilities, and infrastructure. Traditionally, such resources have been designed and implemented as centralized fixed utility services, with almost no options for specialization and customization, especially by processes at the
2012 SC Companion: High Performance Computing, Networking Storage and Analysis, 2012
In this paper we describe the design, and implementation of the Open Science Data Cloud, or OSDC.... more In this paper we describe the design, and implementation of the Open Science Data Cloud, or OSDC. The goal of the OSDC is to provide petabyte-scale data cloud infrastructure and related services for scientists working with large quantities of data. Currently, the OSDC consists of more than 2000 cores and 2 PB of storage distributed across four data centers connected by 10G networks. We discuss some of the lessons learned during the past three years of operation and describe the software stacks used in the OSDC. We also describe some of the research projects in biology, the earth sciences, and social sciences enabled by the OSDC.
This work presents the implementation of a tabular interpolation approach to estimate empirical S... more This work presents the implementation of a tabular interpolation approach to estimate empirical Shannon entropy on programmable data plane ASICs using P4. The technique transforms the complex computations of the random projection into fast lookup over precomputed tables in the match-action pipeline. Likewise, the interpolation heuristic further reduces the table size substantially. Thus, more tables can be accommodated, achieving higher estimation accuracy. Simulations based on real-world network traffic traces are performed to evaluate the estimation accuracy. The scheme is deployed in a Barefoot Tofino2 switch connected to the International Center for Advanced Internet Research (iCAIR) national testbed. The system can estimate the entropy of network traffic accurately at 400 Gbps throughput.
We describe the GENI Experiment Engine, a Distributed-Platform-as-a-Service facility designed to ... more We describe the GENI Experiment Engine, a Distributed-Platform-as-a-Service facility designed to be implemented on a distributed testbed or infrastructure. The GEE is intended to provide rapid and convenient access to a distributed infrastructure for simple, easy-to-configure experiments and applications. Specifically, the design goal of the GEE is to permit experimenters and application writers to: (a) allocate a GEE slicelet; (b) deploy a simple experiment or application; (c) run the experiment; (d) collect the results; and (e) tear down the experiment, starting from scratch, within five minutes. The GEE consists of a set of cooperating services over the GENI infrastructure, which together with rapidly-allocated slicelets and a rapidly-allocated network offers a complete, ready to use, sliceable platform over the GENI Infrastructure. The GEE is designed to use off-the-shelf components and infrastructure; unlike previous PaaS offerings, it can be nested nicely inside a GENI slice, or any other IaaS infrastructure. Further, the GEE's southbound interface is extremely small and lightweight, making it portable to other underlying infrastructures.
To realize P4 language benefits: “Protocol Independent, Target Independent, Field Reconfigurable”... more To realize P4 language benefits: “Protocol Independent, Target Independent, Field Reconfigurable” [1] in a network research and development environment, a few P4 research institutions around the world formed a partnership to design and implement the international P4 Experimental Networks (Figure 1). The participating institutions can share distributed P4 resources over international research and education networks, as well as initiate international P4 research collaboration projects. Initial network scenarios in this research testbed are Software-Defined Network Exchanges (SDXs), Network and Cloud Testbed Networks, Science Networks, and Campus Research Networks. These four network scenarios require multi-tenant support. Current P4Runtime specifications include a Multi-Controller design for twofold: Control Plane Partition and Redundancy [2]. However, the current P4Runtime does not implement Control Plane partition to support multi-tenant service on a switch. This project modified the current design to support a provider and multiple tenant model for the four international P4 Experimental Networks (i-P4EN)scenarios and other applications.
Proceedings of the 5th International Workshop on P4 in Europe
This work presents the implementation of a tabular interpolation approach to estimate empirical S... more This work presents the implementation of a tabular interpolation approach to estimate empirical Shannon entropy on programmable data plane ASICs using P4. The technique transforms the complex computations of the random projection into fast lookup over precomputed tables in the match-action pipeline. Likewise, the interpolation heuristic further reduces the table size substantially. Thus, more tables can be accommodated, achieving higher estimation accuracy. Simulations based on real-world network traffic traces are performed to evaluate the estimation accuracy. The scheme is deployed in a Barefoot Tofino2 switch connected to the International Center for Advanced Internet Research (iCAIR) national testbed. The system can estimate the entropy of network traffic accurately at 400 Gbps throughput.
2017 20th Conference on Innovations in Clouds, Internet and Networks (ICIN), 2017
This paper describes a Bioinformatics Software Defined Network Exchange (SDX) or BioSDX, which ha... more This paper describes a Bioinformatics Software Defined Network Exchange (SDX) or BioSDX, which has been designed, deployed, and demonstrated by a multi-organizational research consortium to enable bioinformatics knowledge discovery supported by dynamic networking services. This BioSDX uses precision networking to support precision medicine. The BioSDX is based on recent technical developments in infrastructure abstraction that enables new types of tools and services utilizing programmable network infrastructure through high levels of resource virtualization. Combined with close integration of programmable cloud computing facilities, the BioSDX is an important advance in supporting the new paradigm of data intensive bioinformatics across multiple disciplines, including computational genomics and precision medicine.
2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS), 2020
Increasingly, scientific research discovery depends on the analysis of extremely large amounts of... more Increasingly, scientific research discovery depends on the analysis of extremely large amounts of data. This data is gathered through very large scientific instruments, very large numbers of small devices, and mid-scale instruments. These devices generate exponentially increasing volumes of data. Such data is often obtained, analyzed, stored, visualized, and transferred at and among sites worldwide by large collaborations of science communities. Managing and moving extremely large volumes of data across today's networks is a special multidimensional challenge. Also, existing data management and movement services and tools often are inadequate to address all specific requirements. This paper describes the various components of this challenge and how those issues can be addressed by ROBIN, a unique comprehensive set of integrated services designed specifically for managing and moving extremely large amounts of data over long distances, e.g., thousands of miles around the globe. This paper also describes the results of initial experiments using that set of integrated services.
Increasingly, science requires the gathering, transport, analysis and storage of extremely large ... more Increasingly, science requires the gathering, transport, analysis and storage of extremely large volumes of data, a key resource for new knowledge discovery. High energy physics, bioinformatics, genomics, oceanography, atmospheric sciences, weather modeling, astrophysics, geophysical sciences, earth sciences, materials science, chemistry, molecular dynamics, neurosciences, and many other disciplines use petabytes of information for sophisticated analytic processing, models, and simulations. Also, many of these science communities are creating next generation instruments that will generate many additional petabytes of data (exebytes in aggregate), including high luminosity synchrotrons, including the High Luminosity Large Hadron Collider, the Large Synoptic Survey Telescope, the Square Kilometer Array, neutrino detectors, genomic sequencers and others.
In part because of the limitations of the TCP/IP protocols used by the current Internet, the Futu... more In part because of the limitations of the TCP/IP protocols used by the current Internet, the Future Internet has become an attractive network research topic. Recently, several projects focused on the Future Internet have been launched around the globe based on the OpenFlow platform, such as those that are part of the GENI project in the U.S., the OFELIA project of FP7 in the E.U., and the FIRST project in Korea. Because OpenFlow allows researchers to design and develop innovative protocols, it could be applied in different situations for constructing desired network environments. OpenFlow is not limited to single controller environment within a single network domain. It allows network engineers to construct a multi-controller environment across various network domains. In this paper, we design and implement an automatic network topology discovery mechanism based on a multi-controller OpenFlow network. The result has been applied to a large-scale OpenFlow testbed implemented across r...
2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS), 2019
Transferring big data over Wide Area Networks (WANs) is challenging because optimization is depen... more Transferring big data over Wide Area Networks (WANs) is challenging because optimization is dependent on the specifics of multiple parameters. Network services, paths, and technologies have different characteristics, including loss rate, latency, and available capacity. Yet, frameworks currently used to configure and orchestrate transfer systems, measure performance, and analyze results have limited capabilities. We propose a framework, DTN-as-a-Service (DaaS), for high-performance network data transfers using and integration of techniques, including virtualization, network provisioning, and performance data analysis. This framework has a modular design for supporting multiple transfer tools, optimizers and orchestrators for the data transfer environment, including \textit{Docker} and \textit{Kubernetes}. We present a \textit{Jupyter} based workflow for high-speed network data transfer in data-intensive science and evaluate the performance of the transfer with a simple programmable visualizer implemented in the framework. This framework has been implemented as a prototype at two recent SC supercomputing conferences. With the increase in the number and the capacity of WAN links at the conferences (multiple 100 Gbps WAN circuits), the challenges involved in setting up, testing, debugging, verifying and running applications on high-performance systems connecting to the conference SCinet WAN circuits also increase. The SCinet implementation of the DaaS framework for the conference community allowed users to control hardware, software, and network infrastructure for high-speed network data transfer, primarily for large scale applications. Through the evaluation of the framework in our test setup, we demonstrated that NVMe over Fabrics with TCP is twice as efficient compared to using conventional TCP in high-speed NVMe-to-NVMe transfers. We also implemented a 400 Gbps LAN experiment to evaluate the DaaS framework.
Significant efforts have been devoted to creating large scale compute and network testbeds for st... more Significant efforts have been devoted to creating large scale compute and network testbeds for studying future Internet challenges. Besides large geographic span, the common emphasis is programmability, allowing researchers to reserve or create, via software, flexible sets of compute and network resources over specified topologies to execute research prototypes of new protocols, processes, and applications. Also emphasized are virtualization, instrumentation, and software defined networking (SDN) capabilities of the infrastructure. SDN in particular stimulated significant interests in academia, industry, and public sectors to re-imagine the future computing and networking infrastructure landscape and roadmap while it becomes increasingly utilized in production environments. Amidst these interests, one can start to capture desirable characteristics to glimpse the potential architecture of the future Internet. In this paper, we discuss the significance of compute-network interaction across complex, highly customized federated architecture in the future Internet. Infrastructure federation has been happening across multiple dimensions. Federation expands the scope of infrastructure, geographically and administratively, for use by members of different organizations. For example, federation initiatives are underway among: 1) US Global Environment for Network Innovations (GENI), Europe Future Internet Research and Experimentation (FIRE), and future Internet testbeds in Asia, South America, and Canada, 2) university production infrastructure, 3) US cities, 4) US public research institutes, and 5) commercial infrastructure. While requirements and objectives differ, they must all address a common set of issues. Such federation suggests the fundamental needs of applications to interact with compute and network resources across a generic, federated, future Internet environment.
A complete system for distributing quantum entangled signals over fiber based on commercially ava... more A complete system for distributing quantum entangled signals over fiber based on commercially available equipment will be demonstrated. Measurements are collected and controlled from a single location using an embedded optical data link.
The increase in real time ultra-high definition video services presents a challenging issue to cu... more The increase in real time ultra-high definition video services presents a challenging issue to current network infrastructures, because of its high bandwidth usage, which saturate network links. The required bandwidth is related to strict QoS requirements for digital media. There are systems in place currently to help reduce these problems, such as transcoders and application layer multicasting. However, these approaches are limited because they are usually implemented as static resources. In contrast, by using the OpenFlow based system presented in this paper, it is possible to provide a more effective approach using dynamic resources-by both optimally placing transcoders in the network, as well as by migrating them to different locations while the streaming is taking place. This migration mechanism provides a near seamless switchover with minimal interruption to the clients.
As many of you know, one of the outcomes of the Monterey Conference this past fall was the creati... more As many of you know, one of the outcomes of the Monterey Conference this past fall was the creation of an informal group which later called itself the "MFuG" or the Monterey Future's Group. This group has met several times since Monterey and developed a paper which discusses the networking needs and supporting technical requirements of the university community as it is now evolving. Doug Gale and a great many in MFuG contributed to this effort. In addition, there was work by a small group of folks (Stuart Lynn, David Wasley, Russ Hobby, Mike Staman) to write a concept paper describing a National Higher Education Networking Organization. Finally, at the request of the NTTF Steering Committee, Bill Graves wrote a background piece in preparation for the joint NTTF/FARNET meeting on April 15th and 16th.
Communication services, architecture, and technologies are rapidly evolving in response to applic... more Communication services, architecture, and technologies are rapidly evolving in response to application demand and research innovation. These changes are motivating a fundamentally new approach to the design and provisioning of services, facilities, and infrastructure. Traditionally, such resources have been designed and implemented as centralized fixed utility services, with almost no options for specialization and customization, especially by processes at the
2012 SC Companion: High Performance Computing, Networking Storage and Analysis, 2012
In this paper we describe the design, and implementation of the Open Science Data Cloud, or OSDC.... more In this paper we describe the design, and implementation of the Open Science Data Cloud, or OSDC. The goal of the OSDC is to provide petabyte-scale data cloud infrastructure and related services for scientists working with large quantities of data. Currently, the OSDC consists of more than 2000 cores and 2 PB of storage distributed across four data centers connected by 10G networks. We discuss some of the lessons learned during the past three years of operation and describe the software stacks used in the OSDC. We also describe some of the research projects in biology, the earth sciences, and social sciences enabled by the OSDC.
This work presents the implementation of a tabular interpolation approach to estimate empirical S... more This work presents the implementation of a tabular interpolation approach to estimate empirical Shannon entropy on programmable data plane ASICs using P4. The technique transforms the complex computations of the random projection into fast lookup over precomputed tables in the match-action pipeline. Likewise, the interpolation heuristic further reduces the table size substantially. Thus, more tables can be accommodated, achieving higher estimation accuracy. Simulations based on real-world network traffic traces are performed to evaluate the estimation accuracy. The scheme is deployed in a Barefoot Tofino2 switch connected to the International Center for Advanced Internet Research (iCAIR) national testbed. The system can estimate the entropy of network traffic accurately at 400 Gbps throughput.
We describe the GENI Experiment Engine, a Distributed-Platform-as-a-Service facility designed to ... more We describe the GENI Experiment Engine, a Distributed-Platform-as-a-Service facility designed to be implemented on a distributed testbed or infrastructure. The GEE is intended to provide rapid and convenient access to a distributed infrastructure for simple, easy-to-configure experiments and applications. Specifically, the design goal of the GEE is to permit experimenters and application writers to: (a) allocate a GEE slicelet; (b) deploy a simple experiment or application; (c) run the experiment; (d) collect the results; and (e) tear down the experiment, starting from scratch, within five minutes. The GEE consists of a set of cooperating services over the GENI infrastructure, which together with rapidly-allocated slicelets and a rapidly-allocated network offers a complete, ready to use, sliceable platform over the GENI Infrastructure. The GEE is designed to use off-the-shelf components and infrastructure; unlike previous PaaS offerings, it can be nested nicely inside a GENI slice, or any other IaaS infrastructure. Further, the GEE's southbound interface is extremely small and lightweight, making it portable to other underlying infrastructures.
To realize P4 language benefits: “Protocol Independent, Target Independent, Field Reconfigurable”... more To realize P4 language benefits: “Protocol Independent, Target Independent, Field Reconfigurable” [1] in a network research and development environment, a few P4 research institutions around the world formed a partnership to design and implement the international P4 Experimental Networks (Figure 1). The participating institutions can share distributed P4 resources over international research and education networks, as well as initiate international P4 research collaboration projects. Initial network scenarios in this research testbed are Software-Defined Network Exchanges (SDXs), Network and Cloud Testbed Networks, Science Networks, and Campus Research Networks. These four network scenarios require multi-tenant support. Current P4Runtime specifications include a Multi-Controller design for twofold: Control Plane Partition and Redundancy [2]. However, the current P4Runtime does not implement Control Plane partition to support multi-tenant service on a switch. This project modified the current design to support a provider and multiple tenant model for the four international P4 Experimental Networks (i-P4EN)scenarios and other applications.
Proceedings of the 5th International Workshop on P4 in Europe
This work presents the implementation of a tabular interpolation approach to estimate empirical S... more This work presents the implementation of a tabular interpolation approach to estimate empirical Shannon entropy on programmable data plane ASICs using P4. The technique transforms the complex computations of the random projection into fast lookup over precomputed tables in the match-action pipeline. Likewise, the interpolation heuristic further reduces the table size substantially. Thus, more tables can be accommodated, achieving higher estimation accuracy. Simulations based on real-world network traffic traces are performed to evaluate the estimation accuracy. The scheme is deployed in a Barefoot Tofino2 switch connected to the International Center for Advanced Internet Research (iCAIR) national testbed. The system can estimate the entropy of network traffic accurately at 400 Gbps throughput.
2017 20th Conference on Innovations in Clouds, Internet and Networks (ICIN), 2017
This paper describes a Bioinformatics Software Defined Network Exchange (SDX) or BioSDX, which ha... more This paper describes a Bioinformatics Software Defined Network Exchange (SDX) or BioSDX, which has been designed, deployed, and demonstrated by a multi-organizational research consortium to enable bioinformatics knowledge discovery supported by dynamic networking services. This BioSDX uses precision networking to support precision medicine. The BioSDX is based on recent technical developments in infrastructure abstraction that enables new types of tools and services utilizing programmable network infrastructure through high levels of resource virtualization. Combined with close integration of programmable cloud computing facilities, the BioSDX is an important advance in supporting the new paradigm of data intensive bioinformatics across multiple disciplines, including computational genomics and precision medicine.
2020 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS), 2020
Increasingly, scientific research discovery depends on the analysis of extremely large amounts of... more Increasingly, scientific research discovery depends on the analysis of extremely large amounts of data. This data is gathered through very large scientific instruments, very large numbers of small devices, and mid-scale instruments. These devices generate exponentially increasing volumes of data. Such data is often obtained, analyzed, stored, visualized, and transferred at and among sites worldwide by large collaborations of science communities. Managing and moving extremely large volumes of data across today's networks is a special multidimensional challenge. Also, existing data management and movement services and tools often are inadequate to address all specific requirements. This paper describes the various components of this challenge and how those issues can be addressed by ROBIN, a unique comprehensive set of integrated services designed specifically for managing and moving extremely large amounts of data over long distances, e.g., thousands of miles around the globe. This paper also describes the results of initial experiments using that set of integrated services.
Increasingly, science requires the gathering, transport, analysis and storage of extremely large ... more Increasingly, science requires the gathering, transport, analysis and storage of extremely large volumes of data, a key resource for new knowledge discovery. High energy physics, bioinformatics, genomics, oceanography, atmospheric sciences, weather modeling, astrophysics, geophysical sciences, earth sciences, materials science, chemistry, molecular dynamics, neurosciences, and many other disciplines use petabytes of information for sophisticated analytic processing, models, and simulations. Also, many of these science communities are creating next generation instruments that will generate many additional petabytes of data (exebytes in aggregate), including high luminosity synchrotrons, including the High Luminosity Large Hadron Collider, the Large Synoptic Survey Telescope, the Square Kilometer Array, neutrino detectors, genomic sequencers and others.
In part because of the limitations of the TCP/IP protocols used by the current Internet, the Futu... more In part because of the limitations of the TCP/IP protocols used by the current Internet, the Future Internet has become an attractive network research topic. Recently, several projects focused on the Future Internet have been launched around the globe based on the OpenFlow platform, such as those that are part of the GENI project in the U.S., the OFELIA project of FP7 in the E.U., and the FIRST project in Korea. Because OpenFlow allows researchers to design and develop innovative protocols, it could be applied in different situations for constructing desired network environments. OpenFlow is not limited to single controller environment within a single network domain. It allows network engineers to construct a multi-controller environment across various network domains. In this paper, we design and implement an automatic network topology discovery mechanism based on a multi-controller OpenFlow network. The result has been applied to a large-scale OpenFlow testbed implemented across r...
2019 IEEE/ACM Innovating the Network for Data-Intensive Science (INDIS), 2019
Transferring big data over Wide Area Networks (WANs) is challenging because optimization is depen... more Transferring big data over Wide Area Networks (WANs) is challenging because optimization is dependent on the specifics of multiple parameters. Network services, paths, and technologies have different characteristics, including loss rate, latency, and available capacity. Yet, frameworks currently used to configure and orchestrate transfer systems, measure performance, and analyze results have limited capabilities. We propose a framework, DTN-as-a-Service (DaaS), for high-performance network data transfers using and integration of techniques, including virtualization, network provisioning, and performance data analysis. This framework has a modular design for supporting multiple transfer tools, optimizers and orchestrators for the data transfer environment, including \textit{Docker} and \textit{Kubernetes}. We present a \textit{Jupyter} based workflow for high-speed network data transfer in data-intensive science and evaluate the performance of the transfer with a simple programmable visualizer implemented in the framework. This framework has been implemented as a prototype at two recent SC supercomputing conferences. With the increase in the number and the capacity of WAN links at the conferences (multiple 100 Gbps WAN circuits), the challenges involved in setting up, testing, debugging, verifying and running applications on high-performance systems connecting to the conference SCinet WAN circuits also increase. The SCinet implementation of the DaaS framework for the conference community allowed users to control hardware, software, and network infrastructure for high-speed network data transfer, primarily for large scale applications. Through the evaluation of the framework in our test setup, we demonstrated that NVMe over Fabrics with TCP is twice as efficient compared to using conventional TCP in high-speed NVMe-to-NVMe transfers. We also implemented a 400 Gbps LAN experiment to evaluate the DaaS framework.
Uploads
Papers by Joe Mambretti