Award Abstract # 1440745
CC*IIE Integration: Dynamically Optimizing Research Data Workflow with a Software Defined Science Network

NSF Org: OAC
Office of Advanced Cyberinfrastructure (OAC)
Recipient: YALE UNIV
Initial Amendment Date: September 9, 2014
Latest Amendment Date: August 22, 2018
Award Number: 1440745
Award Instrument: Standard Grant
Program Manager: Kevin Thompson
kthompso@nsf.gov
 (703)292-4220
OAC
 Office of Advanced Cyberinfrastructure (OAC)
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: October 1, 2014
End Date: September 30, 2019 (Estimated)
Total Intended Award Amount: $788,605.00
Total Awarded Amount to Date: $788,605.00
Funds Obligated to Date: FY 2014 = $788,605.00
History of Investigator:
  • Yang Yang (Principal Investigator)
    yry@cs.yale.edu
  • Robert Bjornson (Co-Principal Investigator)
  • Andrew Sherman (Co-Principal Investigator)
  • Robert Starr (Co-Principal Investigator)
  • David Galassi (Former Co-Principal Investigator)
Recipient Sponsored Research Office: Yale University
150 MUNSON ST
NEW HAVEN
CT  US  06511-3572
(203)785-4689
Sponsor Congressional District: 03
Primary Place of Performance: Yale University
AKWatson Hall
New Haven
CT  US  06520-8285
Primary Place of Performance
Congressional District:
03
Unique Entity Identifier (UEI): FL6GV84CKN57
Parent UEI: FL6GV84CKN57
NSF Program(s): Information Technology Researc
Primary Program Source: 01001415DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 1640, 8002
Program Element Code(s): 164000
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Intelligent management of campus research networks has become a major challenge for many institutions, as their networks grow rapidly in size and complexity in order to meet the demands of on-campus scientists who are conducting research, collaborating with peers, and fulfilling their mission of scientific education. Traditional, static network management approaches are no longer adequate, since they often result in low efficiency, poor usability, and unpredictable network application performance.

The goal of this project is to design and deploy a novel intelligent network cyberinfrastructure that greatly expands the ability of scientists to rapidly and efficiently move the large quantities of data required for computation- and data-intensive scientific workflows. To ensure a broad impact, the project includes specific focus on a range of science drivers in diverse fields such as astronomy, climatology, and genomics.

The project achieves its goal by leveraging and validating several prior networking research and development efforts. These include Maple, a novel Software Defined Networking (SDN) programming framework developed at Yale, and an Application Layer Traffic Optimization (ALTO) protocol and framework pioneered at Yale and now incorporated in a proposed standard for the Internet by the Internet Engineering Task Force. Maple simplifies network programming for end-to-end, complex, dynamically constructed network services, while ALTO enables network applications to adapt dynamically, according to network states, to deliver network efficiency and application quality of service. In addition, the project builds on prior Yale and NSF investments in high-speed physical network cyberinfrastructure, the widely-adopted InCommon authentication framework, and IPv6 technology.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 12)
Qiao Xiang, James Aspnes, Franck Le, Chin Guok, Linghe Kong, Dennis Yu, and Y. Richard Yang "Optimizing in the Dark: Learning an Optimal Solution Through a Simple Request Interface" The Thirty-Third AAAI Conference on Artificial Intelligence} (AAAI-19) , 2019 https://aaai.org/ojs/index.php/AAAI/article/view/3984/3862
Danny Alex lachos Perez, Christian Esteve Rothenberg, Qiao Xiang, Börje Ohlman, Sabine Randriamasy, Farni Boten, and Y. Richard Yang "Supporting Multi-domain Use Cases with ALTO" IETF Applied Networking Research Workshop 2019 (ANRW 2019) , 2019 https://irtf.org/anrw/2019/slides-anrw19-final28.pdf
Geng Li, Akrit Mudvari, Kerim Gokarslan, Patrick Baker, Sastry Kompella, Franck Le, Kelvin Marcus, Vinod Mishra, Jeremy Tucker Y. Richard Yang and Paul Yu "Magnalium: Highly reliable SDC Networks by Composing Multiple Control Planes" DAIS 2019 , 2019 https://ieeexplore.ieee.org/abstract/document/8784088
Geng Li, Franck le, Yeon-sup Lim, Junqi Wang, and Y. Richard Yang "Update Algebra: Toward Continuous, Non-Blocking Composition of Network Updates in SDN" IEEE INFOCOM , 2019 10.1109/INFOCOM.2019.8737618
H. Newman, M. Spiropulu, J. Balcas, T. Hendricks, D. Kcira, I. Legrand, A. Mughal, S. Novaes, A. Baruchi, R. Iope, B. Leal, K. Gao, M. Wang, Q. Xiang, Y.R. Yang, J. Zhang "A Next Generation Terabit/sec SDN Architecture and Data Intensive Applications for High Energy Physics and Exascale Science." INDIS Workshop , 2017
Kai Gao, Qiao Xiang, X. Tony Wang, Jun Bi and Y. Richard Yang "An Objective-Driven On-Demand Network Abstraction for Adaptive Applications" IEEE/ACM Transactions on Networking (TON) , v.27 , 2019 , p.805 10.1109/TNET.2019.2899905
K. Gao, C. Gu, Q. Xiang, X. Wang, and Y.R. Yang "MERSA: Towards On-demand Minimal Equivalent Routing State Abstraction as a Northbound API" IEEE ICNP Extended Abstracts , 2016
K. Gao, C. Gu, Q. Xiang, Y.R. Yang, and J. Bi "FAST: A Simple Programming Abstraction for Complex State-Dependent SDN Programming" ACM SIGCOMM Posters , 2016
Qiao Xiang, J. Jensen Zhang, X. Tony Wang, Y. Jace Liu, Chin Guok, Franck Le, John McAuley, Harvey Newman, and Y. Richard Yang "Toward Fine-Grained, Privacy-Preserving, Efficient Multi-Domain Network Resource Discovery" IEEE Journal on Selected Areas in Communications (JSAC) Special Issue on Series on Network Softwarization & Enablers , 2019
Qiao Xiang, X. Tony Wang, J. Jensen Zhang, Harvey Newman, Y. Jace Liu, and Y. Richard Yang "Unicorn: Unified Resource Orchestration for Multi-domain, Geo-distributed Data Analytics" Future Generation of Computer Systems , v.93 , 2019 , p.188
S. Chen, A. Voellmy, X. Wang, Y. R. Yang "Magellan: Generating Multi-Table Datapath from Datapath Oblivious Algorithmic SDN Policies" ACM SIGCOMM Posters , 2016
(Showing: 1 - 10 of 12)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

This project (CC*IIE Integration: Dynamically Optimizing Research Data Workflow with a Software Defined Science Network, Award# 1440745) develops a programmable, efficient network cyberinfrastructure (CI) at Yale to support scientists both at Yale and broadly to greatly expand the ability of scientists to rapidly and efficiently move the large quantities of data required for complex computation- and data-intensive scientific workflows. A network infrastructure based on traditional designs can be too too inefficient and static (e.g., multiple, static, redundant networks) to support such workflows.

The intellectual foundation of the project is two successful research innovations at Yale: (1) Maple, which is a novel, high-level network programming framework to introduce programmability into network management. By changing a network from static to programmable, the network gains an essential ability to introduce end-to-end, complex, dynamically constructed network services. (2)  Application-layer traffic optimization (ALTO), which is a Proposed Standard of the Internet to integrate network traffic optimization and application-layer traffic optimization. Since the flexibility and efficiency of an infrastructure depends on both the resource provider (network) and the resource consumer (applications), by introducing ALTO, the infrastructure can achieve flexibility and efficiency which cannot be achieved by either network or applications alone.

The project makes significant intellectual progress in realizing the design beyond the initial foundation. (1) It extends the state-of-art programmable networking from single domain (i.e., one network) to multi-domain, geo-distributed software-defined networking to better support key science drivers (e.g., LHC), which consist of multiple organizations, where each organization has its own network (domain) and its local policies to limit complete programmability. (2) It introduces systematic foundations to understand the capabilities of existing framework (e.g., a learning based BoxOpt framework on the feasibility of using existing networking resource models) and the fundamental limitation of any programmable networking (e.g., a network programming capacity theorem). (3) It introduces fundamental primitives to handle challenges including handling complex dynamicity (e.g., the Update Algebra and the Trident uniform, consistent programming model), predicting resource availability (e.g., a model based Prophet model, and a secure multi-party computation based on multi-domain resource aggregation model), and introducing reliability (i.e., the novel multi-control-plane architecture).

The project also devote substantial efforts to achieve broader impacts. In particular, the key goal of the project is to demonstrate the benefits of a single, flexible network and the benefits have motivated Yale to deploy related techniques on the whole Yale network, achieving a major outcome of this project. The project works closely with the LHC/CMS science driver collaborators, and demonstrated the benefits of the systems at multiple settings including substantial demos at SuperComputing. The project also leads to substantial progress in the design and implementation of the ALTO Internet standards, which has proven extremely valuable for broader impacts. In particular, ALTO was the foundational framework for a large-scale deployment between one of the largest eyeball networks and a leading hyper-giant (paper see http://people.csail.mit.edu/gsmaragd/publications/CoNEXT2019/; a succint summary of impact of ALTO see slide 36 of http://people.csail.mit.edu/gsmaragd/publications/CoNEXT2019/CoNEXT2019-presentation.pdf). The project adopts a open-source, public-domain policy, and makes all of your systems publicly available.

 

 


Last Modified: 01/10/2020
Modified by: Yang R Yang

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page