| NSF Org: |
OAC Office of Advanced Cyberinfrastructure (OAC) |
| Recipient: |
|
| Initial Amendment Date: | September 10, 2012 |
| Latest Amendment Date: | September 10, 2012 |
| Award Number: | 1245926 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Kevin Thompson
kthompso@nsf.gov (703)292-4220 OAC Office of Advanced Cyberinfrastructure (OAC) CSE Directorate for Computer and Information Science and Engineering |
| Start Date: | January 1, 2013 |
| End Date: | December 31, 2015 (Estimated) |
| Total Intended Award Amount: | $511,497.00 |
| Total Awarded Amount to Date: | $511,497.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
104 AIRPORT DR STE 2200 CHAPEL HILL NC US 27599-5023 (919)966-3411 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
NC US 27599-1350 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): |
Information Technology Researc, Campus Cyberinfrastructure |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
Workflows, especially data-driven workflows and workflow ensembles are becoming a centerpiece of modern computational science. However, scientists lack the tools that integrate the operation of workflow-driven science applications on top of dynamic infrastructures that link campus, institutional and national resources into connected arrangements targeted at solving a specific problem. These tools must (a) orchestrate the infrastructure in response to application demands, (b) manage application lifetime on top of the infrastructure by monitoring various workflow steps and modifying slices in response to application demands, and (c) integrate data movement with the workflows to optimize performance.
Project ADAMANT (Adaptive Data-Aware Multi-domain Application Network Topologies) brings together researchers from RENCI/UNC Chapel Hill, Duke University and USC/ISI and two successful software tools to solve these problems: Pegasus workflow management system and ORCA resource control framework, developed for NSF GENI. The integration of Pegasus and ORCA enables powerful application- and data-driven virtual topology embedding into multiple institutional and national substrates (providers of cyber-resources, like computation, storage and networks). ADAMANT leverages ExoGENI - an NSF-funded GENI testbed, as well as national providers of on-demand bandwidth services (NLR, I2, ESnet) and existing OSG computational resources to create elastic, isolated environments to execute complex distributed tasks. This approach improves the performance of these applications and, by explicitly including data movement planning into the application workflow, enables new unique capabilities for distributed data-driven "Big Science" applications.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
Project ADAMANT integrated together a scientific workflow management system called Pegasus and an infrastructure orchestration system called ORCA. Together the two systems can dynamically manage computational, network and storage infrastructure to support the performance requirements of workflows running under Pegasus, thus making scientists who use Pegasus more productive. The infrastructure managed by ORCA for Pegasus has Quality of Service guarantees and responds just like hand-built infrastructure, except it can be created with a push of a button. What’s more, it is cheaper to own, because it allows multiple workflow applications to take advantage of its capabilities, without resorting to creating dedicated infrastructure for each. Additionally, today’s computational science is hampered by the need to manage data transfers and storage manually and ADAMANT automates many of these steps and helps optimize data movement across the networks. We used real astronomy and bio-medical applications to demonstrate the advantages of this approach.
This project gained a lot of interest from the various researcher communities and was presented at major conferences, like, for example, SuperComputing. It also won ‘Innovations in Networking’ award from CENIC in 2013. A number of students contributed to the project and gained experience in working in interdisciplinary teams on complex science problems.
The individual components that were added to Pegasus and ORCA are also used independent of ADAMANT. For example Pegasus ShadowQ, used in ADAMANT for providing performance feedback information from the workflow, is also used to predict completion times of workflows. Algorithms that were designed for ORCA to help manage compute, storage and networking infrastructure in response to Pegasus requirements were incorporated into ORCA code base and served as a foundation of a set of unique capabilities that are now available to the users of ExoGENI testbed, that is managed by ORCA. These capabilities allow computer science and distributed systems experimenters to create new types of experimental environments that flexibly respond to their needs and allow them to create more sophisticated experiments to test their ideas.
Last Modified: 05/19/2016
Modified by: Ilya Baldin
Please report errors in award information by writing to: awardsearch@nsf.gov.
