Award Abstract # 2016619
CCRI: Planning: Collaborative Research: Infrastructure for Enabling Systematic Development and Research of Scientific Workflow Management Systems

NSF Org: CNS
Division Of Computer and Network Systems
Recipient: UNIVERSITY OF SOUTHERN CALIFORNIA
Initial Amendment Date: August 14, 2020
Latest Amendment Date: October 20, 2020
Award Number: 2016619
Award Instrument: Standard Grant
Program Manager: Marilyn McClure
mmcclure@nsf.gov
 (703)292-5197
CNS
 Division Of Computer and Network Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: October 1, 2020
End Date: September 30, 2021 (Estimated)
Total Intended Award Amount: $70,000.00
Total Awarded Amount to Date: $70,000.00
Funds Obligated to Date: FY 2020 = $70,000.00
History of Investigator:
  • Rafael Ferreira da Silva (Principal Investigator)
    rafsilva@isi.edu
Recipient Sponsored Research Office: University of Southern California
3720 S FLOWER ST FL 3
LOS ANGELES
CA  US  90033
(213)740-7762
Sponsor Congressional District: 34
Primary Place of Performance: University of Southern California
4676 Admiralty Way, Suite 1001
Marina de Rey
CA  US  90292-6611
Primary Place of Performance
Congressional District:
36
Unique Entity Identifier (UEI): G88KLJR3KYT5
Parent UEI:
NSF Program(s): CCRI-CISE Cmnty Rsrch Infrstrc
Primary Program Source: 01002021DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7359
Program Element Code(s): 735900
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Scientific workflows are used almost universally across research and engineering disciplines and have underpinned some of the most significant discoveries of the past several decades (e.g., first detection of gravitational waves from colliding black holes, the discovery of the Higgs boson, and the detection of an exotic nuclear decay). Workflow management systems (WMSs) are software systems that provide abstraction and automation for facilitating access to and management of distributed and heterogeneous compute and storage resources. They enable a broad range of researchers to easily define sophisticated computational processes and to then execute them efficiently on parallel and distributed computing platforms. Unfortunately, in spite of widespread adoption of workflows, the technology landscape is segmented and presents significant barriers to entry due to the existence of dozens of seemingly comparable, yet incompatible, systems. The research landscape is also disjoint, making it difficult to compare and contrast approaches, verify and reproduce results, and build upon existing work.

This project will engage with representatives from the workflows community ? including researchers, developers, science and engineering users, and cyberinfrastructure experts. Through targeted community surveys and focused workshops, the project will gather a diverse set of perspectives, create a community-owned WMS inventory and common knowledge taxonomy, define an experimental methodology for measuring WMS capabilities, and develop a blueprint for a community research infrastructure. This proposed infrastructure has the potential to truly democratize workflows research, enabling researchers, postdocs, and students, irrespective of their institutions, to access cutting-edge infrastructure for comparison, evaluation, and verification of workflows research results.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Coleman, Taina and Casanova, Henri and Ferreira da Silva, Rafael "WfChef: Automated Generation of Accurate Scientific Workflow Generators" 17th IEEE EScience Conference , 2021 Citation Details
Coleman, Taina and Casanova, Henri and Gwartney, Ty and Ferreira da Silva, Rafael "Evaluating Energy-Aware Scheduling Algorithms for I/O-Intensive Scientific Workflows" Computational Science - ICCS 2021 , 2021 https://doi.org/10.1007/978-3-030-77961-0_16 Citation Details
Coleman, Tainã and Casanova, Henri and Pottier, Loïc and Kaushik, Manav and Deelman, Ewa and Ferreira da Silva, Rafael "WfCommons: A framework for enabling scientific workflow research and development" Future Generation Computer Systems , v.128 , 2022 https://doi.org/10.1016/j.future.2021.09.043 Citation Details
Ferreira da Silva, Rafael and Casanova, Henri and Chard, Kyle and Altintas, Ilkay and Badia, Rosa M and Balis, Bartosz and Coleman, Taina and Coppens, Frederik and Di Natale, Frank and Enders, Bjoern and Fahringer, Thomas and Filgueira, Rosa and Fursin, G "A Community Roadmap for Scientific Workflows Research and Development" 2021 IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS) , 2021 https://doi.org/10.1109/WORKS54523.2021.00016 Citation Details
Silva, Rafael Ferreira and Pottier, Loic and Coleman, Taina and Deelman, Ewa and Casanova, Henri "WorkflowHub: Community Framework for Enabling Scientific Workflow Research and Development" 2020 IEEE/ACM Workflows in Support of Large-Scale Science (WORKS) , 2020 https://doi.org/10.1109/WORKS51914.2020.00012 Citation Details

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

Scientific workflows are used almost universally across scientific domains and have underpinned some of the most significant discoveries of the past several decades. Workflow management systems (WMSs) provide abstraction and automation which enable a broad range of researchers to easily define sophisticated computational processes and to then execute them efficiently on parallel and distributed computing systems. As a result of widespread workflow adoption, the workflows research and development community has grown: there are now hundreds of independent WMSs, thousands of researchers and developers, and a rapidly growing corpus of workflows research publications. This progress has occurred in spite of the absence of a community research infrastructure, which has had wide-ranging consequences.

During this planning project, we conducted a series of events called "Workflows Community Summits" (January, April, and November 2021, https://workflowsri.org). The overarching goal of these summits was to (i) develop a coherent view of the state of the art, (ii) identify key research challenges, (iii) articulate a vision for potential community activities, and (iv) explore technical approaches for realizing (part of) this vision. The summits were attended by more than 130 participants, including international lead researchers and developers, from distinct workflow systems and user communities (over 30+ workflow systems were represented). The outcomes of the three Workflows Community Summits have been compiled and published in three technical reports and one research paper:

  • Workflows Community Summit: Bringing the Scientific Workflows Research Community Together (http://dx.doi.org/10.5281/zenodo.4606958): The first "Workflows Community Summit" was held online on January 13, 2021. The summit included 48 invited participants from a select group of international (Austria, Brazil, Germany, Netherlands, Spain, UK, and USA) lead researchers from distinct workflow management systems and users, and representatives from the National Science Foundation (NSF), and Department of Energy (DoE). The overarching goal of the summit was to develop a view of the state of the art and identify crucial research challenges in the workflow community. The summit focused on six themes: (i) FAIR computational workflows, (ii) Training and education for workflow users, (iii) AI workflows, (iv) Exascale challenges and beyond, (v) APIs, interoperability, reuse, and standards, and (vi) Building a workflows community. The end goal was to identify other challenges, if need be, and outline short- and long-term community efforts that could help tackle these challenges.

  • Workflows Community Summit: Advancing the State-of-the-art of Scientific Workflows Management Systems Research and Development (http://dx.doi.org/10.5281/zenodo.4915801): The second "Workflows Community Summit" took place on April 7, 2021. The summit included 75 participants from a group of international researchers and developers (Australia, Austria, Brazil, France, Germany, Italy, Netherlands, Norway, Poland, Spain, Switzerland, UK, and USA)  from distinct workflow management systems and users, and representatives from the NSF and DoE. While the first summit focused on establishing a high level vision, this second summit explored technical approaches for realizing (part of) that vision. Consequently, we strived to engage participants who are actively involved in workflow system development. Based on the outcomes of the first summit, we identified three technical topics for discussion: (i) Definition of common workflow patterns and benchmarks (both for determining workflow system functionality and for providing users with tutorial examples); (ii) Identifying paths toward interoperability of workflow systems; and (iii) Improving workflow systems' interface with legacy and emerging HPC software and hardware stacks.

  • Workflows Community Summit: Tightening the Integration between Computing Facilities and Scientific Workflows (http://dx.doi.org/10.5281/zenodo.5815332): The third "Workflows Community Summit" took place on November 8, 2021. The summit included 14 participants from various computing centers and facilities. Based on the outcomes of the first two summits, we identified the need to share outcomes with computing centers and facilities, as well as to capture their perspectives regarding workflow systems and applications. To this end, this third summit was organized as a focused session with a small group of participants with the aim to understand how workflows are currently being used at each facility, how facilities would like to interact with workflow developers and users, how workflows fit with facility roadmaps, and what opportunities there are for tighter integration between facilities and workflows.

  • Research Paper: A Community Roadmap for Scientific Workflows Research and Development, 2021 DOI:  https://doi.org/10.1109/WORKS54523.2021.00016.

As an outcome of this planning project, we have established a Workflows Community Initiative (https://workflows.community) to bring the workflows community together (users, developers, researchers, and facilities) and provide community resources and capabilities to enable scientists and workflow systems developers to discover software products, related efforts, events, technical reports, etc. and engage in community-wide efforts to tackle workflows grand challenges.

 

 


Last Modified: 01/24/2022
Modified by: Rafael Ferreira Da Silva

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page