Indiana University and Jetstream partners fundamentally changed the landscape of the National Science Foundation cyberinfrastructure (CI) ecosystem through the Jetstream project. The Jetstream project served as a first-of-a-kind environment as a production cloud system, while simultaneously acting as a pilot deployment, positively influencing the NSF Advanced Computing Systems and Services (ACSS) program that followed. Jetstream provided on-demand computing and storage anytime, anywhere. By design and in practice, Jetstream was created as programmable cyberinfrastructure in that users could reconfigure the system to suit their needs; this took many forms, from multi-user platforms that leveraged Jetstream to configure their services, to single individuals who utilized Matlab interactively. Jetstream was a very different system from other XSEDE resources, a highly-usable system with a core focus on those in the long tail of science. 

Jetstream provided services directly to 18,714 researchers and educators, including 8,836 students from June 2016 through July 2022 as part of 1,220 projects in 69 fields of science for individuals at 399 institutions during this time period. The 63 science gateways that utilized Jetstream indirectly supported over 183,197 people over the life of the system, most without even knowing their use of the resource. As a training and instructional resource Jetstream was used seven times more than any other XSEDE resource in terms of Educational Allocations. Over the life of Jetstream, 216 courses and workshops utilized Jetstream as part of their instruction. These activities included multi-day workshops for early career researchers, continuing education for researchers, semester long courses for undergraduate and graduate students, and semester-long capstone projects for masters level students. Forty-six courses were conducted multiple times on Jetstream.

The Jetstream environment was accepted under OpenStack Liberty and retired with the Rocky release on CentOS 7.8. Overall, the system went through seven major OpenStack releases without regularly scheduled downtime or unscheduled interruptions. Through the six years of operations Jetstream had an overall availability of 98.54% including planned and unplanned outages, and an uptime of 99.9967% where the system was operating in some form but at a reduced capacity. This was coupled with above average user satisfaction and importance, obtained via annual user surveys (1-5 scale) the "Importance of Jetstream for Research or Education" grew year over year, concluding with a rating >4, along with the quality/speed to support question response averaging >4.5. 

The services pioneered on Jetstream or by the project team have also had a significant impact on currently funded Category I resources as well as within discipline-specific projects that leverage the environment including: dynamically scaling virtual clusters used by the Galaxy project, providing non-command-line access to CI resources by default through single sign-on (including providing resources to spawn a new OpenStack interface, Exosphere), providing the first virtual GPUs, providing the first object storage system in the NSF CI ecosystem, and the first cloud native orchestration interface to a production NSF resource (e.g. OpenStack Magnum used with Kubernetes and Docker Swarm). 

Jetstream's impact on student, researcher, and educator success took multiple forms:

• As was widely published in the mainstream media, a group of scientists produced the first image of a black hole using Event Horizon Telescope (EHT) observations of the center of the galaxy M87. As part of this work, Jetstream was used to develop cloud-based data analysis pipelines that were critical for combining and sharing data taken from the geographically-distributed observatories. It was noted that the cloud pipeline used for analysis could not have been developed without Jetstream.

• Jetstream joined the COVID-19 HPC Consortium in March 2020 to contribute resources to the pandemic response. The team participated in joint reporting efforts to the NSF throughout the pandemic with numerous projects taking advantage of the resource. These COVID-19 related projects include gateways such as Galaxy (used for some of the earliest genomic analysis) and ChemCompute (used by students for remote learning), as well as for international medical records projects like OpenMRS, and regional information such as projections by University of Texas, San Antonio for Texas hospital bed and ICU bed usage. 

• Jetstream hosted an Research Experience for Undergraduates program for five consecutive years from 2017 through 2021 for 4-6 students per year (25 students slots total) that resulted in tangible outcomes for the students in the form of project papers, and opportunities to present at local events to their peers, as well as national forums such as the PEARC conference series and even at the SC Conference Series, the International Conference for High Performance Computing, Networking, Storage, and Analysis. The program focused recruitment on individuals from MSIs, HSI, and EPSCoR jurisdictions. 

Last Modified: 03/23/2023
Modified by: David Y Hancock