| NSF Org: |
OAC Office of Advanced Cyberinfrastructure (OAC) |
| Recipient: |
|
| Initial Amendment Date: | April 29, 2020 |
| Latest Amendment Date: | April 29, 2020 |
| Award Number: | 2004012 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Andrey Kanaev
akanaev@nsf.gov (703)292-2841 OAC Office of Advanced Cyberinfrastructure (OAC) CSE Directorate for Computer and Information Science and Engineering |
| Start Date: | May 1, 2020 |
| End Date: | April 30, 2024 (Estimated) |
| Total Intended Award Amount: | $439,505.00 |
| Total Awarded Amount to Date: | $439,505.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
5000 FORBES AVE PITTSBURGH PA US 15213-3890 (412)268-8746 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
PA US 15213-3815 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): | Software Institutes |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
The transfer of large data sets between computing resources is an integral component of the scientific workflow. Multiple tools have been developed to aid in this task but, despite significant performance bottlenecks, secure shell (SSH) based tools like secure copy protocol (SCP) and secure file transfer protocol (SFTP) remain popular due to ubiquity, ease of use, and minimal administrative burdens. HPN-SSH, the project tool, was initially developed in 2004 to address this need. This award will allow the PIs to expand on the original work of HPN-SSH and address new challenges created by advances in computer technology and the needs of users. They will incorporate hardware accelerated encryption; efficiently use modern CPUs by dynamically sizing the number of threads; accelerate the cryptographic workflow by allowing it to work in parallel; investigate making the default open source secure shell OpenSSH cipher use multiple cores; create a ?resume on failure? feature enabling users to restart transfers from the point of failure; and incorporate networking metrics to aid in troubleshooting and performance analysis of HPN-SSH. HPN-SSH will support and enhance research efforts across a wide range of scientific domains by lowering the costs of entry to big data and remote computation without compromising security or functionality. These benefits will extend to business and industry, educational communities, and the general public as well.
The transfer of large data sets between computing resources is an integral component of the scientific workflow. Multiple tools have been developed to aid in this task but, despite significant performance bottlenecks, SSH based tools remain popular due to ubiquity, ease of use, and minimal associated costs. To address these bottlenecks we developed HPN-SSH; a series of patches that enable high performance throughput for the OpenSSH application. These patches were initially released in 2004 and have become widely used throughout the research, academic, financial, and technology communities. This award gives the PIs opportunity to foster innovative development in HPN-SSH that will benefit the community by significantly increasing performance. With this grant the PIs will: incorporate on-die hardware accelerated encryption in their multithreaded AES counter cipher; efficiently use multicore CPUs by dynamically sizing the number of threads; introduce pipelining and parallelization into the cryptographic workflow; investigate the parallelization of the default OpenSSH cipher CHACHA20; create a ?resume on failure? feature enabling users to restart transfers from the point of failure; and incorporate inline network telemetry to aid in troubleshooting and performance analysis. This work will also advance the field of computer science through the development and improvement of parallelization methods to enhance the performance of cryptographic routines. As most widely used cryptographic libraries and methods are highly serial in nature they are unable to take advantage of multicore processors. As processor speed has remained relatively stable over the past ten years we must distribute the cryptographic workload over multiple cores in order to significantly increase throughput. HPN-SSH will democratize access and extend the reach of the national cyberinfrastructure by lowering the costs of entry without compromising security or functionality. These benefits will extend to business and industry, educational communities, and the general public.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
HPN-SSH is a high performance implementation of the SSH protocol that is built on the foundation of OpenSSH. HPN-SSH provides users of all types access to a secure, fast, and easy to use data transfer tool that is fully interoperable with all protocol compliant implementations of SSH. First released in 2004 HPN-SSH can now provide more than 50 times the throughput of OpenSSH. Throughput rates in excess of 5Gbps for a fully encrypted stream across a wide area network are not uncommon.
The Elements: HPN-SSH award had multiple goals focused on improving the performance, stability, and functionality of HPN-SSH. These included; the enhancement of the parallelized AES-CTR cipher to make use of the AES instruction set available on modern CPUs, the development of in-line network reporting that returns data from the TCP stack on both sides of the connection, the creation of a parallelized ChaCha20-Poly1305 cipher, the automatic resumption of failed transfers when using SCP, the distribution of easy to install and maintain binary packages of HPN-SSH for Linux based distributions, and overall performance improvements.
The HPN-SSH team was able to successfully deliver on all of these goals significantly improving throughput performance while often reducing the computational resources required by prior versions. For example, the introduction of AES-NI support in the parallelized AES-NI cipher, along with other enhancements, improved throughput by 30% while reducing CPU usage as much as 75%. Similarly, the development of the parallel ChaCha20-Poly1305 cipher improved throughput by at least 20%. Package repositories are available for popular distributions such as Fedora, Debian, Ubuntu, Alpine, and Gentoo.
During the course of this award HPN-SSH was also able to significantly expand the user community with thousands of new downloads and installations being recorded. We also incorporated numerous suggestions from users and code from developers into HPN-SSH further strengthening our community. Additionally, multiple people have come forward as volunteers to maintain HPN-SSH packages. These volunteers include maintainers for FreeBSD, Gentoo, and Alpine. Lastly, we were able to collaborate with other researchers resulting in a specialized version of HPN-SSH supporting the astronomy community.
HPN-SSH will continue to be developed, maintained, and released as an open source project to serve the needs of our community and the public as a whole. The source code will be maintained on github.com.
Last Modified: 09/03/2024
Modified by: Christopher J Rapier
Please report errors in award information by writing to: awardsearch@nsf.gov.
