| NSF Org: |
OAC Office of Advanced Cyberinfrastructure (OAC) |
| Recipient: |
|
| Initial Amendment Date: | September 15, 2017 |
| Latest Amendment Date: | September 15, 2017 |
| Award Number: | 1726260 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Alejandro Suarez
alsuarez@nsf.gov (703)292-7092 OAC Office of Advanced Cyberinfrastructure (OAC) CSE Directorate for Computer and Information Science and Engineering |
| Start Date: | October 1, 2017 |
| End Date: | September 30, 2020 (Estimated) |
| Total Intended Award Amount: | $259,731.00 |
| Total Awarded Amount to Date: | $259,731.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
3201 BURTON ST SE GRAND RAPIDS MI US 49546-4301 (616)526-6000 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
3201 Burton SE Grand Rapids MI US 49546-4408 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): | Major Research Instrumentation |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
This project will replace Calvin College's aging computational cluster with a newer, more flexible cluster for multidisciplinary research. The new cluster is designed to facilitate six research projects, as well as research training activities at Calvin, including a nascent Data Science program. The resource will also offer the flexibility to support research activities beyond the local campus, specifically at Macalester, St. Olaf, and Wheaton Colleges. By enabling new, as well as enhancing existing research and research training activities at Calvin, the cluster is expected to improve the technological capabilities of the United States cyber-workforce by equipping hundreds of students each year with domain-specific high-performance computing skills in computer science, data science, mathematics, statistics, and the sciences.
Specifically, the cluster will enable research in: a thread safe graphics library (TSGL), used to create real-time visualizations of parallel algorithms; an agent-based economic model to evaluate the effects of different policies during transitions from non-renewable to renewable energy sources; new computational chemistry models used to explore the photophysical properties of common coumarins; accuracies of various economic models; 3D routing systems for first-responders and improved 3D sniper-line-of-sight models used by homeland security personnel in emergency situations; and enhancing Sivvu.org, a new web service for ERFA (Equilibrium Restricted Factor Analysis), a popular computational chemistry technique. In addition, faculty and student research projects at Calvin, Macalester, St Olaf, and Wheaton colleges will also use the new cluster.
Furthermore the cluster will enable new degree programs in Data Science and Statistics at Calvin. Students in these programs will use the new cluster to store and process large data sets. Additionally, the cluster will enhance existing courses, such as CS 374, a course in High Performance Computing (HPC) where students will use the cluster to learn to program with MPI, OpenMP, CUDA, and similar HPC tools.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
This project--MRI: Acquisition of a Multidisciplinary Beowulf Cluster--was successfully completed at Calvin University from 2017-2020. PI Adams designed the tera-scale Beowulf cluster borg.calvin.edu (named after computer scientist Anita Borg) using current commodity hardware to meet his and his co-PIs? research needs. After soliciting bids for his design, the system was built by Advanced Clustering Technologies of Kansas City, MO, and was installed at Calvin University in Spring 2018.
The Borg system includes: a head node for launching jobs onto the system; twenty compute nodes, each with 96 GB of RAM and 16-core Intel Xeon Gold CPUs; a virtualization node for running virtual machine based web services; and two high-memory nodes. The high-memory "GPU node" has 768 GB of RAM, a 16-core CPU, and four Titan-V GPUs providing a total of 20,480 CUDA cores for data-parallel applications, plus 2560 tensor cores for artificial intelligence / machine learning applications. The high-memory "CPU node" has 512 GB of RAM, 96 CPU cores for CPU-intensive applications, and 2560 CUDA cores for data-parallel applications. The system thus provides a total of 464 CPU cores, over 23,000 GPU cores, over 3 TB of RAM, and over 100 TB of tiered NVMe/HDD tiered storage. A 100 Gbps Omnipath network is used for internal data communication between the cluster's nodes; a 40/10 Gbps Ethernet network is used for administrative communication within the cluster and for communication with the outside world.
To support high-speed communication between Borg and remote users, Calvin's Information Technology (CIT) department upgraded Calvin's network infrastructure between Borg and other campus hubs (including the campus gateway) to 40/10 Gbps Ethernet. Calvin also upgraded the electrical and climate control systems in the room housing Borg, and purchased a new uninterruptible power supply (UPS) system to power Borg.
Researchers from Calvin and other universities are using Borg to accelerate their research projects, including:
- Researchers with traditional distributed-memory parallel research applications (e.g., MPI applications) use Borg's head node and compute nodes.
- Researchers with traditional shared-memory parallel research applications (e.g., OpenMP applications) that are memory- and CPU-intensive use Borg's high-memory CPU node.
- Researchers who need to process large data sets (e.g., using CUDA) and/or train machine-learning systems (e.g., using TensorFlow) use Borg's high-memory GPU node.
- Researchers needing to run web services that utilize any of these subsystems run those services in virtual machines that run on Borg's virtualization node.
- Researchers who need a place to store and quickly access large data sets use Borg's tiered storage system.
Software packages that researchers use on the cluster include:
- Amber: a suite of tools for biomolecular simulation
- CUDA: Nvidia?s framework for GPU computing
- eQueue: a web-interface for running particular kinds of jobs on Borg
- Gaussian: a software suite for computational chemistry, and WebMO: a web-interface for Gaussian
- The GNU software suite: gcc, g++, and so on.
- Intel's software suite for cluster computing: Intel Parallel Studio, Intel C, C++, and Fortran compilers, etc.
- Message Passing Interface (MPI) implementations: OpenMPI, MPICH, and Intel MPI
- OpenMP for implicit multithreading in C, C++, or Fortran
- Python for general purpose scripting, used for custom astronomy simulations, TensorFlow applications, etc.
- R for statistical computations and R Studio, a web-interface for R
- Sivu: a custom-written service for modeling spectroscopic titration data
- SLURM: a scheduler for reserving particular nodes and running software applications on them
- Societies: a custom-written agent-based economics model
- Tensor-flow for computations involving machine learning
Borg is thus being used to accelerate the research projects of faculty and undergraduate students in the areas of artificial intelligence (machine learning), astronomy, computational chemistry, computer science, economics, geographical information systems (GIS), and statistics.
While primarily devoted to research uses, Borg is also being used for workforce development, specifically:
- to train undergraduate computer science and data science students in high performance computing methods.
- to train undergraduate chemistry students in inorganic computational chemistry methods.
- to store the large datasets by which GIS students are trained (remotely) in the use of GIS software.
- to host the development of a free, interactive, parallel computing textbook by the team at CSinParallel.org.
Borg is maintained by Calvin CS sysadmin Chris Wieringa, whose help has been invaluable throughout this project.
Last Modified: 12/31/2020
Modified by: Joel C Adams
Please report errors in award information by writing to: awardsearch@nsf.gov.
