Project Outcomes Report

Parallel computing is indispensable to achieving the high computing performance necessary for science and engineering, as well as, more recently, machine learning/AI.  Unfortunately achieving high performance and efficiency can be extremely challenging and typically requires specialists.  This is only expected to get worse with the advent of heterogeneous architectures. The main purpose of this planning project was to begin to address this problem.  

This project provided a framework for integrating teams from Northwestern, Illinois Institute of Technology, and Carnegie Mellon to develop a large-scale proposal to NSF’s PPoSS program.  The overall concept was to combine the “theory down” and “architecture up” approaches to parallelism that are exemplified by the teams, and to explore what it would mean to develop an “IR-out” approach, centered around a compilation framework.  This “IR-out” (intermediate representation) approach would target future heterogeneous parallel machines and attempt to simultaneously democratize programming of such machines while still achieving expert-level performance.   

In addition to providing the ability to plan together, including in several workshops, the project also supported the collaborative teams ability to start work on several technical challenges together.   The most visible of these is in heartbeat scheduling, which is an approach to modulating the amount of available parallelism in a program, one that has strong theoretical bounds.    In the joint work, the team developed (a) two different compilation technologies (one based on assembly-level transforms, the other based on IR-level transforms) to enable heartbeat scheduling with minimal programmer effort, and (b) special kernel support (both a custom kernel, and a Linux kernel module) to provide the necessary runtime heartbeat signals effectively.  

Significant training opportunities resulted from the project.  7 Ph.D. students (4 NU, 2 IIT, 1 CMU), 1 MS student (IIT), 4 REU students (2 NU, 2 IIT),  and 1 other undergraduate (IIT) were involved in the project.   Several of the MS students and undergraduates joined Ph.D. programs.    The project’s three workshops each included about 30 attendees, providing greater exposure for students at every site, but particularly for Northwestern, which hosted each one and thus could easily send a broad range of students at all levels.    Northwestern’s undergraduate operating systems course was entirely redesigned, leveraging the Nautilus kernel framework to create labs.   A Northwestern graduate course in kernel and other low-level software development was created, and has so far trained over 100 students in this esoteric topic.  A Northwestern course on compiler analysis and transformation was created, and Northwestern’s compiler sequence was  also considerably enhanced.

The results of this project were very promising, including multiple joint publications and software, much of which is publicly available.   The collaborative team was also successful in having the planned-for large-scale project supported by the NSF (with an additional element funded by DOE), and the resulting Constellation Project is now highly active. 

Last Modified: 11/26/2023
Modified by: Peter A Dinda