Memory heterogeneity means multiple memory components with different properties (such as memory bandwidth, latency, capacity, and computing ability) form a memory system. Memory heterogeneity is becoming common because of the needs of increasing memory capacity or providing higher performance in a cost-effective way. Memory heterogeneity raises challenges on deciding the optimal placement of data objects on heterogeneous memory (HM). Recent studies indicate substantial difficulty of matching applications with HM because of the complex and fast changing nature of HM as well as application input sensitivity and phase behaviors.

Intellectual merit. We study system-level solutions to make best usage of HM for high performance. Our solutions are essentially based on the ideas of introducing limited application semantics information to direct data migration and allocation. Using application semantics, we are able to break fundamental tradeoff between memory profiling overhead and accuracy, and decide when to trigger data migration to maximize the overlap between data migration and computation to minimize data migration overhead.

Furthermore, we study application-level solutions to make best usage of HM for high performance.  We propose new data structures and algorithms to reduce expensive accesses to slow memory as much as possible. Those solutions are application-specific, but bring much higher performance than system-level solutions; Those solutions focus on critical and common applications, which justifies highly customization of the solutions for those applications. Both the system-level solutions and application-level solutions investigate principles on how a large amount of memory pages should be profiled to capture spatial and temporal localities without paying large overhead and how page migration should happen to fully utilize fast memory.

Broader impact. This project enables applications to fully tap the large memory capacity provided by HM. Some of those applications are critical to the nation interests (such as the DOE application WarpX, a large-scale plasma simulation code); Some of them are critical to the business (such as deep learning training and fast information retrieval). With our solutions, those applications are able to run in unprecedented scales on a single machine, even performing better than on multiple machines. This project has been highlighted by several medias and companies (e.g., towardsdatascience.com, Microsoft, and linkreseacher.com). This project lays foundation for many HPC applications (including compute-intensive applications with small memory) to leverage HM with large memory capacity. This project is among the first efforts that reveal using limited application semantics can be significantly helpful to improve application performance on HM.

Furthermore, this project provides research opportunities to undergraduate students to gain hands-on experiences on software-hardware co-designs. This project is also based on collaboration with Lawrence Berkeley National Lab and Lawrence Livermore National Lab. The project has impacts on how the future supercomputer infrastructure should be built. Collaborating with the national labs, we provide training opportunities to graduate students and prepare them for future career in the HPC field. Since the HPC field, which is critical to the national interests, lacks workforce. Our project is helpful to address this pressing problem.

Last Modified: 04/13/2021
Modified by: Dong Li