This project under the NSF InTrans (Innovation Transition) Program extended the previous project “Customizable Domain-Specific Computing” awarded in 2009 under the NSF Expeditions in Computing Program, which looked beyond parallelization and focused on domain-specific customization as a new disruptive technology for significant power-performance efficiency improvement. The proposed research in both projects explored the extensive use of accelerators, as custom-designed accelerators often provide 10-100X performance/energy efficiency over the general-purpose processors. Such an accelerator-rich architecture (ARA) presents a fundamental departure from the classical von Neumann architecture, which a common pipeline for the execution of different instructions, providing an efficient solution when the computing resource is scarce. In contrast, the ARA features heterogeneity and customization for energy efficiency; this is better suited for today’s technology where the silicon resource is abundant and energy consumption is the limiting constraint. This project greatly influenced the research community and the computing industry in making recent shift towards customizable computing, e.g. evident by the large-scale deployment of FPGAs in Microsoft’s and Amazon’s datacenters.

In this project, the customization is carried out at multiple levels of computing hierarchy, including the chip-level, server-node level, and datacenter level. At the chip-level, the research proposed the use of customizable ARAs, where a sea of heterogeneous accelerators can be programmed and composed for customized acceleration, in junction with a customizable memory hierarchy and network-on-chip (see Figure 1). It demonstrated orders-of-magnitude energy efficient improvement computational kernels in multiple application domains.  At the server-node level, it investigates various CPU+FPGA computing platforms, where off-the-shelf FPGAs (field-programmable gate-arrays) can be used to implement customized accelerators in a flexible and cost-efficient way.  It provided detailed quantitative characterization of multiple commonly used CPU-FPGA platforms, and demonstrated substantially speedups on a wide range of applications, from string matching, to data compression, and to deep learning.   At the datacenter level, it explored multiple heterogeneous datacenter prototypes with FPGA accelerators, including CPU-only clusters with Xeon or Atom processors, a Xeon cluster with FPGA accelerator attached to the PCI-E bus, and a cluster of embedded ARM processors with on-chip FPGA fabrics (see Figure 2).  The research demonstrated several times energy reduction with FPGA acceleration for multiple big-data applications when compared to the baseline CPU-only cluster.

In parallel to the ARA architecture exploration, this research devoted substantial effort to develop the compilation and runtime support for ARAs.  In particular, significant progress has been made on source-to-source transformations and optimizations, such as the automated generation of various micro-architecture templates (e.g. see Figure 3) and automated design space exploration, so that one can make effective use of modern high-level synthesis tools for high-performance accelerator designs with high productivity.   It also developed the efficient runtime systems for deploying FPGA and GPU accelerators into state-of-the-art big-data computing frameworks like Hadoop and Spark. It supports both node-level accelerator management for CPU+FPGA+GPU co-scheduling (see Figure 4) and cluster-level accelerator management for global accelerator sharing/management.

Finally, this project demonstrated the benefits of ARAs in two important healthcare application domains.   One is precision medicine, where the research focused on accelerating the next-generation sequencing alignment and analysis pipeline for discovering the genomic causes of various diseases, which suffered long computation time. The FPGA and GPU accelerators developed in this project lead to 5-25X acceleration for multiple compute-intensive kernels in the pipeline (e.g. see Figure 5).  Another application domain is medical imaging.  In particular, the research focused on the automated detection of lung cancer, including 1) super-resolution enhancement of suspected indeterminate pulmonary nodules (IPNs), denoising the appearance of nodules to better characterize the spatial/visual properties for classification as benign/malignant; and 2) convolutional neural networks for identifying IPNs in low-dose computed tomography studies (see Figure 6). These efforts are helping to advance strategies for making lung cancer screening both more consistent and scalable through computer-aided detection techniques powered by various customized accelerators from this research.

In terms of broader impact, this project involved 31 graduate students and 5 postdocs.   The multi-disciplinary nature of the research enabled these young researchers and engineers to be well positioned as the leaders in the new era of heterogeneous accelerated computing.    Moreover, this project led to one book, over 30 journal publications, over 50 conference publications, many invited talks and keynote speeches, which greatly facilitated the wide dissemination of the knowledge obtained from this research.  It also led to multiple open-source releases, such as the ARA simulator, various accelerator implementations, multiple source-to-source compilation tools and runtime management tools for FPGA and GPU accelerator design and deployment, and several application pipelines.   At the start of this InTrans award, Intel Corporation was the sole co-sponsor.  At the end of this project, there are over ten companies making financial contributions at various levels,  providing another channel of know transfer with the impact to the industry.

Last Modified: 10/18/2018
Modified by: Jason Cong