Computer designers have traditionally separated the role of storage and compute units. Memories stored data. Processors’ logic units computed them. Is this separation necessary? A human brain does not separate the two so distinctly. Why should a processor? Over two-thirds of a processor die (caches) and all of main memory is devoted to temporary storage. Today, none of these memory elements can compute. But, could they?  Our research, with this NSF CAREER Award, addresses this fundamental question regarding the role of memory and proposes to impose a dual responsibility on them: store and compute data. 

Data stored in memory arrays share wires (bit-lines) and signal sensing apparatus (sense-amps). We observe that logic operations can be computed over these shared structures which allows us to re-purpose thousands of cache memory arrays into over a million of bit-serial arithmetic-logic units. Thus, we morph existing memory into massive vector compute units, providing parallelism several orders of magnitude higher than a contemporary GPU.  Additionally, it saves energy spent shuffling data between storage and compute units -- a significant concern in BigData applications. In-memory computing is a significant departure from processing-in-memory (PIM) technologies which do not reuse memory structures for computing, but simply move conventional compute units near memory.

Caches that compute can be a game changer for Artificial Intelligence (AI). They can add accelerator capabilities to general-purpose processors, without the significant die area cost of a dedicated accelerator like Google’s TPU. For example, we showed that compute-enabled caches in Intel Xeon can improve processor efficiency by 629 times for convolutional neural networks (CNNs). This result received significant attention from the industry. 

 Emerging non-volatile memories such as Resistive Memory (ReRAM) can also be repurposed into very large vector parallel units. We asked, could in-memory computing units be used as general-purpose data-parallel accelerators, much like GPUs? Our research showed that the answer is a resounding yes. We built novel compiler technology that can transform arbitrary data-parallel computation expressed in Google’s TensorFlow into binaries that can run directly on ReRAM. We showed that in-memory computing can be over 700x more efficient than GPUs.  

 Finite state automata (FSA) are widely used as a computation model for approximate/exact regular expression matching in several application domains such as data analytics, network security, bioinformatics, and computational finance. We proposed a new way to repurpose DRAM memory as a Finite State Automata (FSA) accelerator. It addressed a very hard problem: parallelization of inherently sequential FSA. Our follow-up work extended this solution to turn processor caches in the FSA accelerator. To achieve this, we developed novel SRAM-based interconnects for automata processing. Our solution provides a speedup of 25x compared to state-of-the-art custom accelerators and more than 3000x compared to off-the-shelf stock processors.  

The automata research led us to Genomics, which was a killer application for FSA acceleration. Genomics can transform precision health over the next decade. We can detect cancer several years earlier through simple blood tests, without invasive biopsies. We can identify infectious pathogens and avoid the indiscriminate use of broad-spectrum antibiotics. In the last few years,  partially supported by this award, we have made significant advancements in accelerating whole genome sequencing and ultra-rapid real-time sequencing for cancer diagnostics.  We have demonstrated that hardware-software co-design approaches can provide orders of magnitude acceleration in some of the most time-consuming steps in Genomics.

This project yielded several notable outcomes with broad impacts. Firstly, it supported two students in earning their PhD degrees in Computer Science and Engineering from the University of Michigan. It also partially supported several other students, including undergraduate research efforts. Secondly, the PI collaborated closely with Intel Corp. for technology transfer and also co-founded a precision health startup. Third, the PI created a permanent EECS undergraduate parallel programming class being taught to students at the University of Michigan.  Lastly, the research was widely disseminated through numerous conferences and journal publications, seminars at universities and industrial research labs, and the creation of benchmarks and artifacts to facilitate further research by the broader community.

Last Modified: 07/09/2024
Modified by: Reetuparna Das