Modern workloads feature data-dependent computation and control flow patterns that often vary considerably with changing datasets and configuration parameters. This has greatly reduced the effectiveness of conventional compiler optimizations that rely on hoisting program invariants identifiable at compile-time to eliminate redundant computation and further apply machine-specific optimizations to the residual code.  Run-time optimizations have largely been profile-guided and conservative in nature, limiting their ability to adapt to changing execution profiles as datasets evolve, notwithstanding high profiling overheads.

The project makes the key observation that modern datasets offer remarkable predictability, because data access and control flow patterns in many workload-dataset pairs manifest as invariants across long execution intervals, unmasking an untapped pocket of opportunity for speculatively optimizing such code paths at run-time, by exposing and eliminating dead code, entirely within the processor, based on dynamically predicted data and control invariants, enabling the continuous optimization of inherently sequential code, without the need for profiling, compiler metadata, or other source-level information.

Speculative Superoptimization is a minimally-invasive, prediction-driven microarchitectural technique that advances state-of-the-art dynamic binary optimization schemes in several important ways. First, it leverages the rich contextual knowledge available from a wide array of in-processor speculation techniques such as branch prediction, address prediction, and value prediction to track deterministic computational patterns in hot code regions and further transform an instruction sequence in execution into a compact stream of speculatively optimized instructions.  Second, it opens the door to the deployment of aggressive and speculative optimization techniques that have been traditionally deemed unsafe, thanks to the processor's ability to rollback execution to a safe point in the event of a misprediction by flushing speculatively optimized instructions and redirecting execution to the corresponding stream of unoptimized instructions that are co-hosted in the processor's micro-op cache.  Third, by exploiting predictability as the basis for optimization, it offers seamless adaptability to changing datasets and workload patterns, unlike existing runtime optimizations employed by software giants such as Google, Microsoft, and Meta that either require extensive profiling or tend to make conservative assumptions when they lack adequate context about the dynamic execution behavior.

The results from this work have been disseminated through top-tier conference and journal publications, such as MICRO, ISCA, and USENIX Security, and through tech talks at industry and academic institutions alike.  In addition, the software artifacts for this work have been made available through Github, for greater community use.  Notably, the simulator release includes a faithful implementation of a modern x86 front-end (including the micro-op cache and micro-op fusion) along with new speculative features such as value prediction and loop stream detection, providing researchers a strong foundation to build on.

The PIs also regularly teach graduate and undergraduate Computer Architecture courses with a significant emphasis on quantitative methods and performance analysis.  The PIs also offer specialization courses on hardware security and acceleration.

The PIs are also extremely committed towards diversity efforts in Computer Science and other STEM disciplines.  They provided research mentorship to three graduate students from India, China, Iran, and the US, in addition to multiple undergraduate students, one of whom is a female student who was recognized nationally by CRA's outstanding undergraduate researcher award honorable mention.

Last Modified: 04/29/2024
Modified by: Ashish Venkat