Modern data centers run various applications with different performance needs, ranging from highly latency-sensitive applications to throughput-sensitive batch (big data and machine learning) workloads. To meet the growing performance demands of these applications, data center operators are scaling up network speeds. However, ensuring that the applications can drive server network interfaces (NICs) at full line rates while not impacting each other’s performance objectives, and while maintaining optimal levels of server/network utilization and efficiency, continues to remain a challenge. The main reason is that existing solutions—such as hardware/software parallelism to aid application multiplexing, and offloading application computation onto network hardware—serve as specialized point optimizations. Because of loose coordination across layers of the network stacks where these techniques apply, they can often lead to detrimental outcomes such as high and variable latency, unreasonable and wasteful CPU utilization, and severe network congestion.

To allow multiple applications on a server to drive the NIC at high speed, our project aims to rethink the mechanisms employed in various layers of the server networking stack and create new interfaces and abstractions between the layers and applications. The goal is to sustain high line rates while meeting application latency/throughput/fairness objectives and infrastructure-wide utilization and efficiency goals.

Intellectual merit: Toward this goal, our project made several successful breakthroughs. We developed a new NIC that allows application computation to be offloaded from CPUs to network hardware, enabling optimal CPU utilization and high application and network performance. We created hardware modules that can be hosted on this NIC to host other performance- and resource-intensive functionalities such as load balancing, core allocation, and application scheduling. Finally, we showed how the NICs can host data-intensive applications spanning big data processing and machine learning domains. Our impacts and advances not only networking, but also operating systems, computer architecture, and programming languages.

Broader impacts: Results from this project have been successfully incorporated into graduate and undergraduate courses at UW Madison and UT Austin. The outcomes of this work are now being considered for production deployments at hyperscaler networks including Google and Microsoft. The project contributed to the PhD thesis of four students across UW and UT, including two female PhD students. It also led to the honors thesis of two undergraduate students at UT Austin.

Last Modified: 12/15/2023
Modified by: Aditya Akella