Today's networks rely on many human-engineered control and resource management heuristics, but the complexity and heterogeneity of modern systems make it difficult to design effective heuristics for different settings. This project developed principled approaches for building systems that autonomously learn to make resource management decisions and adapt to various environments without human intervention.

Intellectual Merit

Learning-based Systems and Benchmarks

Pensieve: A system that uses reinforcement learning (RL) to adapt video bitrate algorithms based on the performance of past choices, tailoring control policies to network and video characteristics.

Decima: An RL-based job scheduler for data processing clusters that learns workload-specific scheduling policies for jobs represented as dataflow graphs. Decima's learned policies outperform standard heuristics, providing significant cost savings by enabling large-scale data processing clusters to run at higher utilization. To invent Decima, we developed new techniques, including a variance reduction method for robust RL training in environments with stochastic input processes.

Other contributions in this area include:

• Placeto: An RL system for learning generalizable task placement strategies for parallelizing neural network training jobs on multiple GPUs.

• Neo, Bao, Flow-loss: Learning-based relational query optimizers and robust learned cardinality estimation for database management systems.

• Park: An open benchmark suite for research on learning-augmented computer systems.

Advances in Simulation and Modeling

CausalSim: Addresses a fundamental problem in trace-driven simulation, where replaying an observed traces, such as network throughput measurements, can be flawed because interventions (such as new protocols) that are simulated could have changed the trace. CausalSim uses a novel causal inference algorithm to learn the relationship between interventions and trace observations, modifying the trace during simulation to reflect the effect of interventions. 

m3: A scale-free, fast, and accurate model for estimating the tail latency of network transfers in datacenters. m3 trains a neural network to approximate the packet-level simulation function, using a simple flow-level simulator to generate a compact feature map capturing the important characteristics of a network scenario.

Continual, Online Learning in Systems and Their Applications

Many systems must operate in non-stationary environments where workload or operating conditions change over time. Learning-based systems can adapt their models over time to continually optimize for current conditions, enabling the system to leverage simpler models that perform well over a narrower set of inputs and enhancing robustness by reducing reliance on training data coverage.

MMNet: Revisits MIMO detection from an online learning perspective, using a neural network architecture based on iterative soft-thresholding algorithms and an online training algorithm that exploits the locality of channel matrices, accelerating training by more than two orders of magnitude.

AMS: Offloads retraining of specialized DNNs for video analytics to a remote server that continually adapts a small student model running on the edge device for real-time video. AMS introduces techniques to reduce the bandwidth required for over-the-network model adaptation.

RECL: Reduces compute resources required for model retraining in video analytics by reusing specialized DNNs whenever possible, enabling systems to scale to more video streams with limited resources.

SRVC: Combines existing video compression algorithms with a lightweight, content-adaptive super-resolution (SR) neural network, significantly boosting performance with low computation cost. SRVC compresses input video into two bitstreams: a content stream and a model stream, dynamically specializing the SR network for short video segments to achieve significant compression efficiency.

Broader Impacts

This project resulted in 20 technical papers published at top venues in the networking, systems, and machine learning communities. Projects like Pensieve, Decima, and Neo are among the earliest successes in learning-based systems, spurring considerable follow-on research and serving as benchmarks for evaluating new ideas in learned systems.

Our research on learning-based systems has led to broader results in machine learning. For instance, in Decima, the stochastic nature of real-world workloads creates significant variance in the reward signal, making it difficult for RL algorithms to assess the quality of different actions. Decima introduced input-specific baselines to tackle this problem, a concept we generalized for Input-driven Markov Decision Processes where an exogenous, stochastic input process impacts system dynamics. CausalSim led to a theoretical study of causal models with bijective generation mechanisms (BGMs), proving counterfactual identifiability of BGMs in several settings and proposing a practical learning algorithm that generalizes CausalSim's method.

This project has helped several graduate students and postdocs. Five PhD. students have transitioned into the technology industry and one postdoc is now in a faculty position. 

Last Modified: 06/29/2024
Modified by: Mohammad Alizadeh