Machine learning (ML) has emerged as an important workload for cloud platforms and data centers given its wide variety of compelling use-cases, such as weather prediction, intelligent chatbots, self-driving cars, coding and debugging, and many others.  Training machine learning models is expensive, as it often requires processing large-scale datasets across many distributed servers for long periods. Thus, training large-scale ML models is often done on cloud platforms, which have access to the necessary resources.  To reduce cost, cloud platforms offer transient servers, which they may revoke at any time, but are 70-90% cheaper than traditional on-demand servers. Transience may also arise in other contexts, such as in edge devices with unreliable energy sources or network connectivity. Unfortunately, leveraging low-cost transient servers introduces challenges for training distributed ML, as it often uses a bulk-synchronous processing model, which causes revoked servers to block others from continuing past pre-defined synchronization barriers until a newly provisioned replacement server reaches the barrier.  Unfortunately, while asynchronous processing can eliminate this blocking and improve system performance, it does not maintain the algorithmic properties of synchronous algorithms.  To address the problem, this project developed models, algorithms, and frameworks for both understanding and optimizing distributed ML training on transient servers. Specifically, the project developed 1) a number of models that quantify the impact of transient servers on distributed ML in various contexts as a function of revocation rates, degree of parallelism, synchronization method, and straggler mitigation technique; 2) multiple frameworks that use different synchronization models and fault-tolerance techniques for mitigating the performance impact of transient server revocations, including asynchronous and flexible synchronous models; 3) implementations using the frameworks of a number of common ML algorithms to demonstrate their performance improvement under transient server revocations.  The techniques the project developed increased the understanding of how to optimize distributed ML when running on cheap transient servers in the cloud with frequent revocations, which can significantly reduce the cost of training ML models. 

Broader impacts of the project included advancing the state-of-the-art in training ML models at low-cost, which is increasingly important given the accelerating demand for ML since the project's inception. The project's results were disseminated to the community through a large number of presentations and publications on the project's models, algorithms, and frameworks in multiple contexts.  The project also supported the incorporation of topics on transient servers, distributed ML, and synchronization methods into undergraduate and graduate courses. The project supported lectures each year on cloud computing for multiple high school summer programs for students interested in computer science and engineering. The project partially supported seven research assistants.  One doctoral student supported through this award received the department's Outstanding Dissertation Award and joined industry as software engineer after graduation. 

Last Modified: 01/27/2025
Modified by: David Irwin