The goal of this project was to enable better performance, availability, and cost-effectiveness for web services deployed in the cloud. Specifically, our goal was to harness the benefits of spreading an application's deployment across the data centers of multiple cloud providers, as opposed to the default practice of using a single cloud provider.

Intellectual Merit:

The research contributions made in this project are primarily on three fronts.

First, we demonstrated that distributing an application's data across multiple cloud providers can offer better cost versus performance tradeoffs, than when using a single provider, for two reasons: 1) the union of data centers in multiple providers offers a greater geographical density of data centers, thereby requiring fewer copies of every data item to meet latency bounds or making previously infeasible latency bounds achievable, and 2) the discrepancies in pricing across providers, if exploited judiciously, can lead to significant cost savings without any performance penalties.

Second, by conducting a large-scale measurement study of popular cloud providers, Amazon Web Services and Microsoft Azure, we showed that there is significant latency variance when using their storage services, owing to the multi-tenant nature of these services. Our analysis of these measurements revealed that the variance in latencies stems from many sources: latency on Internet paths, latency within the data center network, and even latency within the storage service. Therefore, to cost-effectively overcome the latency variance inherent to cloud storage services, we showed that it is key to combine the use of multiple forms of redundancy (e.g., store copies of an object in multiple data centers and issue multiple requests to each copy). Utilizing multiple cloud providers proved to be key here so that, for an object stored in one data center, redundant requests can be issued to copies stored in a nearby data center from a different provider than a distant data center from the same provider.

Third, to make the benefits of multi-cloud storage accessible to a broad class of applications, we demonstrated how to enable globally consistent replication of data across data centers without requiring any support from applications or from cloud providers. Though multiple cloud providers have no incentive to sync data across each other's data centers, we showed that it is feasible to match the performance that would be feasible if there hypothetically existed a geo-distributed storage service that spans providers. Key to achieving this property was our redesign of existing protocols for executing reads and writes on replicated data without sacrificing data consistency. By accounting for the limited interface offered by cloud storage within each data center, our new replication protocol enables a client-side library to efficiently perform globally consistent accesses of data which is replicated across the data centers of multiple cloud providers.

Broader Impact:

The work done on this project formed the core of one PhD student's dissertation. The findings from this project have been communicated to the research community via several top-tier conference publications as well as to the industry via talks at cloud service providers such as Microsoft, Google, and Nutanix. Moreover, we have publicly shared several measurement datasets gathered as part of our work, so that other researchers can also benefit from our measurements.

The PI created a new undergraduate class on distributed systems at the University of Michigan. All of the lecture materials from this class have been posted on a public website for the benefit of students and instructors at other institutions. The PI also gave lectures on distributed computing at Big Data Summer Institutes organized at the University of Michigan.

Last Modified: 06/30/2018
Modified by: Harsha V Madhyastha