Cloud computing promises vast compute, storage, and communication capabilities to individual users as abstract services for very low financial cost. However, the attractive pricing offered by public clouds introduces “hidden” costs in terms of privacy, reliability, and flexibility that are difficult to mitigate. Public cloud vendors seek to create barriers to exit (“lock-in”) for their customers as a way of ensuring revenue long term while offering their services in an on-demand market.  However, once a customer's data is ensconced in a public cloud, it quickly becomes difficult or infeasible to move it, delete it, or access it from different providers.

We address this “data gravity” with new research investigations into DatGeo -- a geo-distributed datastore that is designed to facilitate the federated, concurrent usage of multiple clouds. DatGeo addresses privacy concerns by allowing data to be partitioned among separate cloud providers, but accessed via a single datastore API at the cloud platform level.  It supports cross-site datastore transactions using Replicated Commit to implement geo-distribution. It also extends the datastore to include facilities for geo-partitioning to enhance privacy and geo-replication to enhance reliability beyond what is available from a single cloud provider.  Our approach has been empirical and systems based, resulting in significant open source/data research software artifacts as well as publications that we believe have furthered our collective understanding of federated data management.

Specifically, over the lifetime of this grant, our team has developed a number of new approaches for data management at a global scale.  This includes new approaches to data caching, placement, replication, and scalability as well as support for multi-cloud use (with new approaches for efficient monitoring, anomaly detection, and management of cross-cloud dependencies).  We have also investigated advances that have leveraged permissionless and permissioned blockchains in new ways to facilitate failure tolerance, distributed deployment sharing, and energy efficient optimization.  Our team has also leveraged these building blocks to develop new adaptive scheduling systems for machine learning applications and programming systems that enhance programmer productivity. Finally, our research has been informed by and validated using real applications of societal import, including those from the domains of digital agriculture, environmental sustainability, and smart homes. Our  research artifacts have been made available as open source software and data,  and we have published our work in over 50 high quality venues.  Moreover this project as enabled us to train a large number of students from diverse backgrounds in the next generation of distributed systems. 

Last Modified: 09/01/2023
Modified by: Chandra Krintz