New applications that promise to revolutionize how people interact with technology in their daily lives have unique characteristics such as strict latency requirements and frequent mobility that are ill suited for today's centralized clouds. We designed a new Mobile Elastic Edge Cloud (ME2C) architecture to provide a programmable, converged architecture that can be customized for the diverse needs of emerging applications.

One of the primary contributions is to provide support on an edge platform for machine learning applications which require the computational support provided by GPU accelerators at the edge. Since many real-time applications require low-latency, high throughput machine learning (ML) functions resident in the network and in the cloud to perform learning and inference, having sufficient computing capability at the edge can be a significant enabler. We developed a better multiplexing on GPUs on edge cloud platforms to support running ML applications, especially Deep Neural Networks on high-end GPUs, such as the nVIDIA V100 GPUs. We created GSLICE, a framework to have controlled spatial and temporal sharing of the GPU by multiple DNNs. We have a true virtualization of the GPU that can be used my many of the applications using the edge cloud. We also used this to improve the tuning of DNNs, using Slice-Tune, which utilizes GSLICE as a base framework.

The second area we have worked on has been to provide function as a service (serverless computing) on resource-constrained edge platforms. We have worked on Container Network Interface (CNI) plug-in evaluations, developing a better framework for autoscaling and function placement as well as load balancing in serverless edge computing, and also looking at using the extended Berkeley Packet Filter (eBPF) as a way to develop an improved side car for serverless computing in KNative. We extended the Knative serverless framework to improve its performance, scalability, and fairness for an edge cloud computing serverless platform that we call "Mu", primarily focused on the control plane. We also looked at supporting IoT workloads on edge clouds using Knative serverless frameworks and how to improve the support for it. We also looked at how CNIs can be improved to provide fast initialization of virtualization for an agile edge cloud computing environment. Finally, we developed an eBPF-based shared-memory framework called SPRIGHT that substantially enhances the data plane for serverless computing.

A third area was to develop improved energy management in data centers, in particular leveraging the affinity of functions that communicate to place and migrate them so that related functions that communicate frequently are co-located. Our work in this area involved improving power management in Data Centers, especially smaller data centers. There were several aspects we worked on on. We combined Dynamic Voltage and Frequency Scaling (DVFS) techniques to take advantage of slack in latency of search queries and also Machine Learning techniques to predict the per-query service time and the error in the prediction to determine the right time to boost the CPU frequency.  We also worked on improving our framework for improved consolidation of data centers while carefully factoring in the cost of migration of containerized applications from one server to another in a data center using graph-partitioning to improve placement and migration of containers.

We have published several papers in all these areas. Two students got their PhDs (one advised by me, one advised by another colleague although I worked a lot with the student) and a 3^rd PhD student is close to graduation at the time the project ended. I also supported a post-doctoral fellow who went on to a faculty position. The project also supported 3 undergraduate student researchers as part of a REU supplement. One undergraduate student (from an under-represented minority) worked on understanding and evaluating cloud orchestration environments and container networking interfaces (CNI). A second undergraduate student worked on understanding how to use edge clouds for automatic speech recognition, especially in the context of emergency management. A third undergraduate student examined using edge clouds, and shared memory for performing aggregation for Federated Learning. Multiple MS students also worked on projects related to this grant.

The research on this project was done in close collaboration with our partner, Prof. Tim Wood of George Washington University. The project has led to several publications in international conferences and also journal publications. I believe that our ideas on GPU multiplexing will significantly improve how GPUs are used in edge clouds. Similarly, the control plane and data plane development will improve KNative and other Kubernetes based cloud environments to have a better autoscaling algorithm, placement/scheduling and an improved shared-memory based data plane. The use of eBPF to improve the sidecar functionality of serverless frameworks will also have long term impact.

Overall, this has been a very successful series of efforts supported by the NSF grant, and we are very grateful to NSF for this support.

Last Modified: 06/29/2023
Modified by: Kadangode K Ramakrishnan