ABSTRACT

This proposal enables low-power edge computers, such as mobile phones, drones, and Internet-of-Things devices, to benefit society. Computer vision is the technology to automatically analyze images and videos. Computer vision on these devices can keep humans safe, for example by spotting dangers in a factory or at a construction site. This project addresses two challenges that hamper practical adoption of computer vision on edge devices. The first challenge is that current computer vision approaches require powerful computers, but these computers are too far away and have long response time. This project brings the computers to the places where data is acquired. The project makes computer vision more efficient, so that visual data can be analyzed by small edge devices like phones and drones. The second challenge is that building complex software for computer vision is difficult. This project provides software engineering support for emerging computer vision technologies. As a result of addressing these two challenges, computer vision on the edge can become feasible.

Bringing computer vision (CV) to devices on the network edge is an essential component of realizing NSF's goal of distributed cyberinfrastructure. This project makes CV on the edge feasible and enables scientific and engineering innovation through improved response time, reduced need for network coverage, and decreased storage costs. This project solves two critical challenges that hinder the transition of edge-based CV into practice. (1) This project makes CV more efficient and edge-friendly. Current CV techniques (e.g., deep neural networks) assume server-class resources (such as graphics processing units, gigabytes of memory); these resources are not available at the edge. This project reduces the resource requirements needed for CV. The methods consider alternative neural network architectures and eliminate redundancies while processing visual data. This project also develops CV-specific distribution techniques to enable edge devices to collaborate on large vision tasks. (2) This project provides software engineering support for CV technologies. Solving real-world CV problems requires engineering new CV applications, often by re-implementing research model architectures as components in new designs. This project develops a library of exemplary CV model implementations for low-power platforms. These exemplars can be used as high-quality components in new CV applications. The project identifies factors that promote and inhibit the reproducibility of CV models. This project also identifies engineering best practices by surveying and interviewing experts in low-power CV and by studying their errors.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Please report errors in award information by writing to: awardsearch@nsf.gov.