ABSTRACT

Computational imaging is a field that combines optics, electronics, and computational processing to capture new forms of visual information. The theoretical underpinning of any computational imaging technique is its model for light propagation. Vision and graphics research generally adopts a geometric optics approximation of light transport, where light travels along straight lines until it undergoes a scattering event or gets absorbed. At small scales, the wave-like properties of light become far more noticeable and require modeling light as a wave at the cost of additional computational overhead. In practice, imaging systems are neither perfectly incoherent (as in geometric optics) nor completely coherent (as in wave optics). Light is more aptly modeled as a state mixture: a statistical ensemble of coherent waves. This project develops a comprehensive theory and corresponding imaging techniques to leverage state mixtures for a wide variety of imaging applications, including the development of new computational microscopes, 3D sensors, and optical vibration sensors. Moreover, this project connects techniques used in both the incoherent and coherent settings, drawing upon imaging research developed across multiple different research communities. In addition, the project will develop a low-cost, open-source projector-camera platform to easily implement a collection of important computational imaging techniques. This platform will be used to educate students at all levels (graduate, undergraduate, and K-12).

This project tackles three independent, but tightly coupled, research thrusts focused on (1) mixed-state sensing, (2) mixed-state illumination, and (3) mixed-state scene analysis. The first thrust explores state mixtures observed at the sensor. Many imaging techniques depend on the incident light being coherent (such as in coherent diffraction imaging), and do not account for the incident light having multiple wavelengths or polarization states. This thrust develops flexible imaging systems and reconstruction algorithms that can uniquely recover complex fields from incoherent mixtures of multiple coherent fields. The second thrust investigates a new form of transport probing based on programming mixed-state illumination. Transport probing provides a camera the ability to enhance or attenuate specific light paths. This thrust develops a fundamentally different approach to probing that involves selectively interfering pairs of light paths, which has applications in 3D sensing and the separation of direct and global light paths. The third thrust uses mixed-state scene analysis techniques to infer scene dynamics, which includes optically amplifying and measuring the imperceptible vibrations of objects. A focus of this thrust is to capture vibration measurements at higher spatial resolutions, in a more light-efficient fashion, and across up to six dimensions of motion, providing more and better information about visual scenes.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Please report errors in award information by writing to: awardsearch@nsf.gov.