
NSF Org: |
OAC Office of Advanced Cyberinfrastructure (OAC) |
Recipient: |
|
Initial Amendment Date: | January 11, 2022 |
Latest Amendment Date: | July 9, 2025 |
Award Number: | 2144764 |
Award Instrument: | Continuing Grant |
Program Manager: |
Juan Li
jjli@nsf.gov (703)292-2625 OAC Office of Advanced Cyberinfrastructure (OAC) CSE Directorate for Computer and Information Science and Engineering |
Start Date: | June 15, 2022 |
End Date: | May 31, 2027 (Estimated) |
Total Intended Award Amount: | $500,000.00 |
Total Awarded Amount to Date: | $395,476.00 |
Funds Obligated to Date: |
FY 2025 = $102,050.00 |
History of Investigator: |
|
Recipient Sponsored Research Office: |
4400 UNIVERSITY DR FAIRFAX VA US 22030-4422 (703)993-2295 |
Sponsor Congressional District: |
|
Primary Place of Performance: |
4400 UNIVERSITY DR FAIRFAX VA US 22030-4422 |
Primary Place of
Performance Congressional District: |
|
Unique Entity Identifier (UEI): |
|
Parent UEI: |
|
NSF Program(s): | CAREER: FACULTY EARLY CAR DEV |
Primary Program Source: |
01002526DB NSF RESEARCH & RELATED ACTIVIT 01002223DB NSF RESEARCH & RELATED ACTIVIT |
Program Reference Code(s): |
|
Program Element Code(s): |
|
Award Agency Code: | 4900 |
Fund Agency Code: | 4900 |
Assistance Listing Number(s): | 47.070 |
ABSTRACT
Video analytics plays a pivotal role in science and engineering monitoring. Monitoring videos captured by remote cameras are typically live streamed to servers for analysis because of the limited computational capabilities of camera devices. From wildlife tracking and coastline event detection to airport suspect recognition and victim search in disaster response, such automated video analytics systems have been deployed widely to assist human operators. The recent advancement of 360 degree cameras enables a new paradigm of panoramic video analytics that can cover the 360 degree surroundings of a monitoring site and can address the errors in and missing analysis abilities of traditional 2D video analytics. However, realizing this vision requires live streaming massive panoramic video data to servers for online analytics, which cannot be supported by the current cyberinfrastructure (CI). The mismatch between the 360 degree video bit rate and available network bandwidth can cause lagging or failed analysis, diminishing the benefits of panoramic video analytics. This project will create a framework of video compression, streaming, and recovery for achieving the vision of panoramic video analytics in science and engineering monitoring. The new CI will allow scientists and engineers to conduct online panoramic video analytics and enable innovative applications that are otherwise unattainable. The research outcomes will support the development of a remote learning tool for imaging analytics, course curriculum and undergraduate research in media computing, and educational videos for public outreach.
This project investigates a machine centered video computing framework in order to enable online panoramic video analytics. Unlike traditional human centered video frameworks where pixels are processed to preserve extensive aesthetic details for human viewing, the proposed CI compresses, streams, and recovers feature points for machine analytics. Because of this fundamental change, the proposed framework is able to greatly outperform legacy video CIs and support panoramic video analytics. To this end, a deep learning based 360 degree video codec will be built to distill the spatiotemporal characteristics of video features and optimize both compression ratio and analytics accuracy. Second, an adaptive 360 degree video bitrate streaming system will be designed to ensure continuous delivery of full 360 degree video frames by prioritizing regions of interest preferred by machines. Third, a 360 degree video recovery scheme will be developed to restore noisy and delayed video data while considering the time constraints in the online analytics models. Finally, interdisciplinary collaboration will be done with application area scientists and engineers to carry out the project plans for evaluation and validation of the panoramic video framework on real world problems.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
Please report errors in award information by writing to: awardsearch@nsf.gov.