| NSF Org: |
IIS Division of Information & Intelligent Systems |
| Recipient: |
|
| Initial Amendment Date: | August 15, 2017 |
| Latest Amendment Date: | August 15, 2017 |
| Award Number: | 1748387 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Jie Yang
jyang@nsf.gov (703)292-4768 IIS Division of Information & Intelligent Systems CSE Directorate for Computer and Information Science and Engineering |
| Start Date: | August 15, 2017 |
| End Date: | July 31, 2020 (Estimated) |
| Total Intended Award Amount: | $200,000.00 |
| Total Awarded Amount to Date: | $200,000.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
1850 RESEARCH PARK DR STE 300 DAVIS CA US 95618-6153 (530)754-7700 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
One Shields Ave Davis CA US 95616-5200 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): | Robust Intelligence |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
This project investigates how synthetic data created using computer graphics can be used for developing algorithms that understand visual data. Synthetic data provides flexibility that is difficult to obtain with real-world imagery, and enables opportunities to explore problems that would be difficult to solve with real-world imagery alone. This project develops new algorithms for reasoning about object occlusions, and for self-supervised representation learning, in which useful image features are developed without the aid of human-annotated semantic labels. The project provides new algorithms that have the potential to benefit applications in autonomous systems and security. In addition to scientific impact, the project performs complementary educational and outreach activities that engage students in research and STEM.
This research explores novel algorithms that learn from synthetic data for visual reasoning and representation learning. While the use of synthetic data has a long history in computer vision, it has mainly been used to complement natural image data to solve standard tasks. In contrast, this project uses synthetic data to make advances in relatively unexplored problems, in which ground-truth is difficult to obtain given real-world imagery. The project consists of three major thrusts, each of which exploits the fact that a user has full control of everything that happens in a synthetic dataset. In Thrust I, it investigates a novel approach to representation learning using synthetic data, and in Thrust II, it extends the algorithm to disentangle task-specific and general-purpose features. Finally, in Thrust III, it explores a novel approach for reasoning about object occlusions.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
The goal of this research project was to explore the use of synthetic visual data for visual scene understanding. In synthetic data, the various properties of objects, scenes, lighting, physics, etc. are fully controllable. The main idea was to create algorithms that can leverage this property for improved visual recognition. As part of this goal, we investigated novel discriminative approaches for leveraging synthetic data to learn robust visual representations, and novel generative approaches for generating realistic synthetic image and video data.
In terms of intellectual merit, there are three key areas of technical contributions. The first is the development of novel generative models that can learn disentangled representations with minimal supervision; specifically, learning to disentangle background, object shape, appearance, and pose for controllable synthetic image generation. The second is the development of novel weakly-supervised visual learning approaches that leverage synthetic data and external knowledge bases for representation learning and object detection. The third is the development of new approaches for privacy and security applications, including a video anonymizer and fake credit card synthesis and analysis method. The work produced 10 peer reviewed papers in top-tier computer vision, machine learning, and security conferences, and new publicly available codebases for the algorithms which are linked from https://www.cs.ucdavis.edu/~yjlee/. The research results were also regularly presented by the PI at international meetings and university seminars.
In terms of broader impact, the main project outcomes are graduate and undergraduate student mentorship and training, outreach activities to promote wider participation of young students in CS and STEM education, and broad scientific impact of the algorithms. In particular, the project helped train PhD, MS, and undergraduate students in conducting and presenting research in the topics of this project. One PhD student completed his PhD and four non-student researchers accepted new PhD and research industry positions. The project's outreach component contributed to efforts that widen middle school and high school student participation in STEM.
Last Modified: 11/27/2020
Modified by: Yong Jae Lee
Please report errors in award information by writing to: awardsearch@nsf.gov.
