| NSF Org: |
CNS Division Of Computer and Network Systems |
| Recipient: |
|
| Initial Amendment Date: | August 17, 2019 |
| Latest Amendment Date: | April 14, 2022 |
| Award Number: | 1908658 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Marilyn McClure
mmcclure@nsf.gov (703)292-5197 CNS Division Of Computer and Network Systems CSE Directorate for Computer and Information Science and Engineering |
| Start Date: | October 1, 2019 |
| End Date: | September 30, 2023 (Estimated) |
| Total Intended Award Amount: | $500,000.00 |
| Total Awarded Amount to Date: | $532,000.00 |
| Funds Obligated to Date: |
FY 2020 = $16,000.00 FY 2022 = $16,000.00 |
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
601 UNIVERSITY DR SAN MARCOS TX US 78666-4684 (512)245-2314 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
601 University Dr. San Marcos TX US 78666-4684 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): |
Special Projects - CNS, CSR-Computer Systems Research |
| Primary Program Source: |
01001920DB NSF RESEARCH & RELATED ACTIVIT 01002021DB NSF RESEARCH & RELATED ACTIVIT |
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
Deep artificial neural network technology has been widely used to solve many challenging tasks in computer vision, natural language processing, speech recognition, and more. Most of today's deep learning algorithms are designed for high performance servers and running in the cloud. As the edge devices (e.g., mobile phones and smart watches) become more capable and the advantages of on-device artificial intelligence (AI) (e.g. protecting privacy, working without a network, processing data locally in real-time) become more evident, bringing AI to the edge will be inevitable. However, the limited resources (e.g., computation, memory, and battery) of edge devices bring a whole new level of challenges: (1) On-device AI must keep the model size small without sacrificing accuracy; (2) On-device AI must keep the power usage low; (3) Future on-device AI should enable efficient processing and analysis on multi-modal data (e.g., video, audio, and text); and (4) On-device AI should be interpretable and reproducible. This project aims to address these challenges by (1) exploring innovative machine learning algorithms (e.g., multi-task learning) for multi-modal data analysis; (2) exploring multi-modal pruning algorithms (reducing the neural network size without compromising accuracy) that can be applied on edge devices; (3) investigating and explaining how pruning works and using the derived theory to guide further pruning optimization; and (4) improving the energy efficiency of on-device AI algorithms and developing energy-aware scheduling algorithms for on-device AI apps.
The research outcomes of this project directly benefit mobile users by accelerating the deployment of efficient AI algorithms on edge devices. Texas State University is a large Hispanic Serving Institution (HSI), which provides a unique platform to engage underrepresented students in science, technology, engineering, and math (STEM) research. We will enlist the Texas State University Houston-Louis Stokes Alliance for Minority Participation Scholars Program and the Women in Science and Engineering Program to enhance the research and education experiences of underrepresented students in STEM fields.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
The goals of this NSF project include: (1) exploring innovative machine learning algorithms (e.g., multi-task learning) for multi-modal data analysis; (2) exploring multimodal pruning algorithms (reducing the neural network size without compromising accuracy) that can be applied on edge devices; (3) investigating and explaining how pruning works and using the derived theory to guide further pruning optimization; and (4) reducing the energy consumption and carbon emissions of AI algorithms.
The accomplished goals and outcomes of this project are outlined below:
Products that address intellectual merit:
-
Research Activities: Various research projects were conducted, encompassing multi-modal and cross-modal learning, pruning and transfer learning, convolutional neural network distillation, network binarization via contrastive learning, learning omnidirectional flow in 360 videos, evaluating the energy efficiency and carbon impact of large language models, and more. These projects generated novel algorithms and studies, which advanced various facets of modern AI research, from data creation, model pruning and distillation, multi-modal learning, to carbon emission reduction.
-
Publications: 15 peer-reviewed papers have been published in esteemed journals and IEEE/ACM-sponsored conferences/workshops, including IEEE Transactions on Computers, the International Journal of Computer Vision, the European Conference on Computer Vision, and the IEEE International Conference on Edge Computing and Scalable Cloud.
-
Released Data: Research findings are made public through conferences, workshops, and the EgoK360 dataset was released to the public to support research in egocentric 360 video understanding.
Products that address broader impacts:
-
Education materials: The content and outcomes of the project have been incorporated as education materials in several undergraduate/graduate courses offered by the PIs in their specific institutions, such as Machine Learning, Green Computing, Advanced Topics on Computer Vision and Multimedia. The courses and education materials benefitted a large group of undergraduate and graduate students in their programs of studies and future career development.
-
Student mentoring: A total of 10 students, including 6 Ph.D. students, and 4 undergraduate students participated in research projects and received professional training on efficient AI and machine learning research. Three Ph.D. students graduated and were hired by prominent companies like Tesla and DataBricks. One supported undergraduate student (Hispanic) has decided to pursue his Ph.D. degree in 2024.
-
Presentations and media coverages: The project has resulted in invited presentations at conferences, research institutions, and industry, as well as coverages at various media channels. The presentations and media coverages serve as important means of research dissemination to the society for potential technology transfer.
Last Modified: 11/30/2023
Modified by: Ziliang Zong
Please report errors in award information by writing to: awardsearch@nsf.gov.
