| NSF Org: |
ECCS Division of Electrical, Communications and Cyber Systems |
| Recipient: |
|
| Initial Amendment Date: | August 26, 2019 |
| Latest Amendment Date: | May 19, 2020 |
| Award Number: | 1947419 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Anthony Kuh
ECCS Division of Electrical, Communications and Cyber Systems ENG Directorate for Engineering |
| Start Date: | August 1, 2019 |
| End Date: | July 31, 2024 (Estimated) |
| Total Intended Award Amount: | $237,297.00 |
| Total Awarded Amount to Date: | $253,297.00 |
| Funds Obligated to Date: |
FY 2020 = $16,000.00 |
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
777 GLADES RD BOCA RATON FL US 33431-6424 (561)297-0777 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
777 GLADES RD BOCA RATON FL US 33431-6424 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): | EPCN-Energy-Power-Ctrl-Netwrks |
| Primary Program Source: |
01002021DB NSF RESEARCH & RELATED ACTIVIT |
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.041 |
ABSTRACT
The recent big wave of artificial intelligence (AI) not only provided tremendous advancements ranging from fundamental research to a wide range of exciting applications, but also presents enormous amounts of opportunities as well as challenges to the community. Among many of the AI techniques, adaptive dynamic programming and reinforcement learning (ADP/RL) is widely considered as one of the key methodologies for learning-based intelligent decision-making process.
The objective of this project is to develop an innovative autonomous hierarchical ADP/RL approach for decision making in complex environments. By autonomously providing a hierarchical representation of sub-goals for improved learning and exploration capability, the proposed research provides a new approach to systematically and adaptively develop an optimal multi-step hierarchical temporal abstraction sequence, rather than the one-step primitive action in traditional methods. The research method advances the foundations, principles, architectures, and algorithms for autonomous learning and hierarchical control, which will facilitate the capability of learning and generalization for decision-making. This project provides unique opportunities to attract and educate future professionals by bridging the connections of ADP/RL and energy systems, and for students to work on cutting-edge problems. The team consists of two PIs with strong collaborations and complementary expertise in computational intelligence, machine learning, autonomous control, and the smart grid.
This research advances the scientific foundations and methodologies of intelligent decision making in complex environments with high-dimensionality, big data, and uncertainty. The collaborations with industry integrates fundamental research into a microgrid application providing critical technical innovations to the energy sector. In addition, the developed ADP/RL based intelligent decision making method can benefit other types of complex engineering systems. Furthermore, the research results of this project are also expected to fulfill a critical need in the community by training and preparing future workforce in the cross-disciplinary areas of machine learning and energy systems. The integrative outreach and education activities will provide unique opportunities to attract women and minorities into the intelligent system and smart grid field.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
This project enhances learning strategies based on reinforcement learning (RL) and adaptive dynamic programming (ADP), pushing the boundaries of autonomous decision-making in complex environment. Specifically, the team has developed a wide array of computational intelligence and machine learning methods with the focus on RL and ADP. The major contributions include the development of (1) new neural-RL/ADP-based control approaches for nonlinear hierarchical systems, effectively addressing complex uncertainties and dynamic environment to enable robust and adaptive decision-making across multiple layers of control, (2) hierarchical control structures integrated with RL/ADP learning techniques, demonstrated through its application on the StarCraft computer game with heterogeneous agents, (3) computationally efficient inverse RL algorithms to facilitate the learning with minimal human intervention, (4) effective federated RL methods to facilitate the collaboration of distributed agents in complex environment while keeping the private data on the edge, and (5) evaluation plans from theoretical analysis, algorithms evaluation and robotics testbed validation. The outcomes of this project are significant, advancing knowledge in machine learning and computational intelligence, with a focus on RL/ADP-based learning techniques and their wide-ranging applications in complex engineering systems.
This project produced numerous peer-reviewed research articles in major journals and pioneer international conference in the field, including IEEE Transactions on Artificial Intelligence (TAI), IEEE Transactions on Neural Networks and Learning Systems (TNNLS), IEEE Transactions on Emerging Topics in Computational Intelligence (TETCI), IEEE Internet of Things Journal (IoT Journal), IEEE International Joint Conference on Neural Networks (IJCNN), and IEEE Symposium Series on Computational Intelligence (SSCI). The research outcomes resulted in multiple awards and recognitions, including International Neural Network Society (INNS) Aharon Katzir Young Investigator Award (2021) and Florida Atlantic University (FAU) College of Engineering & Computer Science Junior Faculty Research Award (2022). This project has created numerous opportunities for student training and mentoring by actively engaging both graduate and undergraduate students in research. The PI has also effectively integrated research materials from this project into senior design projects and direct independent studies, increasing undergraduate involvement. In particular, two senior design groups have respectively won Faculty's Choice Award in 2024 and People's Choice Award in 2021 at FAU Engineering showcase. The project has also supported undergraduate research initiatives through REU supplements. Additionally, the project has reached out to the broader community, emphasizing the involvement of high school students and underrepresented minorities through various outreach activities, including FAU high school research seminar, Engineer the Future Day academic showcase, and Diversity Faculty Sharing Experience Workshop in IEEE International Conference on Image Processing.
Last Modified: 10/23/2024
Modified by: Xiangnan Zhong
Please report errors in award information by writing to: awardsearch@nsf.gov.
