| NSF Org: |
OIA OIA-Office of Integrative Activities |
| Recipient: |
|
| Initial Amendment Date: | August 20, 2018 |
| Latest Amendment Date: | January 31, 2020 |
| Award Number: | 1833005 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Jose Colom
jcolom@nsf.gov (703)292-7088 OIA OIA-Office of Integrative Activities O/D Office Of The Director |
| Start Date: | October 1, 2018 |
| End Date: | September 30, 2019 (Estimated) |
| Total Intended Award Amount: | $261,503.00 |
| Total Awarded Amount to Date: | $50,899.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
940 ADMINISTRATION LN BROOKINGS SD US 57007-0001 (605)688-6696 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
Ames Research Center Moffett Field CA US 94035-0001 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): | EPSCoR Research Infrastructure |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.083 |
ABSTRACT
Nontechnical description:
Data efficiency and learning speed are two of the major bottlenecks for applying biologically-inspired control methods in many domains. The project's goal is to address these fundamental challenges by introducing a new adaptive dynamic programming-based learning control framework and integrate it into space robot navigation and scouting applications such as the Mars Rover. The scientific contribution of this project will promote interdisciplinary research in computational intelligence, machine learning, control and robotics. In addition to space applications, the proposed structure can also be applied to robot-assisted pedestrian evacuation application and cyber-physical power systems and is expected to impact general systems beyond this project period. Due to geographic isolation, South Dakota doesn't have a National Aeronautics and Space Administration (NASA) research center, and research collaboration opportunities on space technology is very limited. This project will expand the principle investigator (PI)'s research capacity through an extended visit and collaboration with NASA Ames Research Center located in San Jose, CA, and transform the PI's career path from theoretical algorithm/architecture development towards a new direction in complex space applications. Meanwhile, the outcomes of this project align well with the South Dakota's and South Dakota State University's strategic plans. The collaboration fits well with NASA's mission to Mars and technology roadmaps.
Technical description:
The proposed project will fundamentally advance the learning and association of biologically-inspired control methods. Three major contributions to the scientific field are expected. First, a new experience network is proposed and systematically integrated into a model-free adaptive dynamic programming-based learning control framework. The PI will design an experience replay tuple (i.e., state-action-reward pair) based on backward temporal difference information from historical data. This design can avoid the model network/prediction noted in existing literature and significantly save computation resources. Second, instead of a uniform sampling method, the PI proposes a prioritized sampling method based on the Bellman's estimation error. This new method is expected to enhance the controller's reflective learning performance with useful long-short term memory. The stability and convergence properties will also be analyzed. Third, this project is closely tied with NASA on robot and optimal control for space program. This new learning control structure will be integrated for robot navigation, exploration and scouting in unknown spaces. The PI and the collaborator will use both a virtual reality platform and a real Rover facility to analyze the control performance of the proposed algorithm at NASA Ames. The PI's outreach and dissemination plans will cultivate the scientific curiosity of K-12 students and motivate their interest in STEM programs. Moreover, the integration of the project's cutting-edge research results into the PI's new courses will aid retention of current STEM students. Specific plans include a workshop for a local middle school, a distance course for demographically diverse institutions, and development of new courses.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
Project outcomes for intellectual merit:
The project's research goal is to improve the data efficiency and learning speed by introducing a new adaptive dynamic programming (ADP)-based learning control framework and integrate it into space robot navigation and other cyber-physical system applications. Computational intelligence-based control is a subfield of computation/artificial intelligence, machine learning and control theory. Improving the data efficiency and learning speed of the ADP-based control system will positively impact aforementioned scientific areas.
The ADP controller usually needs a long training period because the data usage efficiency is relatively low by discarding the samples once used. Prioritized experience replay promotes important experiences and is more efficient in learning control process. In this project, the PI integrates the prioritized experience replay design into heuristic dynamic programming (HDP). The proposed approach is tested for two case studies: a cart-pole balancing task and a triple-link pendulum balancing task. The proposed approach has improved the required average number of trials to succeed by 60.56% for cart-pole, and 56.89% for triple-link balancing tasks, in comparison with traditional HDP approach. Moreover, theoretical convergence analysis is conducted to guarantee the stability of the proposed control design.
In addition, the proposed techniques are also integrated to the fields of the robot-assisted pedestrian evacuation and cyber physical power systems. The results are reported in relevant journal and conference papers as indicated in the previous year's report. The PI was invited to present the research results to the major conferences in the IEEE Computational Intelligence Society. In July 2019, the PI visited the NASA Ames Research Center and worked with Drs. Terry Fong, Brian Coltin and Michael Furlong. The PI participated the Intelligent Robotics Group meeting every week and also contributed to relevant space robot experiments.
Project outcomes for broader impacts:
The PI co-organized the outreach workshop at Nebraska Indian Community College (NICC) in April 2019 and presented to American Indian students of autonomous mobile robot driven by new artificial intelligence and adaptive dynamic programming algorithms. Both minority students and faculty members were involved in the workshop and discussed the possible summer internships at the PI's group.
Meanwhile, the PI was the faculty advisor for the Robotics Club (2018-2019), a student-centered organization for external competition. The PI oversaw the regularly weekly meetings and provided technical guidance for mobile robot assembling, control algorithm programming, hardware experiment demonstrations and others. On average, there were about 30 undergraduate students attending the meeting from majors of Electrical, Computer, and Mechanical Engineering programs.
In addition, the PI also organized a panel session of "Intelligent Learning Control Systems and Robotics", at The 2019 IEEE International Electro/Information Technology (EIT) Conference, Brookings, SD, May 2019. This panel session brought researchers and experts in the field to discuss the new trend of artificial intelligence/machine learning, control and robotics. Attendees (students and young faculty) had a chance to hear the updated research results from outside and meet with domain experts.
The PI's outreach and dissemination activities have cultivated the scientific curiosity of the general public and students. For example, most students from the Robotics Club either found good jobs in STEM areas or purchased a graduate study in the STEM directions. In addition, the PI also leveraged his resources to involve American Indian students and faculty members, and provide them updated research results of intelligent systems. Some students showed interest to complete the full Bachelor's degree in STEM area.
Selected major awards during the grant period
International Neural Network Society (INNS) Aharon Katzir Young Investigator Award, ceremonially presented at 2019 IJCNN conference, Budapest, Hungary.
Featured Paper on Research Frontier of IEEE CIS Newsletter, Issue 82, November 2019.
Z. Ni and S. Paul, "A Multi-Stage Game in Smart Grid Security: A Reinforcement Learning Solution," IEEE Transactions on Neural Networks and Learning Systems (TNNLS). Volume: 30, Issue: 9, pp.2684 - 2695, Sept. 2019.
Last Modified: 10/09/2020
Modified by: Zhen Ni
Please report errors in award information by writing to: awardsearch@nsf.gov.
