Award Abstract # 1925043
NRI: INT: COLLAB: Program Verification and Synthesis for Collaborative Robots

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: UNIVERSITY OF WISCONSIN SYSTEM
Initial Amendment Date: August 19, 2019
Latest Amendment Date: April 27, 2021
Award Number: 1925043
Award Instrument: Standard Grant
Program Manager: Cang Ye
cye@nsf.gov
 (703)292-4702
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: September 1, 2019
End Date: August 31, 2025 (Estimated)
Total Intended Award Amount: $958,887.00
Total Awarded Amount to Date: $974,887.00
Funds Obligated to Date: FY 2019 = $958,887.00
FY 2021 = $16,000.00
History of Investigator:
  • Bilge Mutlu (Principal Investigator)
    bilge@cs.wisc.edu
  • Aws Albarghouthi (Co-Principal Investigator)
  • Allison Sauppe (Co-Principal Investigator)
Recipient Sponsored Research Office: University of Wisconsin-Madison
21 N PARK ST STE 6301
MADISON
WI  US  53715-1218
(608)262-3822
Sponsor Congressional District: 02
Primary Place of Performance: University of Wisconsin-Madison
1210 West Dayton Street
Madison
WI  US  53706-1685
Primary Place of Performance
Congressional District:
02
Unique Entity Identifier (UEI): LCLSJAGTNZQ7
Parent UEI:
NSF Program(s): NRI-National Robotics Initiati
Primary Program Source: 01001920DB NSF RESEARCH & RELATED ACTIVIT
01002122DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 8086, 9251
Program Element Code(s): 801300
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Developing applications for robots currently requires advanced knowledge and skills in both robotics and software engineering. As a result, development of robot applications is currently only accessible to a very small segment of the population, excluding those who might have the deepest insight about potential application domains for robots (e.g., experts in logistics, manufacturing, marketing), those who might be better suited to design robot behaviors for human interaction (e.g., designers, artists, psychologists), and those who might be using and repurposing robots day-to-day (e.g., end-users, supervisors, maintenance personnel). This project aims to design and implement a suite of tools and methods, powered by developments in the field of programming languages, to enable the programming of interactive robots with minimal prior experience. These tools and methods will assist all stages of the programming process, including ideation of what the robot behavior should be, coding of the program, its testing on the robot to guarantee certain properties, and adapting the program for deployments in different contexts. The research will make robot programming accessible to the broader public and "democratize" it by substantially lowering barriers to engagement in technology development. Through collaborations with domain experts, the principles, methods, and tools created by the project will be applied to real-world robotics applications, transferring the scientific contributions of the project to robotics organizations. Through an integrated education program, computer science training and curriculum will be enhanced across three institutions by introducing new courses and course modules on robotics software development. To facilitate public understanding of and engagement with science and technology, a robot programming environment for elementary- and middle-school-aged children will be developed along with a structured curriculum for its use in an outreach summer program, and these will be widely disseminated as open-source materials.

The key innovation of the project is the development of novel paradigms for programming that use state-of-the-art program analysis and synthesis methods to support a wide range of programming tasks across the development cycle for robotics applications. Specifically, the project will support each stage of the cycle in three research thrusts with the following objectives: (1) facilitating the capture of user intent by translating designer and end-user intent captured through acting and dialogue into robot programs; (2) supporting iterative development and testing by facilitating the development, refinement, analysis, and synthesis of robust, reliable, and effective robot programs; (3) optimizing deployment by enabling the adaptation of existing robot programs into new environments, scenarios of use, and social contexts. The outcomes of these research thrusts will be integrated into a suite of robot programming tools within the ROS framework and deployed in real-world robot programming tasks in collaboration with industrial partners.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 20)
Praveena, Pragathi and Schoen, Andrew and Gleicher, Michael and Porfirio, David and Mutlu, Bilge "Petri Nets for the Iterative Development of Interactive Robotic Systems" Proceedings of the AAAI Symposium Series , v.2 , 2024 https://doi.org/10.1609/aaaiss.v2i1.27724 Citation Details
Kim, Callie Y and Lee, Christine P and Mutlu, Bilge "Understanding Large-Language Model (LLM)-powered Human-Robot Interaction" , 2024 https://doi.org/10.1145/3610977.3634966 Citation Details
Murray, Michael and Walker, Nick and Nanavati, Amal and Alves-Oliveira, Patricia and Filippov, Nikita and Sauppe, Allison and Mutlu, Bilge and Cakmak, Maya "Learning Backchanneling Behaviors for a Social Robot via Data Augmentation from Human-Human Conversations" Proceedings of the 5th Conference on Robot Learning (PMLR) , v.164 , 2022 Citation Details
Porfirio, D. and Stegner, L. and Cakmak, M. and Sauppé, A. and Albarghouthi, A. and Mutlu, B. "Figaro: A Tabletop Authoring Environment for Human-Robot Interaction" Human factors in computing systems , 2021 https://doi.org/ Citation Details
Porfirio David and Cakmak, Maya and Sauppé, Allison and Albarghouthi, Aws and Mutlu, Bilge "Interaction Templates: A Data-Driven Approach for Authoring Robot Programs" PLATEAU: 12th Annual Workshop at theIntersection of PL and HCI , 2022 https://doi.org/10.1184/R1/20031965.v1 Citation Details
Porfirio, David and Sauppé, Allison and Albarghouthi, Aws and Mutlu, Bilge "Transforming Robot Programs Based on Social Context" CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems , 2020 https://doi.org/10.1145/3313831.3376355 Citation Details
Porfirio, David and Sauppé, Allison and Cakmak, Maya and Albarghouthi, Aws and Mutlu, Bilge "Crowdsourcing Task Traces for Service Robotics" HRI '23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction , 2023 https://doi.org/10.1145/3568294.3580112 Citation Details
Porfirio, David and Stegner, Laura and Cakmak, Maya and Sauppé, Allison and Albarghouthi, Aws and Mutlu, Bilge "Sketching Robot Programs On the Fly" HRI '23: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction , 2023 https://doi.org/10.1145/3568162.3576991 Citation Details
Schoen, Andrew and Henrichs, Curt and Strohkirch, Mathias and Mutlu, Bilge "Authr: A Task Authoring Environment for Human-Robot Teams" UIST '20: Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology , 2020 https://doi.org/10.1145/3379337.3415872 Citation Details
Schoen, Andrew and Mutlu, Bilge "OpenVP: A Customizable Visual Programming Environment for Robotics Applications" , 2024 https://doi.org/10.1145/3610977.3637477 Citation Details
Schoen, Andrew and Sullivan, Dakota and Zhang, Ze Dong and Rakita, Daniel and Mutlu, Bilge "Lively: Enabling Multimodal, Lifelike, and Extensible Real-time Robot Motion" HRI '23: Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction , 2023 https://doi.org/10.1145/3568162.3576982 Citation Details
(Showing: 1 - 10 of 20)

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page