Award Abstract # 2024802
Collaborative Research: NRI: INT: Transparent and Intuitive Teleoperation Interfaces for the Future Nursing Robots and Workers

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: WORCESTER POLYTECHNIC INSTITUTE
Initial Amendment Date: August 19, 2020
Latest Amendment Date: July 5, 2022
Award Number: 2024802
Award Instrument: Continuing Grant
Program Manager: Wendy Nilsen
wnilsen@nsf.gov
 (703)292-2568
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: September 1, 2020
End Date: August 31, 2024 (Estimated)
Total Intended Award Amount: $731,329.00
Total Awarded Amount to Date: $731,329.00
Funds Obligated to Date: FY 2020 = $245,336.00
FY 2021 = $245,328.00

FY 2022 = $240,665.00
History of Investigator:
  • Zhi Li (Principal Investigator)
    zli11@wpi.edu
  • Jeanine Skorinko (Co-Principal Investigator)
  • Yunus Telliel (Co-Principal Investigator)
  • Cagdas Onal (Co-Principal Investigator)
  • Jie Fu (Co-Principal Investigator)
Recipient Sponsored Research Office: Worcester Polytechnic Institute
100 INSTITUTE RD
WORCESTER
MA  US  01609-2280
(508)831-5000
Sponsor Congressional District: 02
Primary Place of Performance: Worcester Polytechnic Institute
100 Institute Rd
Worcester
MA  US  01609-2247
Primary Place of Performance
Congressional District:
02
Unique Entity Identifier (UEI): HJNQME41NBU4
Parent UEI:
NSF Program(s):
Primary Program Source: 01002021RB NSF RESEARCH & RELATED ACTIVIT
01002122RB NSF RESEARCH & RELATED ACTIVIT

01002223RB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 8086
Program Element Code(s):
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

The recent pandemic outbreaks, including Ebola, Zika and the 2019 Novel Coronavirus (2019-nCoV), urge tele-medicine to go beyond mere tele-presence, to achieve robots that perform real-world nursing assistance tasks that require the coordinated control of manipulation, locomotion, and active teleoperation. Remotely-controlled nursing robots provide a promising alternative for quarantine and remote patient care. However, the traditional and contemporary human-robot interfaces fundamentally limit the performance and user experience of nursing robot teleoperation, and may reinforce burden and safety concerns that discourage healthcare workers to adopt robots. To address this problem, this project will (1) develop an innovative integration of transparent and intuitive teleoperation interface, to support the freeform and coordinated motion control of the remote nursing robots, and (2) integrate this interface with the robot intelligence to enable nursing professionals to learn robot teleoperation with minimal training, and to reduce the physical and cognitive workload using shared autonomy. The proposed project will promote the progress of science in human-robot interfaces for robot teleoperation, and advance the quality, availability and sustainability of healthcare in the present and future pandemic crisis. This project will have significant impacts on the domain of nursing, which consists of 2.9 million registered nurses and 160,000 nurse practitioners across the U.S. It will revolutionize patient-care in quarantine, and has the potential to extend to in-home care, clinics, and hospitals given the upcoming shortage of nursing workforce. The fundamental research also generalizes to other worker domains with robot tele-operations, including warehouse, social service, and maintenance. The proposed research will forge substantial collaboration among faculty and students in robotics engineering, nursing and social science.

This project consists of two research themes. Research Theme 1 will develop a soft-robot teleoperation interface architecture and systematic human-inspired motion mapping strategies, to support the intuitive and transparent mapping of the motion, force, and perception information between humans and robots. The proposed interface will enable transparent and legible robot behavior of reaching-to-grasp, loco-manipulation, and the control of active telepresence. Research Theme 2 will develop the intelligence of the interface, to enable interactive learning and mutual adaptation between humans and robots. Based on game-theoretic planning, it will develop adaptive shared autonomous strategies that use human-robot communication via haptic feedback. It will employ active tele-presence to enhance the training and reduce workload in tele-operation of the human operator. The integrated interface will be evaluated in comprehensive user studies with registered nurses, nursing faculty and nursing students. The evaluation will assess the performance and user experience, including human-robot teaming, using efficiency, workload and interface effort metrics. It will also evaluate the social impacts of the proposed human-robot interface on the acceptance and adoption of nursing robots by the current and future nursing workforce.

This proposal was funded with the National Institute for Occupational Safety and Health (NIOSH) in the Center for Disease Control and Prevention (CDC).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 15)
Achyuthan Unni Krishnan, Tsung-Chi Lin "Design Interface Mapping for Efficient Free-form Tele-manipulation" IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022) , 2022 Citation Details
Genua, Lorena Maria and Boguslavskii, Nikita and Li, Zhi "Multilateral Multimodal Human-Robot Collaboration for Robotic Nursing Assistance: Prototype System and Preliminary User Study" , 2024 https://doi.org/10.1109/RO-MAN60168.2024.10731266 Citation Details
Krishnan, Achyuthan Unni and Lin, Tsung-Chi and Li, Zhi "Design Interface Mapping for Efficient Free-form Tele-manipulation" 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2022 https://doi.org/10.1109/IROS47612.2022.9982149 Citation Details
Krishnan, Achyuthan Unni and Lin, Tsung-Chi and Li, Zhi "Human Preferred Augmented Reality Visual Cues for Remote Robot Manipulation Assistance: from Direct to Supervisory Control" , 2023 https://doi.org/10.1109/IROS55552.2023.10341969 Citation Details
Li, Lening and Rahmani, Hazhar and Fu, Jie "Probabilistic Planning with Prioritized Preferences over Temporal Logic Objectives" , 2023 https://doi.org/10.24963/ijcai.2023/22 Citation Details
Lin, Tsung-Chi and Krishnan, Achyuthan Unni and Li, Zhi "Comparison of Haptic and Augmented Reality Visual Cues for Assisting Tele-manipulation" IEEE International Conference on Robotics and Automation (ICRA) , 2022 https://doi.org/10.1109/ICRA46639.2022.9811669 Citation Details
Lin, Tsung-Chi and Krishnan, Achyuthan Unni and Li, Zhi "How People Use Active Telepresence Cameras in Tele-manipulation" 2021 IEEE International Conference on Robotics and Automation (ICRA) , 2021 https://doi.org/10.1109/ICRA48506.2021.9561243 Citation Details
Lin, Tsung-Chi and Krishnan, Achyuthan Unni and Li, Zhi "Intuitive, Efficient and Ergonomic Tele-Nursing Robot Interfaces: Design Evaluation and Evolution" ACM Transactions on Human-Robot Interaction , 2022 https://doi.org/10.1145/3526108 Citation Details
Lin, Tsung-Chi and Krishnan, Achyuthan Unni and Li, Zhi "Perception and Action Augmentation for Teleoperation Assistance in Freeform Telemanipulation" ACM Transactions on Human-Robot Interaction , v.13 , 2024 https://doi.org/10.1145/3643804 Citation Details
Lin, Tsung-Chi and Krishnan, Achyuthan Unni and Li, Zhi "The Impacts of Unreliable Autonomy in Human-Robot Collaboration on Shared and Supervisory Control for Remote Manipulation" IEEE Robotics and Automation Letters , v.8 , 2023 https://doi.org/10.1109/LRA.2023.3287039 Citation Details
Lin, Tsung-Chi and Unni Krishnan, Achyuthan and Li, Zhi "Perception-Motion Coupling in Active Telepresence: Human Behavior and Teleoperation Interface Design" ACM Transactions on Human-Robot Interaction , v.12 , 2023 https://doi.org/10.1145/3571599 Citation Details
(Showing: 1 - 10 of 15)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

Through this project we developed and evaluated integrative and intelligent human-robot interfaces which can effectively reduce the workload and learning efforts of operators novice to remote robot operation, so that they can easily and intuitively control mobile humanoid robots for nursing assistance tasks. These interfaces include: 1) a soft robotic glove – a wearable interface that can provide intuitive force and contact feedback in the control of dexterous robotic manipulation using multi-finger robot gripper; 2) a human-to-robot motion mapping interface for intuitive and effortless tele-manipulation control, which provides humans adaptive perception assistance (through haptic and augmented reality visual cues) and adaptive action assistance (through autonomous primitive actions and motion constraints) based on human’s manipulation intents; 3) a novel human-robot interface for multimodal multilateral human-robot collaboration that enables multiple remote and local human nurses to collaborative supervise and assist autonomous nursing robots and enhances the human-human/human-robot collaboration through augmented reality and verbal communication.


As for its engineering contribution, we developed novel prototypes of nursing robot platforms and interfaces, as well as the first open-source nursing robot virtual testbed. As for its theoretical contributions, our project investigated how to leverage multimodal feedback to mediate the interactive learning between humans and robot autonomy, and develop principled methods for preference learning in order to provide user-adaptive assistive teleoperation interfaces.  


Our convergent research efforts evaluated the usability of the tele-nursing robot interfaces and social impacts of tele-nursing robot technologies on nursing workers, educators and students. Our project achieved synergistic interdisciplinary research and education collaboration between robotics, social science and nursing. 


 


Last Modified: 12/05/2024
Modified by: Zhi Li

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page