Award Abstract # 1925371
NRI: INT: COLLAB: Muscle Ultrasound Sensing for Intuitive Control of Robotic Leg Prostheses

NSF Org: CMMI
Division of Civil, Mechanical, and Manufacturing Innovation
Recipient: UNIVERSITY OF UTAH
Initial Amendment Date: August 20, 2019
Latest Amendment Date: August 20, 2019
Award Number: 1925371
Award Instrument: Standard Grant
Program Manager: Alex Leonessa
CMMI
 Division of Civil, Mechanical, and Manufacturing Innovation
ENG
 Directorate for Engineering
Start Date: September 1, 2019
End Date: August 31, 2024 (Estimated)
Total Intended Award Amount: $600,987.00
Total Awarded Amount to Date: $600,987.00
Funds Obligated to Date: FY 2019 = $600,987.00
History of Investigator:
  • Tommaso Lenzi (Principal Investigator)
    t.lenzi@utah.edu
Recipient Sponsored Research Office: University of Utah
201 PRESIDENTS CIR
SALT LAKE CITY
UT  US  84112-9049
(801)581-6903
Sponsor Congressional District: 01
Primary Place of Performance: University of Utah
1550 MEK (1495 E 100 S.), office
Salt Lake City
UT  US  84112-0030
Primary Place of Performance
Congressional District:
01
Unique Entity Identifier (UEI): LL8GLEVH6MG3
Parent UEI:
NSF Program(s): NRI-National Robotics Initiati
Primary Program Source: 01001920DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 8013, 8086
Program Element Code(s): 801300
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.041

ABSTRACT

The research objective of this project is to enable volitional control over lower-limb prostheses through the integration of sonomyographic sensing - the ultrasound imaging of amputated (i.e., residual) limb muscle morphology - to control the Utah Lightweight Leg. This powered prosthetic leg is comprised of powered ankle and knee modules, and is roughly half the weight of contemporary technologies. The project team will use sonomyographic sensors in combination with mechanical sensors to infer the user's intent in anticipation of ambulation mode or joint motion, for example locomotor transitions from walking over level ground to ramps or stairs. The team will then perform human subject experiments comparing the ability of participants with transfemoral amputation to ambulate with and without various sonomyographic control algorithms enabled. If successful, the project will have positive impact on national health and welfare by improving the lives of individuals with amputation in terms of their independence and ambulation abilities, and by mitigating undesirable secondary effects of amputation such as a fear of falling and long-term joint health. Additional broader impacts of the work include enhanced undergraduate and graduate research experiences for veterans and underrepresented minorities, as well as outreach activities to K-12 students.

Robotic leg prostheses can overcome the limitations of conventional passive prostheses by generating net-positive energy during the gait cycle and actively regulating joint motion. However, scientific barriers must be overcome for robotic leg prosthesis to safely and effectively operate in real-world settings. The goal of this project is to fill the knowledge gap regarding the integration of the user's volition in the control of lightweight robotic ankle and knee prostheses. The research team will measure muscle contractions of the user's residual limb using wearable ultrasound probes. Specific objectives of this project are: 1) to identify optimal design guidelines to integrate sonomyographic sensing into state-of-the-art powered knee-ankle prostheses; 2) to determine specific algorithms that best anticipate the user's intention to perform different ambulation modes in a timely, accurate, and reliable manner; and 3) to understand how to optimally combine information gathered from sonomyography and mechanical sensors to control a robotic leg prosthesis within specific ambulation modes. Algorithms will be implemented on a lightweight robotic ankle and knee prosthesis to evaluate the hypothesis that providing users with anticipatory volitional control will lead to enhanced performance in complex and uncertain environments, thereby fostering seamless integration of robotic prostheses with human users.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 13)
Cowan, Marissa and Creveling, Suzi and Sullivan, Liam M and Gabert, Lukas and Lenzi, Tommaso "A Unified Controller for Natural Ambulation on Stairs and Level Ground with a Powered Robotic Knee Prosthesis" , 2023 https://doi.org/10.1109/IROS55552.2023.10341691 Citation Details
Creveling, Suzi and Cowan, Marissa and Sullivan, Liam M and Gabert, Lukas and Lenzi, Tommaso "Volitional EMG Control Enables Stair Climbing with a Robotic Powered Knee Prosthesis" , 2023 https://doi.org/10.1109/IROS55552.2023.10341615 Citation Details
Hood, Sarah and Gabert, Lukas and Lenzi, Tommaso "Powered Knee and Ankle Prosthesis With Adaptive Control Enables Climbing Stairs With Different Stair Heights, Cadences, and Gait Patterns" IEEE Transactions on Robotics , v.38 , 2022 https://doi.org/10.1109/TRO.2022.3152134 Citation Details
Hunt, Grace and Hood, Sarah and Lenzi, Tommaso "Stand-Up, Squat, Lunge, and Walk With a Robotic Knee and Ankle Prosthesis Under Shared Neural Control" IEEE Open Journal of Engineering in Medicine and Biology , v.2 , 2021 https://doi.org/10.1109/OJEMB.2021.3104261 Citation Details
Hunt, Grace R. and Hood, Sarah and Gabert, Lukas and Lenzi, Tommaso "Can a powered knee-ankle prosthesis improve weight-bearing symmetry during stand-to-sit transitions in individuals with above-knee amputations?" Journal of NeuroEngineering and Rehabilitation , v.20 , 2023 https://doi.org/10.1186/s12984-023-01177-w Citation Details
Hunt, Grace R. and Hood, Sarah and Gabert, Lukas and Lenzi, Tommaso "Effect of Increasing Assistance From a Powered Prosthesis on Weight-Bearing Symmetry, Effort, and Speed During Stand-Up in Individuals With Above-Knee Amputation" IEEE Transactions on Neural Systems and Rehabilitation Engineering , v.31 , 2023 https://doi.org/10.1109/TNSRE.2022.3214806 Citation Details
Mendez, Joel and Hood, Sarah and Gunnel, Andy and Lenzi, Tommaso "Powered knee and ankle prosthesis with indirect volitional swing control enables level-ground walking and crossing over obstacles" Science Robotics , v.5 , 2020 https://doi.org/10.1126/scirobotics.aba6635 Citation Details
Mendez, Joel and Murray, Rosemarie and Gabert, Lukas and Fey, Nicholas P. and Liu, Honghai and Lenzi, Tommaso "A-Mode Ultrasound-Based Prediction of Transfemoral Amputee Prosthesis Walking Kinematics via an Artificial Neural Network" IEEE Transactions on Neural Systems and Rehabilitation Engineering , v.31 , 2023 https://doi.org/10.1109/TNSRE.2023.3248647 Citation Details
Mendez, Joel and Murray, Rosemarie and Gabert, Lukas and Fey, Nicholas P. and Liu, Honghai and Lenzi, Tommaso "Continuous A-Mode Ultrasound-Based Prediction of Transfemoral Amputee Prosthesis Kinematics Across Different Ambulation Tasks" IEEE Transactions on Biomedical Engineering , 2023 https://doi.org/10.1109/TBME.2023.3292032 Citation Details
Murray, Rosemarie and Mendez, Joel and Gabert, Lukas and Fey, Nicholas P. and Liu, Honghai and Lenzi, Tommaso "Ambulation Mode Classification of Individuals with Transfemoral Amputation through A-Mode Sonomyography and Convolutional Neural Networks" Sensors , v.22 , 2022 https://doi.org/10.3390/s22239350 Citation Details
Rabe, Kaitlin G. and Lenzi, Tommaso and Fey, Nicholas P. "Performance of Sonomyographic and Electromyographic Sensing for Continuous Estimation of Joint Torque During Ambulation on Multiple Terrains" IEEE Transactions on Neural Systems and Rehabilitation Engineering , v.29 , 2021 https://doi.org/10.1109/TNSRE.2021.3134189 Citation Details
(Showing: 1 - 10 of 13)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

This project improved robotic prosthetic legs by allowing users to control them more naturally using signals from their own muscles and nerves. Our work focused on people with above-knee amputations, helping them move more smoothly and intuitively improving their mobilty and quality of life.

To achieve this, we created a new type of controller that blends the user’s own movement intentions with robotic assistance. This controller was tested using electromyography (EMG), a method that measures tiny electrical signals in muscles. Initially, the system allowed users to adjust the robotic leg’s support only when their foot was on the ground—such as when standing up from a chair or squatting. Over time, we improved the technology so users could also control movements while their leg was in the air, like during walking. Our research showed that this controller helped people with amputations climb stairs more naturally, even while carrying a backpack. It also allowed them to smoothly switch between activities like walking, stair climbing, and sitting down, providing better functionality than previous prosthetic controllers.

To further improve how well the system detects user intentions, we explored sonomyography, a technique that uses ultrasound (sound waves) to monitor muscle movements. We tested two types of ultrasound technology—A-mode and B-mode—and found that while B-mode provides more detailed images, but it is also more complex and expensive. Since our goal was to create a practical system that can be used in real life, we developed a lightweight, battery-powered A-mode ultrasound device that works in real-time. With this system, we collected extensive data from individuals with above-knee amputations as they performed tasks like walking and climbing stairs using passive prosthetic legs. Using this data, we trained artificial intelligence (AI) models to predict how the robotic knee and ankle should move based on the user’s muscle activity. Our results showed that a person with an above-knee amputation could successfully control the speed of a robotic knee just by using muscle signals detected through ultrasound—without needing additional sensors or external controls. This was the first successful demonstration of direct neural control of a robotic prosthesis using AI, paving the way for more natural and intuitive prosthetic limb technology.

This project also played a role in education and workforce development. It supported three PhD students and two master’s students, all U.S. citizens. Additionally, it provided research opportunities for undergraduate students through senior design projects and summer internships focused on prosthetics and robotics.


Last Modified: 03/11/2025
Modified by: Tommaso Lenzi

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page