Award Abstract # 2145473
CAREER: Musculoskeletal Modeling with Wearable Sensors and Smartphone Cameras

NSF Org: CBET
Division of Chemical, Bioengineering, Environmental, and Transport Systems
Recipient: CARNEGIE MELLON UNIVERSITY
Initial Amendment Date: January 24, 2022
Latest Amendment Date: June 27, 2024
Award Number: 2145473
Award Instrument: Continuing Grant
Program Manager: Amanda O. Esquivel
aesquive@nsf.gov
 (703)292-0000
CBET
 Division of Chemical, Bioengineering, Environmental, and Transport Systems
ENG
 Directorate for Engineering
Start Date: March 1, 2022
End Date: February 28, 2027 (Estimated)
Total Intended Award Amount: $562,597.00
Total Awarded Amount to Date: $562,597.00
Funds Obligated to Date: FY 2022 = $496,613.00
FY 2024 = $65,984.00
History of Investigator:
  • Eni Halilaj (Principal Investigator)
    ehalilaj@andrew.cmu.edu
Recipient Sponsored Research Office: Carnegie-Mellon University
5000 FORBES AVE
PITTSBURGH
PA  US  15213-3890
(412)268-8746
Sponsor Congressional District: 12
Primary Place of Performance: Carnegie-Mellon University
5000 Forbes Avenue
PITTSBURGH
PA  US  15213-3890
Primary Place of Performance
Congressional District:
12
Unique Entity Identifier (UEI): U3NKNFLNQ613
Parent UEI: U3NKNFLNQ613
NSF Program(s): Disability & Rehab Engineering,
BMMB-Biomech & Mechanobiology
Primary Program Source: 01002223DB NSF RESEARCH & RELATED ACTIVIT
01002425DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 028E, 1045, 9102
Program Element Code(s): 534200, 747900
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.041

ABSTRACT

This Faculty Early Career Development (CAREER) award focuses on democratizing gait analysis for rehabilitation research and therapy. Mobility is a hallmark of human health, yet mobility limitations continue to reduce personal independence and overall quality of life in nearly a third of Americans. Musculoskeletal conditions alone cost the United States? economy 5% of its overall gross domestic product. Traditionally, mobility has been studied in gait laboratories, with expensive equipment, trained personnel, and time-consuming data processing pipelines, limiting studies in size, setting, and monitoring time. Recent advances in computer vision and wearable sensing could bring motion tracking with smartphones and wearable sensors to laboratories, clinics, and patient homes at negligible costs, growing studies from tens to thousands of subjects. Current approaches for motion tracking from videos and wearables, however, remain insufficiently accurate for research and clinical translation. One of the challenges is that they are primarily data-driven, relying on limited training data that do not include patients with mobility limitations. This project will merge the complementary strengths of artificial intelligence (AI) and physics-based modeling into a new motion-tracking paradigm that is widely accessible, dynamically robust, and equitably accurate across human demographics and abilities.

The technical objectives of this project are to (1) create and evaluate a new computational framework for musculoskeletal modeling with wearable sensors and smartphone videos, and (2) use this framework to answer fundamental questions about the role of cyclic loading on cartilage health. This work will empower biomechanical engineers and rehabilitation specialists by fundamentally changing the amount and type of data they use to answer scientific questions around mobility limitations. The technical contributions will integrate multibody dynamics with state-of-the-art machine learning models to enable accurate motion tracking from wearable sensors and smartphone cameras. Additionally, subject-specific anatomy from medical images will be incorporated into these modeling frameworks to enable the estimation of internal biomechanics. The scientific contributions will advance the field of orthopaedic rehabilitation by generating previously unavailable knowledge on how subject-specific joint mechanics modulate cartilage response to cyclic loading in natural environments, which will be critical in the design of personalized rehabilitation technologies for osteoarthritis prevention. The project will also leverage the pop-science appeal and practical utility of AI to inspire, train, and retain the next generation of multidimensionally diverse rehabilitation engineers. Activities include (1) collaborative outreach events with Facebook Reality Labs to expose Pittsburgh area K-12 students to futuristic motion capture facilities (Inspire), (2) educational software that synergistically generates annotated datasets to improve computer-vision algorithms, while teaching high-school students about trustworthy AI (Train), and (3) an AI toolkit that promotes equity and inclusion in higher education (Retain).

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Pearl, Owen and Shin, Soyong and Godura, Ashwin and Bergbreiter, Sarah and Halilaj, Eni "Fusion of video and inertial sensing data via dynamic optimization of a biomechanical model" Journal of Biomechanics , v.155 , 2023 https://doi.org/10.1016/j.jbiomech.2023.111617 Citation Details
Phan, Vu and Song, Ke and Silva, Rodrigo Scattone and Silbernagel, Karin G. and Baxter, Josh R. and Halilaj, Eni "Seven Things to Know about Exercise Classification with Inertial Sensing Wearables" IEEE Journal of Biomedical and Health Informatics , 2024 https://doi.org/10.1109/JBHI.2024.3368042 Citation Details
Shin, Soyong and Kim, Juyong and Halilaj, Eni and Black Michael J. "WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion" Proceedings of IEEE/CVF Computer Vision and Pattern Recognition , 2024 Citation Details
Shin, Soyong and Li, Zhixiong and Halilaj, Eni "Markerless Motion Tracking With Noisy Video and IMU Data" IEEE Transactions on Biomedical Engineering , v.70 , 2023 https://doi.org/10.1109/TBME.2023.3275775 Citation Details

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page