Motivation and objectives: Physical and neurological disorders span a wide spectrum, from acute to mild and are manifested through different physical disabilities. In this project, the focus is on designing an adaptive, easy-to-use intelligent instrument that supports personalized rehabilitation therapy for individuals suffering from brain injury, motor disabilities, cognitive impairments, or demonstrate psychosocial symptoms. We call this instrument iRehab. The instrument integrates rehabilitation modules, i.e., sensors, software, and effectors.  It enables monitoring and analysis of the user’s rehabilitation activity and performance with the aim to provide effective personalization and guidance through the rehabilitation process. The instrument then generates recommendations to the therapist as to the intensity, length, and difficulty of the rehabilitation regimen. The goals of this project were to (a) build an intelligent closed-loop instrument for adaptive rehabilitation that can monitor, collect and analyze different types of user activity data; (b) measure cognitive and physical ability of the user while performing a task-driven rehabilitation session and adapt the difficulty or intensity of the task accordingly; (c) perform analysis on the collected sensor data to enable personalization of the rehabilitation and generate recommendations to the therapist.

Summary of Activities: During the period of the grant, special therapy activities were designed (including games) in order to assess and monitor rehabilitation compliance and performance through a common front end and thus allow the therapist and the user to choose the appropriate type of activity to follow. Data logging of the game performance was automated and algorithms were designed to enable multimodal data fusion and sensor data analysis in order to determine whether a user is ready for the next exercise level or needs a different type of stimulus. Real time monitoring facilities were developed to enable exercise adaptation and personalization regarding the user’s motor, mental and psychological state. The development of iRehab took into consideration computational as well as practical considerations: (a) accurate assessment of behavior during rehabilitation, as prescribed by the therapist; (b) cost and ease of use, (c) the types of rehabilitation equipment available (ranging from simple to advanced), and (d) the goals of the therapy, as defined by the therapist (e.g., improve motor ability of left upper limb). Different types of rehabilitation equipment were used, including a robotic arm device that generates detailed and exact torque sensing output at any instance of movement. The robotic device enabled refinement and validation of the analysis of motion tracking data in combination with other multisensory data collected. The robotic arm was instructed to move along a trajectory in 3D space, and the user, while wearing a brain-imaging sensor, was able to experience the motion trajectory.

Project Outcomes and Findings: The main project outcome is an evidence-based framework that collects and analyzes patient multisensory data while a person is engaged in a specific rehabilitation activity. Different metrics were designed to determine the person’s ability to perform and respond to the tasks given. Findings include the development of specific tasks or stimuli to assess motor and mental ability, promote user engagement and analyze and visualize a person’s response and performance in order to further refine and personalize treatment.  The focus was on the upper limbs that model traditional physical therapy exercises. Different types of incentives were integrated to promote user engagement. Interfaces were developed to analyze and visualize the user’s performance data. Machine learning algorithms were applied to analyze the fused multisensory data. Virtual reality tools were used to visualize the process and provide immersive exercise experience. Interactive user interfaces were designed to enable the therapist to control and guide the therapy. 

Intellectual Merit: The project led to the development of advanced motion analysis algorithms and the development of new types of multi-sensing game-driven data collection and analysis methods with broad impact on a variety of neurological disorders, facilitating the diagnosis and treatment.  The project provided new interdisciplinary research opportunities across the fields of human-centered computing, computer vision, assistive technology, robotics, machine learning, and neuroimaging. It advanced research in personalized physical therapy and big data and led to new insights in the use of robot-based rehabilitation, fusion of disparate sensor data (robotic, vision, neuroimaging).  The project also opened the way for advancements in building prosthetic devices and other assistive technologies to improve the quality of life in a fundamental way.

Broader Impacts: The project developed software and hardware to enhance individual sensory and cognitive capabilities, improve the remote monitoring and rehabilitation therapy delivery, and provide for personalized computer and robot enabled rehabilitation. The project also developed assessment and intervention methods for different types of chronic disorders, including neurological disorders that can help reduce healthcare costs and improve the quality of therapy practices.  The project also offered new insights into building adaptive rehabilitation systems that can accelerate the success of therapy of a large spectrum of injuries and diseases causing motor, neurological and cognitive disorders.

Last Modified: 11/02/2018
Modified by: Fillia S Makedon