Award Abstract # 1565310
CHS: Large: Collaborative Research: Computational Science for Improving Assessment of Executive Function in Children

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: YALE UNIV
Initial Amendment Date: September 15, 2016
Latest Amendment Date: May 31, 2018
Award Number: 1565310
Award Instrument: Standard Grant
Program Manager: Ephraim Glinert
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: October 1, 2016
End Date: June 30, 2022 (Estimated)
Total Intended Award Amount: $1,205,294.00
Total Awarded Amount to Date: $1,221,294.00
Funds Obligated to Date: FY 2016 = $1,205,294.00
FY 2018 = $16,000.00
History of Investigator:
  • Morris Bell (Principal Investigator)
    morris.bell@yale.edu
  • Bruce Wexler (Co-Principal Investigator)
Recipient Sponsored Research Office: Yale University
150 MUNSON ST
NEW HAVEN
CT  US  06511-3572
(203)785-4689
Sponsor Congressional District: 03
Primary Place of Performance: CT Mental Health Center
34 Park Street
New Haven
CT  US  06519-1187
Primary Place of Performance
Congressional District:
03
Unique Entity Identifier (UEI): FL6GV84CKN57
Parent UEI: FL6GV84CKN57
NSF Program(s): HCC-Human-Centered Computing
Primary Program Source: 01001617DB NSF RESEARCH & RELATED ACTIVIT
01001819DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7367, 7925, 9251
Program Element Code(s): 736700
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

The identification of cognitive impairments in early childhood provides the best opportunity for successful remedial intervention, because brain plasticity diminishes with age. Attention deficit hyperactivity disorder (ADHD) is a psychiatric neurodevelopmental disorder that is very hard to diagnose or tell apart from other disorders. Symptoms include inattention, hyperactivity, or acting impulsively, all of which often result in poor performance in school and persist later in life. In this project, an interdisciplinary team of computer and neurocognitive scientists will develop and implement transformative computational approaches to evaluate the cognitive profiles of young children and to address these issues. The project will take advantage of both physical and computer based exercises already in place in 300 schools in the United States and involving thousands of children, many of whom have been diagnosed with ADHD or other learning disabilities. Project outcomes will have important implications for a child's success in school, self-image, and future employment and community functioning. The PIs will discover new knowledge about the role of physical exercise in cognitive training, including correlations between individual metrics and degree of improvement over time. They will identify important new metrics and correlations currently unknown to cognitive scientists, which will have broad impact on other application domains as well. And the PIs will develop an interdisciplinary course on computational cognitive science and one on user interfaces for neurocognitive experts.

The research will involve four thrusts. The PIs will devise new human motion analysis and computer vision algorithms that can automatically assess embodied cognition during structured physical activities, and which will constitute a breakthrough in improving the accuracy and efficiency of cognitive assessments of young children. Intelligent mining techniques will be used to discover new knowledge about the role of physical exercise in cognitive training and to find correlations between individual metrics and degree of improvement over time. A methodology will be developed using advanced multimodal sensing to collect and process huge amounts of evidence based assessment data with intelligent mechanisms that learn about a child's executive function capabilities and help uncover possible causes of cognitive dysfunctions. And a closed loop cognitive assessment system will be designed and implemented to understand and monitor a child's progress over time and provide recommendations and decision support to cognitive experts so they can make better treatment decisions.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

4. Muppala B, Tsiakas K, Fleury C, Weinstein A, Bell MD "Activate Test of Embodied Cognition (ATEC): A new automated system of cognitively demanding physical tasks to assess development of executive functioning." J Am Acad Child and Adolescent Psychiatry , v.58 , 2019 , p.S159
5. Bell MD, Abujelala M, Weinstein A, Fleury C, Pittman B "An automated digital assessment of executive functioning using cognitively demanding physical tasks (ATEC): reliability, concurrent validity and discriminant validity in a community sample." J Am Acad Child and Adolescent Psychiatry. , 2020
Gattupalli, Srujana and Babu, Ashwin Ramesh and Brady, James Robert and Makedon, Fillia and Athitsos, Vassilis "Towards Deep Learning based Hand Keypoints Detection for Rapid Sequential Movements from RGB Images" Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference , 2018 978-1-4503-6390-7
Konstantinos Tsiakas1, Michalis Papakostas1, Michail Theofanidis1, Morris Bell3, Rada Mihalcea4, ShouyiWang2, Mihai Burzo5, and Fillia Makedon1 "An Interactive Multisensing Framework for PersonalizedHuman Robot Collaboration and Assistive Training UsingReinforcement Learning" PETRA ?17 June 21-23, 2017, Island of Rhodes, Greece© 2017 ACM. ISBN 978-1-4503-5227-7/17/06. . . 15.00 , 2017 DOI: http://dx.doi.org/10.1145/3056540.3076191
Kuanar, Shiba and Athitsos, Vassilis and Pradhan, Nityananda and Mishra, Arabinda and Rao, KR "Cognitive Analysis of Working Memory Load from EEG, by a Deep Recurrent Neural Network" IEEE International Conference on Acoustics, Speech and Signal Processing , 2018
Miao, Xin and Zhen, Xiantong and Liu, Xianglong and Deng, Cheng and Athitsos, Vassilis and Huang, Heng "Direct Shape Regression Networks for End-to-End Face Alignment" Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018
Rajavenkatanarayanan, Akilesh and Babu, Ashwin Ramesh and Tsiakas, Konstantinos and Makedon, Fillia "Monitoring task engagement using facial expressions and body postures" Proceedings of the 3rd International Workshop on Interactive and Spatial Computing , 2018
Tsiakas, Konstantinosand Abujelala, Maherand Rajavenkatanarayanan, Akileshand Makedon, Fillia "User Skill Assessment Using Informative Interfaces for Personalized Robot-Assisted Training" Learning and Collaboration Technologies. Learning and Teaching , 2018 978-3-319-91152-6
Xiang, Wei and Zhang, Dong-Qing and Yu, Heather and Athitsos, Vassilis "Context-Aware Single-Shot Detector" IEEE Winter Conference on Applications of Computer Vision , 2017 978-1-5386-4886-5

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

This project aimed to develop a new method of assessing executive function in children using cognitively demanding physical tasks that require thinking in action.  Motion capture technology and scoring algorithms were developed by the computer science team to evaluate children?s performance. The result was a novel approach to cognitive assessment called the Activate Test of Embodied Cognition (ATEC). Conventional tests of cognitive functioning rely on pencil and paper, or computer administered tasks (like IQ tests) and have modest relationships to children?s functioning in daily life.  Embodied cognition is a construct based on the fact that neurocognition develops along with and by way of physical movement, is sustained by physical activity and that the mind and body interact continuously in daily functioning.

During this project, our team developed 32 ATEC tasks that measure balance, motor speed, rhythm and coordination, attention, working memory, response inhibition and self-regulation. Video instructions for each task were scripted and recorded and then a separate video was created to administer the task (like an exercise video). These were embedded in original software, downloadable on conventional computer equipment. Originally, Kinect cameras were used for motion capture, but the computer science team learned that accurate scoring algorithms could be achieved using conventional webcam data. This finding allowed ATEC to become an easily transferrable technology. 

The computer science team was able to develop automatic scoring for several of the tasks and achieved better than 90% accuracy compared to expert scoring.  These were scientific accomplishments, published in the proceedings of leading computer science journals. Algorithm development has not been completed for all the tasks and will become more accurate as more data accumulates. When these scoring algorithms are embedded in the software, ATEC will then provide automated scoring of ATEC performance.  Meanwhile, a scoring pamphlet for hand scoring all the tasks was developed and used for research purposes.

ATEC data was collected from a community sample of 55 children, age 5 to 11 years, at their neighborhood schools. Children were assessed twice, two weeks apart, for determining reliability, and they were also given standard cognitive tests.  Their parents completed questionnaires on their functioning in daily life. ATEC total scores were significantly correlated with age and with concurrent measures of executive functioning. It showed discriminant validity between At-Risk children and normal range children on parent ratings of overall competency, attention problems, cognitive regulation problems and overall executive function problems.  ATEC total score was also found to be a better predictor of these problems than conventional cognitive tests. ATEC had excellent test-retest reliability with only a small practice effect.  This means that ATEC is valid and reliable and can be used to test changes as a child grows older or through interventions aimed at improving executive functions. These findings were published in the journal, Child Neuropsychology in 2021.

In the final two years of the project, an adult version of ATEC was developed to measure decline in cognitive functioning and is being studied with older adults and those with specific cognitive disorders including acquired brain injury and dementia. The adult version and child version combined are called the Automated Test of Embodied Cognition (ATEC), and a patent application for the computer application has been submitted to the US Patent Office and Trademark Office under that name. 

ATEC is the first assessment instrument to systematically measure the construct of Embodied Cognition.  It is the first to use motion capture technology and machine learning algorithms to provide users a high- fidelity, computerized administration and scoring system.  As a transferrable technology, it can be widely used as a downloadable computer application, and it can be easily translated into other languages for international use.  It is also possible that it can be developed for telehealth and that a modified version can be used for in-home assessments. 

 


Last Modified: 11/14/2022
Modified by: Morris Bell

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page