Award Abstract # 1646470
CPS: Breakthrough: Wearables With Feedback Control

NSF Org: CNS
Division Of Computer and Network Systems
Recipient: RECTOR & VISITORS OF THE UNIVERSITY OF VIRGINIA
Initial Amendment Date: September 2, 2016
Latest Amendment Date: September 2, 2016
Award Number: 1646470
Award Instrument: Standard Grant
Program Manager: Sylvia Spengler
sspengle@nsf.gov
 (703)292-7347
CNS
 Division Of Computer and Network Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: September 1, 2016
End Date: August 31, 2020 (Estimated)
Total Intended Award Amount: $425,000.00
Total Awarded Amount to Date: $425,000.00
Funds Obligated to Date: FY 2016 = $425,000.00
History of Investigator:
  • John Stankovic (Principal Investigator)
    jas9f@virginia.edu
Recipient Sponsored Research Office: University of Virginia Main Campus
1001 EMMET ST N
CHARLOTTESVILLE
VA  US  22903-4833
(434)924-4270
Sponsor Congressional District: 05
Primary Place of Performance: University of Virginia
P. O. Box 400195
Charlottesville
VA  US  22904-4195
Primary Place of Performance
Congressional District:
05
Unique Entity Identifier (UEI): JJG6HU8PA4S5
Parent UEI:
NSF Program(s): CPS-Cyber-Physical Systems
Primary Program Source: 01001617DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7918, 8234
Program Element Code(s): 791800
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Recently there is an increasing availability of smart wearables including smart watches, bands, buttons and pendants. Many of these devices are part of human-in-the-loop Cyber Physical Systems (CPS). With future fundamental advances in the intersection of communications, control, and computation for these energy and resource limited devices, there is a great potential to revolutionize many CPS applications. Examples of possible applications include detecting and controlling hand washing to prevent transmission of infections or bacteria, monitoring and using interventions to keep factory workers safe, detecting activities in the home for monitoring the elderly, and improving rehabilitation of stroke victims via controlled exercises. However, to date, much of the work on wearables concentrates on only sensing, collecting and presenting data. For use in CPS it is necessary to consider the increased use of new sensing modalities, to apply feedback to close the control loop, and to focus on the fundamental issues of how both the environment and human behavior affect the cyber. In particular, since humans are intimately involved with wearables it is necessary to increase understanding of how human behaviors affect and can be affected by the control loops and how the systems can maintain safety.

This work develops generic underlying algorithms for processing smart wearable data rather than one-off solutions, it extends the understanding and control of wearable systems by addressing humans-in-the-loop behaviors, and it explicitly focuses on the impact of the environment and human behavior on the cyber. Novel ideas are proposed for each of these areas along with a structure for their integration. For example, the algorithmic approach to support more robust, accurate and efficient activity recognition using wearable devices is based on five fundamental concepts: (i) Direction Agnostic Modeling, (ii) Direction Aware Modeling, (iii) Spatial Reachability, (iv) Spatiotemporal Segmentation, and (v) Dynamic Space Time Warping. For dealing with humans-in the-loop behaviors, Model Predictive Control (MPC) is extended to semantic based MPC. This solves control problems that are not amenable to electromechanical laws and employs machine learning. Many CPS projects do not explicitly address how the uncertain world affects how the cyber must be developed in order to perform robustly and safely. A new *-aware software development paradigm focuses on physical-cyber CPS issues as central tenets and that serves as an integrating platform for all the proposed work. The *-aware paradigm focuses on how software must be made robust to handle the physical world, while meeting safety and adaptability requirements

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Emi, I. and Mondol, A. and Stankovic, J. "QuActive: A Grammar Based Micro-Activity Modeling Framework for Detecting Activity, Monitroing Activity Quality and Notifying the User" ICCPS2 , 2017 Citation Details
Mondol, A and Stankovic, J. "HAWAD: Hand Washing Detection using Wrist Wearable Inertial Sensors" International Conference on Distributed Computing in Sensor Systems and workshops , 2020 Citation Details
Samyoun, S. and Mondol, A and Stankovic, J. "Stress Detection via Sensor Translation" International Conference on Distributed Computing in Sensor Systems and workshops , 2020 https://doi.org/10.1109/DCOSS49796.2020.00017 Citation Details

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

There is an increasing availability of smart wearables including smart watches, bands, buttons and pendants. Many of these devices are part of human-in-the-loop Cyber Physical Systems (CPS). With future fundamental advances in the intersection of communications, control, and computation for these energy and resource limited devices, there is a great potential to revolutionize many CPS applications. Examples of possible applications include detecting and controlling hand washing to prevent transmission of infections or bacteria, monitoring and using interventions to keep factory workers safe, detecting activities in the home for monitoring the elderly, and improving rehabilitation of stroke victims via controlled exercises. However, to date, much of the work on wearables concentrates on only sensing, collecting and presenting data. For use in CPS it is necessary to consider the increased use of new sensing modalities, to apply feedback to close the control loop, and to focus on the fundamental issues of how both the environment and human behavior affect the cyber. In particular, since humans are intimately involved with wearables it is necessary to increase understanding of how human behaviors affect and can be affected by the control loops and how the systems can maintain safety.

This work first developed generic underlying algorithms for processing smart wearable data rather than one-off solutions.  For example, it developed an algorithmic approach to support more robust, accurate and efficient activity recognition using wearable devices based on five fundamental concepts called: (i) Direction Agnostic Modeling, (ii) Direction Aware Modeling, (iii) Spatial Reachability, (iv) Spatiotemporal Segmentation, and (v) Dynamic Space Time Warping. It applied such principles to multiple wearable solutions and to new visualization primitives. One solution detected handwashing conformance and was tested in the Pediatrics Intensive Care Unit of the University of Virginia Hospital. A voice-based cognitive assistant on smart watches was also developed to benefit stroke patients as they return home and monitored as a form of telemedicine. In another implemented application, using wearables for automatic detection of eating meals was incorporated into a study on obesity performed in more than 20 homes in Los Angeles. A smart watch solution was also developed that detects stress using the accelerometer and gyroscope only, and shown to correlate well with ground truth.


Last Modified: 09/07/2020
Modified by: John A Stankovic

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page