Skip to feedback

Award Abstract # 1565604
CRII: CHS: WiFi-Based Human Behavior Sensing and Recognition System for Aging in Place

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: MICHIGAN STATE UNIVERSITY
Initial Amendment Date: June 15, 2016
Latest Amendment Date: August 28, 2017
Award Number: 1565604
Award Instrument: Continuing Grant
Program Manager: Ephraim Glinert
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: August 15, 2016
End Date: July 31, 2019 (Estimated)
Total Intended Award Amount: $171,643.00
Total Awarded Amount to Date: $171,643.00
Funds Obligated to Date: FY 2016 = $90,800.00
FY 2017 = $80,843.00
History of Investigator:
  • Mi Zhang (Principal Investigator)
    zhang.13664@osu.edu
Recipient Sponsored Research Office: Michigan State University
426 AUDITORIUM RD RM 2
EAST LANSING
MI  US  48824-2600
(517)355-5040
Sponsor Congressional District: 07
Primary Place of Performance: Michigan State University
428 S. Shaw Street Room 3530
East Lansing
MI  US  48824-1226
Primary Place of Performance
Congressional District:
07
Unique Entity Identifier (UEI): R28EKN92ZTZ9
Parent UEI: VJKZC4D1JN36
NSF Program(s): CRII CISE Research Initiation
Primary Program Source: 01001617DB NSF RESEARCH & RELATED ACTIVIT
01001718DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7367, 8228
Program Element Code(s): 026Y00
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

As "baby boomers" age, the United States will experience considerable growth in its elderly population over the coming years. Studies consistently confirm that the majority of older adults would prefer to remain in their own homes for as long as possible. Therefore, there is a critical need for home-based assisted living technologies capable of continuously yet unobtrusively monitoring activities of daily living (ADLs) and detecting abnormal events, both to reduce the cost of elder care and to enhance the quality of life. Current human behavior monitoring systems for aging in place, which are typically based on cameras, smartphone/wearable devices, or ambient sensors, have fundamental limitations such as high cost and invasion of privacy that prevent them from being widely deployed. The PI's objective in this project is to build on his prior work to establish a research program to investigate a new approach to aging in place that harnesses the now-ubiquitous commercial home WiFi signals to monitor ADLs and detect abnormal events. The central idea is that different human activities cause different changes in WiFi signals; by analyzing these changes, the activity that caused the change can be recognized. This work will have broad societal impact both within the United States and abroad, by contributing to new techniques and systems for WiFi-based human behavior sensing and recognition in both single-subject and multi-subject scenarios. If the new system is effective, it will provide a non-intrusive, device-free, low-cost and privacy-preserving assisted living technology for aging in place. The PI will integrate research results from this project into both his undergraduate and graduate courses, as well as the K-12 education program; furthermore, the hardware and software developed in this research will be open-source, and the dataset collected during this project will be made available to others for further research.

The PI plans to exploit the fine-grained PHY layer Channel State Information (CSI) extracted from the WiFi signals as the basis for a unified scheme for monitoring both the most common stationary and moving activities performed daily by older adults in their homes. He will detect stationary activities by tracking the minute but periodic chest movements caused by breathing, and he will extract frequency domain features to robustly recognize the same moving activity even with different movement directions or at different locations. The PI will develop Markov models to recognize complex ADLs, and he will leverage the breathing and physical body movement information to detect abnormal behaviors including accidental falls and disturbed sleep that are potential issues relating to aging in place. Ultimately, the PI will extend his techniques to recognize ADLs of multiple persons performed at the same time. To successfully achieve these objectives, the PI will need to overcome a number of significant technical challenges, for example detecting minute changes in the WiFi signal due to stationary activities such as working at a computer or watching TV while seated on a sofa. Robustly recognizing the same moving activity (e.g., housecleaning) performed in different ways or at different locations will also be tricky, because different movement directions or different layouts at different locations cause different disturbances to WiFi signals.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Biyi Fang, Jillian Co, and Mi Zhang. "DeepASL: Enabling Ubiquitous and Non-Intrusive Word and Sentence-Level Sign Language Translation" ACM Conference on Embedded Networked Sensor Systems (SenSys'17). , 2017
Biyi Fang, Nicholas D. Lane, Mi Zhang, Aidan Boran, and Fahim Kawsar. "BodyScan: Enabling Radio-based Sensing on Wearable Devices for Contactless Activity and Vital Sign Monitoring" The 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys) , 2016
Biyi Fang, Nicholas D. Lane, Mi Zhang, Aidan Boran, and Fahim Kawsar. "Poster: Towards Radio-based Sensing on Wearables" The 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys) Companion , 2016
Biyi Fang, Nicholas D. Lane, Mi Zhang, and Fahim Kawsar. "HeadScan: A Wearable System for Radio-based Sensing of Head and Mouth-related Activities" The 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) , 2016
Biyi Fang, Xiao Zeng, and Mi Zhang "NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Vision" ACM International Conference on Mobile Computing and Networking (MobiCom) , 2018
Fazlay Rabbi, Taiwoo Park, Biyi Fang, Mi Zhang, and Youngki Lee. "When Virtual Reality Meets Internet of Things in the Gym: Enabling Immersive Interactive Machine ExercisesFazlay Rabbi*, Taiwoo Park*, Biyi Fang, Mi Zhang, and Youngki Lee." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) , v.2 , 2018
Xiao Zeng, Kai Cao, and Mi Zhang. "MobileDeepPill: A Small-Footprint Mobile Deep Learning System for Recognizing Unconstrained Pill Images" ACM International Conference on Mobile Systems, Applications, and Services (MobiSys'17). , 2017

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

The goal of this award was to create a device-free wireless sensing system named WiCare that uses Wi-Fi signal as the sensing modality to non-intrusively monitor elderly people's activities of daily living (ADLs), vital signs, and detect abnormal behaviors such as accidental falls and disrupted sleep to support aging in place.

The project resulted in a number of machine/deep learning-based wireless signal processing pipelines for recognizing activities of daily living involving walking, standing up, sitting down, being stationary, and falling. Significant advances were made in developing deep learning-based models for processing wireless signals, where deep neural network models which are traditionally used for processing images and audios are repurposed to process multi-channel wireless signals. Fundamental contributions were also made to the area of deep neural network compression. The project contributed to the development of novel deep neural network model compression techniques that are able to successfully reduce the computational and memory intensity of deep neural network models such that the deep learning based wireless signal processing pipeline can be implemented in resource constrained wireless devices such as WiFi routers. Furthermore, this project contributed to the design and development of data augmentation techniques for enhancing the robustness of deep learning based wireless signal processing pipelines to real-world interferences. This result is important to demonstrate the feasibility of deploying the developed wireless sensing systems in real-world scenarios such as homes.

The project also contributed to the collection of two high-quality datasets. The first dataset was collected in well-controlled laboratory settings whereas the second dataset was collected in real-world environments. These two datasets not only provide the training samples and testbenches to develop and evaluate our machine/deep learning-based wireless sensing pipelines, but only provide insights into how real-world interferences affects the performance of the wireless sensing systems. The results are expected to aid in identifying more interference-robust techniques to enhance the performance of the system.

Collectively, the project contributed to one book chapter, one journal publication, and six conference publications, one PhD dissertation, and one Master’s thesis. It provided training opportunities to four PhD students, two Master students, and one undergrad student. The device-free wireless sensing technology developed in this project has a significant societal impact on reducing costs of elder care as well as enhancing the quality of life of older adults.  Finally, the findings of this project have been integrated into the teaching materials of the PI’s graduate course and have been disseminated through the presentations at a number of premier international conferences, the invited talks at universities and companies, and the coverage of leading national and international media.

 


Last Modified: 11/25/2019
Modified by: Mi Zhang

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page