| NSF Org: |
CNS Division Of Computer and Network Systems |
| Recipient: |
|
| Initial Amendment Date: | February 25, 2016 |
| Latest Amendment Date: | April 7, 2020 |
| Award Number: | 1552924 |
| Award Instrument: | Continuing Grant |
| Program Manager: |
Alexander Sprintson
CNS Division Of Computer and Network Systems CSE Directorate for Computer and Information Science and Engineering |
| Start Date: | March 1, 2016 |
| End Date: | February 28, 2022 (Estimated) |
| Total Intended Award Amount: | $542,403.00 |
| Total Awarded Amount to Date: | $542,403.00 |
| Funds Obligated to Date: |
FY 2017 = $149,738.00 FY 2018 = $56,221.00 FY 2019 = $116,954.00 FY 2020 = $121,701.00 |
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
7 LEBANON ST HANOVER NH US 03755-2170 (603)646-3007 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
6211 Sudikoff Laboratory Hanover NH US 03755-3510 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): |
CSR-Computer Systems Research, Networking Technology and Syst |
| Primary Program Source: |
01001718DB NSF RESEARCH & RELATED ACTIVIT 01001819DB NSF RESEARCH & RELATED ACTIVIT 01001920DB NSF RESEARCH & RELATED ACTIVIT 01002021DB NSF RESEARCH & RELATED ACTIVIT |
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
The ability to sense and detect human movement is critical to the development of data-driven mobile health systems. It can help detect disease and foster behavioral changes to cultivate healthy lifestyles. Existing sensing technologies either require users to constantly wear or carry on-body potentially cumbersome devices, are vulnerable to electromagnetic interference, or present severe privacy risks involving leaking of sensitive data and images. This project takes a entirely different approach to addressing these issues. It exploits the use of ubiquitous light as a low-cost, unobtrusive, and accurate sensing medium capable of simultaneously sensing people and their surrounding context. The proposed vision " LightSense " consists of off-the-shelf LED lights on the ceiling and a few low-cost photodiode sensors sprinkled in the environment. The photodiodes passively capture light blockage created by the human body and reconstruct fine-grained user behaviors in real time. LightSense leverages light to turn a space into a cognitive space, which recognizes our presence, senses our behaviors such as postures and high-level activities while monitoring our health status indicators such as levels of stress.
LightSense is empowered by Visible Light Communication (VLC) that turns the visible light into computational light. It contains the following novel systems and algorithmic designs: 1) a novel VLC network architecture with LED panels and sparse photodiodes to ease system deployment; 2) algorithmic and systems designs to separate light rays from dense LEDs, optimize the placement of photodiodes, and overcome the blockage of other objects (e.g., furniture, other users); 3) a new VLC primitive that allows light communication and sensing to be sustained even under extremely low light conditions; and 4) learning algorithms to infer physical activities, derive movement characteristics, and monitor psychological state. LightSense will be evaluated using real-scale testbeds and user studies. Results from this project will establish the foundational pieces to define a new research space (visible light sensing), and will generate far-reaching impact on promoting innovative interaction designs and enabling new types of precise health monitoring.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
The overall goal is to advance the state of art on human behavioral sensing and object tracking by exploring a new sensing paradigm that exploits ubiquitous visible light to sense users/objects and their contextual information.
== Intellectual merits
During the project period, the team have made following intellectual contributions.
1. Human sensing
1) The design, implementation, and evaluation of a light sensing system that reconstructs skeleton poses in real time, without needing cameras or on-body sensors. The optimization of photodiode placement to mitigate the impact of furniture blockage on light sensing. Refinement on the skeleton pose reconstruction algorithm to deal with a mobile user with unknown orientation and location.
2) Algorithmic design to enable the reconstruction of hand skeleton poses using a table lamp augmented by light sensing. It consists of an LED panel inside the lampshade and a grid of 16 low-cost photodiodes embedded in the lamp base.
3) The design, implementation, and evaluation of a self-powered module that reuses ambient light for both finger gesture sensing and energy harvesting. It consists of arrays of photodiodes operating in the photovoltaic mode, harvesting energy from ambient light while recognizing finger gestures based on changes in instantaneously harvested power. Recognition algorithm was designed to enable robust gesture recognition in diverse ambient light settings.
4) The design, implementation, and evaluation of battery-free light sensing systems that track gaze direction and pupil movement/size. Novel hardware designs to sense light reflected by the eye while addressing ambient light dynamics. Algorithmic designs and machine learning models to address user diversity and adapt system's sensing frequency.
5) The design, implementation, and evaluation of a wearable technology that uses near-infrared light to sense body distance and relative body orientation during face-to-face social interactions. A fusion algorithm is designed to fuse light signals and inertial sensor readings to improve sensing robustness.
6) The design, implementation, and evaluation of an ear-worn system that detects faces touches in sensitive/muscosal zones. It combines thermal infrared sensing to detect an approaching hand and physiological sensing to detect impedance changes caused by skin deformation during a touch. A convolutional neural network model is developed for multi-modal sensing.
2. Object Sensing/Tracking
1) Integration of light and inertial sensing to achieve accurate and robust object tracking. A novel light cover design casts polarization patterns in the space and provide external landmarks for calibrating inertial sensor's drift errors.
2) The design, implementation, and evaluation of a novel optical tag design that is passive and imperceptible. Algorithmic designs encode data while dealing with partial occlusion, detect the tag from background scenery, and decode data against ambient light interference and off-axis viewing.
3) The design, implementation, and evaluation of a laser-based localization technique that allows an aerial drone to directly locate underwater robots without any relays on water's surface. A novel optic-fiber sensing ring senses extremely-weak retro-reflected light from the underwater robot. A pin-hole based incident angle sensing scheme deals with the sensing skew at the air-water boundary. A laser-optimized backscatter communication modulates retro-reflected light to send back robot's angle and depth information.
== Broad impacts
Research outcomes have been disseminated 15 publications in top conferences and workshops. The project has also led to one technical report and two PhD theses. Results have also been integrated into PI's courses.
They have triggered or been cited by active follow-up works. Dataset, source code, and demo video are available at project website: http://dartnets.cs.dartmouth.edu/career
Project demo videos were produced and disseminated in the lab YouTuble channel: https://www.youtube.com/channel/UCQhj_t5VL-SnOAj24EJcwaw
Research results have been disseminated in PI's 13 seminar talks, 5 keynotes, and 5 invited talks. The project has provided opportunities to train 1 post-doc researcher, 5 PhD students, 4 Master students, and 10 undergraduate students.
The team have broadly disseminated results in local community and engaged the public.
- In 2016, the PI participated in Workshop on Exploring Graduate Study in Computer Science and delivered a talk on light communication and sensing to a group of 20 undergraduate students.
- In 2016, 2017, and 2019, the PI participated in the Hour of Code session at the local Ray Elementary School , for 40+ Grade-5 students, 87 Grade-5 students, and 89 Grade-4 students, respectively.
- In February 2019, the PI participated in the 2nd Annual "Introduce a Girl to an Engineer" day at the American Precision Museum in Windsor, Vermont, where the PI introduced the concept of light communication and sensing.
- In March 2019, project results are featured in an NSF Science Nation video (https://www.nsf.gov/news/special_reports/science_nation/visiblelightcommunication.jsp).
- In April 2019, the PI delivered a keynote introducing the research of this project at the 2019 TechWomen | TechGirls annual awards luncheon, the New Hampshire Tech Alliance.
- In October 2019, the PI has disseminated the results via the STEM Pathways program at Hanover High School. The PI introduced the concept of light communication and sensing to 20+ high school students.
Last Modified: 05/23/2022
Modified by: Xia Zhou
Please report errors in award information by writing to: awardsearch@nsf.gov.
