The goal of this award was to design and develop iSee, a smartphone/Fitbit-based depression behavior monitoring and analytics platform with the objective to enhance the efficiency of delivering mental health services at college and university counseling centers.

The project resulted in multiple outcomes. First, the project contributed to user studies on clinicians and college students at the MSU counseling center to assess the needs of both clinicians and students and understand the benefits and obstacles of the proposed technology. Major results from clinicians' interviews included those clinicians considered our behavioral monitoring technology as an extension of counseling service during working hours; they generally preferred to provide support messages to students with depression based on their tracked data; and they offered specific suggestions to make better use of tracking technology such as goal setting and positive reinforcement. Major results from students' interviews included challenges in managing their depression conditions (e.g., lack of motivation and energy), their current self-tracking practice and self-management techniques, and suggestions to motivate continued use of our behavioral monitoring technology.

Second, the project contributed to the design and development of a new smartphone/Fitbit-based sensor-rich platform that can continuously and passively collect data from a wide range of smartphone/Fitbit sensors including accelerometer, gyroscope, GPS, audio, Bluetooth, Wi-Fi, screen touch, app usage, and sleep duration and quality. The sensor-rich platform has been successfully implemented in both iOS and Android devices, which cover majority of the college student users. Moreover, the project contributed to the design and development of  human-system interfaces for the delivery of the enhanced counseling services to both clinicians and patients.

Third, the project resulted in the development of five depression behavioral markers from the sensor data collected by the developed sensor-rich platform. Specifically, we have successfully extracted phone usage behavioral markers from smartphone touch screen sensor and app usage meta data; we have successfully extracted travel behavioral markers from smartphone GPS sensor; and we have successfully extracted social behavioral markers from phone call records, we have successfully extracted physical activity/sedentary behavioral markers from Fitbit accelerometer; and we have successfully extracted sleep behavioral markers from Fitbit accelerometer and heart rate sensor.

Fourth, the project also contributed 1) a set of deep learning-based algorithms that can infer the severity of an individual's mental health condition based on the extracted robust depression behavioral markers with an average accuracy of 88.7% across all the participants; 2) a set of causality-based algorithms to infer the causality relationship between the extracted depression behavioral markers and the degradation or improvement of mental health conditions; and 3) a number of federated learning techniques that enhances the mental health condition severity prediction accuracy by learning from other peers’ smartphone/Fitbit sensor data. In particular, one key contribution within this theme is the design and development of new automated machine learning (AutoML)-based algorithm for recognizing the severity of an individual's mental health condition. Specifically, this project resulted in a AutoML-based technique that can automatically identify the architectures of the deep learning models through search algorithms including evolutionary algorithm, reinforcement learning, and Bayesian optimization. This automatically designed deep learning model takes the smartphone/Fitbit sensor data as its input, transforms the sensor data into markers, and generates classification results to recognize the severity of an individual's mental health condition. Another key contribution within this theme is the design of new data augmentation techniques that can automatically identify the optimal policies of data augmentation operators through gradient matching. This technique improves the classification accuracy of the deep learning model to infer severity of an individual's mental health condition. The third highlighted contribution within this theme is the design of federated learning technique based on masking and mask fusion that is able to train a personalized deep learning model by learning from both the individual's own data as well as the data of people who share similar behavioral patterns without sharing data between each other.

Collectively, the project contributed to sixteen conference and journal publications, three PhD dissertations, and two Master’s thesis. It provided training opportunities to six PhD students, two Master students, and one undergrad student. The human-centered smart service system (iSee) developed in this project has a significant societal impact on reducing costs of mental health delivery as well as enhancing students’ quality of lives.  Finally, the findings of this project have been integrated into the teaching materials of the PI’s graduate course and have been disseminated through the presentations at a number of premier international conferences, the invited talks at universities and companies, and the coverage of leading national and international media.

Last Modified: 01/23/2023
Modified by: Mi Zhang