Award Abstract # 1712580
Collaborative Research: Collaborative Learning for Multimodal Data

NSF Org: DMS
Division Of Mathematical Sciences
Recipient: OHIO STATE UNIVERSITY, THE
Initial Amendment Date: May 10, 2017
Latest Amendment Date: May 10, 2017
Award Number: 1712580
Award Instrument: Standard Grant
Program Manager: Gabor Szekely
DMS
 Division Of Mathematical Sciences
MPS
 Directorate for Mathematical and Physical Sciences
Start Date: July 1, 2017
End Date: June 30, 2021 (Estimated)
Total Intended Award Amount: $124,963.00
Total Awarded Amount to Date: $124,963.00
Funds Obligated to Date: FY 2017 = $124,963.00
History of Investigator:
  • Yunzhang Zhu (Principal Investigator)
    Zhu.219@osu.edu
Recipient Sponsored Research Office: Ohio State University
1960 KENNY RD
COLUMBUS
OH  US  43210-1016
(614)688-8735
Sponsor Congressional District: 03
Primary Place of Performance: Ohio State University
1928 Neil Avenue, Cockins Hall
Columbus
OH  US  43210-1016
Primary Place of Performance
Congressional District:
03
Unique Entity Identifier (UEI): DLWBSLWAJWR1
Parent UEI: MN4MDDMN8529
NSF Program(s): STATISTICS
Primary Program Source: 01001718DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s):
Program Element Code(s): 126900
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.049

ABSTRACT

A multimodal paradigm has become increasingly important given today's explosive growth of information, which often arises from, for instance, automatic image categorization and personalized prediction. Multimodal data has a wide spectrum of applications in medical diagnostics, social networking, multimedia, information filtering, personalized advertising, consumers' recommendations, virtually in any electronic commerce and entertainment platform. This research aims to develop statistical theory, methods, and computational tools to integrate multimodal data for prediction and description. The development will lead to the higher accuracy of learning, which will ultimately enhance information storage, sorting and filtering. Moreover, the research project has an education component to train graduate students in emerging areas. The research products will be disseminated through publications and presentations.

The proposed research aims to develop statistical techniques to utilize conditional dependence structures for integrating multimodal data. It will proceed in the areas of collaborative learning and personalized prediction. In each area, regression, classification, and ranking will be performed collaboratively based on pairwise conditional dependencies between the response components, modeled by a directed graph or an undirected graph. Special efforts will be devoted to the joint learning of data of multiple modalities and extraction of latent structures with an adjustment for covariates. Target applications include image categorization and recommender systems, where the proposed techniques will be applied to understand the content of an image and to predict personalized preference over a large number of items. Furthermore, The research will develop computational tools and design methods that have desirable statistical properties.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Che, Yuezhang and Zhu, Yunzhang and Shen, Xiaotong "Multilabel Classification With Multivariate Time Series Predictors" IEEE Transactions on Signal Processing , v.68 , 2020 https://doi.org/10.1109/TSP.2020.3027277 Citation Details
Tang, Shuhan F. and Craigmile, Peter and Zhu, Yunzhang "Spectral Estimation Using Multitaper Whittle Methods With a Lasso Penalty" IEEE Transactions on Signal Processing , v.67 , 2019 10.1109/TSP.2019.2932879 Citation Details
Zhu, Yunzhang "A convex optimization formulation for multivariate regression" Advances in neural information processing systems , 2020 Citation Details
Zhu, Yunzhang and and Liu, Renxiong "An algorithmic view of l2-regularization and some path-following algorithms" Journal of machine learning research , 2021 Citation Details
Zhu, Yunzhang and Li, Lexin "Multiple Matrix Gaussian Graphs Estimation" Journal of the Royal Statistical Society Series B: Statistical Methodology , v.80 , 2018 https://doi.org/10.1111/rssb.12278 Citation Details
Zhu, Yunzhang and Shen, Xiaotong and Jiang, Hui and Wong, Wing Hung "Collaborative multilabel classification*" Journal of the American Statistical Association , 2021 https://doi.org/10.1080/01621459.2021.1961783 Citation Details
Zhu, Yunzhang and Shen, Xiaotong and Pan, Wei "On High-Dimensional Constrained Maximum Likelihood Inference" Journal of the American Statistical Association , 2018 10.1080/01621459.2018.1540986 Citation Details

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

The big-data era leads to the creation of data science at the interface of computer science and statistics. While big data demand scalable methods and their associated theories, it also requires extraction and integration of the knowledge and information across heterogeneous data of different types, particularly multimodal data involving images, audios, and texts. The research project sheds some new insight into leveraging multivariate dependence structures to integrate multimodal data to improve our capability of analyzing this type of complex data. On this ground, we develop statistical theory and methodology utilizing the conditional dependence structure, that is, label dependence in classification and pairwise component dependence in regression. As a result of our effort, we can generalize our personalized prediction to unseen and novel examples.  

This collaborative project has incorporated research results in teaching to create an exciting opportunity for students to learn state-of-art technology and emerging research areas. During the project period, the principal investigators have mentored 3 M.S. students and 10 Ph.D. students. Concerning research products, the project has produced about 25 journal publications and several publicly available software packages for end-users. The principal investigators have also delivered more than 20 invited presentations at national and international conferences, workshops, and institutional colloquia.

 

 


Last Modified: 08/18/2021
Modified by: Yunzhang Zhu

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page