Award Abstract # 1910983
III: Small: Collaborative Research: Scalable Deep Bayesian Tensor Decomposition

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: UNIVERSITY OF UTAH
Initial Amendment Date: September 7, 2019
Latest Amendment Date: September 7, 2019
Award Number: 1910983
Award Instrument: Standard Grant
Program Manager: Sylvia Spengler
sspengle@nsf.gov
 (703)292-7347
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: October 1, 2019
End Date: September 30, 2023 (Estimated)
Total Intended Award Amount: $298,370.00
Total Awarded Amount to Date: $298,370.00
Funds Obligated to Date: FY 2019 = $298,370.00
History of Investigator:
  • Shandian Zhe (Principal Investigator)
    zhe@cs.utah.edu
Recipient Sponsored Research Office: University of Utah
201 PRESIDENTS CIR
SALT LAKE CITY
UT  US  84112-9049
(801)581-6903
Sponsor Congressional District: 01
Primary Place of Performance: University of Utah
50 S Central Campus Dr. Rm 3190
Salt Lake City
UT  US  84112-9205
Primary Place of Performance
Congressional District:
01
Unique Entity Identifier (UEI): LL8GLEVH6MG3
Parent UEI:
NSF Program(s): Info Integration & Informatics
Primary Program Source: 01001920DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7364, 7923
Program Element Code(s): 736400
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Many applications in the real world, such as online
shopping, recommendation, social media and information
security, involve interactions among different
entities. For example, online shopping behaviors can
be simply described by the interactions between
customers, commodities and shopping web sites. These
interactions are naturally represented by tensors,
which are arrays of multiple dimensions. Each
dimension represents a type of entities (e.g.,
customers or commodities), and each element describes
a particular interaction (e.g, purchased/not
purchased). The project aims to develop flexible and
efficient tensor decomposition approaches that can
discover a variety of complicated relationships
between the entities in tensors, handle a tremendous
amount of data from practical applications, and adapt
to rapid data growth. The developed approaches can be
used to promote many important prediction and
knowledge discovery tasks, such as improving the
recommendation accuracy, predicting advertisement
click rates, understanding how misinformation propagation
through social media, and detecting malicious cell-
phone apps.

Despite the success of the existing tensor
decomposition approaches, they use multilinear
decomposition forms or shallow kernels, and are
incapable of capturing highly complicated
relationships in data. However, complex and nonlinear
relationships, effects and patterns are ubiquitous,
due to the diversity and complexity of the practical
applications. Furthermore, there is a lack of
efficient, scalable nonlinear decomposition algorithms
to handle static tensors nowadays at unprecedented
scales, and dynamic tensors that grow fast and
continuously. The project aims to develop scalable
deep Bayesian tensor decomposition approaches that
maximize the flexibility to capture all kinds of
complex relationships, efficiently process static data
at unprecedented scales and rapid data streams, and
provide uncertainty quantification for both embedding
estimations and predictions. The research will be
accomplished through: (1) the design of new Bayesian
tensor decomposition models that incorporate deep
architectures to improve the capability of estimating
intricate functions, (2) the development of
decentralized, asynchronous learning algorithms to
process extremely large-scale static tensors, (3) the
development of online incremental learning algorithms
to handle rapid data streams and to produce responsive
updates upon receiving new data, without retraining
from scratch, and (4) comprehensive evaluations on
both synthetic and real-world big data. The proposed
research will contribute a markedly improved tensor
decomposition toolset that are powerful to estimate
arbitrarily complex relationships, scalable to static
tensors at unprecedented scales (e.g., billions of
nodes and trillions of entries) and to fast data
streams with efficient incremental updates. Moreover,
as Bayesian approaches, the toolset are resilient to
noise, provide posterior distributions for
uncertainty quantification, and integrate all possible
outcomes into robust predictions. Once the toolsets are
available, the understanding of the high-order
relationships in tensors, and the mining of associated
patterns, such as communities and anomalies, will be
enormously enhanced; the predictive performance for
the quantify of interests, such as social links,
click-through-rates, and recommendation, will be
dramatically promoted.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Fang, Shikai and Narayan, Akil and Kirby, Robert M. and Zhe, Shandian "Bayesian Continuous-Time Tucker Decomposition" The 39th International Conference on Machine Learning (ICML) , 2022 Citation Details
Fang, Shikai and Wang, Zheng and Pan, Zhimeng and Liu, Ji and Zhe, Shandian "Streaming Bayesian Deep Tensor Factorization" Proceedings of the 38th International Conference on Machine Learning , 2021 Citation Details
Fang, Shikai and Zhe, Shandian "Bayesian Streaming Sparse Tucker Decomposition" Proceedings of the 37th Conference on Uncertainty in Artificial Intelligence (UAI) , 2021 Citation Details
Li, Shibo Li and Xing, Wei and Kirby, Robert M. and Zhe, Shandian "Scalable Gaussian Process Regression Networks" International Joint Conference on Artificial Intelligence - Pacific Rim International Conference on Artificial Intelligence (IJCAI-PRICAI) , 2020 Citation Details
Pan, Zhimeng and Wang, Zheng and Zhe, Shandian "Streaming Nonlinear Bayesian Tensor Decomposition" The Conference on Uncertainty in Artificial Intelligence (UAI) , 2020 Citation Details
Tillinghast, Conor and Fang, Shikai and Zhang, Kai and Zhe, Shandian "Probabilistic Neural-Kernel Tensor Decomposition" 2020 IEEE International Conference on Data Mining (ICDM) , 2020 https://doi.org/10.1109/ICDM50108.2020.00062 Citation Details
Tillinghast, Conor and Wang, Zheng and Zhe, Shandian "Nonparametric Sparse Tensor Factorization with Hierarchical Gamma Processes" The 39th International Conference on Machine Learning (ICML) , 2022 Citation Details
Tillinghast, Conor and Zhe, Shandian "Nonparametric Decomposition of Sparse Tensors" Proceedings of the 38th International Conference on Machine Learning , 2021 Citation Details
Wang, Zheng and Xing, Wei and Kirby, Robert M. and Zhe, Shandian "Multi-Fidelity High-Order Gaussian Processes for Physical Simulation" Proceedings of The 24th International Conference on Artificial Intelligence and Statistics , 2021 Citation Details
Wang, Zheng and Zhe, Shandian "Nonparametric Factor Trajectory Learning for Dynamic Tensor Decomposition" The 39th International Conference on Machine Learning (ICML) , 2022 Citation Details

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

The project on tensor decomposition stands as a cornerstone in the realm of data science, aiming to unearth valuable representations and intricate knowledge from the pervasive high-order multi-way interaction data encountered in real-world applications. This investigation leverages deep architectures, probabilistic modeling, and efficient computational techniques to forge a new suite of tensor decomposition tools. These tools are expressly designed to tackle critical challenges associated with analyzing tensor data in the current era of big data, where complexities range from highly nonlinear and nonsmooth relationships to the efficient handling of large volumes of static data and rapidly generated dynamic data streams, as well as reliable uncertainty quantification.

The outcomes of the project are multifaceted:

  1. High-Quality Research Publications:

    • The project has produced a substantial body of research, generating ten high-quality publications. These publications bridge the gap between classical tensor decomposition methods and the practical needs arising in real-world tensor data analysis in the current era of big data.
  2. GitHub Repositories:

    • Eleven GitHub repositories have been created, encompassing the comprehensive implementation and documentation of the developed tensor decomposition methods. These repositories are shared with the broader data science and machine learning community, fostering collaboration and knowledge exchange.
  3. Benchmark Datasets and Evaluation Frameworks:

    • A curated list of benchmark datasets, testing baselines, performance metrics, and evaluation results is released alongside the GitHub repositories. This initiative enables the community and researchers to continually enhance and refine the tools and methods developed throughout the project.
  4. Educational Contributions:

    • Two machine learning and data science courses have been built and conducted as part of the project. These courses not only disseminate foundational machine learning knowledge but also expose students to cutting-edge research in probabilistic learning. By fostering engagement among students from diverse backgrounds, the project encourages their participation in this evolving field, motivating some of them to pursue higher education.

In summary, the project not only advances the theoretical and practical aspects of tensor decomposition but also actively contributes to the wider scientific community through publications, open-source implementations, benchmark datasets, and educational initiatives.


Last Modified: 01/15/2024
Modified by: Shandian Zhe

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page