The primary objective of the project was developing modern large scale methods for finding relevant dimensions, directions to subspaces in data. 

The first part of the project concerned development of stochastic and distributed methods for handling large scale linear dimensionality reduction methods, both based on single-view data (as in PCA) and based on multi-view data (as in CCA and PLS).  Finding the most important linear directions in the data, or the directions that are most conserved in multiple views, has been a basic building block in statistics, data analysis and machine learning, for decades.  Multi-view methods in particular are important in leveraging the relationship between different modalities, such as audio and visual queues, or the relationship between multiple related tasks.  We develop methods that allow doing so on a much larger scale where traditional linear-algebraic methods cannot go. These methods employ recent developments on stochastic optimization, that is working on only samples from the data at each step instead of the entire dataset, which might be too large to handle in its entirety.  The methods also allow using multiple computers in a distributed fashion, which is often necessary for handling massive data sets.  Due to the importance of multi-view dimensionality reduction techniques and the increase in data set sizes in the past decade, these advances can have broad impact across scientific and engineering applications.

In the second part of the project we went beyond linear models and investigated generalizations and extensions to dimensionality reduction, and more generally to the notion of “dimension”, and to learning low dimensional representations.   We investigated theoretical notions of generalized dimensions, and deep learning approaches to learning low dimensional representations, as well as other notions of finding relevant directions.  In particular, we consider the important problem of generalizing from a few environments to other very different environments (e.g. generalizing a pedestrian detection system trained on data from a few cities with certain cameras, to other cities with different visual characteristics and for images captured by cameras with different optics).  A possible approach to such generalization is by learning relevant dimensions in the data that are invariant across environments.  Our research elucidates the form of invariants that can and cannot be captured using such methods.

Related to the main theme of the project, we also investigated several other directions that emerged during the course of the research: (1) learning predictors that are robust to malicious perturbations in the input; and (2) methods for obtaining valid statistical answers to arbitrarily adaptive statistical queries.

Beyond its direct impact through research, multiple graduate students and post-doctoral researchers received training and mentorship through the project, including a female post-doctoral researcher.  The two post-doctoral researchers involved in the project are now working in top technology companies (Google and Microsoft), helping bring research ideas to practice.

Last Modified: 05/31/2022
Modified by: Nathan Srebro