| NSF Org: |
IIS Division of Information & Intelligent Systems |
| Recipient: |
|
| Initial Amendment Date: | September 14, 2015 |
| Latest Amendment Date: | September 14, 2015 |
| Award Number: | 1546452 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Sylvia Spengler
sspengle@nsf.gov (703)292-7347 IIS Division of Information & Intelligent Systems CSE Directorate for Computer and Information Science and Engineering |
| Start Date: | January 1, 2016 |
| End Date: | September 30, 2021 (Estimated) |
| Total Intended Award Amount: | $610,432.00 |
| Total Awarded Amount to Date: | $610,432.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
110 INNER CAMPUS DR AUSTIN TX US 78712-1139 (512)471-6424 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
201 E. 24th Street, C0200 Austin TX US 78712-1229 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): | Big Data Science &Engineering |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
With an ever increasing ability to collect and archive data, massive data sets are becoming increasingly common. These data sets are often too big to fit into the main memory of a single computer, and so there is a great need for developing scalable and sophisticated machine learning methods for their analysis. In particular, one has to devise strategies to distribute the computation across multiple machines. However, stochastic optimization and inference algorithms that are so effective for large-scale machine learning appear to be inherently sequential.
The main research goal of this project is to develop a novel "nomadic" framework that overcomes this barrier. This will be done by showing that many modern machine learning problems have a certain "double separability" property. The aim is to exploit this property to develop convergent, asynchronous, distributed, and fault tolerant algorithms that are well-suited for achieving high performance on commodity hardware that is prevalent on today's cloud computing platforms. In particular, over a four year period, the following will be developed: (i) parallel stochastic optimization algorithms for the multi-machine cloud computing setting, (ii) theoretical guarantees of convergence, (iii) open source code under a permissive license, (iv) application of these techniques to a variety of problem domains such as topic models and mixture models. In addition, a cohort of students who can transfer their skills to both industry and academia will be trained, and a graduate level course on scalable machine learning will be developed.
The proposed research will enable practitioners in different application areas to quickly solve their big data problems. The results of the project will be disseminated widely through papers and open source software. Course material will be developed for the education of students in the area of Scalable Machine Learning, and the course will be co-taught at UCSC and UT Austin. The project will recruit women and minority students.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
As data grows in size and complexity, it is a contemporary challenge to develop scalable, robust and distributed algorithms for big data analytics. In paticular, data sets are often too big to fit into the main memory of a single computer, and so there is a great need for developing scalable and sophisticated machine learning methods for their analysis.
For this project, we have explored novel scalable algorithms and framework for large-scale machine learning problems, such as matrix completion, topic modeling, kernel machines, extreme classification, federated learning, and sequence-to-sequence prediction. We have published papers and released software for these problems in addition to training students with expertise in these areas.
In particular, we have developed (i) nomadic distributed, decentralized algorithms for very large-scale matrix completion, topic modeling and mixture modeling, (ii) communication efficient distributed block minimization algorithms for large-scale nonlinear kernel machines, (iii) parallel primal-dual sparse methods for large-scale extreme classification poblems, (iv) robust and efficient federated learning methods, and (v) stabilized, scalable methods for training sequence-to-sequence deep learning models. These algorithms allow us to train larger machine learning models for practical problems in natural language processing, recommender systems, object recognition, and information retrieval.
The results from this funded project have been disseminated through publications in various venues, such as KDD, NeurIPS, ICML, AISTATS, and IEEE Transactions, which are leading conferences and journals in machine learning and data mining. In terms of education and training, multiple Ph.D. students, including one female student, obtained their degree supported by funding from this project.
Last Modified: 02/13/2022
Modified by: Inderjit S Dhillon
Please report errors in award information by writing to: awardsearch@nsf.gov.
