Skip to feedback

Award Abstract # 1704624
CIF:Medium:Collaborative Research: Geometric Network Information Theory

NSF Org: CCF
Division of Computing and Communication Foundations
Recipient: THE LELAND STANFORD JUNIOR UNIVERSITY
Initial Amendment Date: June 22, 2017
Latest Amendment Date: June 22, 2017
Award Number: 1704624
Award Instrument: Standard Grant
Program Manager: Phillip Regalia
pregalia@nsf.gov
 (703)292-2981
CCF
 Division of Computing and Communication Foundations
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: August 1, 2017
End Date: July 31, 2021 (Estimated)
Total Intended Award Amount: $451,330.00
Total Awarded Amount to Date: $451,330.00
Funds Obligated to Date: FY 2017 = $451,330.00
History of Investigator:
  • Ayfer Ozgur (Principal Investigator)
    aozgur@stanford.edu
Recipient Sponsored Research Office: Stanford University
450 JANE STANFORD WAY
STANFORD
CA  US  94305-2004
(650)723-2300
Sponsor Congressional District: 16
Primary Place of Performance: Stanford University
3160 Porter Drive
Palo Alto
CA  US  94304-1212
Primary Place of Performance
Congressional District:
16
Unique Entity Identifier (UEI): HJD6G4D6TJY5
Parent UEI:
NSF Program(s): Comm & Information Foundations
Primary Program Source: 01001718DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7924, 7935
Program Element Code(s): 779700
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

The proliferation of communication networks means that their efficiency and security now affects almost all aspects of our daily lives. At the societal level, reliable and efficient networks are of central importance to technological development, national economic growth and national defense. In light of this, the problem of understanding fundamental information limits of networked devices is more acute now than ever. Unfortunately, it has been a longstanding challenge in information theory to systematically extend its success beyond classical point-to-point exchanges of information. Despite significant research effort since the inception of the field in 1948, the information limits for most network settings, even for networks as fundamental as a three node relay channel, have remained open to date.

This project aims to change the way many network problems are analyzed today in information theory. Traditional information-theoretic approaches for quantifying fundamental performance limits of networks often fail because the current toolset of information theory is not sufficiently refined to characterize tensions that emerge in networks due to competing objectives. Looking beyond traditional methods, this project will study the role that inequalities from geometric and functional analysis can play in characterizing tensions between information measures. The research project will be complemented with a strong inter-university education program, which includes joint and collaborative student and post-doctoral researcher advising.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 26)
Asoodeh, Shahab and Chen, Wei-Ning and Calmon, Flavio P. and Ozgur, Ayfer "Differentially Private Federated Learning: An Information-Theoretic Perspective" 2021 IEEE International Symposium on Information Theory (ISIT) , 2021 https://doi.org/10.1109/ISIT45174.2021.9518124 Citation Details
Bai, Yikun and Wu, Xiugang and Ozgur, Ayfer "Information Constrained Optimal Transport: From Talagrand, to Marton, to Cover" 2020 IEEE International Symposium on Information Theory (ISIT) , 2020 10.1109/ISIT44484.2020.9174478 Citation Details
Barnes, Leighton Pate and Chen, Wei-Ning and Ozgur, Ayfer "Fisher Information Under Local Differential Privacy" IEEE Journal on Selected Areas in Information Theory , v.1 , 2020 https://doi.org/10.1109/JSAIT.2020.3039461 Citation Details
Barnes, Leighton Pate and Han, Yanjun and Ozgur, Ayfer "A Geometric Characterization of Fisher Information from Quantized Samples with Applications to Distributed Statistical Estimation" 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , 2018 10.1109/ALLERTON.2018.8635899 Citation Details
Barnes, Leighton Pate and Inan, Huseyin A. and Isik, Berivan and Ozgur, Ayfer "rTop- k : A Statistical Estimation Approach to Distributed SGD" IEEE Journal on Selected Areas in Information Theory , v.1 , 2020 https://doi.org/10.1109/JSAIT.2020.3042094 Citation Details
Barnes, Leighton Pate and Ozgur, Ayfer "Fisher Information and Mutual Information Constraints" 2020 IEEE International Symposium on Information Theory (ISIT) , 2021 https://doi.org/10.1109/ISIT45174.2021.9518061 Citation Details
Barnes, Leighton Pate and Ozgur, Ayfer "The Courtade-Kumar Most Informative Boolean Function Conjecture and a Symmetrized Li-Médard Conjecture are Equivalent" 2020 IEEE International Symposium on Information Theory (ISIT) , 2020 10.1109/ISIT44484.2020.9174063 Citation Details
Barnes, Leighton Pate and Wu, Xiugang and Ozgur, Ayfer "A solution to cover's problem for the binary symmetric relay channel: Geometry of sets on the hamming sphere" 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , 2017 10.1109/ALLERTON.2017.8262827 Citation Details
Chen, Wei-Ning and "Breaking the Communication-Privacy-Accuracy Trilemma" Advances in neural information processing systems , v.33 , 2020 Citation Details
Han, Yanjun and Mukherjee, Pritam and Ozgur, Ayfer and Weissman, Tsachy "Distributed Statistical Estimation of High-Dimensional and Non-parametric Distributions" 2017 IEEE International Symposium on Information Theory (ISIT) , 2018 Citation Details
Han, Yanjun and Ozgur, Ayfer and Weissman, Tsachy "Geometric Lower Bounds for Distributed Parameter Estimation under Communication Constraints" Proceedings of Machine Learning Research , v.75 , 2018 Citation Details
(Showing: 1 - 10 of 26)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

From cellular networks, to the IoT and the Internet, the reliability and efficiency of communication networks now affect almost all aspects of our daily lives. The nodes in these networks exchange information with each other to accomplish tasks such as communication, estimation and learning. What are the fundamental laws that govern information flow in networks and how can a desired task be achieved most efficiently? This project answered such questions by developing new mathematical tools which emphasize a viewpoint from high-dimensional geometry and applying them to communication networks.


There were many new scientific results that were discovered and published during the project, with details in the publications as well as summarized in annual reports of the project. We highlight two major directions:

1) The project led to the solution of a central open problem in network information theory that  has remained open for 30 years. This problem concerns the capacity of a three node communication network called a relay channel. Our solution relied on combining measure concentration with a probabilistic geometric analysis of typical sets, which allowed us to establish new and surprising relations between the information measures involved in the problem. We have also shown that tools from geometric and functional analysis, such as reverse hypercontractivity and optimal transport, can be successfully used to prove impossibility results for network information problems.

2) We considered a collection of networked statistical estimation problems modeling bandwidth and privacy constraints in distributed and federated learning systems, and showed that the functional and geometric analysis techniques developed in the project can be successfully used to analyse these systems. In these problems, data is distributed across many nodes in a network and must be communicated to a centralized estimator under communication, privacy, or mutual information constraints. We showed how a geometric interpretation of Fisher information from the processed statistical samples can derive tight minimax lower bounds for many distributed estimation problems of interest.

The research was complemented with a strong cirriculum development effort at the home institution that focused on integrating education and research at the undegraduate level, and enhancing and promoting  an understanding of the science of information among a diverse set of undegraduate students accross campus.


Last Modified: 06/08/2022
Modified by: Ayfer Ozgur

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page