Award Abstract # 2040880
FAI: Foundations of Fair AI in Medicine: Ensuring the Fair Use of Patient Attributes

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: PRESIDENT AND FELLOWS OF HARVARD COLLEGE
Initial Amendment Date: January 25, 2021
Latest Amendment Date: May 20, 2021
Award Number: 2040880
Award Instrument: Standard Grant
Program Manager: Wendy Nilsen
wnilsen@nsf.gov
 (703)292-2568
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: July 1, 2021
End Date: June 30, 2025 (Estimated)
Total Intended Award Amount: $625,000.00
Total Awarded Amount to Date: $625,000.00
Funds Obligated to Date: FY 2021 = $625,000.00
History of Investigator:
  • Flavio Calmon (Principal Investigator)
    flavio@seas.harvard.edu
  • Elena Glassman (Co-Principal Investigator)
  • Berk Ustun (Co-Principal Investigator)
Recipient Sponsored Research Office: Harvard University
1033 MASSACHUSETTS AVE STE 3
CAMBRIDGE
MA  US  02138-5366
(617)495-5501
Sponsor Congressional District: 05
Primary Place of Performance: Harvard University SEAS
33 Oxford Street, MD 347
Cambridge
MA  US  02138-2933
Primary Place of Performance
Congressional District:
05
Unique Entity Identifier (UEI): LN53LCFJFL45
Parent UEI:
NSF Program(s): Fairness in Artificial Intelli,
Smart and Connected Health
Primary Program Source: 01002122DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 8018, 075Z
Program Element Code(s): 114Y00, 801800
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Machine learning models support decisions that affect millions of patients in the U.S. healthcare system in diagnosing illnesses, facilitating triage in emergency rooms, and informing supervision at intensive care units. In such applications, models will often include group attributes such as age, weight, and employment status to capture differences between patient subgroups. Standard techniques to build models with group attributes typically improve aggregate performance across the entire patient population. As a result, however, such models may lead to worse performance for specific groups. In such cases, the model may assign these groups preventable inaccurate predictions that undermine medical care and health outcomes. This project aims to prevent this harm by developing tools to ensure the fair use of group attributes in predictive models. The goal is to ensure that a model uses group attributes in a way that yields a tailored performance benefit for every group.

Currently deployed machine learning models in medicine may exhibit fair use violations that undermine health outcomes. This project mitigates fair use violations at key stages in the deployment of machine learning in medicine: verification, model development, and communication. First, it develops tools to check if a model ensures fair use. These tools include theoretical guarantees that characterize when common approaches to model development produce fair use violations, and statistical tests to verify if a model violates fair use before and during deployment. Second, it develops algorithms for learning models with fair use guarantees. Algorithms will be tailored for salient use cases in medicine, paired with open-source software, and applied to build decision support tools for real-world medical applications. Third, it creates tools to inform key stakeholders (regulators, physicians, and patients) about a model's fair use guarantees. The project draws on machine learning, information theory, optimization, human-centered design, as well as expertise in deploying models in clinical settings. The resulting toolkit for ensuring fair use of group attributes in medicine will be embedded in real-world systems through collaborations with medical researchers and industry.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 13)
Alghamdi, Wael and Asoodeh, Shahab and Calmon, Flavio P and Felipe_Gomez, Juan and Kosut, Oliver and Sankar, Lalitha "Schrödinger Mechanisms: Optimal Differential Privacy Mechanisms for Small Sensitivity" , 2023 https://doi.org/10.1109/ISIT54713.2023.10206616 Citation Details
Alghamdi, Wael and Calmon, Flavio P "Measuring Information From Moments" IEEE Transactions on Information Theory , v.70 , 2024 https://doi.org/10.1109/TIT.2022.3202492 Citation Details
Alghamdi, W. and Hsu, H. and Jeong H. and Wang H. and Michalak, P.W. and Asoodeh, S. and Calmon F.P. "Beyond Adult and COMPAS: Fair Multi-Class Prediction via Information Projection" Advances in neural information processing systems , 2022 Citation Details
Arawjo, Ian and Swoopes, Chelse and Vaithilingam, Priyan and Wattenberg, Martin and Glassman, Elena L "ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing" , 2024 https://doi.org/10.1145/3613904.3642016 Citation Details
Gero, Katy Ilonka and Swoopes, Chelse and Gu, Ziwei and Kummerfeld, Jonathan K and Glassman, Elena L "Supporting Sensemaking of Large Language Model Outputs at Scale" , 2024 https://doi.org/10.1145/3613904.3642139 Citation Details
Gomez, Juan Felipe and Machado, Caio and Paes, Lucas Monteiro and Calmon, Flavio "Algorithmic Arbitrariness in Content Moderation" , 2024 https://doi.org/10.1145/3630106.3659036 Citation Details
Kulynych, Bogdan and Hsu, Hsiang and Troncoso, Carmela and Calmon, Flavio P. "Arbitrary Decisions are a Hidden Cost of Differentially Private Training" FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , 2023 https://doi.org/10.1145/3593013.3594103 Citation Details
Jeong, Haewon and Wang, Hao and Calmon, Flavio P. "Fairness without Imputation: A Decision Tree Approach for Fair Prediction with Missing Values" Proceedings of the AAAI Conference on Artificial Intelligence , v.36 , 2022 https://doi.org/10.1609/aaai.v36i9.21189 Citation Details
Hsu, H. and Calmon, F.P. "Rashomon Capacity: A Metric for Predictive Multiplicity in Classification" Advances in neural information processing systems , 2022 Citation Details
Wang, Hao and Huang, Yizhe and Gao, Rui and Calmon, Flavio P. "Analyzing the Generalization Capability of SGLD Using Properties of Gaussian Channels" Advances in Neural Information Processing Systems 34 (NeurIPS 2021) , v.34 , 2021 Citation Details
Wang, Hao and Gao, Rui and Calmon, Flavio P. "Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channels" Journal of machine learning research , v.24 , 2023 Citation Details
(Showing: 1 - 10 of 13)

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page