
NSF Org: |
IIS Division of Information & Intelligent Systems |
Recipient: |
|
Initial Amendment Date: | January 25, 2021 |
Latest Amendment Date: | May 20, 2021 |
Award Number: | 2041009 |
Award Instrument: | Standard Grant |
Program Manager: |
Sylvia Spengler
sspengle@nsf.gov (703)292-7347 IIS Division of Information & Intelligent Systems CSE Directorate for Computer and Information Science and Engineering |
Start Date: | February 1, 2021 |
End Date: | January 31, 2024 (Estimated) |
Total Intended Award Amount: | $375,000.00 |
Total Awarded Amount to Date: | $375,000.00 |
Funds Obligated to Date: |
|
History of Investigator: |
|
Recipient Sponsored Research Office: |
9500 GILMAN DR LA JOLLA CA US 92093-0021 (858)534-4896 |
Sponsor Congressional District: |
|
Primary Place of Performance: |
CA US 92093-0934 |
Primary Place of
Performance Congressional District: |
|
Unique Entity Identifier (UEI): |
|
Parent UEI: |
|
NSF Program(s): | Fairness in Artificial Intelli |
Primary Program Source: |
|
Program Reference Code(s): |
|
Program Element Code(s): |
|
Award Agency Code: | 4900 |
Fund Agency Code: | 4900 |
Assistance Listing Number(s): | 47.070 |
ABSTRACT
With the increasing use of artificial intelligence (AI) systems in life-changing decisions, such as hiring or firing of individuals or the length of jail sentences, there has been an increasing concern about the fairness of these systems. There is a need to guarantee that AI systems are not biased against segments of the population. This project aims to mitigate AI bias in the domain of computer vision, a driving application for much of the recent advances in a popular form of AI known as deep learning. Computer vision systems are increasingly prevalent in areas of society ranging from healthcare to law enforcement: from apps that analyze skin pictures for melanoma detection to face recognition systems used in criminal investigations. These systems are subject to three major sources of bias: biased data, biased annotations, and biased models. Biased data follows from poor image collection practices, typically the under-representation of certain population groups. Biased annotation follows from the use of annotation platforms with untrained image labelers, who tend to produce annotations that reflect their own image interpretations, rather than objective labels. Biased models can ensue from either the existence of data or annotation biases on the datasets used to train the models, or the choice of biased model architectures. The three bias components have received different attention in the literature, with most previous work focusing on the mitigation of model bias. However, this usually boils down to downplaying groups for which there is a lot of data and promoting groups for which data is scarce. This practice can hurt overall system performance. The remaining sources of bias, datasets and annotation, have received very little algorithmic attention.
The project aims to overcome this problem, by introducing a new framework to jointly address the three sources of bias within one unified bias mitigation architecture. This architecture aims to train fair classifiers by iterative optimization of three distinct modules: 1) Dataset bias mitigation algorithms that identify and downweigh biased examples and seek additional examples in a large pool of data to counterbalance the associated biases. 2) Label bias mitigation systems based on machine teaching algorithms that establish clear, replicable, and auditable procedures to teach annotators how to label images without label bias. 3) Model auditing techniques based on counterfactual visual explanations that enable the visualization of the factors contributing to model decisions and why they are biased. The three modules combine into an architecture for joint dataset, label, and model bias mitigation by iterative optimization of datasets, annotators, and models to minimize bias. The project will generate software for dataset bias mitigation, unbiased annotator training, explanations and visualizations, model auditing, and fair model training, which will be made available from the investigator website. This will be complemented with datasets for the design of various form of bias mitigation algorithms, and tools to help practitioners detect and combat bias. Several activities are also planned to broaden the participation of underrepresented K-12 and undergraduate students in the STEM field. They will include the participation of a team of such students, recruited from University of California San Diego programs that aim to increase the participation of these groups in STEM, and aim to provide these students with early exposure to the challenges of real-world engineering, fair machine learning, and deep learning systems.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
Computer vision systems are increasingly prevalent in areas of society ranging from healthcare to law enforcement. These systems are subject to three major sources of bias: biased data, biased annotations, and biased models. Biased data follows from poor image collection practices, typically the under-representation of certain population groups. Biased annotation follows from the use of annotation platforms with untrained image labelers, who tend to produce annotations that reflect their own image interpretations, rather than objective labels. Biased models can ensue from either the existence of data or annotation biases on the datasets used to train the models, or the choice of biased model architectures. The three bias components have received different attention in the literature, with most previous work focusing on the mitigation of model bias. However, this usually boils down to downplaying groups for which there is a lot of data and promoting groups for which data is scarce. This practice can hurt overall system performance. The remaining sources of bias, datasets and annotation, have received very little algorithmic attention.
The project addressed this problem by introducing a new framework to jointly address the three sources of bias within one unified bias mitigation architecture. This architecture, illustrated in Figure 1, aims to train fair classifiers by iterative optimization of three distinct modules: 1) Dataset bias mitigation algorithms that identify and downweigh biased examples and seek additional examples in a large pool of data to counterbalance the associated biases. 2) Label bias mitigation systems based on machine teaching algorithms that establish clear, replicable, and auditable procedures to teach annotators how to label images without label bias. 3) Model auditing techniques based on visual explanations that enable the visualization of the factors contributing to model decisions and why they are biased.
Beyond the architecture, the project produced technical contributions along all the individual directions. Figure 2 illustrates a new approach to explainable AI, denoted as deliberative explanations. These aim to expose the deliberations carried out by a neural network to arrive at a prediction, by uncovering the insecurities of the network about the latter. The explanation consists of a list of insecurities, each composed of 1) an image region, and 2) an ambiguity formed by the pair of classes responsible for the network uncertainty about the region. Since insecurity detection requires quantifying the difficulty of network predictions, deliberative explanations combine ideas from the literature on visual explanations and assessment of classification difficulty.
Figure 3 illustrates the problem of machine teaching, where the goal is to teach non-experts to label images of expert domains, in this case butterfly species. The annotators are instructed through a visual interface, showing annotated examples chosen by a teaching algorithm. A new formulation of machine teaching was proposed under the assumption of an optimal student, where optimality is defined in the usual machine learning sense of empirical risk minimization. It was shown that, if allowed unbounded effort, the optimal student always learns the optimal predictor for a classification task. Hence, the role of the optimal teacher is to select the teaching set that minimizes student effort. This was formulated as a problem of functional optimization where, at each teaching iteration, the teacher seeks to align the steepest descent directions of the risk of (1) the teaching set and (2) entire example population. The optimal teacher, denoted MaxGrad, was then shown to maximize the gradient of the risk on the set of new examples selected per iteration. MaxGrad teaching algorithms were finally provided for both binary and multiclass tasks. Figure 4 illustrates the steps of the MaxGrad algorithm.
A source of bias is poor model performance for different populations due to the uneven data coverage. For example, extensive progress has been achieved for translation between the "major" languages (English, French, Spanish) but fewer advances have been observed for less resourced languages, lacking the very large datasets and models needed to train translation systems. Since "an image is worth a thousand words," the addition of images should reduce the required number of examples needed for training. While existing multimodal methods show promising performance over text-only translation systems, they require paired text and image as input during inference, which limits their applicability to real-world scenarios. We introduced a visual hallucination framework, called VALHALLA, which requires only source sentences at inference time and instead uses hallucinated visual representations for multimodal machine translation, as illustrated in Figure 5. Given a source sentence an autoregressive hallucination transformer predicts a visual representation, and the combined text and hallucinated representations are utilized to generate the target translation. We train the hallucination transformer jointly with the translation transformer using cross-entropy losses and an additional loss that encourages consistency between predictions using either ground-truth or hallucinated visual representations. The model architecture, illustrated in Figure 6, was shown to be successful in several translation datasets.
Last Modified: 04/25/2024
Modified by: Nuno M Vasconcelos
Please report errors in award information by writing to: awardsearch@nsf.gov.