Award Abstract # 2000638
Collaborative Research: Exploring Algorithmic Fairness and Potential Bias in K-12 Mathematics Adaptive Learning

NSF Org: DUE
Division Of Undergraduate Education
Recipient: UNIVERSITY OF ILLINOIS
Initial Amendment Date: July 28, 2020
Latest Amendment Date: July 28, 2020
Award Number: 2000638
Award Instrument: Standard Grant
Program Manager: Dawn Rickey
drickey@nsf.gov
 (703)292-4674
DUE
 Division Of Undergraduate Education
EDU
 Directorate for STEM Education
Start Date: October 1, 2020
End Date: September 30, 2025 (Estimated)
Total Intended Award Amount: $987,015.00
Total Awarded Amount to Date: $987,015.00
Funds Obligated to Date: FY 2020 = $987,015.00
History of Investigator:
  • Nigel Bosch (Principal Investigator)
    pnb@illinois.edu
  • Steven Ritter (Co-Principal Investigator)
Recipient Sponsored Research Office: University of Illinois at Urbana-Champaign
506 S WRIGHT ST
URBANA
IL  US  61801-3620
(217)333-2187
Sponsor Congressional District: 13
Primary Place of Performance: University of Illinois at Urbana-Champaign
506 S. Wright Street
Urbana
IL  US  61801-3620
Primary Place of Performance
Congressional District:
13
Unique Entity Identifier (UEI): Y8CWNJRCNN91
Parent UEI: V2PHZ2CSCH63
NSF Program(s): Discovery Research K-12,
ECR-EDU Core Research
Primary Program Source: 04002021DB NSF Education & Human Resource
Program Reference Code(s): 8212, 8244, 8817
Program Element Code(s): 764500, 798000
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.076

ABSTRACT

Students in middle school and high school often use adaptive learning software as part of their math education experience. Adaptive learning software works by automatically measuring how much students have learned about the topic, as well as their learning process and experiences, and then adjusting the instruction accordingly. This project will investigate potential ways in which adaptive learning software might be biased against students from certain groups, and how such biases can be reduced. Adaptive learning offers an opportunity to provide high quality instruction that is personalized to the needs of individual learners, but little is known about who benefits most from adaptive learning technologies. This project will address this issue by observing and interviewing students who use adaptive math learning software to discover what aspects of their identity are most salient in the adaptive learning context. This project will then investigate possible algorithmic biases related to the identities that students express. Findings from the project will contribute to understanding of the most relevant aspects of student identity in adaptive learning contexts, and how those identities affect their learning experience. Finally, this project will address the biases that are identified, thereby providing a more equitable mathematics education experience for students.

Modern adaptive learning platforms individualize learning support and improve learner outcomes by using algorithms that are typically derived through machine learning. Previous work has studied biases in educational model accuracy for large groups (e.g., ethnic and gendered categories, urban vs. rural, etc.); however, large groups may have a great deal of heterogeneity, and little is known about which specific groups of students suffer from biases in model accuracy and why. This project will approach the problem of potential bias in three steps. First, the project will begin by collecting data on educational software usage patterns (i.e., logs of actions and classroom observations of student experiences) for students using MATHia, a math education platform used by over half a million students across the United States. As part of this data collection, students will describe their identity in open-ended survey responses and interviews, which will be analyzed to discover identity characteristics that shape their learning experiences. Second, existing machine learning models will be applied to these data to predict knowledge, engagement, and self-regulated learning behaviors, and the predictions will be analyzed to reveal cases where models are systematically biased. Third, the project will compare various pre-processing, in-processing, and post- processing methods for bias reduction, and study the effects of the improved algorithms when applied in MATHia. Results from this project will contribute to scientific understanding of the role of student identity in adaptive learning software, biases in machine learning for educational software, and the effects of applying machine learning methods for bias reduction. This project is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development, with co-funding by the Discovery Research PreK-12 (DRK-12) program.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Almoubayyed, Husni and Fancsali, Stephen and Ritter, Steve "Generalizing Predictive Models of Reading Ability in Adaptive Mathematics Software" Proceedings of the 16th International Conference on Educational Data Mining , 2023 Citation Details
Belitz, Clara and Lee, HaeJin and Nasiar, Nidhi and Fancsali, Stephen E and Ritter, Steve and Almoubayyed, Husni and Baker, Ryan S and Ocumpaugh, Jaclyn and Bosch, Nigel "Hierarchical Dependencies in Classroom Settings Influence Algorithmic Bias Metrics" , 2024 https://doi.org/10.1145/3636555.3636869 Citation Details
Belitz, Clara and Ocumpaugh, Jaclyn and Ritter, Steven and Baker, Ryan_S and Fancsali, Stephen_E and Bosch, Nigel "Constructing categories: Moving beyond protected classes in algorithmic fairness" Journal of the Association for Information Science and Technology , v.74 , 2022 https://doi.org/10.1002/asi.24643 Citation Details
Jiang, Lan and Belitz, Clara and Bosch, Nigel "Synthetic Dataset Generation for Fairer Unfairness Research" , 2024 https://doi.org/10.1145/3636555.3636868 Citation Details
Levin, Nathan and Baker, Ryan and Nasiar, Nidhi and Fancsali Stephen and Hutt, Stephen "Evaluating Gaming Detector Model Robustness Over Time" Proceedings of the 15th International Conference on Educational Data Mining, International Educational Data Mining Society , 2022 https://doi.org/10.5281/zenodo.6852961 Citation Details
Stinar, Frank and Xiong, Zihan and Bosch, Nigel "An Approach to Improve k-Anonymization Practices in Educational Data Mining" Journal of Educational Data Mining , 2024 https://doi.org/10.5281/zenodo.11056083 Citation Details
Zambrano, Andres Felipe and Zhang, Jiayi and Baker, Ryan S "Investigating Algorithmic Bias on Bayesian Knowledge Tracing and Carelessness Detectors" , 2024 https://doi.org/10.1145/3636555.3636890 Citation Details

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page