Award Abstract # 2315626
Development of Assessment Tools for Evaluating Organic Chemistry Learners' Understanding of Reaction Mechanisms

NSF Org: DUE
Division Of Undergraduate Education
Recipient: UNIVERSITY OF SOUTH FLORIDA
Initial Amendment Date: September 6, 2023
Latest Amendment Date: September 6, 2023
Award Number: 2315626
Award Instrument: Standard Grant
Program Manager: Dawn Rickey
drickey@nsf.gov
 (703)292-4674
DUE
 Division Of Undergraduate Education
EDU
 Directorate for STEM Education
Start Date: October 1, 2023
End Date: September 30, 2026 (Estimated)
Total Intended Award Amount: $299,134.00
Total Awarded Amount to Date: $299,134.00
Funds Obligated to Date: FY 2023 = $299,134.00
History of Investigator:
  • Jeffrey Raker (Principal Investigator)
    jraker@usf.edu
Recipient Sponsored Research Office: University of South Florida
4202 E FOWLER AVE
TAMPA
FL  US  33620-5800
(813)974-2897
Sponsor Congressional District: 15
Primary Place of Performance: University of South Florida
4202 E FOWLER AVE
TAMPA
FL  US  33620-9951
Primary Place of Performance
Congressional District:
15
Unique Entity Identifier (UEI): NKAZLXLL7Z91
Parent UEI:
NSF Program(s): IUSE
Primary Program Source: 04002324DB NSF STEM Education
Program Reference Code(s): 8209, 9178
Program Element Code(s): 199800
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.076

ABSTRACT

This project aims to serve the national interest by developing and implementing computer-based scoring models to evaluate students? learning about reaction mechanisms in college-level organic chemistry courses. Developing a good understanding of reaction mechanisms is critical for students? success in organic chemistry courses, which are often a barrier for students seeking to complete STEM degrees and advance their professional goals. To address this challenge, the project team will design predictive scoring models to help instructors provide feedback to students during their studies, as well as to instructors during class sessions. The investigators will collect students? written explanations for what they think is happening for organic chemistry reaction mechanisms; develop computer-based scoring models using machine learning text analyses to evaluate the written explanations; and offer professional development opportunities (both in-person and asynchronously online) for organic chemistry instructors to reflect upon how they assess students? understanding of reaction mechanisms and how they can incorporate the new technologies into their organic chemistry courses.

The project team from the University of South Florida (USF) will: (1) develop machine learning text analysis scoring models, (2) use the scoring models to explore how student understanding of reaction mechanisms develops across the two-semester organic chemistry course sequence, and (3) facilitate professional development opportunities for organic chemistry instructors. This work will explore how machine learning technologies can be used to develop predictive scoring models for chemistry topics such as the role of nucleophiles or electrophiles in organic chemistry reaction mechanisms. The work will extend current predictive scoring models that only evaluate written explanations for a single assessment prompt to the development of a set of models that can be used to evaluate written explanations for a wide array of prompts. Additionally, this work will capture how faculty members teaching organic chemistry engage with and respond to (1) learners? written explanations of what is happening in a reaction mechanism and (2) the education research literature that highlights opportunities to improve students? learning of reaction mechanisms. The project activities have the potential to positively impact the learning and STEM retention of 7,000+ students at the USF study site, an emerging Hispanic-serving institution with a diverse population, including investigating differential impacts on learners for whom English is not their first language. Through this work, the project team will also strive to encourage other researchers to explore more generalized predictive models that address broader concepts and skills (e.g., argumentation and explanation) in STEM courses. The NSF IUSE: EDU program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page