Award Abstract # 1515586
Improving Math Identity for Underrepresented Populations: An Implementation and Impact Study of After-School Math Plus

NSF Org: DRL
Division of Research on Learning in Formal and Informal Settings (DRL)
Recipient: IMPAQ INTERNATIONAL, LLC
Initial Amendment Date: September 16, 2015
Latest Amendment Date: February 14, 2019
Award Number: 1515586
Award Instrument: Continuing Grant
Program Manager: Julie Johnson
jjohnson@nsf.gov
 (703)292-8624
DRL
 Division of Research on Learning in Formal and Informal Settings (DRL)
EDU
 Directorate for STEM Education
Start Date: October 1, 2015
End Date: December 31, 2019 (Estimated)
Total Intended Award Amount: $1,703,284.00
Total Awarded Amount to Date: $1,901,837.00
Funds Obligated to Date: FY 2015 = $600,886.00
FY 2016 = $639,673.00

FY 2017 = $661,278.00
History of Investigator:
  • David Silver (Principal Investigator)
    dsilver@rti.org
  • Cheri Fancsali (Co-Principal Investigator)
  • Michaela Gulemetova (Co-Principal Investigator)
  • Merle Froschl (Co-Principal Investigator)
  • Cheri Fancsali (Former Principal Investigator)
  • Andrea Beesley (Former Principal Investigator)
  • Barbara Sprung (Former Co-Principal Investigator)
Recipient Sponsored Research Office: IMPAQ International, LLC
10420 LITTLE PATUXENT PKWY STE 300
COLUMBIA
MD  US  21044-3636
(443)259-5401
Sponsor Congressional District: 03
Primary Place of Performance: IMPAQ International, LLC
10420 Little Patuxent Pkwy, #300
Columbia
MD  US  21044-2104
Primary Place of Performance
Congressional District:
03
Unique Entity Identifier (UEI): HPLEKBSHLEM4
Parent UEI: CSMWEEVYAPF4
NSF Program(s): AISL
Primary Program Source: 04001718DB NSF Education & Human Resource
04001617DB NSF Education & Human Resource

04001516DB NSF Education & Human Resource
Program Reference Code(s):
Program Element Code(s): 725900
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.076

ABSTRACT

Most students who pursue math have chosen to do so by high school. Elementary and middle school experiences are thus vitally important in attracting students to STEM. Research consistently points to after-school as a golden opportunity to increase students' exposure to high-quality math learning opportunities and to develop the key influencers of math participation and persistence: interest and identity. However, more research on how and under what conditions after-school programs can foster these factors is needed. The role of identity in math education has been particularly neglected. The proposed research project addresses this gap by studying the implementation and outcomes of After-School Math PLUS (ASM+), an after-school math program designed to address all aspects of math identity and thus have a positive effect on this key influencer of math participation and achievement. "Improving Math Identity" is a Research-in-Service to Practice project funded by the Advancing Informal STEM Learning (AISL) Program which seeks to advance new approaches to, and evidence-based understanding of, the design and development of STEM learning in informal environments.

The team will study the impact of ASM+ through a rigorous randomized controlled trial of 30 elementary-level after-school sites in South Carolina serving predominately low-income and minority students (15 treatment using ASM+; 15 control using Mixing in Math). Sites selected into the study must serve fourth and fifth graders and must operate five days a week. Through an implementation study, data will be collected in order to assess the program and understand the experiences of group leaders and students in the ASM+ program and at comparison sites. Data sources include surveys, interviews, observations, and administrative data collected from the treatment and control sites. The study will investigate how and to what extent ASM+ develops fourth and fifth grade students' math identity and increases math engagement and interest. It will explore whether increasing identity, engagement, and interest leads to greater skill development and academic achievement.

This research is being conducted by IMPAQ International LLC, a social science and public policy research and evaluation firm in collaboration with Educational Equity at FHI 360, a global development and education organization. The research addresses the need to enhance students' math identity at an early age and, as a result, change students' educational and career aspirations. The ultimate goal is to broaden participation in STEM by underrepresented groups. Results will inform the development of interventions designed to motivate and retain students in STEM, particularly in informal settings. Knowledge gained from this research will be broadly disseminated to practitioners, researchers, program developers, and policy makers.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Beesley, A. D., Fancsali, C., and Gulemetova, M. "Building mathematical identity after school: Year 1 of a cluster-randomized trial." Paper presented at the Society for Research in Educational Effectiveness Spring 2018 conference, Washington, DC. , 2018
Beesley, A. D. "Recruitment, retention, and partnership challenges in rigorous afterschool evaluation designs" Panel paper presented at the Annual Conference of the American Evaluation Association, Washington, DC. , 2017
Beesley, A. D. "Recruitment, retention, and partnership challenges in rigorous OST evaluation designs." Panel paper presented at the Annual Meeting of the American Educational Research Association, New York, NY. , 2018
Gulemetova, M. & Beesley, A. D. "Working with a survey vendor ? Why, when, and how" Poster presented at the Annual Conference of the American Evaluation Association, Washington, DC. , 2017
Murchison, L., Brohawn, K., Fancsali, C., Beesley, A. D., & Stafford, E. "The Unique Challenges of Afterschool Research: A Practical Guide for Evaluators and Practitioners" Afterschool Matters , 2019
Murchison, L., Fancsali, C., Stafford, E., Beesley, A., & Brohawn, K. "The unique challenges of conducting afterschool evaluations: lessons learned from the field" Afterschool Matters , 2019

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

IMPAQ International, Educational Equity at FHI 360, the Research Alliance for New York City Schools, and SRI International worked in partnership to broaden the understanding of mathematics identity, how it is measured, how it evolved over time, its relationship to mathematics achievement, and its potential to be developed in afterschool environments, for both students and educators. The 4-year project employed a randomized-controlled trial design in an implementation and impact study examining the effects of mathematics identity-supporting activities from an innovative afterschool curriculum--Afterschool Math Plus (ASM+)--on fourth and fifth grade students' mathematics identity and achievement.

ASM+ is a hands-on mathematics activity program with built-in literacy and career connections designed to enhance the key influencers of mathematics identity. The intervention included training and technical support for educators and an inquiry-based curriculum comprised of four themes, each expected to be delivered over the course of a semester. The curriculum, training, and technical support were offered to both treatment and control sites. Control sites received a version of the curriculum that did not include specific activities hypothesized to develop mathematics identity: literacy connections, connections to role models, family outreach, and a culminating event where students become experts and share their work in a public exhibit.

The study investigated how and to what extent the treatment-group activities developed students' mathematics identity. We also explored whether increasing student mathematics identity was associated with improved academic achievement in mathematics, how math identity changes over time, and whether afterschool educators' mathematics identity was related to student mathematics identity. The research was motivated by the need for evidence about how to enhance students' mathematics identity at an age before they have disengaged from math, and as a result, change students' educational and career aspirations. As part of this project, we also developed and tested a student and afterschool educator survey designed to measure mathematics identity.

The results of our impact analyses showed no statistically significant effects of the treatment--the mathematics identity-supporting activities--on mathematics identity or achievement. The lack of impact is likely related to low fidelity of implementation due to unstable funding for afterschool sites to plan and commit to afterschool curricula, limited staff capacity to implement program activities and participate in research, and high educator and student turnover. It may also be that the intervention as experienced by students in this study was not intense enough to have a measurable impact on students' mathematics identity. It is unclear if the mathematics identity activities would have had an impact if fidelity of implementation were stronger and more students received the full dosage (four themes taught over four semesters) as designed. 

To investigate questions related to the change in mathematics identity over time, we used longitudinal data with up to five observations per student over the course of two years, to estimate a growth model of mathematics identity. The results showed that on average, student mathematics identity did not increase over the study period. Further, we saw no overall differences in initial levels and growth rates by gender or grade level. The lack of gender differences runs counter to the literature that suggests boys report greater mathematics identity than girls. We did find significant differences by ethnicity. Black, Hispanic, and students of other ethnicities had on average higher initial identity levels than white students in our study, and their identity levels remained higher over time. However, these overall averages mask the large variation we found by individual students and sites. While some students saw improvement in mathematics identity, others saw decreases. Further, we found that sites and students with the lowest initial levels of mathematics identity exhibited the greatest growth rates. Our analyses did not reveal any patterns in the variation to help us understand what may have led to increased mathematics identity for some students and decreases for others.  

We also found a significant positive relationship between mathematics identity and student achievement. An increase of prior mathematics achievement by one standard deviation was associated with an increase of 0.11 (p=.02) points in mathematics identity at follow up. Similarly, a one-point increase in baseline mathematics identity was associated with an increase of mathematics achievement equal to 0.15 (p<.001) standard deviations at follow up. Contrary to our expectation, we found no significant relationship between educator mathematics identity and student mathematics identity.  

This study has contributed to a broader understanding of the development and measurement of students' mathematics identity. To date, we have disseminated our findings through presentations at professional conferences such as SREE, AERA, AEA and NCTM and peer-reviewed journal articles. We are finalizing two papers for submission to academic journals.

 


Last Modified: 02/24/2020
Modified by: Michaela Gulemetova

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page