Project Title: Measuring Growth in Trusting Relationships between Students and Educators with Natural Language Processing and Machine Learning Technology

Sown To Grow (STG) successfully completed its NSF SBIR Phase I project aimed at leveraging Natural Language Processing (NLP) and Machine Learning (ML) to assess and enhance student-teacher relationships. The project focused on analyzing student reflections and teacher responses within the STG platform to measure the depth of interactions and foster trust. Key achievements include the development of three distinct NLP/ML models, early deployment of these models in school settings, and a refined strategic direction for Phase II.

Key Achievements:

1. NLP/ML Model Development:

   • Student Self-Disclosure Algorithm: Developed to measure the level of personal information students share in their reflections, achieving 71% accuracy.

   • Reflective Depth Algorithm: Created to assess the depth and complexity of student reflections, also achieving 75% accuracy.

   • Teacher Response Progression Algorithm: Built to categorize teacher responses based on their level of trust-building and support, achieving 90% accuracy.

2. Early Deployment and Applications:

   • Teacher Notification Emails: Implemented a system that notifies teachers of student reflections requiring timely and supportive responses. An A/B test showed a statistically significant increase in teachers reading student reflections in the treatment group.

   • Positive School Highlight Reports: Used model-driven insights to generate reports highlighting school-wide growth in student-teacher relationships, shared with administrators.

   • Celebration of Exemplary Teacher Engagement: Piloted surfacing school highlights on the admin landing page, recognizing teachers providing high-quality responses.

   • End-of-year data deep dives: Conducted with administrators to share insights on the quality of implementation measured through these rubrics and to gather feedback on technology.

3. Strategic Refinement for Phase II:

   • Initial plans to develop a student-teacher trust metric were adjusted. Data analysis revealed that stronger student reflections and more authentic teacher responses were needed first. The focus shifted to supporting and encouraging these behaviors in Phase II.

   • Identified and mitigated key risks related to rubric development complexity, achieving high model accuracy in a qualitative space, and demonstrating the feasibility of assessing student-teacher connections formatively.

Methodology and Process:

The project followed a structured lifecycle for Data Science and ML projects:

• Rubric Development: Collaborated with pedagogy experts to create rubrics for analyzing student reflections (Self-Disclosure and Reflective Depth) and teacher responses.

• Data Labeling: Trained team members to label data using the rubrics and conducted inter-rater reliability exercises.

• Data Science Exploration: Developed and tested various ML algorithms, including deep learning, tree-based models, and KNN. Feature engineering and addressing data imbalances were crucial.

• ML Engineering: Built inference pipelines to process historical and live data, integrating model outputs into the product.

Challenges and Lessons Learned:

• Imbalanced Data: Underrepresentation of certain categories (e.g., vulnerable reflections) posed challenges for model training.

• Qualitative Measurement: Measuring subjective concepts required careful feature engineering and model selection.

• Ethical Considerations: Recognized the importance of not oversimplifying student-teacher relationships when creating metrics.

Phase II Focus:

Building on Phase I successes, Phase II will focus on:

1. Supporting deeper student-teacher connections through product features and educator coaching.

2. Refining student and teacher rubrics based on additional data and insights.

3. Improving existing algorithms and developing the trust metric algorithm.

4. Transitioning to real-time infrastructure using Amazon Web Services.

5. Implementing model monitoring and continuous improvement processes.

Broader Impact and Sustainability:

• STG is expanding its MTSS functionality, aligning with broader educational needs.

• The company has strong customer retention and secures large-scale district partnerships.

• STG is involved in community school initiatives and grant programs, ensuring sustainable funding.

• The company raised a Seed Growth round and is building a diverse and experienced team.

Conclusions:

Phase I successfully demonstrated the technical feasibility and practical value of using NLP and ML to support student-teacher relationships. STG has a strong foundation for Phase II, with clear objectives and a strategic plan for expanding the impact of its platform. The company is committed to ethical, human-centered AI in education and continues to grow its business and impact.

Last Modified: 05/12/2025
Modified by: Disha Gupta