Modern learning, particularly those in STEM domains, are characterized by the use of integrated tools.  Students must interact with a range of platforms that provide access to static information, peer and instructor support, development environments, and tutoring.  Students' performance in the course is governed by how they work with and integrate across these tools.  This qualitative change imposes new demands on both parties.  Instructors must learn how to select and coordinate those tools and, they must learn how to monitor students' actions across the platforms so that they can maintain a good overall understanding of the learners progress.  This understanding can best be facilitated by new platforms which provide opportunities for support.  For students this also imposes new challenges as they must learn the necessary skills to integrate information across these platforms and to know when and how to target their help-seeking and coordination while learning skills of problem-solving.  Our goal in conducting this research was to enhance our understanding of how students and instructors deal with these constellations of tools and how we can better support them.  Our research studies broadly fall into two categories, modeling, and design. 

This research project has led to three broad outcomes that both advance our understanding of our research domain and make broad positive impacts on students' educational experiences and opportunities.  

Our first outcome is the development of CONCERT, a novel platform for collecting, integrating, and analyzing student-classroom interaction data.  The Concert platform is designed to ingest data both manually and automatically from multiple data sources including the Learning Management System, Ticketing System for help management, online support forums and cloud-based development tools.  The platform will then link this data with students both within and across classes and to allow instructors to monitor students' work in the class through novel visualizations of individual and gestalt interactions, track group performance, and manage course resources.  

Our second outcome is the development of a rich dataset showing how students make use of support opportunities, notably static materials as hosted on the LMS, and other support opportunities including in-person office hours, and help requests, and then evaluating their understanding of it.  We captured this raw data from multiple sources, the existing Learning Management Systems, the ticketing systems used for office hours requests, online support forums, development tools, and testing platforms.  This rich student-system interaction data serves to illuminate students? study habits, their help seeking and communication, and it helps to show how existing resources are utilized for support.  By collecting and analyzing this data, and supplementing it with in-person interviews we were able to develop a deep picture of how and when students seek help, and how that relates to their current work.  This dataset has yielded insights for our work and the work of others and will continue to support research into the future.   

Finally, we have conducted a series of research studies on CS courses offered both via in-person interaction and as fully online courses, including courses that transitioned from in-person to online learning due to the COVID pandemic.  These research studies have yielded insights around how students use learning materials, how they seek help across in-person and online support opportunities, how shifting learning support to online contexts changes students? experiences and outcomes; and how best to manage these resources.  We have also conducted detailed analyses of students? problem-solving processes, and studied how they make use of instructor-provided resources.  This research has gone in parallel with research on how instructors understand and monitor this coursework both within and across platforms.  These results have enhanced our understanding of learning in these novel domains and supported crucial changes to classroom operations and teaching methods including the orchestration of help resources.

This project has made both intellectual and broader social contributions of benefit to society.  With respect to intellectual contributions it has enhanced our understanding of classroom orchestration, instructional methods, and study processes. It has also yielded novel techniques for user-system interaction modeling, data mining, and data integration which will advance research in education, data mining, and AI in education.  With respect to social benefits this work has supported the development of educational infrastructure that will support instructors in managing their blended and online courses and better allocating resources, teamwork, and support to meet students? needs. This in turn will lead to better student experiences and long-term student outcomes thus enhancing the flow of good STEM trained students to the wider society.  

Last Modified: 01/23/2023
Modified by: Collin Lynch