Award Abstract # 1628976
DIP: Improving Collaborative Learning in Engineering Classes Through Integrated Tools

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: UNIVERSITY OF ILLINOIS
Initial Amendment Date: September 7, 2016
Latest Amendment Date: September 7, 2016
Award Number: 1628976
Award Instrument: Standard Grant
Program Manager: Tatiana Korelsky
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: September 1, 2016
End Date: August 31, 2021 (Estimated)
Total Intended Award Amount: $1,349,576.00
Total Awarded Amount to Date: $1,349,576.00
Funds Obligated to Date: FY 2016 = $1,349,576.00
History of Investigator:
  • Emma Mercier (Principal Investigator)
    mercier@illinois.edu
  • Luc Paquette (Co-Principal Investigator)
Recipient Sponsored Research Office: University of Illinois at Urbana-Champaign
506 S WRIGHT ST
URBANA
IL  US  61801-3620
(217)333-2187
Sponsor Congressional District: 13
Primary Place of Performance: University of Illinois at Urbana-Champaign
1901 S. First Street, Suite A
Champaign
IL  US  61820-7473
Primary Place of Performance
Congressional District:
13
Unique Entity Identifier (UEI): Y8CWNJRCNN91
Parent UEI: V2PHZ2CSCH63
NSF Program(s): Science of Learning,
S-STEM-Schlr Sci Tech Eng&Math,
Cyberlearn & Future Learn Tech
Primary Program Source: 01001617DB NSF RESEARCH & RELATED ACTIVIT
1300XXXXDB H-1B FUND, EDU, NSF
Program Reference Code(s): 1340, 8045, 8244, 8842
Program Element Code(s): 004Y00, 153600, 802000
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

The Cyberlearning and Future Learning Technologies Program funds efforts that support envisioning the future of learning technologies and advance what we know about how people learn in technology-rich environments. Development and Implementation (DIP) Projects build on proof-of-concept work that shows the possibilities of the proposed new type of learning technology to build and refine a minimally-viable example of their proposed innovation that allows them to understand how such technology should be designed and used in the future and that allows them to answer questions about how people learn, how to foster or assess learning, and/or how to design for learning. This project is focused on the teaching of collaborative problem solving activities in introductory engineering courses and builds on a prior project to design tools for collaborative sketching in these courses. The project is based on a recognition of the importance of collaborating in engineering, the need for student to learn this skill, the value of collaborative learning tasks for engaging students in authentic problem solving activities, and the difficulty that graduate student teaching assistants (TAs) encounter when trying to teach in this way. There are two parts to the technology innovation. The first part is a set of tools for the teaching assistants, to help them manage the classroom technologies, and to help them understand how to intervene in groups who are struggling with the content or collaborative processes. The second part is a set of tools for the students. Building on the collaborative sketch software previously developed, prompts to support their collaborative processes will be embedded in the software students will use, based on analysis of the logfiles that help determine who needs what prompts when. Research goals include understanding how receiving prompts changes the nature of students' collaborative activity, and how receiving insight into the difficulties students are having helps TAs learn about to foster collaborative learning in their classes.

The PIs are addressing the difficulties encountered implementing collaborative learning activities in engineering courses by designing and studying tools for TAs and students in these classes. Through an iterative design approach, the PIs will design and study tools for TAs to orchestrate the classroom and collaboration activities and to tools for students which support their collaborative problem solving processes. The PIs will investigate the use of learning analytics in evaluating the collaborative practices of students using these tools; in particular, logfiles will be examined for collaborative indicators based on prior research on collaborative processes, then clustered to look for patterns of engagement, and finally used to create regression models of successful collaboration processes using machine learning techniques. Cross-validation of the models will be done with both logfile and video data to avoid overfitting. These insights will be provided to TAs to examine whether such information is helpful in determining how and when to intervene in groups. Findings from the research will provide insight into: 1) The knowledge that TAs need in order to successfully implement collaborative problem solving in undergraduate courses; 2) Whether TAs can learn more about collaborative problem solving with the support of tools aimed at helping them implement this form of pedagogy; 3) Whether students can learn collaborative problem solving skills through embedded prompts during multi-week collaborative activities and 4) The potential of analytics in determining when and how to reduce the collaboration supports from groups.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 24)
Hur, P., Bosch, N., Paquette, L., Mercier, E. "Harbingers of Collaboration? The Role of Early-Class Behaviors in Predicting Collaborative Problem Solving" International Conference on Educational Data Mining (EDM) , 2020 https://eric.ed.gov/?id=ED607905
Hur, P., Bosch, N., Paquette, L., & Mercier, E. "Harbingers of collaboration? The role of early-class behaviors in predicting collaborative problem solving." International Conference on Educational Data Mining (EDM 2020) , 2020 , p.104
Lawrence, L. & Mercier, E. "Co-design of an orchestration tool: Supporting engineering teaching assistants as they facilitate collaborative learning." Interaction Design and Architecture(s) Journal , v.42 , 2019 , p.111
Lawrence, L. Mercier, E. "A Review of the Evolving Definition of Orchestration: Implications for Research and Design" A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning inCollaborative Settings, 13th International Conference on Computer Supported Collaborative Learning (CSCL) 2019 , v.2 , 2019 , p.829
Lawrence, L., & Mercier, E "A review of the evolving definition of orchestration: Implications for research and design." International Conference on Computer Supported Collaborative Learning , 2018 https://repository.isls.org//handle/1/1683
Lawrence, L., Mercier, E. "Co-Design of an Orchestration Tool: Supporting Engineering Teaching Assistants as they Facilitate Collaborative Learning." Interaction Design and Architecture (s) , v.42 , 2019 , p.111 http://ixdea.uniroma2.it/inevent/events/idea2010/index.php?s=10&a=10&link=ToC_42_P&link=42_6_abstract
Lawrence, L.,Mercier, E. "A Review of the Evolving Definition of Orchestration: Implications for Research and Design" 13th International Conference on Computer Supported Collaborative Learning-A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings, CSCL 2019 , 2019 , p.829 https://doi.dx.org/10.22318/cscl2019.829
Lawrence, L., Tucker, T,,Mercier, E. "Examining the Influence of Instructor Interventions on Group Collaboration." Proceedings of the 14th International Conference on Computer-Supported Collaborative Learning-CSCL 2021. , 2021 , p.161 https://doi.dx.org/10.22318/cscl2021.161
Paquette, L.,Bosch, N.,Mercier, E.,Jung, J.,Shehab, S., Tong, Y. "Matching Data-Driven Models of Group Interactions to Video Analysis of Collaborative Problem Solving on Tablet Computers" Proceedings of International Conference of the Learning Sciences, ICLS , v.1 , 2018 , p.312 https://doi.dx.org/10.22318/cscl2018.312
Paquette, L., Bosch, P.N., Mercier, E., Jung, J. *Shehab, S. & *Tong, Y "Matching Data-Driven Models of Group Interactions to Video Analysis of Collaborative Problem Solving on Tablet Computers." International Conference of the Learning Sciences , 2018 https://repository.isls.org//handle/1/719
Paquette, L., Bosch, P.N., Mercier, E., Jung, J. *Shehab, S. & *Tong, Y. "Matching Data-Driven Models of Group Interactions to Video Analysis of Collaborative Problem Solving on Tablet Computers." Rethinking Learning in the Digital Age: Making the Learning Sciences Count, 13th International Conference of the Learning Sciences (ICLS) 2018 , v.1 , 2018 https://doi.dx.org/10.22318/cscl2018.312
(Showing: 1 - 10 of 24)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

It is essential that students graduate from STEM programs with strong collaboration skills in order to effectively participate in the workforce of today and tomorrow. Additionally, the use of collaborative problem-solving activities allows students to learn how to use the language of their discipline, and how to approach complex problems. However, implementing successful collaboration in classrooms is not simple, particularly in large undergraduate required courses, which are often taught by graduate student teaching assistants with little or no teaching preparation. 

In this project, we focused on collaboration in an introductory required engineering course, that enrolls about 1,000 students per year, with students meeting in discussion sections of about 30 students each week. Building on our prior work, we used a tablet application that allows group members’ work to be synchronized within their group, allowing for the creating of joint representations during collaborative problem solving. We built an instructor dashboard that allowed the instructors to track activity of each group, and in the second iteration, provided live alerts to problematic behavior.

From the student software, we collected log data of groups of about 100 students in group of 3-4 solving collaborative problems each week, as well as video data of their interactions. We identified interaction behaviors in 20-second clips of video, and then created models of log data using data mining techniques, that corresponded with the interactions.  

We identified two key behaviors in the log data: off-topic activity and silent on-task (everyone was working, but not talking) that could be reliably identified to produce a proof-of-concept tool. In the next semester, instructors were sent alerts when these behaviors were identified in the logs. The instructors was alerted to monitor the group for the behavior, and if they agreed the behavior was present, they confirmed the alert and received a suggestion for how to intervene. Initial analysis found that instructors who valued collaboration were more likely to use the tool and find the prompts useful. Future work will extend the live prompt tools to a broader range of behaviors. 

The project had an impact on the teaching of engineering at the University of Illinois Urbana Champaign by adapting tasks to support collaborative learning, and providing guides for future instructors for how to successfully implement collaborative learning.  Over 300 students used the tools we developed, while over 3000 students per year (across math and engineering) use tasks developed during this project.


Last Modified: 02/23/2022
Modified by: Emma Mercier

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page