Award Abstract # 1740775
Collaborative Research: Community-Building and Infrastructure Design for Data-Intensive Research in Computer Science Education

NSF Org: DRL
Division of Research on Learning in Formal and Informal Settings (DRL)
Recipient: UNIVERSITY OF PITTSBURGH - OF THE COMMONWEALTH SYSTEM OF HIGHER EDUCATION
Initial Amendment Date: August 15, 2017
Latest Amendment Date: August 15, 2017
Award Number: 1740775
Award Instrument: Standard Grant
Program Manager: Wu He
wuhe@nsf.gov
 (703)292-7593
DRL
 Division of Research on Learning in Formal and Informal Settings (DRL)
EDU
 Directorate for STEM Education
Start Date: September 1, 2017
End Date: February 28, 2022 (Estimated)
Total Intended Award Amount: $271,216.00
Total Awarded Amount to Date: $271,216.00
Funds Obligated to Date: FY 2017 = $271,216.00
History of Investigator:
  • Peter Brusilovsky (Principal Investigator)
    peterb@mail.sis.pitt.edu
Recipient Sponsored Research Office: University of Pittsburgh
4200 FIFTH AVENUE
PITTSBURGH
PA  US  15260-0001
(412)624-7400
Sponsor Congressional District: 12
Primary Place of Performance: University of Pittsburgh
350 Thackeray Hall
Pittsburgh
PA  US  15260-0001
Primary Place of Performance
Congressional District:
12
Unique Entity Identifier (UEI): MKAGLD59JRL1
Parent UEI:
NSF Program(s): ECR-EDU Core Research
Primary Program Source: 04001718DB NSF Education & Human Resource
Program Reference Code(s): 7433, 8083
Program Element Code(s): 798000
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.076

ABSTRACT

The Building Community and Capacity in Data Intensive Research in Education program seeks to enable research communities to develop visions, teams, and capabilities dedicated to creating new, large-scale, next-generation data resources and relevant analytic techniques to advance fundamental research for areas of research covered by the Education and Human Resources Directorate. Successful proposals will outline activities that will have significant impacts across multiple fields by enabling new types of data-intensive research. Online educational systems, and the large-scale data streams that they generate, have the potential to transform education as well as our scientific understanding of learning. Computer Science Education (CSE) researchers are increasingly making use of large collections of data generated by the click streams coming from eTextbooks, interactive programming environments, and other smart content. However, CSE research faces barriers that slow progress: 1) Collection of computer science learning process and outcome data generated by one system is not compatible with that from other systems. 2) Computer science problem solving and learning (e.g., open-ended coding solutions to complex problems) is quite different from the type of data (e.g., discrete answers to questions or verbal responses) that current educational data mining focuses on. This project will build community and capacity among CSE researchers, data scientists, and learning scientists toward reducing these barriers and facilitating the full potential of data-intensive research on learning and improving computer science education. The project will bring together CSE tool building communities with learning science and technology researchers towards developing a software infrastructure that supports scaled and sustainable data-intensive research in CSE that contributes to basic science of human learning of complex problem solving. The project will support community-building and infrastructure capacity-building whose ultimate goal is to develop and disseminate infrastructure that facilitates three aspects of CSE research: (1) development and broader re-use of innovative learning content that is instrumented for rich data collection, (2) formats and tools for analysis of learner data, and (3) best practices to make large collections of learner data and associated analytics available to researchers in CSE, data science, or learning science. To achieve these goals, a large community of researchers will be engaged to define, develop, and use critical elements of this infrastructure toward addressing specific data-intensive research questions.The project will host workshops, meetings, and online forums leveraging existing communities and building new capacities toward significant research outcomes and lasting infrastructure support.

This project will provide an infrastructure that can support various kinds of research in CSE domain as a one-stop-shop, and will be the first to focus on full-cycle educational research infrastructure in any domain. CSE tool developers and educators will become more productive at creating and integrating advanced technologies and novel analytics. Learning researchers will have better tools for analyzing the huge amounts of learner data that modern digital education software produces. Data scientists will have rich new datasets in which to explore new machine learning and statistical techniques. Collectively, these efforts will reduce barriers to educational innovation and support scientific discoveries about the nature of complex learning and how best to enhance it. The project will support scientific investigations through community meetings and mini-grants to others addressing questions such as: What is the optimal ratio of solution examples and problem-solving practice? How do computational thinking skills emerge? In what quanta are programming skills acquired? Can automated tutoring of programming be effective at scale in enhancing student learning?. Many of the innovations developed under this project will directly impact learning in any discipline. Educational software will more quickly be developed in the future, that more easily generates meaningful learner data, which in turn can be more easily analyzed.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 26)
Akhuseyinoglu, Kamil and Brusilovsky, Peter "Data-Driven Modeling of Learners Individual Differences for Predicting Engagement and Success in Online Learning" the 29th ACM Conference on User Modeling, Adaptation and Personalization , 2021 https://doi.org/10.1145/3450613.3456834 Citation Details
Akhuseyinoglu, Kamil and Brusilovsky, Peter "Exploring Behavioral Patterns for Data-Driven Modeling of Learners' Individual Differences" Frontiers in Artificial Intelligence , v.5 , 2022 https://doi.org/10.3389/frai.2022.807320 Citation Details
Akhuseyinoglu, Kamil and Hardt, Ryan and Barria-Pineda, Jordan and Brusilovsky, Peter and Pollari-Malmi, Kerttu and Sirkiä, Teemu and Malmi, Lauri "A Study of Worked Examples for SQL Programming" Proceedings of the 27th ACM Conference on Innovation and Technology in Computer Science Education , v.1 , 2022 Citation Details
Albó, Laia and Barria-Pineda, Jordan and Brusilovsky, Peter and Hernández-Leo, Davinia "Concept-Level Design Analytics for Blended Courses" Proceedings of 14th European Conference on Technology Enhanced Learning (EC-TEL 2019) , 2019 10.1007/978-3-030-29736-7_40 Citation Details
Albó, Laia and Barria-Pineda, Jordan and Brusilovsky, Peter and Hernández-Leo, Davinia "Knowledge-Based Design Analytics for Authoring Courses with Smart Learning Content" International Journal of Artificial Intelligence in Education , 2021 https://doi.org/10.1007/s40593-021-00253-3 Citation Details
Barria-Pineda, Jordan and Akhuseyinoglu, Kamil and Brusilovsky, Peter and Pollari-Malmi, Kerttu and Sirkiä, Teemu and Malmi, Lauri "Personalized Remedial Recommendations for SQL Programming Practice System" Proceedings of Workshop on Adaptation and Personalization in Computer Science Education at the 28th ACM Conference on User Modeling, Adaptation and Personalization , 2020 10.1145/3386392.3399312 Citation Details
Barria-Pineda, Jordan and Akhuseyinoglu, Kamil and elem-elap, Stefan and Brusilovsky, Peter and Klasnja Milicevic, Aleksandra and Ivanovic, Mirjana "Explainable Recommendations in a Personalized Programming Practice System" Lecture notes in computer science , v.12748 , 2021 https://doi.org/10.1007/978-3-030-78292-4_6 Citation Details
Brusilovsky, Peter and Koedinger, Ken and Joyner, David A. and Price, Thomas W. "Building an Infrastructure for Computer Science Education Research and Practice at Scale" Proceedings of the Seventh ACM Conference on Learning at Scale , 2020 10.1145/3386527.3405936 Citation Details
Brusilovsky, Peter and Malmi, Lauri and Hosseini, Roya and Guerra, Julio and Sirkiä, Teemu and Pollari-Malmi, Kerttu "An integrated practice system for learning programming in Python: design and evaluation" Research and Practice in Technology Enhanced Learning , v.13 , 2018 10.1186/s41039-018-0085-9 Citation Details
Chau, Hung and Barria-Pineda, Jordan and Brusilovsky, Peter "Course-Adaptive Content Recommender for Course Authoring" 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3?5, 2018 , 2018 10.1007/978-3-319-93846-2_9 Citation Details
Ellis, Margaret and Shaffer, Clifford A. and Edwards, Stephen H. "Approaches for Coordinating eTextbooks, Online Programming Practice, Automated Grading, and More into One Course" Proceedings of the 2019 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE'19) , 2019 10.1145/3287324.3287487 Citation Details
(Showing: 1 - 10 of 26)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

SPICE is a community-building effort that brings together Computer Science Education (CSEd) researchers, data scientists, and learning scientists to reduce barriers that educators historically have faced in adopting innovative educational software. We developed the means for CS educators to more easily adopt and integrate new smart tools for supporting learning. This infrastructure is, in turn, supporting scaled and sustainable data-intensive research in CSEd that contributes to the basic science of human learning of complex problem-solving. Over the life of our project, we engaged the CSEd community through some 20 workshops, training, and community events; websites containing developer resources, tutorials, and software demonstrations; working groups to intensively work on key standards and infrastructure issues; and data set collection and distribution.

Over the past four years, SPLICE has championed and helped popularize software interoperability standards within the CSEd community, such that use of Learning Tools Interoperability (LTI) is now normal and expected within the community. We have worked to make it easy for our community of innovative educational tool developers to distribute their projects within the broader CS Education community. Our working groups helped to create and distribute standards for collecting data related to small programming exercises, an important and widespread artifact in CS education, and we have made progress on standards to allow faculty to package course curriculum for sharing with others. We have helped to standardize data collection from eTextbooks and associated interactive exercises. We have encouraged researchers to collect and then share nearly 60 CSEd datasets so that other researchers can harness the massive quantities of student analytics data that modern education systems generate, but which historically has gone to waste.

The SPLICE Live Catalog is an innovative way to allow potential adopters of CS Education software to try out the software themselves within the context of an actual Learning Management System. As the community continues to contribute interoperable software to the live catalog, it encourages wider adoption of LTI and similar standards. We directly interacted with hundreds of members of the community through a series of over a dozen workshops, plus special events such as the weeklong LearnLab Summer Schools and a series of CSEDM Data Challenges at the annual Educational Data Mining conference. We supported a number of development efforts through mini-grants and through working groups.


 


Last Modified: 05/30/2022
Modified by: Peter L Brusilovsky

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page