Skip to feedback

Award Abstract # 1623094
EXP: Collaborative Research: Extracting Salient Scenarios from Interaction Logs (ESSIL)

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: NEW YORK HALL OF SCIENCE
Initial Amendment Date: August 26, 2016
Latest Amendment Date: May 19, 2021
Award Number: 1623094
Award Instrument: Standard Grant
Program Manager: Amy Baylor
abaylor@nsf.gov
 (703)292-5126
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: September 1, 2016
End Date: June 30, 2022 (Estimated)
Total Intended Award Amount: $55,000.00
Total Awarded Amount to Date: $55,000.00
Funds Obligated to Date: FY 2016 = $55,000.00
History of Investigator:
  • Stephen Uzzo (Principal Investigator)
    uzzo@momath.org
  • Leilah Lyons (Former Principal Investigator)
Recipient Sponsored Research Office: New York Hall of Science
4701 111TH ST
CORONA
NY  US  11368-2950
(718)595-9173
Sponsor Congressional District: 06
Primary Place of Performance: New York Hall of Science
47-01 111TH STREET
Corona
NY  US  11368-2950
Primary Place of Performance
Congressional District:
06
Unique Entity Identifier (UEI): UQ7FBRE34HS5
Parent UEI:
NSF Program(s): Cyberlearn & Future Learn Tech
Primary Program Source: 01001617DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 8045, 8841
Program Element Code(s): 802000
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

The Extracting Salient Scenarios from Interaction Logs (ESSIL) project proposes to develop a new type of educational technology to support students' learning about complex systems from their participation in a multi-person immersive simulation. Many important challenges we face today as a society -- including responding to climate change, managing global economies, city planning, disease outbreaks -- are "complex systems" problems, meaning that important phenomena in each (for instance trends in weather, stock bubbles, traffic jams, disease transmission) result not from a single cause, but because many small causes combine together. Participating in a simulation has the potential to help students understand the principles of complex systems, but because different principles surface depending on how each simulation unfolds, it can be difficult for teachers to adjust their lesson plans on the fly to highlight the principles that emerge in a given simulation run. To address this challenge, ESSIL will develop methods to create "automatic salient recaps," as a way to help learners and their teachers make better sense of simulations. These recaps, which will be automatically generated, provide a story of "what happened" in the simulation in a way that both helps students remember their experience and reveals important scientific principles. Teachers and other facilitators will use these recaps, along with an accompanying discussion guide, to support productive learning conversations about the scientific principles incorporated in a simulation. The recaps will be developed for a large-scale immersive simulation installed at the New York Hall of Science (NYSCI), potentially improving the educational experience of thousands of daily visitors. The capabilities developed to produce them have widespread applicability, because logs of student interactions are routinely produced by many educational systems. The project is supported by the Cyberlearning and Future Learning Technologies Program, which funds efforts that will help envision the next generation of learning technologies and advance what we know about how people learn in technology-rich environments. Cyberlearning Exploration (EXP) Projects explore the viability of new kinds of learning technologies by designing and building new kinds of learning technologies and studying their possibilities for fostering learning and challenges to using them effectively.

The immersive simulation context for the project is Connected Worlds, an embodied, multi-person ecology simulation at NYSCI, with pedagogical goals around sustainability and systems thinking. Using logs from groups of students interacting with Connected Worlds, ESSIL will construct selective recaps of their experience that both are personally salient to them (by including memorable details of their experience) and have explanatory coherence (to enable their discussion of important interconnections in the simulation's underlying model). Artificial Intelligence-based methods will be developed to 1) identify salient changes in the state of the simulation during student interaction and 2) construct qualitative models of causal chains that could have led to these changes. These qualitative models will be used to generate salient recaps and discussion guides based on them, which will be provided to teachers whose classes are visiting NYSCI. The effectiveness of the innovation will be investigated by comparing visiting students' conversations with and without ESSIL-generated discussion supports and by interrogating their resulting models of the Connected Worlds system through concept maps.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

Hoernle, N., Gal, Y., Grosz, B., Protopapas, P. and Rubin, A. "Modeling the Effects of Students' Interactions with Immersive Simulations using Markov Switching Systems" Educational Data Mining Conference , 2018
Mallavarapu, Aditi and Lyons, Leilah and Uzzo, Stephen and Thompson, Wren and Levy-Cohen, Rinat and Slattery, Brian "Connect-to-Connected Worlds: Piloting a Mobile, Data-Driven Reflection Tool for an Open-Ended Simulation at a Museum" CHI '19 Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems , 2019 10.1145/3290605.3300237 Citation Details
Mallavarapu, AditiLyons, Leilah BUzzo, Stephen M.Thompson, WrenLevy-Cohen, RinatSlattery, Brian "Connect to Connected Worlds: Piloting a Mobile, Data-Driven Reflection Tool for an Open-Ended Simulation at a Museum" ACM Conference on Human Factors in Computing Systems (CHI '19) , 2019 https://doi.org/10.1145/3290605.3300237

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

The goal of ESSIL was to further the educational goals of multi-person immersive participatory simulations.  The work was carried out in the context of a spectacular multi-story ecosystem simulation called Connected Worlds at the New York Hall of Science Museum.  Connected Worlds consists of four biomes that share water from a few common sources; participants can route the ?water? using props and can plant seeds, which will sprout if there is sufficient water.  Animals appear in each biome after a variety of plants have begun to grow.  A rendering of the Connected Worlds exhibit is attached.

Participatory simulations, in which participants act on the simulation and thus impact what happens, can be highly engaging, but supporting participants? ability to understand a simulation?s underlying principles is a significant challenge.  Especially with a simulation like Connected Worlds, in which dozens of people can participate at once, it is difficult for any one person to have a sense of the whole or even to remember enough about what went on to discern cause and effect relationships.

We approached this problem in several ways.  Working with classes of middle school students who experienced Connected Worlds on field trips to the museum, we first designed a pre-visit introduction to Connected Worlds and a structured activity for the visit itself, in which students were assigned to biome teams and had specific roles within those teams.  We then designed an automatically-generated visual recap that used the logs kept by the system as the simulation progressed.  The visual recap comprised a line graph that showed the amount of water in each biome over time, bar graphs that tracked the appearance of plants and animals in each biome and a birds-eye view video of the simulation itself.  All of the graphs and the video were linked, so that they all reflected the state of the system at the same time.  A sophisticated data science algorithm was used to identify periods of generally increasing or decreasing water flow to each biome and indicate them on the water level line graph.  An example of a data visualization is attached.

Our experiments with a class of students who visited Connected Worlds demonstrated that a post-visit reflection activity supported by our visualization enabled students to recall their experience, analyze it more deeply, and begin to build a conceptual model of the workings of the simulation.  However, the experience also demonstrated some gaps in students? thinking, which we addressed by designing an additional intervention: an annotation task, in which students would create a single representation of their experience by superimposing indications of animal and plant events on the water graph.  While we did not have the opportunity to try out this activity with students due to the pandemic, we elicited feedback from museum interpreters who had had experience facilitating visitors? interactions with Connected Worlds. A sample annotation is attached.

Our in-depth study of Connected Worlds, as well as our experience designing and studying interventions to support students? understanding of its underlying model, led to a set of insights about the opportunities and challenges of participatory simulations, useful to simulation designers, curriculum designers and learning scientists.


Last Modified: 10/06/2022
Modified by: Stephen M Uzzo

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page