Award Abstract # 2119549
Using Augmented Reality and Artificial Intelligence to Improve Teaching and Learning Spatial Transformations in STEM Disciplines

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: TEXAS A & M UNIVERSITY
Initial Amendment Date: September 8, 2021
Latest Amendment Date: June 20, 2025
Award Number: 2119549
Award Instrument: Standard Grant
Program Manager: Paul Tymann
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: October 1, 2021
End Date: September 30, 2025 (Estimated)
Total Intended Award Amount: $849,971.00
Total Awarded Amount to Date: $865,971.00
Funds Obligated to Date: FY 2021 = $849,971.00
FY 2023 = $16,000.00
History of Investigator:
  • Wei Yan (Principal Investigator)
    wyan@tamu.edu
  • Philip Yasskin (Co-Principal Investigator)
  • Jeffrey Liew (Co-Principal Investigator)
  • Francis Quek (Former Co-Principal Investigator)
  • Dezhen Song (Former Co-Principal Investigator)
  • Heather Burte (Former Co-Principal Investigator)
Recipient Sponsored Research Office: Texas A&M University
400 HARVEY MITCHELL PKY S STE 300
COLLEGE STATION
TX  US  77845-4375
(979)862-6777
Sponsor Congressional District: 10
Primary Place of Performance: Texas A&M University
College Station
TX  US  77843-4242
Primary Place of Performance
Congressional District:
10
Unique Entity Identifier (UEI): JF6XLNB4CDJ5
Parent UEI:
NSF Program(s): HSI-Hispanic Serving Instituti,
IUSE,
Cyberlearn & Future Learn Tech
Primary Program Source: 01002324DB NSF RESEARCH & RELATED ACTIVIT
04002122DB NSF Education & Human Resource
Program Reference Code(s): 092Z, 8045, 8209, 9251
Program Element Code(s): 077y00, 199800, 802000
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

The mathematics that are used to describe spatial transformations can be very difficult for undergraduate students. While moving something in the physical world may be easy to understand, describing the same operation with math in the digital world can be daunting. This project will develop new technology using Augmented Reality (AR) and Artificial Intelligence (AI) to improve the teaching and learning of these difficult concepts in STEM disciplines as well as creative endeavors. The project will test the use of new AR/AI technology to enhance students? learning of the mathematics behind spatial transformations. This will improve the use of AR/AI-powered precise motion tracking of objects that can collect high resolution in-situ motion and scene data to enhance learning analytics. The project will identify the AR capabilities that can help students conceive, connect, and compare mathematical representations of motion to overcome the well-documented difficulties students face when learning spatial transformations. Strengthening this skill will support their continued development across many STEM disciplines.

An AR/AI-powered innovative learning environment will be developed and evaluated for its effects on teaching and learning major rotation and orientation representations in the Euclidean space. Different levels of abstraction, including Axis-Angle, Euler Angles, Matrices, and Quaternions will be tested. In workshops participants will play with and transform 3D-printed geometry models to evaluate the effectiveness of the AR/AI technology. The project will assess student learning outcomes by comparing math skills in pre- and post-workshop tests compared to students working through the same tasks without the use of the AR/AI technology. The project will contribute to advancing knowledge across disciplines of spatial and mathematical pedagogies, by exploring: a) the role of novel AR interaction that allows the interplay between physical and virtual manipulatives to engage students in embodied learning; b) the capabilities of AR to make difficult invisible concepts visible for supporting an intuitive and formal understanding of spatial reasoning and mathematical formulation; c) the features of AR that help students see relationships between spatial manipulations and mathematical operations; and d) the potential impact of individual differences in spatial transformations when using AR-assisted mathematical learning.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 14)
Aguilar, Samantha D and Burte, Heather and Stautler, James and Mondal, Sadrita and Qian, Chengyuan and Monjoree, Uttamasha and Yasskin, Philip and Liew, Jeffrey and Song, Dezhen and Yan, Wei "Learning 3D Matrix Algebra Using Virtual and Physical Manipulatives: Qualitative Analysis of the Efficacy of the AR-Classroom" , v.14724 , 2024 Citation Details
Aguilar, Samantha D and Burte, Heather and Yasskin, Philip and Liew, Jeffrey and Yeh, Shu-Hao and Qian, Chengyuan and Song, Dezhen and Monjoree, Uttamasha and Yan, Wei "AR-Classroom: Usability of AR Educational Technology for Learning Rotations Using Three-Dimensional Matrix Algebra" , 2023 https://doi.org/10.1109/FIE58773.2023.10342996 Citation Details
Alhazzaa, Kifah and Yan, Wei "Integrating Parametric Modeling, BIM, and Building Performance Analysis into Augmented Reality for Architectural Design and Education" , 2023 https://doi.org/10.1145/3603421.3603431 Citation Details
Ashour, Ziad and Shaghaghian, Zohreh and Yan, Wei "An Augmented Reality Application and User Study for Understanding and Learning Architectural Representations" 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) , 2023 https://doi.org/10.1109/VRW58643.2023.00241 Citation Details
Burte, Heather and Aguilar, Samantha D and Stautler, James and Mondal, Sadrita and Qian, Chengyuan and Monjoree, Uttamasha and Yasskin, Philip and Liew, Jeffrey and Song, Dezhen and Yan, Wei "Learning 3D Matrix Algebra Using Virtual and Physical Manipulatives: Statistical Analysis of Quantitative Data Evaluating the Efficacy of the AR-Classroom" , v.14724 , 2024 Citation Details
Guo, Fengzhi and Xie, Shuangyu and Wang, Di and Fang, Cheng and Zou, Jun and Song, Dezhen "A Pretouch Perception Algorithm for Object Material and Structure Mapping to Assist Grasp and Manipulation Using a DMDSM Sensor" Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems , 2023 https://doi.org/10.1109/IROS55552.2023.10341560 Citation Details
Shaghaghian, Z. and Burte, H. and Song, D. and Yan, W. "Learning Spatial Transformations and their Math Representations through Embodied Learning in Augmented Reality" Learning and Collaboration Technologies. Novel Technological Environments. HCII 2022. Lecture Notes in Computer Science. , v.13329 , 2022 https://doi.org/10.1007/978-3-031-05675-8_10 Citation Details
Shaghaghian, Zohreh and Burte, Heather and Song, Dezhen and Yan, Wei "An augmented reality application and experiment for understanding and learning spatial transformation matrices" Virtual Reality , v.28 , 2024 https://doi.org/10.1007/s10055-023-00904-x Citation Details
Shaghaghian, Zohreh and Burte, Heather and Song, Dezhen and Yan, Wei "Design and Evaluation of an Augmented Reality App for Learning Spatial Transformations and their Mathematical Representations" 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) , 2022 https://doi.org/10.1109/VRW55335.2022.00155 Citation Details
Tuzun Canadinc, Seda and Yan, Wei "3D-Model-Based Augmented Reality for Enhancing Physical Architectural Models" Proceedings of the 40th Conference on Education and Research in Computer Aided Architectural Design in Europe (eCAADe 2022) , v.2 , 2022 https://doi.org/10.52842/conf.ecaade.2022.2.495 Citation Details
Tuzun Canadinc, Seda and Yan, Wei "Multi-3D-Models Registration-Based Augmented Reality Instructions for Assembly" IEEE Conference on Virtual Reality + 3D User Interfaces Abstracts and Workshops (VRW) , 2024 Citation Details
(Showing: 1 - 10 of 14)

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page