Award Abstract # 1918751
Advancing the Science of Learning Data Science with Adaptive Learning for Future Workforce Development
NSF Org: |
DUE
Division Of Undergraduate Education
|
Recipient: |
UNIVERSITY OF MEMPHIS
|
Initial Amendment Date:
|
January 13, 2020 |
Latest Amendment Date:
|
January 13, 2020 |
Award Number: |
1918751 |
Award Instrument: |
Standard Grant |
Program Manager: |
Nasser Alaraje
nalaraje@nsf.gov
(703)292-8063
DUE
Division Of Undergraduate Education
EDU
Directorate for STEM Education
|
Start Date: |
January 15, 2020 |
End Date: |
August 31, 2025 (Estimated) |
Total Intended Award
Amount: |
$3,439,035.00 |
Total Awarded Amount to
Date: |
$3,439,035.00 |
Funds Obligated to Date:
|
FY 2020 = $3,439,035.00
|
History of Investigator:
|
-
Andrew
Olney
(Principal Investigator)
aolney@memphis.edu
-
Vasile
Rus
(Co-Principal Investigator)
-
Scott
Fleming
(Co-Principal Investigator)
-
Dale
Bowman
(Co-Principal Investigator)
-
Andrew
Tawfik
(Co-Principal Investigator)
|
Recipient Sponsored Research
Office: |
University of Memphis
115 JOHN WILDER TOWER
MEMPHIS
TN
US
38152-0001
(901)678-3251
|
Sponsor Congressional
District: |
09
|
Primary Place of
Performance: |
The University of Memphis
Applicant Services 101 Wilder To
Memphis
TN
US
38152-3370
|
Primary Place of
Performance Congressional District: |
09
|
Unique Entity Identifier
(UEI): |
F2VSMAKDH8Z7
|
Parent UEI: |
|
NSF Program(s): |
IUSE
|
Primary Program Source:
|
04002021DB NSF Education & Human Resource
|
Program Reference
Code(s): |
7967,
9178
|
Program Element Code(s):
|
199800
|
Award Agency Code: |
4900
|
Fund Agency Code: |
4900
|
Assistance Listing
Number(s): |
47.076
|
ABSTRACT

This project aims to serve the national interest by improving training in data science. Data scientists are needed to power the ongoing revolution in Big Data that is transforming virtually every sector of the economy. Progress in training data scientists is currently limited by a lack of understanding about how data science is learned and by a lack of techniques to optimize that learning. This project will advance understanding of how data science is learned by weaving together statistics, programming, and machine learning and experimental results about student learning. It will use this understanding to create an innovative Artificial Intelligence-enabled data science tutor called ?DataWhys.? The DataWhys tutor can be integrated into JupyterLab, an established professional data science tool, and will provide 250 hours of training content.
To advance understanding of how data science is learned and how to optimize that learning, this project will identify the most effective scaffolds for worked examples across varying levels of expertise and identify when scaffolds should be removed. It will then compare a data science intelligent tutoring condition that implements these findings against worked example and pure problem-solving controls. This approach will synthesize previous work in the related fields of statistics, programming, and machine learning education, each of which has used only a few of the scaffolds and techniques that will be comprehensively investigated in this project. In addition to cross-sectional studies with college freshman, STEM majors, and graduate students, longitudinal studies will be conducted in partnership with the data science division of St. Jude Children's Research Hospital and through a summer internship for STEM majors from LeMoyne-Owen College. These longitudinal studies will provide additional evidence regarding workforce relevance through usability metrics and progress in personal learning plans. Source code and training material produced under the project will be publicly shared on GitHub where it can be freely used and modified by anyone under the open-source Apache license. This project is supported by the Accelerating Discovery: Educating the Future STEM Workforce program, which funds projects to educate the STEM workforce in the critical scientific areas defined by the Big Ideas for NSF Investment.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
(Showing: 1 - 10 of 61)
(Showing: 1 - 61 of 61)
Banjade, R and Oli, P and Sajib, MH and Rus, V
"Identifying Gaps in Students Explanations of Code Using LLMs"
, v.14830
, 2024
Citation
Details
Banjade, R. and Oli, P. and Tamang, L.J. and Rus, V.
"Preliminary Experiments with Transformer based Approaches To Automatically Inferring Domain Models from Textbooks"
Proceedings of the 15th International Conference on Educational Data Mining
, 2022
Citation
Details
Bridson, Kathryn and Atkinson, Jeffrey and Fleming, Scott D.
"Delivering Round-the-Clock Help to Software Engineering Students Using Discord: An Experience Report"
Proceedings of the 53rd ACM Technical Symposium on Computer Science Education
, v.1
, 2022
https://doi.org/10.1145/3478431.3499385
Citation
Details
Bridson, Kathryn and Fleming, Scott D.
"Frequent, Timed Coding Tests for Training and Assessment of Full-Stack Web Development Skills: An Experience Report"
Proceedings of the 52nd ACM Technical Symposium on Computer Science Education
, 2021
https://doi.org/10.1145/3408877.3432549
Citation
Details
Chapagain, Jeevan and Risha, Zak and Banjade, Rabin and Oli, Priti and Tamang, Lasang and Brusilovsky, Peter and RUs, Vasile
"SelfCode: An Annotated Corpus and a Model for Automated Assessment of Self-Explanation During Source Code Comprehension"
The International FLAIRS Conference Proceedings
, v.36
, 2023
https://doi.org/10.32473/flairs.36.133385
Citation
Details
DeFalco, J. A. and Blake-Plock, S. and Hampton, A. J.
"The Renovated Room: Ethical Implications of Intentional AI in Learning Technology"
Proceedings of the Ninth Annual GIFT Users Symposium (GIFTsym9)
, 2021
Citation
Details
Gish-Lieberman, Jaclyn Joy and Rockinson-Szapkiw, Amanda and Tawfik, Andrew A. and Theiling, Teresa M.
"Designing for Self-Efficacy: E-Mentoring Training for Ethnic and Racial Minority Women in STEM"
International Journal of Designs for Learning
, v.12
, 2021
https://doi.org/10.14434/ijdl.v12i3.31433
Citation
Details
Hampton, Andrew
"Say What? Learner Reactions to Unexpected Agent Dialogue Moves"
Journal of Applied Instructional Design
, 2022
https://doi.org/10.51869/111/ahjgjgat
Citation
Details
Hampton, Andrew J. and Tawfik, Andrew A.
"Experiential Instruction of Metacognitive Strategies"
Proceedings of the Second International Conference on Adaptive Instructional Systems
, 2020
https://doi.org/10.1007/978-3-030-50788-6_8
Citation
Details
Hollander, John and Olney, Andrew
"Raising the Roof: Situating Verbs in Symbolic and Embodied Language Processing"
Cognitive Science
, v.48
, 2024
https://doi.org/10.1111/cogs.13442
Citation
Details
Johnson, Jill and Olney, Andrew M.
"Using community-based problems to increase motivation in a data science virtual internship"
Proceedings of the 15th International Conference on Educational Data Mining
, 2022
Citation
Details
Jung, Yeonji and Wise, Alyssa Friend
"Probing Actionability in Learning Analytics: The Role of Routines, Timing, and Pathways"
, 2024
https://doi.org/10.1145/3636555.3636914
Citation
Details
Kim, Kyung and Tawfik, Andrew A.
"Different approaches to collaborative problem solving between successful versus less successful problem solvers: Tracking changes of knowledge structure"
Journal of Research on Technology in Education
, 2021
https://doi.org/10.1080/15391523.2021.2014374
Citation
Details
Li, F and Jung, Y and Wise, A F
"I Didnt Pass the Exam Because ...: Testing the Viability of Conceptual Features for Actionable Analytics in the Context of Competency Exam Failure Reflection"
, 2024
Citation
Details
Mainali, Sambriddhi and Garzon, Max and Venugopal, Deepak and Jana, Kalidas and Yang, Ching-Chi and Kumar, Nirman and Bowman, Dale and Deng, Lih-Yuan
"An Information-theoretic approach to dimensionality reduction in data science"
International Journal of Data Science and Analytics
, v.12
, 2021
https://doi.org/10.1007/s41060-021-00272-2
Citation
Details
Oli, Priti and Banjade, Rabin and Chapagain, Jeevan and Rus, Vasile
"Automated Assessment of Students Code Comprehension using LLMs"
, v.257
, 2024
Citation
Details
Oli, Priti and Banjade, Rabin and Lekshmi_Narayanan, Arun Balajiee and Brusilovsky, Peter and Rus, Vasile
"Exploring The Effectiveness of Reading vs. Tutoring For Enhancing Code Comprehension For Novices"
ACM Symposium on Applied Computing, SAC 2024
, 2024
https://doi.org/10.1145/3605098.3636007
Citation
Details
Oli, Priti and Banjade, Rabin and Tamang, Lasang Jimba and Rus, Vasile
"Automated Assessment of Quality of Jupyter Notebooks Using Artificial Intelligence and Big Code"
The International FLAIRS Conference Proceedings
, v.34
, 2021
https://doi.org/10.32473/flairs.v34i1.128560
Citation
Details
Olney, A. M.
"Paraphrasing Academic Text: A Study of Back-Translating Anatomy and Physiology with Transformers."
Proceedings of the 22nd International Conference on Artificial Intelligence in Education
, 2021
https://doi.org/10.1007/978-3-030-78270-2_50
Citation
Details
Olney, A. M.
"Sentence Selection for Cloze Item Creation: A Standardized Task and Preliminary Results."
Joint Proceedings of the Workshops at the 14th International Conference on Educational Data Mining
, v.3051
, 2021
Citation
Details
Olney, A. M. and Fleming, S. D.
"JupyterLab Extensions for Blocks Programming, Self-Explanations, and HTML Injection"
Joint Proceedings of the Workshops at the 14th International Conference on Educational Data Mining
, v.3051
, 2021
Citation
Details
Olney, Andrew M
"Generating multiple choice questions from a textbook: LLMs match human performance on most metrics"
CEUR workshop proceedings
, 2023
Citation
Details
Olney, Andrew M.
"Assessing Readability by Filling Cloze Items with Transformers"
Proceedings of the 22nd International Conference on Artificial Intelligence in Education
, 2022
https://doi.org/10.1007/978-3-031-11644-5_25
Citation
Details
Olney, Andrew M.
"Generating Multiple Choice Questions with a Multi-Angle Question Answering Model"
, 2023
https://doi.org/10.5281/zenodo.7761561
Citation
Details
Olney, Andrew M.
"Generating Response-Specific Elaborated Feedback Using Long-Form Neural Question Answering"
Proceedings of the Eighth ACM Conference on Learning @ Scale
, 2021
https://doi.org/10.1145/3430895.3460131
Citation
Details
Olney, Andrew M. and Fleming, Scott D. and Johnson, Jillian C.
"Learning Data Science with Blockly in JupyterLab"
Proceedings of the 52nd ACM Technical Symposium on Computer Science Education
, 2021
https://doi.org/10.1145/3408877.3439534
Citation
Details
Olney, Andrew M. and Gilbert, Stephen B. and Rivers, Kelly
"Preface to the Special Issue on Creating and Improving Adaptive Learning: Smart Authoring Tools and Processes"
International Journal of Artificial Intelligence in Education
, 2021
https://doi.org/10.1007/s40593-021-00277-9
Citation
Details
Olney, Andrew M and Smith, Jesse and Sen, Saunak and Thomas, Fridtjof and Unwin, H Juliette
"Estimating the Effect of Social Distancing Interventions on COVID-19 in the United States"
American Journal of Epidemiology
, v.190
, 2021
https://doi.org/10.1093/aje/kwaa293
Citation
Details
Pahi, Kritish and Hawlader, Shiplu and Hicks, Eric and Zaman, Alina and Phan, Vinhthuy
"Enhancing active learning through collaboration between human teachers and generative AI"
Computers and Education Open
, v.6
, 2024
https://doi.org/10.1016/j.caeo.2024.100183
Citation
Details
Pavlik Jr., P. I. and Olney, A. M. and Banker, A. and Eglington, L. and Yarbro, J.
"The Mobile Fact and Concept Textbook System (MoFaCTS)"
CEUR workshop proceedings
, v.2674
, 2020
https://doi.org/
Citation
Details
Payne, Linda and Tawfik, Andrew and Olney, Andrew M.
"Computational Thinking in Education: Past and Present"
TechTrends
, v.66
, 2022
https://doi.org/10.1007/s11528-022-00766-1
Citation
Details
Payne, Linda Ann and Tawfik, Andrew and Olney, Andrew
"Datawhys Phase 1: Problem Solving to Facilitate Data Science & STEM Learning Among Summer Interns"
International Journal of Designs for Learning
, v.12
, 2021
https://doi.org/10.14434/ijdl.v12i3.31555
Citation
Details
Pham, Diem-Trang and Phan, Vinhthuy
"MetaBIDx: a new computational approach to bacteria identification in microbiomes"
Microbiome Research Reports
, v.3
, 2024
https://doi.org/10.20517/mrr.2024.01
Citation
Details
Rus, V. and Akhuseyinoglu, K. and Chapagain, J. and Tamang, L.J.
"Prompting for Free Self-Explanations Promotes Better Code Comprehension."
5th Educational Data Mining in Computer Science Education (CSEDM) Workshop in Conjunction with The 14th International Educational Data Mining Conference
, 2021
Citation
Details
Schmidt, Matthew
"Activity Theory as a Lens for Developing and Applying Personas and Scenarios in Learning Experience Design"
Journal of Applied Instructional Design
, 2022
https://doi.org/10.51869/111/msat
Citation
Details
Shakya, A. and Rus, V. and Fancsali, S. and Ritter, S and Venugopal, D.
"NeTra: A Neuro-Symbolic System to Discover Strategies in Math Learning"
Proceedings of The Third Workshop of the Learner Data Institute, The 15th International Conference on Educational Data Mining
, 2022
Citation
Details
Tamang, Lasang Jimba and Alshaikh, Zeyad and Khayi, Nisrine Ait and Oli, Priti and Rus, Vasile
"A Comparative Study of Free Self-Explanations and Socratic Tutoring Explanations for Source Code Comprehension"
Proceedings of the 52nd ACM Technical Symposium on Computer Science Education
, 2021
https://doi.org/10.1145/3408877.3432423
Citation
Details
Tamang, L.J. and Banjade, R. and Chapagain, J. and Rus, V.
"Automatic Question Generation for Scaffolding Self-explanations for Code Comprehension"
Proceedings of the 22nd International Conference on Artificial Intelligence in Education
, 2022
https://doi.org/10.1007/978-3-031-11644-5_77
Citation
Details
Tawfik, Andrew A. and Bradford, Jacque and Gish-Lieberman, Jaclyn and Gatewood, Jessica
"Repeated Measures of Cognitive and Affective Learning Outcomes in Simulation Debriefing"
Journal of Physical Therapy Education
, v.36
, 2022
https://doi.org/10.1097/JTE.0000000000000233
Citation
Details
Tawfik, Andrew A. and Gatewood, Jessica and Armstrong, Laura and Shepherd, Craig E.
"Informal Learning in United States Libraries: A Systematic Review"
TechTrends
, 2022
https://doi.org/10.1007/s11528-022-00811-z
Citation
Details
Tawfik, Andrew A. and Gatewood, Jessica and Gish-Lieberman, Jaclyn J. and Hampton, Andrew J.
"Toward a Definition of Learning Experience Design"
Technology, Knowledge and Learning
, 2021
https://doi.org/10.1007/s10758-020-09482-2
Citation
Details
Tawfik, Andrew A. and Gatewood, Jessica D. and Gish-Lieberman, Jaclyn J. and Keene, Charles W.
"Exploring the Differences Between Experts and Novices on Inquiry-Based Learning Cases"
Journal of Formative Design in Learning
, v.5
, 2021
https://doi.org/10.1007/s41686-021-00062-w
Citation
Details
Tawfik, Andrew A. and Graesser, Arthur and Gatewood, Jessica and Gishbaugher, Jaclyn
"Role of questions in inquiry-based instruction: towards a design taxonomy for question-asking and implications for design"
Educational Technology Research and Development
, v.68
, 2020
https://doi.org/10.1007/s11423-020-09738-9
Citation
Details
Tawfik, Andrew A and Hung, Woei and Giabbanelli, Philippe J.
"Comparing How Different Inquiry-based Approaches Impact Learning Outcomes"
Interdisciplinary Journal of Problem-Based Learning
, v.14
, 2020
https://doi.org/10.14434/ijpbl.v14i1.28624
Citation
Details
Tawfik, Andrew A. and Kim, Kyung and Kim, Dongho
"Effects of case library recommendation system on problem solving and knowledge structure development"
Educational Technology Research and Development
, v.68
, 2020
https://doi.org/10.1007/s11423-020-09737-w
Citation
Details
Tawfik, Andrew A. and Koehler, Adrie A. and Gish-Lieberman, Jaclyn J. and Gatewood, Jessica
"Investigating the depth of problem-solving prompts in collaborative argumentation"
Innovations in Education and Teaching International
, v.58
, 2021
https://doi.org/10.1080/14703297.2021.1966821
Citation
Details
Tawfik, Andrew_A and Payne, Linda and Ketter, Heather and James, Jedidiah
"What instruments do researchers use to evaluate LXD? A systematic review study"
Technology, Knowledge and Learning
, v.30
, 2024
https://doi.org/10.1007/s10758-024-09763-0
Citation
Details
Tawfik, Andrew A. and Payne, Linda and Olney, Andrew M.
"Scaffolding Computational Thinking Through Block Coding: A Learner Experience Design Study"
Technology, Knowledge and Learning
, 2022
https://doi.org/10.1007/s10758-022-09636-4
Citation
Details
Tawfik, Andrew A. and Schmidt, Matthew and Hooper, Chelsy P.
"Role of conjecture mapping in applying a game-based strategy towards a case library: a view from educational design research"
Journal of Computing in Higher Education
, v.32
, 2020
https://doi.org/10.1007/s12528-020-09251-1
Citation
Details
Tawfik, Andrew and Gatewood, Jessica
"Decision Making and Problem-Solving: Implications for Learning Design"
Journal of Applied Instructional Design
, v.11
, 2022
https://doi.org/10.51869/112/atjg
Citation
Details
Tawfik, Andrew and Schmidt, Matthew and Payne, Linda and Huang, Rui
"Advancing understanding of learning experience design: refining and clarifying definitions using an eDelphi study approach"
Educational technology research and development
, v.72
, 2024
https://doi.org/10.1007/s11423-024-10355-z
Citation
Details
Thapa_Magar, Abisha and Fancsali, Stephen E and Rus, Vasile and Murphy, April and Ritter, Steve and Venugopal, Deepak
"Learning Representations for Math Strategies using BERT"
, 2024
https://doi.org/10.1145/3657604.3664711
Citation
Details
West, Richard E. and Tawfik, Andrew A. and Gishbaugher, Jaclyn J. and Gatewood, Jessica
"Guardrails to Constructing Learning: the Potential of Open Microcredentials to Support Inquiry-Based Learning"
TechTrends
, v.64
, 2020
https://doi.org/10.1007/s11528-020-00531-2
Citation
Details
Yarbro, J. T. and Olney, A. M.
"Contextual Definition Generation"
Proceedings of the Third International Workshop on Intelligent Textbooks
, v.2895
, 2021
Citation
Details
Yarbro, J. T. and Olney, A. M.
"WikiMorph: Learning to Decompose Words into Morphological Structures"
Proceedings of the 22nd International Conference on Artificial Intelligence in Education
, 2021
https://doi.org/10.1007/978-3-030-78270-2_72
Citation
Details
Zhu, Xinran and Jung, Yeonji and Chen, Bodong and Hickey, Daniel and Chartrand, Grant and Kalir, Remi and Hodgson, Justin and Andrews, Chris and Wise, Alyssa and Shui, Hong and Chen, Pingting and Avadhanam, Rukmini Manasa
"Bridging Social Annotation Practice with Perspectives from the Learning Sciences and CSCL"
, 2024
https://doi.org/10.22318/cscl2024.530500
Citation
Details
(Showing: 1 - 10 of 61)
(Showing: 1 - 61 of 61)
Please report errors in award information by writing to: awardsearch@nsf.gov.