Award Abstract # 1822494
CAREER: Human-Computer Cooperation for Word-by-Word Question Answering
NSF Org: |
IIS
Division of Information & Intelligent Systems
|
Recipient: |
UNIVERSITY OF MARYLAND, COLLEGE PARK
|
Initial Amendment Date:
|
March 8, 2018 |
Latest Amendment Date:
|
February 18, 2021 |
Award Number: |
1822494 |
Award Instrument: |
Continuing Grant |
Program Manager: |
Tatiana Korelsky
IIS
Division of Information & Intelligent Systems
CSE
Directorate for Computer and Information Science and Engineering
|
Start Date: |
December 1, 2017 |
End Date: |
January 31, 2023 (Estimated) |
Total Intended Award
Amount: |
$488,346.00 |
Total Awarded Amount to
Date: |
$496,346.00 |
Funds Obligated to Date:
|
FY 2017 = $79,715.00
FY 2018 = $108,838.00
FY 2019 = $98,519.00
FY 2020 = $103,861.00
FY 2021 = $105,413.00
|
History of Investigator:
|
-
Jordan
Boyd-Graber
(Principal Investigator)
jbg@umiacs.umd.edu
|
Recipient Sponsored Research
Office: |
University of Maryland, College Park
3112 LEE BUILDING
COLLEGE PARK
MD
US
20742-5100
(301)405-6269
|
Sponsor Congressional
District: |
04
|
Primary Place of
Performance: |
University of Maryland College Park
3112 LEE BLDG 7809 Regents Drive
College Park
MD
US
20742-5103
|
Primary Place of
Performance Congressional District: |
04
|
Unique Entity Identifier
(UEI): |
NPU8ULVAAS23
|
Parent UEI: |
NPU8ULVAAS23
|
NSF Program(s): |
Robust Intelligence
|
Primary Program Source:
|
01001718DB NSF RESEARCH & RELATED ACTIVIT
01001819DB NSF RESEARCH & RELATED ACTIVIT
01001920DB NSF RESEARCH & RELATED ACTIVIT
01002021DB NSF RESEARCH & RELATED ACTIVIT
01002122DB NSF RESEARCH & RELATED ACTIVIT
|
Program Reference
Code(s): |
1045,
7495,
9251
|
Program Element Code(s):
|
749500
|
Award Agency Code: |
4900
|
Fund Agency Code: |
4900
|
Assistance Listing
Number(s): |
47.070
|
ABSTRACT

This CAREER project investigates how humans and computers can work together to answer questions. Humans and computers possess complementary skills: humans have extensive commonsense understanding of the world and greater facility with unconventional language, while computers can effortlessly memorize countless facts and retrieve them in an instant. This proposal helps machines understand who people, places, and characters are; how to communicate this information to humans; and how to allow humans and computers to collaborate in question answering using limited information. A key component of this proposal is answering questions word-by-word: this forces both humans and computers to answer questions using information as efficiently as possible. In addition to embedding these skills in question answering tasks, this proposal has an extensive outreach program to exhibit this technology in interactive question answering competitions for high school and college students.
This research is possible by a new representations of entities in a medium-dimensional embedding that encodes relationships between entities (e.g., the representation of "Goodluck Jonathan" and "Nigeria" encodes that the former is the leader of the latter) to enable the system to answer questions about Nigeria. We validate the effectiveness of these representations both through traditional question answering evaluations and through interactive experiments with human collaboration to ensure that we can visualize these representations effectively. In addition to helping train computers to answer questions, we use opponent modeling and reinforcement learning to help train humans to better answer questions.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
(Showing: 1 - 10 of 26)
(Showing: 1 - 26 of 26)
Alexander Hoyle, Pranav Goel
"Is Automated Topic Model Evaluation Broken?: The Incoherence of Coherence"
Neural Information Processing Systems
, 2021
Citation
Details
Chenglei Si, Chen Zhao
"Whats in a Name? Answer Equivalence For Open-Domain Question Answering"
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
, 2021
Citation
Details
Chen Zhao, Chenyan Xiong
"Distantly-Supervised Dense Retrieval Enables Open-Domain Question Answering without Evidence Annotation"
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
, 2021
Citation
Details
Denis Peskov, Viktor Hangya
"Adapting Entities across Languages and Cultures"
Findings of the Association for Computational Linguistics: EMNLP 2021
, 2021
Citation
Details
Eisenschlos, Julian and Dhingra, Bhuwan and Bulian, Jannis and Börschinger, Benjamin and Boyd-Graber, Jordan
"Fool Me Twice: Entailment from Wikipedia Gamification"
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
, 2021
https://doi.org/10.18653/v1/2021.naacl-main.32
Citation
Details
Elgohary, Ahmed and Peskov, Denis and Boyd-Graber, Jordan
"Can You Unpack That? Learning to Rewrite Questions-in-Context"
Can You Unpack That? Learning to Rewrite Questions-in-Context
, 2019
10.18653/v1/D19-1605
Citation
Details
Elgohary, Ahmed and Zhao, Chen and Boyd-Graber, Jordan
"Dataset and Baselines for Sequential Open-Domain Question Answering"
Empirical Methods in Natural Language Processing
, 2018
Citation
Details
Feng, Shi and Boyd-Graber, Jordan
"What can AI do for me?: Evaluating machine learning interpretations in cooperative play"
IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces
, 2019
10.1145/3301275.3302265
Citation
Details
Feng, Shi and Wallace, Eric and Boyd-Graber, Jordan
"Misleading Failures of Partial-input Baselines"
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
, 2019
10.18653/v1/P19-1554
Citation
Details
Feng, Shi and Wallace, Eric and Grissom II, Alvin and Rodriguez, Pedro and Iyyer, Mohit and Boyd-Graber, Jordan
"Pathologies of Neural Models Make Interpretation Difficult"
Empirical Methods in Natural Language Processing
, 2018
Citation
Details
Han, HyoJung and Carpuat, Marine and Boyd-Graber, Jordan
"SimQA: Detecting Simultaneous MT Errors through Word-by-Word Question Answering"
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
, 2022
https://doi.org/10.18653/v1/2022.emnlp-main.378
Citation
Details
He, Wanrong and Mao, Andrew and Boyd-Graber, Jordan
"Cheaters Bowl: Human vs. Computer Search Strategies for Open-Domain QA"
Findings of the Association for Computational Linguistics: EMNLP 2022
, 2022
https://doi.org/10.18653/v1/2022.findings-emnlp.266
Citation
Details
Iyyer, Mohit and Manjunatha, Varun and Boyd-Graber, Jordan and Davis, Larry
"Learning to Color from Language"
North American Association of Computational Linguistics
, 2018
Citation
Details
Maharshi Gor, Kellie Webster
"Toward Deconfounding the Effect of Entity Demographics for Question Answering Accuracy"
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
, 2021
Citation
Details
Pedro Rodriguez, Jordan Boyd-Graber
"Evaluation Paradigms in Question Answering"
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
, 2021
https://doi.org/10.18653/v1/2021.emnlp-main.758
Citation
Details
Rodriguez, Pedro and Barrow, Joe and Hoyle, Alexander Miserlis and Lalor, John P. and Jia, Robin and Boyd-Graber, Jordan
"Evaluation Examples are not Equally Informative: How should that change NLP Leaderboards?"
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
, 2021
https://doi.org/10.18653/v1/2021.acl-long.346
Citation
Details
Sahil Singla, Eric Wallace
"Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation"
Proceedings of Machine Learning Research
, v.97
, 2019
Citation
Details
Shi, Tianze and Zhao, Chen and Boyd-Graber, Jordan and Daumé III, Hal and Lee, Lillian
"On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries"
Findings of the Association for Computational Linguistics: EMNLP 2020
, 2020
https://doi.org/10.18653/v1/2020.findings-emnlp.167
Citation
Details
Si, Chenglei and Zhao, Chen and Min, Sewon and Boyd-Graber, Jordan
"Re-Examining Calibration: The Case of Question Answering"
Findings of the Association for Computational Linguistics: EMNLP 2022
, 2022
https://doi.org/10.18653/v1/2022.findings-emnlp.204
Citation
Details
Wallace, Eric and Boyd-Graber, Jordan
"Trick Me If You Can: Adversarial Writing of Trivia Challenge Questions"
ACL Student Research Workshop
, 2018
Citation
Details
Wallace, Eric and Feng, Shi and Kandpal, Nikhil and Gardner, Matt and Singh, Sameer
"Universal Adversarial Triggers for Attacking and Analyzing NLP"
Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
, 2019
10.18653/v1/D19-1221
Citation
Details
Wallace, Eric and Rodriguez, Pedro and Feng, Shi and Yamada, Ikuya and Boyd-Graber, Jordan
"Trick Me If You Can: Human-in-the-Loop Generation of Adversarial Examples for Question Answering"
Transactions of the Association for Computational Linguistics
, v.7
, 2019
10.1162/tacl_a_00279
Citation
Details
Zhao, Chen and Xiong, Chenyan and Boyd-Graber, Jordan and Daumé III, Hal
"Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval"
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
, 2021
https://doi.org/10.18653/v1/2021.naacl-main.368
Citation
Details
Zhao, Chen and Xiong, Chenyan and Qian, Xin and Boyd-Graber, Jordan
"Complex Factoid Question Answering with a Free-Text Knowledge Graph"
WWW '20: Proceedings of The Web Conference 2020
, 2020
https://doi.org/10.1145/3366423.3380197
Citation
Details
(Showing: 1 - 10 of 26)
(Showing: 1 - 26 of 26)
Please report errors in award information by writing to: awardsearch@nsf.gov.