Explainable natural language inference is critical for building Natural Language Processing systems that are both reliable and trustworthy. This research aimed at developing methods and datasets that can help build explainable NLP systems in information access applications such as  question answering and relation extraction. As an example, consider a question answering (QA) system that provides an answer to a question. If it is also able to provide the pieces of information that it used to arrive at its answer then a user can assess if the answer is correct. And when the system returns incorrect  answers, a deployer or developer of such a system can attempt to debug why the system failed thus providing a way to improve the system.

One of the main difficulties in developing explainable models is the lack of training  data -- i.e. examples of the input, output and desired explanations. While there were plenty of question and  answer pair datasets, obtaining explanations is much more laborious to create. A related difficulty this presents is that when we build systems that are only trained to only identify the correct answers, they tend to figure out any artifact or spurious correlation that might exist between question and answers in the datasets that they train on. This then results in models that are (A) unreliable -- i.e. ones that don’t do the correct reasoning we expect of them, and (B) not explainable -- i.e. ones that can only provide answers but no useful explanations. This project made multiple contributions in addressing this challenge which can be described in four main threads.

Thread one focused on building QA models that are structured in such a way that they are forced to take good intermediate steps i.e. finding useful sentences that contain necessary information for arriving at the final answer. This naturally lends itself to building models that can provide high quality explanations. We showed how existing ideas in verifying entailment (figuring out if one piece of text supports information in another) can be used to assemble a QA system of this kind.

Thread two focused on minimizing unreliable reasoning by formalizing one type of bad reasoning in QA models and developing new datasets to discourage this behavior. When there is no supervision for intermediate steps models can latch on to any artifact in the data and perform what we call disconnected reasoning. If we can somehow catch when models are not identifying or using all the information they are supposed to use when answering a question then we can address this problem. We developed ways to transform existing datasets to detect and discourage disconnected reasoning.

Another important contribution we make here is to show how to construct reliable and explainable QA datasets. A multihop question, i.e. one which requires multiple pieces of information, can be seen as composed of multiple single-hop questions. This naturally provides a way to construct (and filter) a large collection of multihop questions by connecting single-hop questions where the answer of one question is part of another question. More importantly, this gives us questioons with identified sub-steps, thus providing examples for models to learn to do the intermediate steps and provide supervision for explanations. 

Thread three looked at pushing the scope for what constitutes an explanation for multihop question answering. Previous datasets mostly identified existing spans of text within the inputs to the QA system as explanations. However, this is neither adequate nor concise.  A human providing an explanation would summarize the relevant information and not simply read out the relevant sentences. Again we need examples of such explanations at scale which is expensive and time consuming to obtain. We show that we can make use of existing ideas in abstractive summarization to provide useful compressed summaries which can then be used by a QA model. Further, we introduce a reinforcement  learning framework where we turn notions of explanation quality into rewards to improve the quality of the models ability to find most useful information for answering a question.

Thread four looked at explainable biomedical relation extraction. The main contribution is in formalizing a new task forcing models to learn to not only identify relations between biomedical entities, but also to figure out what is the biomedical mechanism that connects and justifies the inferred relation. The key challenge is the lack of large scale datasets. We show how relatively small amounts of domain expert time can be used to first identify examples of these biomedical mechanisms and then use this to bootstrap creation of a large weakly labeled dataset for this task. 

Overall this project made useful algorithmic advances in terms of methods for building explainable NLP systems and resulted in resources via large scale datasets for question answering and relation extraction which we hope will further advances in these areas.

Last Modified: 12/30/2021
Modified by: Niranjan Balasubramanian