The goal of the SearchIE project was to explore the use of search to solve problems where an annotation-based approach was overly resource- and time-consuming. For example, consider someone reading news and coming across an article listing a few winners of the Iditarod racing (dog sleds) and then wondering who else had won that race previously. A classic information extraction approach to this problem would be to mark a collection of documents with known winners, train a word classification system, and then run that classifier on an entire collection of documents. The time spent on all of those steps is not worthwhile for a passing curiosity or something with so few instances. In the SearchIE project we explore methods that allows the reader to mark a few interesting passages (the list of winners), perhaps be asked to highlight the actual names to make the need clear, and then allow the system to immediately find matches by searching based on underlying features rather than exhaustively scanning the entire collection.

The primary work in this vein considered how to add to a known set of entities, either by providing a handful of sample entities (e.g., winners of the Iditarod, presidents of the U.S.) or 1-2 sentences including an entity and context (e.g., a couple of sentences describing actions of a U.S. president). As highlighted above, our approach is to extract important features from entities or their context and use those to search a collection of documents for the additional entities -- rather than scanning the entire collection word-by-word to find instances that match. We considered this both in a completely automatic setting -- given some examples, find more -- and an interactive setting -- allowing the system to request additional clarifying information from the user. Over the life of this grant, we demonstrated that the types of features used for searching are not the same as those an exhaustive classifier might use and that a strict subset is equally effective, that some useful features are very hard to describe to a user so cannot be used in an interactive setting, and that a person is much more adept than an automatic system at identifying important features and their relative importance for this task.

We extended the ideas of this proposal to the domain of poetry, where classic classification algorithms almost uniformly fail because of atypical structure, punctuation, and capitalization. We showed that our approach can be used to find poems in a large collection of scanned books.

In the early stages of this work, we felt that modern neural approaches could not be applied because of a paucity of training data. Toward the end of the work we developed techniques that generated "weak" (less than 100% correct) training data at a sufficiently large scale that some neural approaches could be trained and evaluated; we found that those techniques successfully addressed this task. We note that that this use of data is routine in the field but had never been successfully used for this problem. We were able to design and successfully use modern "deep learning" approaches to address the task. We were also able to employ reinforcement learning approaches, where a final success/failure reward function is sufficient to determine the parameters of a complex algorithm.

This grant supported the training of five PhD students and indirectly helped train one postdoctoral fellow. It resulted in 14 publications, contributed directly to one completed PhD thesis as well as a handful still being written.

Last Modified: 10/28/2020
Modified by: James M Allan