Recommender systems have become a key tool of everyday life, and we routinely use them for tasks ranging from browsing entertainment options to researching products we may want to purchase. In this project, we developed new methods for training recommender systems based on the feedback the users provided both explicitly and implicitly through their actions. In particular, the project focused on designing recommendation systems for scientific literature, where this particular application provided not only a testbed for the general methods we developed, but also explored the design of the next generation of information systems that will further enable the dissemination of scientific results.

The project was a collaborative effort of researchers at Cornell, Princeton, and Rutgers. Focusing on the results on the Cornell side, the project developed new machine learning algorithms for several aspects of the recommendation problem. The project made many contributions to the design of such learning algorithms, but for conciseness of this report we focus on the following two areas of research.

First, recommendation systems need to strike the right balance between exploiting what they already know about the user, and exploring aspects of the users’ interests that the system is not yet confident about. Making the right trade-offs between exploration and exploitation is important, since too much exploration makes the recommendation system look like it does not know the users’ tastes, and too much exploitation may lead to never discovering all the interests a user may have. Approaching this trade-off between exploration and exploration as a multi-armed bandit problem, we have designed new algorithms and their underlying theory for solving this trade-off optimally under various conditions.

Second, we asked the question of how to reuse data that was collected by the recommendation system in the past. The problem here lies in dealing with the biases that were introduced by the historic version of the recommendation system, as well as the biases that are inherent in how humans provide feedback and make choices. For example, if we want to use the set of papers that the user read while using our historic recommendation system as a feedback signal for learning, then it is important to know what papers the recommender system did recommend and how visible this was to the user. Clearly, a paper that was never discovered by the user cannot make it into the set of paper the user read, even if it was very relevant to the user’s interests. To deal with such biases in a principled and provably correct way, we designed learning methods that explicitly correct for selection biases and that scale to large datasets.

Beyond these research contributions in machine learning, the project developed the my.arxiv.org system as a prototype for the next generation of systems that help researchers discover relevant scientific literature. It allowed us to explore different interfaces and how these interfaces interact with the recommendation algorithms. The lessons learned will be incorporated into the design of the next generation of Arxiv.Org, which is the main repository of scientific papers for a wide range of disciplines in science and engineering.

Last Modified: 05/10/2017
Modified by: Thorsten Joachims