This project developed and implemented a research system, called my.arXiv, to improve scientific collaboration. We worked with the arXiv resource at Cornell University to develop and debug the system.  In use, the system shows scientific papers matching a search and suggests additional scientific preprints (items that have not yet gone through the lengthy publication process) so that researchers can quickly find the "missing link" for their own research. We worked with outstanding computer scientists, who developed a number of ways to link one item to another. In addition to the key words and subject headings of those items, my.arXiv can use the text of the abstract itself, to find related materials.

Researchers at Cornell examined the possibility that our system may learn more about what a user is looking for by suggesting some items that do not seem to be the best match. These items, instead, are chosen because they help the system to resolve some ambiguity about what the researcher is looking for, in other words, to explore what is sought, rather than simply exploiting what is already known. Furthermore If a researcher was never shown any specific item, there is no record of the fact that she would have found it useful. Other Cornell research developed ways to improve recommendations, and to correct for this "items never seen" problem, and scale up to very large collections.

The research at Princeton and then Columbia examined many innovative ways to identify new or emerging subjects, which may not even have standard names yet, much as bio-chemistry was born nearly 100 years ago at the interface of biology and chemistry.  For an individual user, these methods can adapt and track changing interests, and profile those interests in a meaningful way.

The Rutgers research also explored new ways to combine several possible recommendations, and to learn from the users active responses to those recommendations. As shown in an example window preprints are shown together with an estimate of their relevance (the length of the red bar).  The users may click the active title link to learn more about the item. Or indicate that, whatever its value, it has already been seen; or to advise us that it is not what the user needs. Finally, a mouse-drag can move any an item up or down the screen. This tells the system to consider it more (or less) useful than any item that it moves past.

The system has been developed with an extensive "back room" that can be used to design and conduct experiments. These experiments will lead to new knowledge about which algorithms best match the real thought processes of human researchers. For long term study, the My.arXiv system supports user accounts. A researcher with an account can choose to have her preferences and feedback saved from one session to the next, as the system learns more and more about how to help.

In addition to the search for better ways to make recommendations, this project looked into growing concerns about privacy. A researcher on the trail of a scientific breakthrough wants to be sure that using a collaborative resource will not make it easy for other researchers to steal her ideas. Extensive experiments using years of data from the arXiv history have shown a kind of "bad news / good news" result. The bad news is that an adversary might be able to figure out what a researcher  has been examining or recommending; all he must do is submit the very same search query, just before and just after the researcher does. The good news is that for a system as large arXiv, with over one million items, and more than half a million users per week, the chance of leaking a research interest to this kind of attack is almost astronomically small.

Last Modified: 02/26/2019
Modified by: Rebecca N Wright