Award Abstract # 2221733
I-Corps: Historical Photo Identification with Crowdsourcing and Automated Face Recognition

NSF Org: TI
Translational Impacts
Recipient: VIRGINIA POLYTECHNIC INSTITUTE & STATE UNIVERSITY
Initial Amendment Date: April 6, 2022
Latest Amendment Date: April 6, 2022
Award Number: 2221733
Award Instrument: Standard Grant
Program Manager: Ruth Shuman
rshuman@nsf.gov
 (703)292-2160
TI
 Translational Impacts
TIP
 Directorate for Technology, Innovation, and Partnerships
Start Date: April 1, 2022
End Date: March 31, 2023 (Estimated)
Total Intended Award Amount: $49,309.00
Total Awarded Amount to Date: $49,309.00
Funds Obligated to Date: FY 2022 = $49,309.00
History of Investigator:
  • Kurt Luther (Principal Investigator)
    kluther@vt.edu
Recipient Sponsored Research Office: Virginia Polytechnic Institute and State University
300 TURNER ST NW
BLACKSBURG
VA  US  24060-3359
(540)231-5281
Sponsor Congressional District: 09
Primary Place of Performance: Virginia Polytechnic Institute and State University
900 N. Glebe Road
BLACKSBURG
VA  US  22203-1822
Primary Place of Performance
Congressional District:
08
Unique Entity Identifier (UEI): QDE5UHE5XD16
Parent UEI: X6KEFGLHSJX7
NSF Program(s): I-Corps
Primary Program Source: 01002223DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7453
Program Element Code(s): 802300
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.041, 47.084

ABSTRACT

The broader impact/commercial potential of this I-Corps project is the development of a technology to generate significant cultural and economic value, including recognizing the contributions of historically marginalized groups. While we initially focus on the user groups of auction houses, appraisers, and dealers, this work can also apply to the vast volume of unidentified photos in the collections of galleries, libraries, archives, and museums (GLAMs) and genealogical societies, which are generally short-staffed and largely rely on donors and outside researchers to identify photos. Further, the identification workflow can also be extended beyond the American Civil War era to other historical time periods. Finally, this work fosters new, interdisciplinary connections between technology, art, and history through a commercial platform made available to both scholars and the general public.

This I-Corps project is based on the development of technology identifying unknown people in historical photographs. This is a challenging task performed by a broad range of researchers, including journalists, historians, curators, genealogists, archivists, dealers, and collectors. Currently, these researchers largely rely on manual investigative methods such as paging through hundreds of pages of reference books looking for a potential match. AI-based facial recognition algorithms and crowdsourcing offer promise for supporting this task, but key shortcomings, such as false positives, bias, and groupthink, must be overcome. This work explores novel human-AI collaboration techniques that effectively and ethically combine the complementary strengths of human and artificial intelligence to support historical person identification.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

This grant enabled our project team to successfully participate in and complete the National I-Corps program. We conducted over 100 interviews with potential customers across a range of industries both virtually and in person. We developed and refined a business model in response to this feedback and won the People's Choice Award for our team's final presentation. We found that our technology has the potential to enable individuals and organizations to identify, interpret, and value the historical photographs in their collections. We are now exploring next steps and organizational structures for transitioning and commercializing the technology.

The grant supported additional R&D prototyping work, resulting in BackTrace, a new software feature for grouping together photos with similar backgrounds inspired by customer feedback. We conducted an evaluation of BackTrace with real users and found that it helped them find and organize relevant photos. This work was shared with both the academic computer science community and the Civil War photography community. It was accepted for publication as a full paper and a demonstration at the upcoming AAAI HCOMP 2023 conference, and published as an article in the Summer 2023 issue of Military Images Magazine. This work also contributed to two computer science master's theses and one computer science PhD dissertation, all completed in 2023.


Last Modified: 09/05/2023
Modified by: Kurt Luther

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page