Award Abstract # 1421943
RI: Small: Using Humans in the Loop to Collect High-quality Annotations from Images and Time-lapse Videos of Cells

NSF Org: IIS
Division of Information & Intelligent Systems
Recipient: TRUSTEES OF BOSTON UNIVERSITY
Initial Amendment Date: July 23, 2014
Latest Amendment Date: June 26, 2015
Award Number: 1421943
Award Instrument: Continuing Grant
Program Manager: Jie Yang
jyang@nsf.gov
 (703)292-4768
IIS
 Division of Information & Intelligent Systems
CSE
 Directorate for Computer and Information Science and Engineering
Start Date: August 1, 2014
End Date: October 31, 2018 (Estimated)
Total Intended Award Amount: $370,151.00
Total Awarded Amount to Date: $370,151.00
Funds Obligated to Date: FY 2014 = $200,000.00
FY 2015 = $170,151.00
History of Investigator:
  • Margrit Betke (Principal Investigator)
    betke@cs.bu.edu
Recipient Sponsored Research Office: Trustees of Boston University
1 SILBER WAY
BOSTON
MA  US  02215-1703
(617)353-4365
Sponsor Congressional District: 07
Primary Place of Performance: Dept. of Computer Science, Boston University
111 Cummington Mall
Boston
MA  US  02215-2411
Primary Place of Performance
Congressional District:
07
Unique Entity Identifier (UEI): THL6A6JLE1S7
Parent UEI:
NSF Program(s): ADVANCES IN BIO INFORMATICS,
Information Technology Researc,
Cross-BIO Activities,
Robust Intelligence
Primary Program Source: 01001415DB NSF RESEARCH & RELATED ACTIVIT
01001516DB NSF RESEARCH & RELATED ACTIVIT
Program Reference Code(s): 7495, 7923, 8750
Program Element Code(s): 116500, 164000, 727500, 749500
Award Agency Code: 4900
Fund Agency Code: 4900
Assistance Listing Number(s): 47.070

ABSTRACT

Sequences of microscopy images of live cells are analyzed by cell biologists to understand cellular processes, for example, to prevent cancer or design bio-materials for wound healing. Research progress is slowed or compromised when scientists find the image analysis efforts too labor-intensive to do themselves and the automation methods too numerous, unreliable, or difficult to use. The project develops image-analysis software to leverage human and computer resources together, in particular on the internet, to create high-quality image interpretations. Live-cell imaging studies support basic research to understand cellular processes and design biomaterials. The work on statistically significant performance evaluation can have broad impact on the research methodology in computer vision.

The research explores how human and computer resources can be leveraged together, in particular on the internet, to interpret images and videos of cells. Initially, an expansive benchmark study of detection, segmentation, and tracking algorithms for analyzing images of live cells is conducted. Computer-vision approaches to address the major challenges for existing algorithms are then developed, for example, to interpret the emergence of new cells due to mitosis in time-lapse microscopy videos. Methods are designed for quantifying the quality of automatically and manually obtained annotations and the variability between multiple annotations. A tool is built to effectively and efficiently use the expertise of domain specialists, in particular, cell biologists, to judge and select automated methods that analyze cell images. Crowd-sourcing experiments in which internet workers analyze images are designed and conducted. The quality of these lay workers' annotations is compared to the quality of annotations by domain experts and automated methods. Finally, a machine learning system is developed that automatically determines which types of cell images or videos can be analyzed accurately with or without human involvement.

PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH

Note:  When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

(Showing: 1 - 10 of 12)
Danna Gurari, Kun He, Bo Xiong, Jianming Zhang, Mehrnoosh Sameki, Suyog Dutt Jain, Stan Sclaroff, Margrit Betke, and Kristen Grauman "Predicting Foreground Object Ambiguity and Efficiently Crowdsourcing the Segmentation(s)" International Journal of Computer Vision , v.126 , 2018 , p.714 https://doi.org/10.1007/s11263-018-1065-7
Danna Gurari, Kun He, Bo Xiong, Jianming Zhang, Mehrnoosh Sameki, Suyog Dutt Jain, Stan Sclaroff, Margrit Betke, and Kristen Grauman "Predicting Foreground Object Ambiguity and Efficiently Crowdsourcing the Segmentation(s)." International Journal of Computer Vision , v.126 , 2018 , p.714-730 https://doi.org/10.1007/s11263-018-1065-7
Danna Gurari, Suyog Dutt Jain, Margrit Betke and Kristen Grauman "Pull the Plug? Predicting If Computers or Humans Should Segment Images" 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016 10.1109/CVPR.2016.48
D. Gurari, M. Sameki, and M. Betke "Investigating the Influence of Data Familiarity to Improve the Design of a Crowdsourcing Image Annotation System" AAAI Conference on Human Computation & Crowdsourcing (HCOMP), Austin, TX, November, 2016. , 2016
Gentil, M., M. Sameki, D. Gurari, E. Saraee, E. Hasenberg, J. Y. Wong, and M. Betke "Interactive Tracking of Cells in Microscopy Image Sequences" Interactive Medical Image Computation Workshop (IMIC), held in conjunction with MICCAI 2016, in Athens, Greece, October 21, 2016. 8 pages. , 2016
Gurari, D., M. Sameki, Z. Wu, and M. Betke "Mixing Crowd and Algorithm E?orts toSegment Objects in Biomedical Images" Interactive Medical Image Computation Workshop (IMIC), held in conjunction with MICCAI 2016, in Athens, Greece, October 21, 2016. 8 pages.. , 2016
Jianming Zhang, Shugao Ma, Mehrnoosh Sameki, Stan Sclaroff, Margrit Betke, Zhe Lin, Xiaohui Shen, Brian Price, Radomír M?ch "Salient Object Subitizing" International Journal of Computer Vision , v.124 , 2017 10.1007/s11263-017-1011-0
Rana S.M. Saad, Randa Elanwar, N.S. Abdel Kader, Samia Mashali, and Margrit Betke "ASAR 2018 Layout Analysis Challenge: Using Random Forests to Analyze Scanned Arabic Books" 2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR) , 2018 10.1109/ASAR.2018.8480330
Randa Elanwar and Margrit Betke "The ASAR 2018 Competition on Physical Layout Analysis of Scanned Arabic Books(PLA-SAB 2018)" 2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR) , 2018 10.1109/ASAR.2018.8480194
Randa Elanwar, Wenda Qin, and Margrit Betke "Making scanned Arabic documents machine accessible using an ensemble of SVM classifiers" International Journal on Document Analysis and Recognition , v.21 , 2018 , p.59 https://doi.org/10.1007/s10032-018-0298-x
Randa Elanwar, Wenda Qin, and Margrit Betke "Making scanned Arabic documents machine accessible using an ensemble of SVM classifiers." International Journal on Document Analysis and Recognition , v.21 , 2018 , p.59-79 https://doi.org/10.1007/s10032-018-0298-x
(Showing: 1 - 10 of 12)

PROJECT OUTCOMES REPORT

Disclaimer

This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.

This project addressed image analysis problems that are difficult for a computer to solve automatically and extremely time consuming for an expert to solve manually.  The project developed methods for combining computing resources with expert or non-expert work to interpret single images and image sequences of living cells. Non-experts who work with crowdsourcing platforms on the internet participated in experiments with phase contrast microscopy and fluorescence microscopy images of cells. The project developed several systems.   One system automatically decides, for a batch of images, when to replace humans with computers to create coarse outlines of cells, which are required to initialize automated tools. Another system decides when to replace computers with humans to create final, fine-grained cell outlines. Experiments demonstrated the advantage of relying on a mix of human and computer efforts over relying on either resource alone for annotating objects in various diverse datasets.  The project also showed how the allocations of the number of internet workers to an annotation task can be computed optimally based on task features alone, without using worker profiles.  A machine learning system was trained to predict an optimal number of crowd workers needed to maximize the accuracy of the annotations.  The computed allocation can yield large savings in the crowdsourcing budget (up to 49 percent points) while maintaining labeling accuracy.


Last Modified: 03/06/2019
Modified by: Margrit Betke

Please report errors in award information by writing to: awardsearch@nsf.gov.

Print this page

Back to Top of page