| NSF Org: |
IIS Division of Information & Intelligent Systems |
| Recipient: |
|
| Initial Amendment Date: | August 30, 2012 |
| Latest Amendment Date: | August 30, 2012 |
| Award Number: | 1149299 |
| Award Instrument: | Standard Grant |
| Program Manager: |
Kenneth Whang
kwhang@nsf.gov (703)292-5149 IIS Division of Information & Intelligent Systems CSE Directorate for Computer and Information Science and Engineering |
| Start Date: | September 1, 2012 |
| End Date: | August 31, 2017 (Estimated) |
| Total Intended Award Amount: | $409,406.00 |
| Total Awarded Amount to Date: | $409,406.00 |
| Funds Obligated to Date: |
|
| History of Investigator: |
|
| Recipient Sponsored Research Office: |
201 PRESIDENTS CIR SALT LAKE CITY UT US 84112-9049 (801)581-6903 |
| Sponsor Congressional District: |
|
| Primary Place of Performance: |
72 South Central Campus Dr Salt Lake City UT US 84112-9200 |
| Primary Place of
Performance Congressional District: |
|
| Unique Entity Identifier (UEI): |
|
| Parent UEI: |
|
| NSF Program(s): |
Robust Intelligence, EPSCoR Co-Funding |
| Primary Program Source: |
|
| Program Reference Code(s): |
|
| Program Element Code(s): |
|
| Award Agency Code: | 4900 |
| Fund Agency Code: | 4900 |
| Assistance Listing Number(s): | 47.070 |
ABSTRACT
The research objective of this proposal is to create novel computational algorithms and image processing tools that will make it possible for biologists to reconstruct large-scale neural circuits from electron microscopy volumes. Electron microscopy is a key technology in reconstruction of neural circuits at the level of individual neurons and synapses, also known as connectomics. While an important motivation of connectomics is providing anatomical ground truth for neural circuit models, the ability to decipher neural wiring maps at the individual cell level is also important in studies of many neurodegenerative diseases. State-of-the-art image analysis solutions are still far from the accuracy and robustness of human vision and biologists are still limited to studying small neural circuits using mostly manual analysis. The proposed computational models will provide biologists a tool for segmenting individual neurons and detecting other structures such as synapses in very large electron microscopy volumes, and proof reading these automatically produced results in a time efficient manner.
Reconstruction of a neural circuit from an electron microscopy volume involves pixel-by-pixel annotation of these images into classes such as cell membrane, mitochondria and synaptic vesicles and the segmentation of individual neurons in three dimensions. This task demands extremely high accuracy. Even with 99% pixel accuracy, an acceptable accuracy for many other applications, it is virtually certain that almost every neuron in a volume will be incorrectly segmented due to their global, tree-like structure and correspondingly large surface area. Therefore, lack of reliable automated solutions is a critical bottleneck in the field of connectomics. In this project, a novel hierarchical model will be created by combining the representation power of sparse dictionaries and their ease of learning with an inference and proof reading capability. Human experts will contribute to the process by providing ground truth for supervised learning and proof reading of automatically produced results. The combination of deep sparse dictionaries with inference using connection weights from conditional probabilities can provide a very fast way to learn hierarchical models. Several variants of the model will be studied for understanding the relative importance of feature representation, inference, symmetric connections, deep and lateral connections. The model will be applied to general object classification and image parsing problems in computer vision as well as connectomics datasets. Success will be evaluated on real datasets annotated by experts.
PUBLICATIONS PRODUCED AS A RESULT OF THIS RESEARCH
Note:
When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external
site maintained by the publisher. Some full text articles may not yet be available without a
charge during the embargo (administrative interval).
Some links on this page may take you to non-federal websites. Their policies may differ from
this site.
PROJECT OUTCOMES REPORT
Disclaimer
This Project Outcomes Report for the General Public is displayed verbatim as submitted by the Principal Investigator (PI) for this award. Any opinions, findings, and conclusions or recommendations expressed in this Report are those of the PI and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
Connectomics at the microscopic level is the complete mapping of all individual neurons and their synaptic contacts in a region of the neuropil to create its canonical network map. Electron microscopy has sufficient detail for the identification of individual neurons and their synaptic connections required for this task. While electron microscopy images provide rich datasets for connectomics, they also pose formidable challenges to due to their size and complexity. Electron microscopy image datasets for connectomics can reach hundreds of terabytes. Manual analysis of even the smallest connectomics datasets costs years of effort by experts. Therefore, automation of the electron microscopy pipeline is imperative. On the other hand, the images are complex with densely packed neurons that have similar visual appearance. This creates significant challenges for automatic analysis. This project addressed the image analysis and machine learning challenges associated with connectomics. Specifically, we addressed the following challenges in this project:
1) Volumes must be assembled from tens of thousands of individual EM images. To address this challenge, we developed computationally efficient methods for microscopy image registration.
2) Image analysis and machine learning approaches promise to automate the connectomics pipeline; however, they also must face challenges regarding the complexity of the data. Experts have no problem delineating the boundaries of neurons in these datasets in general; however, teaching a computer to do the same requires state-of-the-art machine learning methods that can reason about ambiguities in the data. To address this challenge we developed the contextual hierarchical model for cell membrane detection and the merge tree algorithm for image segmentation.
3) State-of-the art machine learning methods require large amounts of expert annotated training data that can be costly to generate. This is especially true for image segmentation methods that need training data at a fine-grained resolution. To alleviate this problem, we developed novel approaches to semi-supervised machine learning that can produce accurate results with smaller sets of expert generated training data. Our approaches to semi-supervised learning formulate cost functions using the principles of mutual exclusivity between classes of objects and the stability of the classifier output under perturbations to its input.
4) Machine learning methods such as artificial neural networks can also be costly in terms of computer time. To address this challenge, we developed a novel classifier that we named the logistic disjunctive normal network which can be trained faster than a regular neural network. We further adopted this model for shape representations and segmentation.
5) Even the best machine learning algorithms can not be 100% accurate. To address this challenge, we developed efficient proof reading approaches for experts to correct the results of the automatic analysis algorithms.
The broader outcomes and impacts of this project include:
1) Five Ph.D. students were trained supported in part by funds from this project. These students have graduated and found research and engineering positions in top machine learning research laboratories including Google Research and NVIDIA Corporation.
2) We have created a webpage dedicated to this project which includes brief descriptions of the outcomes as well as the research papers published. We have also made the software for our models publicly available, free of charge, on our website.
3) As part of outreach efforts supported by this grant, we have worked with high school students on image analysis and machine learning based projects. One of these high school students placed first in the Intermountain Junior Science and Humanities Symposium with his project titled “Analysis of Montane Rainforest Dwellings with Computer Vision” which aimed to count the number of houses in the Gishwati Rainforest area and eventually correlate this to the approximate number of people in the area and change in rainforest cover.
4) We have used the research supported by this grant to enrich courses in the University of Utah Electrical and Computer Engineering Department’s curriculum and to develop new courses.
5) The software we have developed is being used at the National Center for Microscopy and Imaging Research in San Diego for analyzing the electron microscopy images they collect.
Last Modified: 11/16/2017
Modified by: Tolga Tasdizen
Please report errors in award information by writing to: awardsearch@nsf.gov.
