Pixel and superpixel classifiers have become essential tools for EM segmentation algorithms. Training these classifiers remains a major bottleneck primarily due to the requirement of completely annotating the dataset which is tedious, error-prone and costly. In this paper, we propose an interactive learning scheme for the superpixel classifier for EM segmentation. Our algorithm is "active semi-supervised" because it requests the labels of a small number of examples from user and applies label propagation technique to generate these queries. Using only a small set (<20%) of all datapoints, the proposed algorithm consistently generates a classifier almost as accurate as that estimated from a complete groundtruth. We provide segmentation results on multiple datasets to show the strength of these classifiers.

The aim in high-resolution connectomics is to reconstruct complete neuronal connectivity in a tissue. Currently, the only technology capable of resolving the smallest neuronal processes is electron microscopy (EM). Thus, a common approach to network reconstruction is to perform (error-prone) automatic segmentation of EM images, followed by manual proofreading by experts to fix errors. We have developed an algorithm and software library to not only improve the accuracy of the initial automatic segmentation, but also point out the image coordinates where it is likely to have made errors. Our software, called gala (graph-based active learning of agglomeration), improves the state of the art in agglomerative image segmentation. It is implemented in Python and makes extensive use of the scientific Python stack (numpy, scipy, networkx, scikit-learn, scikit-image, and others). We present here the software architecture of the gala library, and discuss several designs that we consider would be generally useful for other segmentation packages. We also discuss the current limitations of the gala library and how we intend to address them.

In the perception of color, wavelengths of light reflected off objects are transformed into the derived quantities of brightness, saturation and hue. Neurons responding selectively to hue have been reported in primate cortex, but it is unknown how their narrow tuning in color space is produced by upstream circuit mechanisms. We report the discovery of neurons in the Drosophila optic lobe with hue-selective properties, which enables circuit-level analysis of color processing. From our analysis of an electron microscopy volume of a whole Drosophila brain, we construct a connectomics-constrained circuit model that accounts for this hue selectivity. Our model predicts that recurrent connections in the circuit are critical for generating hue selectivity. Experiments using genetic manipulations to perturb recurrence in adult flies confirm this prediction. Our findings reveal a circuit basis for hue selectivity in color vision.

In this work, we propose a learning framework for identifying synapses using a deep and wide multi-scale recursive (DAWMR) network, previously considered in image segmentation applications. We apply this approach on electron microscopy data from invertebrate fly brain tissue. By learning features directly from the data, we are able to achieve considerable improvements over existing techniques that rely on a small set of hand-designed features. We show that this system can reduce the amount of manual annotation required, in both acquisition of training data as well as verification of inferred detections.

Information processing relies on precise patterns of synapses between neurons. The cellular recognition mechanisms regulating this specificity are poorly understood. In the medulla of the Drosophila visual system, different neurons form synaptic connections in different layers. Here, we sought to identify candidate cell recognition molecules underlying this specificity. Using RNA sequencing (RNA-seq), we show that neurons with different synaptic specificities express unique combinations of mRNAs encoding hundreds of cell surface and secreted proteins. Using RNA-seq and protein tagging, we demonstrate that 21 paralogs of the Dpr family, a subclass of immunoglobulin (Ig)-domain containing proteins, are expressed in unique combinations in homologous neurons with different layer-specific synaptic connections. Dpr interacting proteins (DIPs), comprising nine paralogs of another subclass of Ig-containing proteins, are expressed in a complementary layer-specific fashion in a subset of synaptic partners. We propose that pairs of Dpr/DIP paralogs contribute to layer-specific patterns of synaptic connectivity.

We propose a method to facilitate exploration and analysis of new large data sets. In particular, we give an unsupervised deep learning approach to learning a latent representation that captures semantic similarity in the data set. The core idea is to use data augmentations that preserve semantic meaning to generate synthetic examples of elements whose feature representations should be close to one another.
We demonstrate the utility of our method applied to nano-scale electron microscopy data, where even relatively small portions of animal brains can require terabytes of image data. Although supervised methods can be used to predict and identify known patterns of interest, the scale of the data makes it difficult to mine and analyze patterns that are not known a priori. We show the ability of our learned representation to enable query by example, so that if a scientist notices an interesting pattern in the data, they can be presented with other locations with matching patterns. We also demonstrate that clustering of data in the learned space correlates with biologically-meaningful distinctions. Finally, we introduce a visualization tool and software ecosystem to facilitate user-friendly interactive analysis and uncover interesting biological patterns. In short, our work opens possible new avenues in understanding of and discovery in large data sets, arising in domains such as EM analysis.

We aim to improve segmentation through the use of machine learning tools during region agglomeration. We propose an active learning approach for performing hierarchical agglomerative segmentation from superpixels. Our method combines multiple features at all scales of the agglomerative process, works for data with an arbitrary number of dimensions, and scales to very large datasets. We advocate the use of variation of information to measure segmentation accuracy, particularly in 3D electron microscopy (EM) images of neural tissue, and using this metric demonstrate an improvement over competing algorithms in EM and natural images.

The ability to automatically segment an image into distinct regions is a critical aspect in many visual processing applications. Because inaccuracies often exist in automatic segmentation, manual segmentation is necessary in some application domains to correct mistakes, such as required in the reconstruction of neuronal processes from microscopic images. The goal of the automated segmentation tool is traditionally to produce the highest-quality segmentation, where quality is measured by the similarity to actual ground truth, so as to minimize the volume of manual correction necessary. Manual correction is generally orders-of-magnitude more time consuming than automated segmentation, often making handling large images intractable. Therefore, we propose a more relevant goal: minimizing the turn-around time of automated/manual segmentation while attaining a level of similarity with ground truth. It is not always necessary to inspect every aspect of an image to generate a useful segmentation. As such, we propose a strategy to guide manual segmentation to the most uncertain parts of segmentation. Our contributions include 1) a probabilistic measure that evaluates segmentation without ground truth and 2) a methodology that leverages these probabilistic measures to significantly reduce manual correction while maintaining segmentation quality.