@article {69997, title = {A connectomic resource for neural cataloguing and circuit dissection of the larval zebrafish brain}, journal = {bioRxiv}, year = {2025}, month = {2025 Jun 16}, abstract = {

We present a correlated light and electron microscopy (CLEM) dataset from a 7-day-old larval zebrafish, integrating confocal imaging of genetically labeled excitatory (vglut2a) and inhibitory (gad1b) neurons with nanometer-resolution serial section EM. The dataset spans the brain and anterior spinal cord, capturing \>180,000 segmented soma, \>40,000 molecularly annotated neurons, and 30 million synapses, most of which were classified as excitatory, inhibitory, or modulatory. To characterize the directional flow of activity across the brain, we leverage the synaptic and cell body annotations to compute region-wise input and output drive indices at single cell resolution. We illustrate the dataset{\textquoteright}s utility by dissecting and validating circuits in three distinct systems: water flow direction encoding in the lateral line, recurrent excitation and contralateral inhibition in a hindbrain motion integrator, and functionally relevant targeted long-range projections from a tegmental excitatory nucleus, demonstrating that this resource enables rigorous hypothesis testing as well as exploratory-driven circuit analysis. The dataset is integrated into an open-access platform optimized to facilitate community reconstruction and discovery efforts throughout the larval zebrafish brain.

Preprint: https://www.biorxiv.org/content/early/2025/06/15/2025.06.10.658982

}, doi = {10.1101/2025.06.10.658982}, url = {https://www.biorxiv.org/content/early/2025/06/15/2025.06.10.658982}, author = {Petkova, Mariela D. and Januszewski, Micha{\l} and Blakely, Tim and Herrera, Kristian J. and Schuhknecht, Gregor F.P. and Tiller, Robert and Choi, Jinhan and Schalek, Richard L. and Boulanger-Weill, Jonathan and Peleg, Adi and Wu, Yuelong and Wang, Shuohong and Troidl, Jakob and Kumar Vohra, Sumit and Wei, Donglai and Lin, Zudi and Bahl, Armin and Tapia, Juan Carlos and Iyer, Nirmala and Miller, Zachary T. and Hebert, Kathryn B. and Pavarino, Elisa C. and Taylor, Milo and Deng, Zixuan and Stingl, Moritz and Hockling, Dana and Hebling, Alina and Wang, Ruohong C. and Zhang, Lauren L. and Dvorak, Sam and Faik, Zainab and King, Kareem I. and Goel, Pallavi and Wagner-Carena, Julian and Aley, David and Chalyshkan, Selimzhan and Contreas, Dominick and Li, Xiong and Muthukumar, Akila V. and Vernaglia, Marina S. and Carrasco, Teodoro Tapia and Melnychuck, Sofia and Yan, TingTing and Dalal, Ananya and DiMartino, James M. and Brown, Sam and Safo-Mensa, Nana and Greenberg, Ethan and Cook, Michael and Finley-May, Samantha and Flynn, Miriam A. and Hopkins, Gary Patrick and Kovalyak, Julie and Leonard, Meghan and Lohff, Alanna and Ordish, Christopher and Scott, Ashley L. and Takemura, Satoko and Walsh, Claire and Walsh, John J. and Berger, Daniel R. and Pfister, Hanspeter and Berg, Stuart and Knecht, Christopher and Meissner, Geoffrey W. and Korff, Wyatt and Ahrens, Misha B. and Jain, Viren and Lichtman, Jeff W. and Engert, Florian} } @article {68222, title = {Structured sampling of olfactory input by the fly mushroom body.}, journal = {Current Biology}, volume = {32}, year = {2022}, month = {2022 Aug 08}, pages = {3334-3349.e6}, abstract = {

Associative memory formation and recall in the fruit fly Drosophila melanogaster is subserved by the mushroom body (MB). Upon arrival in the MB, sensory information undergoes a profound transformation from broadly tuned and stereotyped odorant responses in the olfactory projection neuron (PN) layer to narrowly tuned and nonstereotyped responses in the Kenyon cells (KCs). Theory and experiment suggest that this transformation is implemented by random connectivity between KCs and PNs. However, this hypothesis has been challenging to test, given the difficulty of mapping synaptic connections between large numbers of brain-spanning neurons. Here, we used a recent whole-brain electron microscopy volume of the adult fruit fly to map PN-to-KC connectivity at synaptic resolution. The PN-KC connectome revealed unexpected structure, with preponderantly food-responsive PN types converging at above-chance levels on downstream KCs. Axons of the overconvergent PN types tended to arborize near one another in the MB main calyx, making local KC dendrites more likely to receive input from those types. Overconvergent PN types preferentially co-arborize and connect with dendrites of αβ and α{\textquoteright}β{\textquoteright} KC subtypes. Computational simulation of the observed network showed degraded discrimination performance compared with a random network, except when all signal flowed through the overconvergent, primarily food-responsive PN types. Additional theory and experiment will be needed to fully characterize the impact of the observed non-random network structure on associative memory formation and recall.

}, issn = {1879-0445}, doi = {10.1016/j.cub.2022.06.031}, author = {Zheng, Zhihao and Li, Feng and Fisher, Corey and Ali, Iqbal J and Sharifi, Nadiya and Calle-Schuler, Steven and Hsu, Joseph and Masoodpanah, Najla and Kmecova, Lucia and Kazimiers, Tom and Perlman, Eric and Nichols, Matthew and Li, Peter H and Jain, Viren and Bock, Davi D} } @article {68507, title = {A connectome and analysis of the adult Drosophila central brain.}, journal = {Elife}, volume = {9}, year = {2020}, month = {2020 Sep 07}, abstract = {

The neural circuits responsible for animal behavior remain largely unknown. We summarize new methods and present the circuitry of a large fraction of the brain of the fruit fly . Improved methods include new procedures to prepare, image, align, segment, find synapses in, and proofread such large data sets. We define cell types, refine computational compartments, and provide an exhaustive atlas of cell examples and types, many of them novel. We provide detailed circuits consisting of neurons and their chemical synapses for most of the central brain. We make the data public and simplify access, reducing the effort needed to answer circuit questions, and provide procedures linking the neurons defined by our analysis with genetic reagents. Biologically, we examine distributions of connection strengths, neural motifs on different scales, electrical consequences of compartmentalization, and evidence that maximizing packing density is an important criterion in the evolution of the fly{\textquoteright}s brain.

}, keywords = {Animals, Brain, Connectome, Drosophila melanogaster, Female, Male, Neurons, Synapses}, issn = {2050-084X}, doi = {10.7554/eLife.57443}, author = {Scheffer, Louis K and Xu, C Shan and Januszewski, Michal and Lu, Zhiyuan and Takemura, Shin-ya and Hayworth, Kenneth J and Huang, Gary B and Shinomiya, Kazunori and Maitlin-Shepard, Jeremy and Berg, Stuart and Clements, Jody and Hubbard, Philip M and Katz, William T and Umayam, Lowell and Zhao, Ting and Ackerman, David and Blakely, Tim and Bogovic, John and Dolafi, Tom and Kainmueller, Dagmar and Kawase, Takashi and Khairy, Khaled A and Leavitt, Laramie and Li, Peter H and Lindsey, Larry and Neubarth, Nicole and Olbris, Donald J and Otsuna, Hideo and Trautman, Eric T and Ito, Masayoshi and Bates, Alexander S and Goldammer, Jens and Wolff, Tanya and Svirskas, Robert and Schlegel, Philipp and Neace, Erika and Knecht, Christopher J and Alvarado, Chelsea X and Bailey, Dennis A and Ballinger, Samantha and Borycz, Jolanta A and Canino, Brandon S and Cheatham, Natasha and Cook, Michael and Dreher, Marisa and Duclos, Octave and Eubanks, Bryon and Fairbanks, Kelli and Finley, Samantha and Forknall, Nora and Francis, Audrey and Hopkins, Gary Patrick and Joyce, Emily M and Kim, SungJin and Kirk, Nicole A and Kovalyak, Julie and Lauchie, Shirley A and Lohff, Alanna and Maldonado, Charli and Manley, Emily A and McLin, Sari and Mooney, Caroline and Ndama, Miatta and Ogundeyi, Omotara and Okeoma, Nneoma and Ordish, Christopher and Padilla, Nicholas and Patrick, Christopher M and Paterson, Tyler and Phillips, Elliott E and Phillips, Emily M and Rampally, Neha and Ribeiro, Caitlin and Robertson, Madelaine K and Rymer, Jon Thomson and Ryan, Sean M and Sammons, Megan and Scott, Anne K and Scott, Ashley L and Shinomiya, Aya and Smith, Claire and Smith, Kelsey and Smith, Natalie L and Sobeski, Margaret A and Suleiman, Alia and Swift, Jackie and Takemura, Satoko and Talebi, Iris and Tarnogorska, Dorota and Tenshaw, Emily and Tokhi, Temour and Walsh, John J and Yang, Tansy and Horne, Jane Anne and Li, Feng and Parekh, Ruchi and Rivlin, Patricia K and Jayaraman, Vivek and Costa, Marta and Jefferis, Gregory Sxe and Ito, Kei and Saalfeld, Stephan and George, Reed and Meinertzhagen, Ian A and Rubin, Gerald M and Hess, Harald F and Jain, Viren and Plaza, Stephen M} } @article {65738, title = {The mind of a mouse.}, journal = {Cell}, volume = {182}, year = {2020}, month = {2020 Sep 17}, pages = {1372-1376}, abstract = {

Large scientific projects in genomics and astronomy are influential not because they answer any single question but because they enable investigation of continuously arising new questions from the same data-rich sources. Advances in automated mapping of the brain{\textquoteright}s synaptic connections (connectomics) suggest that the complicated circuits underlying brain function are ripe for analysis. We discuss benefits of mapping a mouse brain at the level of synapses.

}, issn = {1097-4172}, doi = {10.1016/j.cell.2020.08.010}, author = {Abbott, Larry F and Bock, Davi D and Callaway, Edward M and Denk, Winfried and Dulac, Catherine and Fairhall, Adrienne L and Fiete, Ila and Harris, Kristen M and Helmstaedter, Moritz and Jain, Viren and Kasthuri, Narayanan and LeCun, Yann and Lichtman, Jeff W and Littlewood, Peter B and Luo, Liqun and Maunsell, John H R and Reid, R Clay and Rosen, Bruce R and Rubin, Gerald M and Sejnowski, Terrence J and Seung, H Sebastian and Svoboda, Karel and Tank, David W and Tsao, Doris and Van Essen, David C} } @article {6261, title = {Connectomic reconstruction of the inner plexiform layer in the mouse retina.}, journal = {Nature}, volume = {500}, year = {2013}, month = {2013 Aug 8}, pages = {168-74}, abstract = {

Comprehensive high-resolution structural maps are central to functional exploration and understanding in biology. For the nervous system, in which high resolution and large spatial extent are both needed, such maps are scarce as they challenge data acquisition and analysis capabilities. Here we present for the mouse inner plexiform layer{\textendash}the main computational neuropil region in the mammalian retina{\textendash}the dense reconstruction of 950 neurons and their mutual contacts. This was achieved by applying a combination of crowd-sourced manual annotation and machine-learning-based volume segmentation to serial block-face electron microscopy data. We characterize a new type of retinal bipolar interneuron and show that we can subdivide a known type based on connectivity. Circuit motifs that emerge from our data indicate a functional mechanism for a known cellular response in a ganglion cell that detects localized motion, and predict that another ganglion cell is motion sensitive.

}, keywords = {Amacrine Cells, Animals, Biological, Cell Communication, Computer-Assisted, Connectome, Electron, Image Processing, Inbred C57BL, Mice, Microscopy, Models, Neuropil, Retina, Retinal Ganglion Cells}, issn = {1476-4687}, doi = {10.1038/nature12346}, author = {Helmstaedter, Moritz and Briggman, Kevin L and Turaga, Srinivas C and Jain, Viren and Seung, H Sebastian and Denk, Winfried} } @article {3846, title = {Learning to agglomerate superpixel hierarchies.}, journal = {Neural Information Processing Systems}, volume = {24}, year = {2011}, month = {2011}, pages = {648-56}, abstract = {

An agglomerative clustering algorithm merges the most similar pair of clusters at every iteration. The function that evaluates similarity is traditionally handdesigned, but there has been recent interest in supervised or semisupervised settings in which ground-truth clustered data is available for training. Here we show how to train a similarity function by regarding it as the action-value function of a reinforcement learning problem. We apply this general method to segment images by clustering superpixels, an application that we call Learning to Agglomerate Superpixel Hierarchies (LASH). When applied to a challenging dataset of brain images from serial electron microscopy, LASH dramatically improved segmentation accuracy when clustering supervoxels generated by state of the boundary detection algorithms. The naive strategy of directly training only supervoxel similarities and applying single linkage clustering produced less improvement.

}, author = {Jain, Viren and Turaga, Srinivas and Briggman, Kevin and Helmstaedter, Moritz N. and Denk, Winfried and Seung, Sebastian} } @conference {3858, title = {Boundary learning by optimization with topological constraints.}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition}, year = {2010}, month = {2010}, abstract = {

Recent studies have shown that machine learning can improve the accuracy of detecting object boundaries in images. In the standard approach, a boundary detector is trained by minimizing its pixel-level disagreement with human boundary tracings. This naive metric is problematic because it is overly sensitive to boundary locations. This problem is solved by metrics provided with the Berkeley Segmentation Dataset, but these can be insensitive to topological differences, such as gaps in boundaries. Furthermore, the Berkeley metrics have not been useful as cost functions for supervised learning. Using concepts from digital topology, we propose a new metric called the warping error that tolerates disagreements over boundary location, penalizes topological disagreements, and can be used directly as a cost function for learning boundary detection, in a method that we call Boundary Learning by Optimization with Topological Constraints (BLOTC). We trained boundary detectors on electron microscopic images of neurons, using both BLOTC and standard training. BLOTC produced substantially better performance on a 1.2 million pixel test set, as measured by both the warping error and the Rand index evaluated on segmentations generated from the boundary labelings. We also find our approach yields significantly better segmentation performance than either gPb-OWT-UCM or multiscale normalized cut, as well as Boosted Edge Learning trained directly on our data.

}, author = {Jain, Viren and Bollmann, Benjamin and Richardson, Mark and Berger, Daniel R. and Helmstaedter, Moritz N. and Briggman, Kevin L and Denk, Winfried and Bowden, Jared B. and Mendenhall, John M. and Abraham, Wickliffe C. and Harris, Kristen M and Kasthuri, Narayanan and Hayworth, Ken J. and Schalek, Richard and Tapia, Juan Carlos and Lichtman, Jeff W. and Seung, Hyunjune Sebastian} } @article {3844, title = {Convolutional networks can learn to generate affinity graphs for image segmentation.}, journal = {Neural Computation}, volume = {22}, year = {2010}, month = {2010 Feb}, pages = {511-38}, abstract = {

Many image segmentation algorithms first generate an affinity graph and then partition it. We present a machine learning approach to computing an affinity graph using a convolutional network (CN) trained using ground truth provided by human experts. The CN affinity graph can be paired with any standard partitioning algorithm and improves segmentation accuracy significantly compared to standard hand-designed affinity functions. We apply our algorithm to the challenging 3D segmentation problem of reconstructing neuronal processes from volumetric electron microscopy (EM) and show that we are able to learn a good affinity graph directly from the raw EM images. Further, we show that our affinity graph improves the segmentation accuracy of both simple and sophisticated graph partitioning algorithms. In contrast to previous work, we do not rely on prior knowledge in the form of hand-designed image features or image preprocessing. Thus, we expect our algorithm to generalize effectively to arbitrary image types.

}, keywords = {Algorithms, Artificial Intelligence, Automated, Computer-Assisted, Electron, Image Processing, Mathematical Computing, Mathematical Concepts, Mathematics, Microscopy, Neural Networks (Computer), Pattern Recognition}, issn = {1530-888X}, doi = {10.1162/neco.2009.10-08-881}, author = {Turaga, Srinivas C and Murray, Joseph F and Jain, Viren and Roth, Fabian and Helmstaedter, Moritz and Briggman, Kevin and Denk, Winfried and Seung, Hyunjune Sebastian} } @article {3843, title = {Machines that learn to segment images: a crucial technology for connectomics.}, journal = {Current Opinion in Neurobiology}, volume = {20}, year = {2010}, month = {2010 Oct}, pages = {653-66}, abstract = {

Connections between neurons can be found by checking whether synapses exist at points of contact, which in turn are determined by neural shapes. Finding these shapes is a special case of image segmentation, which is laborious for humans and would ideally be performed by computers. New metrics properly quantify the performance of a computer algorithm using its disagreement with {\textquoteright}true{\textquoteright} segmentations of example images. New machine learning methods search for segmentation algorithms that minimize such metrics. These advances have reduced computer errors dramatically. It should now be faster for a human to correct the remaining errors than to segment an image manually. Further reductions in human effort are expected, and crucial for finding connectomes more complex than that of Caenorhabditis elegans.

}, keywords = {Algorithms, Animals, Artificial Intelligence, Computer-Assisted, Electron, Humans, Image Processing, Microscopy, Nanotechnology, Neurobiology}, issn = {1873-6882}, doi = {10.1016/j.conb.2010.07.004}, author = {Jain, Viren and Seung, Hyunjune Sebastian and Turaga, Srinivas C} } @mastersthesis {3859, title = {Machine learning of image analysis with convolutional networks and topological constraints.}, year = {2009}, month = {2009}, school = {Massachusetts Institute of Technology}, type = {masters}, abstract = {

We present an approach to solving computer vision problems in which the goal is to produce a high-dimensional, pixel-based interpretation of some aspect of the underlying structure of an image. Such tasks have traditionally been categorized as {\textquotedblleft}low-level vision{\textquotedblright} problems, and examples include image denoising, boundary detection, and motion estimation. Our approach is characterized by two main elements, both of which represent a departure from previous work. The first is a focus on convolutional networks, a machine learning strategy that operates directly on an input image with no use of hand-designed features and employs many thousands of free parameters that are learned from data. Previous work in low-level vision has been largely focused on completely handdesigned algorithms or learning methods with a hand-designed feature space. We demonstrate that a learning approach with high model complexity, but zero prior knowledge about any specific image domain, can outperform existing techniques even in the challenging area of natural image processing. We also present results that establish how convolutional networks are closely related to Markov random fields (MRFs), a popular probabilistic approach to image analysis, but can in practice can achieve significantly greater model complexity. The second aspect of our approach is the use of domain specific cost functions and learning algorithms that reflect the structured nature of certain prediction problems in image analysis. In particular, we show how concepts from digital topology can be used in the context of boundary detection to both evaluate and optimize the high-order property of topological accuracy. We demonstrate that these techniques can significantly improve the machine learning approach and outperform state of the art boundary detection and segmentation methods. Throughout our work we maintain a special interest and focus on application of our methods to connectomics, an emerging scientific discipline that seeks highthroughput methods for recovering neural connectivity data from brains. This application requires solving low-level image analysis problems on a tera-voxel or peta-voxel scale, and therefore represents an extremely challenging and exciting arena for the development of computer vision methods.

}, author = {Jain, Viren} } @proceedings {4527, title = {Natural image denoising with convolutional networks.}, year = {2008}, month = {2008 }, address = {Vancouver, B.C., Canada}, author = {Jain, Viren and Seung, H Sebastian} } @proceedings {4528, title = {Supervised learning of image restoration with convolutional networks.}, volume = {2}, year = {2007}, month = {2007 Oct 14}, pages = {1-8}, publisher = {IEEE}, edition = {ICCV}, author = {Jain, Viren and Murray, Joseph F and Roth, Fabian and Turaga, Srinivas and Zhigulin, Valentin and Briggman, Kevin L and Helmstaedter, Moritz N. and Denk, Winfried and Seung, H Sebastian} } @proceedings {4532, title = {Representing part-whole relationships in recurrent neural networks.}, volume = {18}, year = {2005}, month = {2005}, pages = {563-70}, address = {Whistler, B.C.}, abstract = {

There is little consensus about the computational function of top-down synaptic connections in the visual system. Here we explore the hypothesis that top-down connections, like bottom-up connections, reflect partwhole relationships. We analyze a recurrent network with bidirectional synaptic interactions between a layer of neurons representing parts and a layer of neurons representing wholes. Within each layer, there is lateral inhibition. When the network detects a whole, it can rigorously enforce part-whole relationships by ignoring parts that do not belong. The network can complete the whole by filling in missing parts. The network can refuse to recognize a whole, if the activated parts do not conform to a stored part-whole relationship. Parameter regimes in which these behaviors happen are identified using the theory of permitted and forbidden sets [3, 4]. The network behaviors are illustrated by recreating Rumelhart and McClelland{\textquoteright}s {\textquotedblleft}interactive activation{\textquotedblright} model [7].

}, author = {Jain, Viren and Zhigulin, Valentin and Seung, Hyunjune Sebastian} } @proceedings {4533, title = {Exploratory analysis and visualization of speech and music by locally linear embedding.}, volume = {3}, year = {2004}, month = {2004 May 17}, pages = {iii - 984-7}, address = {Montreal, Quebec, Canada}, abstract = {

Many problems in voice recognition and audio processing involve feature extraction from raw waveforms. The goal of feature extraction is to reduce the dimensionality of the audio signal while preserving the informative signatures that, for example, distinguish different phonemes in speech or identify particular instruments in music. If the acoustic variability of a data set is described by a small number of continuous features, then we can imagine the data as lying on a low dimensional manifold in the high dimensional space of all possible waveforms. Locally linear embedding (LLE) is an unsupervised learning algorithm for feature extraction in this setting. In this paper, we present results from the exploratory analysis and visualization of speech and music by LLE.

}, author = {Jain, Viren and Saul, Lawrence K.} } @proceedings {4534, title = {A smorgasbord of features for statistical machine translation.}, year = {2004}, month = {2004 May}, abstract = {

We describe a methodology for rapid experimentation in statistical machine translation which we use to add a large number of features to a baseline system exploiting features from a wide range of levels of syntactic representation. Feature values were combined in a log-linear model to select the highest scoring candidate translation from an n-best list. Feature weights were optimized directly against the BLEU evaluation metric on held-out data. We present results for a small selection of features at each level of syntactic representation.

}, author = {Och, Franz J. and Gildea, Daniel and Khudanpur, Sanjeev and Sarkar, Anoop and Yamada, Kenji and Fraser, Alex and Kumar, Shankar and Shen, Libin and Smith, David and Eng, Katherine and Jain, Viren and Jin, Zhen and Radev, Dragomir} }