Generalist methods for cellular segmentation have good out-of-the-box performance on a variety of image types; however, existing methods struggle for images that are degraded by noise, blurring or undersampling, all of which are common in microscopy. We focused the development of Cellpose3 on addressing these cases and here we demonstrate substantial out-of-the-box gains in segmentation and image quality for noisy, blurry and undersampled images. Unlike previous approaches that train models to restore pixel values, we trained Cellpose3 to output images that are well segmented by a generalist segmentation model, while maintaining perceptual similarity to the target images. Furthermore, we trained the restoration models on a large, varied collection of datasets, thus ensuring good generalization to user images. We provide these tools as 'one-click' buttons inside the graphical interface of Cellpose as well as in the Cellpose API.

Many biological applications require the segmentation of cell bodies, membranes and nuclei from microscopy images. Deep learning has enabled great progress on this problem, but current methods are specialized for images that have large training datasets. Here we introduce a generalist, deep learning-based segmentation algorithm called Cellpose, which can very precisely segment a wide range of image types out-of-the-box and does not require model retraining or parameter adjustments. We trained Cellpose on a new dataset of highly-varied images of cells, containing over 70,000 segmented objects. To support community contributions to the training data, we developed software for manual labelling and for curation of the automated results, with optional direct upload to our data repository. Periodically retraining the model on the community-contributed data will ensure that Cellpose improves constantly.

Many biological applications require the segmentation of cell bodies, membranes and nuclei from microscopy images. Deep learning has enabled great progress on this problem, but current methods are specialized for images that have large training datasets. Here we introduce a generalist, deep learning-based segmentation method called Cellpose, which can precisely segment cells from a wide range of image types and does not require model retraining or parameter adjustments. Cellpose was trained on a new dataset of highly varied images of cells, containing over 70,000 segmented objects. We also demonstrate a three-dimensional (3D) extension of Cellpose that reuses the two-dimensional (2D) model and does not require 3D-labeled data. To support community contributions to the training data, we developed software for manual labeling and for curation of the automated results. Periodically retraining the model on the community-contributed data will ensure that Cellpose improves constantly.

Electrophysiology has long been the workhorse of neuroscience, allowing scientists to record with millisecond precision the action potentials generated by neurons in vivo. Recently, calcium imaging of fluorescent indicators has emerged as a powerful alternative. This technique has its own strengths and weaknesses and unique data processing problems and interpretation confounds. Here we review the computational methods that convert raw calcium movies to estimates of single neuron spike times with minimal human supervision. By computationally addressing the weaknesses of calcium imaging, these methods hold the promise of significantly improving data quality. We also introduce a new metric to evaluate the output of these processing pipelines, which is based on the cluster isolation distance routinely used in electrophysiology.

Two-photon calcium imaging enables measurements of neural activity from individual neurons and their subcellular processes in awake animals. Axial brain motion can change measured fluorescence independently of neural activity and move small structures out of the imaging plane, confounding two-photon calcium measurements in behaving animals. We developed two complementary post-acquisition methods, selected according to whether the imaged structure remains detectable across the encountered depth range. When it does, z-stack correction estimates imaging depth from a separately acquired reference z-stack and uses an axial fluorescence profile specific to each neural structure to rescale fluorescence to a common reference depth. When a structure can leave a single imaging plane, volume-based z-registration reconstructs a common-depth movie from repeated acquisitions of closely spaced planes. In awake mice, both methods tracked experimentally imposed axial displacements within the sampled range, with depth-estimation root mean squared errors (RMSEs) below 1 μm for z-stack correction and 0.69-1.03 μm for volume-based z-registration. Correction changed fluorescence substantially for a subset of somata, with larger effects in axonal boutons. Both methods are implemented in suite2p-in-depth, a Python package that integrates with Suite2p and provides diagnostics for depth estimation, correction quality and data reliability. Together, these methods enable axial-motion artifacts to be assessed and corrected within established Suite2p workflows.

Limited color channels in fluorescence microscopy have long constrained spatial analysis in biological specimens. Here, we introduce cycle Hybridization Chain Reaction (HCR), a method that integrates multicycle DNA barcoding with HCR to overcome this limitation. cycleHCR enables highly multiplexed imaging of RNA and proteins using a unified barcode system. Whole-embryo transcriptomics imaging achieved precise three-dimensional gene expression and cell fate mapping across a specimen depth of ~310 μm. When combined with expansion microscopy, cycleHCR revealed an intricate network of 10 subcellular structures in mouse embryonic fibroblasts. In mouse hippocampal slices, multiplex RNA and protein imaging uncovered complex gene expression gradients and cell-type-specific nuclear structural variations. cycleHCR provides a quantitative framework for elucidating spatial regulation in deep tissue contexts for research and potentially diagnostic applications.

Preprint: https://doi.org/10.1101/2024.05.17.594641

The brain exhibits rich oscillatory dynamics that vary across tasks and states, such as the EEG oscillations that define sleep. These oscillations play critical roles in cognition and arousal, but the brainwide mechanisms underlying them are not yet described. Using simultaneous EEG and fast fMRI in subjects drifting between sleep and wakefulness, we developed a machine learning approach to investigate which brainwide fMRI dynamics predict alpha (8-12 Hz) and delta (1-4 Hz) rhythms. We predicted moment-by-moment EEG power from fMRI activity in held-out subjects, and found that information about alpha power was represented by a remarkably small set of regions, segregated in two distinct networks linked to arousal and visual systems. Conversely, delta rhythms were diffusely represented on a large spatial scale across the cortex. These results identify distributed networks that predict delta and alpha rhythms, and establish a computational framework for investigating fMRI brainwide dynamics underlying EEG oscillations.

Neural recordings using optical methods have improved dramatically. For example, we demonstrate here recordings of over 100,000 neurons from the mouse cortex obtained with a standard commercial microscope. To process such large datasets, we developed Suite2p, a collection of efficient algorithms for motion correction, cell detection, activity extraction and quality control. We also developed new approaches to benchmark performance on these tasks. Our GPU-accelerated non-rigid motion correction substantially outperforms alternative methods, while running over five times faster. For cell detection, Suite2p outperforms the CNMF algorithm in Caiman and Fiola, finding more cells and producing fewer false positives, while running in a fraction of the time. We also introduce quality control steps for users to evaluate performance on their own data, while offering alternative algorithms for specialized types of recordings such as those from one-photon and voltage imaging.

Recent studies in mice have shown that orofacial behaviors drive a large fraction of neural activity across the brain. To understand the nature and function of these signals, we need better computational models to characterize the behaviors and relate them to neural activity. Here we developed Facemap, a framework consisting of a keypoint tracking algorithm and a deep neural network encoder for predicting neural activity. We used the Facemap keypoints as input for the deep neural network to predict the activity of ∼50,000 simultaneously-recorded neurons and in visual cortex we doubled the amount of explained variance compared to previous methods. Our keypoint tracking algorithm was more accurate than existing pose estimation tools, while the inference speed was several times faster, making it a powerful tool for closed-loop behavioral experiments. The Facemap tracker was easy to adapt to data from new labs, requiring as few as 10 annotated frames for near-optimal performance. We used Facemap to find that the neuronal activity clusters which were highly driven by behaviors were more spatially spread-out across cortex. We also found that the deep keypoint features inferred by the model had time-asymmetrical state dynamics that were not apparent in the raw keypoint data. In summary, Facemap provides a stepping stone towards understanding the function of the brainwide neural signals and their relation to behavior.

fMRI signals were traditionally seen as slow and sampled in the order of seconds, but recent technological advances have enabled much faster sampling rates. We hypothesized that high-frequency fMRI signals can capture spontaneous neural activity that index brain states. Using fast fMRI (TR=378ms) and simultaneous EEG in 27 humans drifting between sleep and wakefulness, we found that fMRI spectral power increased during NREM sleep (compared to wakefulness) across several frequency ranges as fast as 1Hz. This fast fMRI power was correlated with canonical arousal-linked EEG rhythms (alpha and delta), with spatiotemporal correlation patterns for each rhythm reflecting a combination of shared arousal dynamics and rhythm-specific neural signatures. Using machine learning, we found that alpha and delta EEG rhythms can be decoded from fast fMRI signals, in subjects held-out from the training set, showing that fMRI as fast as 0.9Hz (alpha) and 0.7Hz (delta) contains reliable neurally-coupled information that generalizes across individuals. Finally, we demonstrate that this fast fMRI acquisition allows for EEG rhythms to be decoded from 3.8s windows of fMRI data. These results reveal that high-frequency fMRI signals are coupled to dynamically varying brain states, and that fast fMRI sampling allows for more temporally precise quantification of spontaneous neural activity than previously thought possible.