Neural recordings using optical methods have improved dramatically. For example, we demonstrate here recordings of over 100,000 neurons from the mouse cortex obtained with a standard commercial microscope. To process such large datasets, we developed Suite2p, a collection of efficient algorithms for motion correction, cell detection, activity extraction and quality control. We also developed new approaches to benchmark performance on these tasks. Our GPU-accelerated non-rigid motion correction substantially outperforms alternative methods, while running over five times faster. For cell detection, Suite2p outperforms the CNMF algorithm in Caiman and Fiola, finding more cells and producing fewer false positives, while running in a fraction of the time. We also introduce quality control steps for users to evaluate performance on their own data, while offering alternative algorithms for specialized types of recordings such as those from one-photon and voltage imaging.

Understanding how neural signals control muscle activity during behavior is a key challenge in motor neuroscience. To this end, recent advances in intramuscular multielectrode arrays have enabled high-quality multichannel recordings of many motor unit action potentials (MUAPs) in freely moving subjects. However, identifying individual MUAP events within multichannel recordings is a significant challenge for existing spike sorting methods, which are typically optimized for identifying action potentials from neurons in the brain. To overcome this challenge, we developed the Enhanced Motor Unit sorter (EMUsort), an extension of Kilosort4 (KS4) that achieves high-performance MUAP spike sorting. We applied EMUsort to high-resolution intramuscular recordings from rat forelimb during locomotion and monkey forelimb during a reaching task. EMUsort improves upon prior methods by addressing key challenges encountered with MUAP datasets, including: 1) long time delays across electrodes due to propagation along muscle fibers, 2) more complex waveform shapes compared to neuronal action potentials, and 3) a high degree of MUAP overlap due to cumulative motor unit recruitment. We compared EMUsort to existing spike sorting methods quantitatively using simulated datasets that closely emulated the rat and monkey datasets we recorded. EMUsort provided median error rate reductions of 67.5% and 49.9% during periods of high motor unit activation for the rat and monkey datasets, respectively. In sum, EMUsort provides a substantial improvement to MUAP spike sorter accuracy, especially during regions of high MUAP overlap, in an easy-to-use software package.

Online monitoring and quantification of neural signals has tremendous value both for neurofeedback experiments and for brain-computer interfaces. Unfortunately, established methods of online monitoring primarily involve the use of thresholded neural activity rather than sorted single-neuron spikes. The recent introduction of large-scale, high-density electrophysiology has enabled the recording of activity from hundreds of neurons simultaneously in both model organisms and human participants. This development highlights the need for a robust and easily implementable system for sorting spikes during data collection for ‘live’ analyses of neuronal signals. Here, we describe a system for live sorting of neuronal activity (LSS) based on the widely used Kilosort platform. The LSS workflow utilizes an initial period of recorded neural data to identify waveform templates using Kilosort 4. LSS then interfaces with the SpikeGLX API to retrieve small batches (e.g. 50 ms) of data and for processing online. We measured the similarity of single-neuron activity sorted live by LSS to that sorted offline in neurophysiological recordings from macaque visual cortex using Neuropixels probes. We show that LSS closely replicates the post-stimulus time histograms and visual response tuning curves of single-neurons obtained using offline sorting. Furthermore, we show that decoding neural signals online with LSS consistently outperforms online decoding of thresholded activity, and that LSS can achieve the same performance as that obtained with offline sorting.

Predictive coding is a theoretical framework that can explain how animals build internal models of their sensory environments by predicting sensory inputs. Predictive coding may capture either spatial or temporal relationships between sensory objects. While the original theory by Rao and Ballard, 1999 described spatial predictive coding, much of the recent experimental data has been interpreted as evidence for temporal predictive coding. Here we directly tested whether the “mismatch” neural responses in sensory cortex are due to a spatial or a temporal internal model. We adopted two common paradigms to study predictive coding: one based on virtual-reality and one based on static images. After training mice with repeated visual stimulation for several days, we performed multiple manipulations, including: 1) we introduced a novel stimulus, 2) we replaced a stimulus with a novel gray wall, 3) we duplicated a trained stimulus, or 4) we altered the order of the stimuli. The first two manipulations induced a substantial mismatch response in neural populations of up to 20,000 neurons recorded across primary and higher-order visual cortex, while the third and fourth ones did not. Thus, a mismatch response only occurred if a new spatial – not temporal – pattern was introduced.

Artificial neural networks (ANNs) have been shown to predict neural responses in primary visual cortex (V1) better than classical models. However, this performance often comes at the expense of simplicity and interpretability. Here we introduce a new class of simplified ANN models that can predict over 70% of the response variance of V1 neurons. To achieve this high performance, we first recorded a new dataset of over 29,000 neurons responding to up to 65,000 natural image presentations in mouse V1. We found that ANN models required only two convolutional layers for good performance, with a relatively small first layer. We further found that we could make the second layer small without loss of performance, by fitting individual "minimodels" to each neuron. Similar simplifications applied for models of monkey V1 neurons. We show that the minimodels can be used to gain insight into how stimulus invariance arises in biological neurons.

Preprint: https://www.biorxiv.org/content/early/2024/07/02/2024.06.30.601394

High-density silicon probes have transformed neuroscience by enabling large-scale neural recordings at single-cell resolution. However, existing technologies have provided limited functionality in nonhuman primates (NHPs) such as macaques. In the present report, we describe the design, fabrication and performance of Neuropixels 1.0 NHP, a high-channel electrode array designed to enable large-scale acute recording throughout large animal brains. The probe features 4,416 recording sites distributed along a 45-mm shank. Experimenters can programmably select 384 recording channels, enabling simultaneous multi-area recording from thousands of neurons with single or multiple probes. This technology substantially increases scalability and recording access relative to existing technologies and enables new classes of experiments that involve electrophysiological mapping of brain areas at single-neuron and single-spike resolution, measurement of spike-spike correlations between cells and simultaneous brain-wide recordings at scale.

Preprint: https://doi.org/10.1101/2023.02.01.526664 

Representation learning in neural networks may be implemented with supervised or unsupervised algorithms, distinguished by the availability of instruction. In the sensory cortex, perceptual learning drives neural plasticity1-13, but it is not known whether this is due to supervised or unsupervised learning. Here we recorded populations of up to 90,000 neurons simultaneously from the primary visual cortex (V1) and higher visual areas (HVAs) while mice learned multiple tasks, as well as during unrewarded exposure to the same stimuli. Similar to previous studies, we found that neural changes in task mice were correlated with their behavioural learning. However, the neural changes were mostly replicated in mice with unrewarded exposure, suggesting that the changes were in fact due to unsupervised learning. The neural plasticity was highest in the medial HVAs and obeyed visual, rather than spatial, learning rules. In task mice only, we found a ramping reward-prediction signal in anterior HVAs, potentially involved in supervised learning. Our neural results predict that unsupervised learning may accelerate subsequent task learning, a prediction that we validated with behavioural experiments.

Preprint: https://www.biorxiv.org/content/early/2024/02/27/2024.02.25.581990

Modern algorithms for biological segmentation can match inter-human agreement in annotation quality. This however is not a performance bound: a hypothetical human-consensus segmentation could reduce error rates in half. To obtain a model that generalizes better we adapted the pretrained transformer backbone of a foundation model (SAM) to the Cellpose framework. The resulting Cellpose-SAM model substantially outperforms inter-human agreement and approaches the human-consensus bound. We increase generalization performance further by making the model robust to channel shuffling, cell size, shot noise, downsampling, isotropic and anisotropic blur. The new model can be readily adopted into the Cellpose ecosystem which includes finetuning, human-in-the-loop training, image restoration and 3D segmentation approaches. These properties establish Cellpose-SAM as a foundation model for biological segmentation.

Motor control in mammals is traditionally viewed as a hierarchy of descending spinal-targeting pathways, with frontal cortex at the top 1–3. Many redundant muscle patterns can solve a given task, and this high dimensionality allows flexibility but poses a problem for efficient learning 4. Although a feasible solution invokes subcortical innate motor patterns, or primitives, to reduce the dimensionality of the control problem, how cortex learns to utilize such primitives remains an open question 5–7. To address this, we studied cortical and subcortical interactions as head-fixed mice learned contextual control of innate hindlimb extension behavior. Naïve mice performed reactive extensions to turn off a cold air stimulus within seconds and, using predictive cues, learned to avoid the stimulus altogether in tens of trials. Optogenetic inhibition of large areas of rostral cortex completely prevented avoidance behavior, but did not impair hindlimb extensions in reaction to the cold air stimulus. Remarkably, mice covertly learned to avoid the cold stimulus even without any prior experience of successful, cortically-mediated avoidance. These findings support a dynamic, heterarchical model in which the dominant locus of control can change, on the order of seconds, between cortical and subcortical brain areas. We propose that cortex can leverage periods when subcortex predominates as demonstrations, to learn parameterized control of innate behavioral primitives.

Generalist methods for cellular segmentation have good out-of-the-box performance on a variety of image types; however, existing methods struggle for images that are degraded by noise, blurring or undersampling, all of which are common in microscopy. We focused the development of Cellpose3 on addressing these cases and here we demonstrate substantial out-of-the-box gains in segmentation and image quality for noisy, blurry and undersampled images. Unlike previous approaches that train models to restore pixel values, we trained Cellpose3 to output images that are well segmented by a generalist segmentation model, while maintaining perceptual similarity to the target images. Furthermore, we trained the restoration models on a large, varied collection of datasets, thus ensuring good generalization to user images. We provide these tools as 'one-click' buttons inside the graphical interface of Cellpose as well as in the Cellpose API.