Artificial neural networks (ANNs) have been shown to predict neural responses in primary visual cortex (V1) better than classical models. However, this performance often comes at the expense of simplicity and interpretability. Here we introduce a new class of simplified ANN models that can predict over 70% of the response variance of V1 neurons. To achieve this high performance, we first recorded a new dataset of over 29,000 neurons responding to up to 65,000 natural image presentations in mouse V1. We found that ANN models required only two convolutional layers for good performance, with a relatively small first layer. We further found that we could make the second layer small without loss of performance, by fitting individual "minimodels" to each neuron. Similar simplifications applied for models of monkey V1 neurons. We show that the minimodels can be used to gain insight into how stimulus invariance arises in biological neurons.

Preprint: https://www.biorxiv.org/content/early/2024/07/02/2024.06.30.601394

Representation learning in neural networks may be implemented with supervised or unsupervised algorithms, distinguished by the availability of instruction. In the sensory cortex, perceptual learning drives neural plasticity1-13, but it is not known whether this is due to supervised or unsupervised learning. Here we recorded populations of up to 90,000 neurons simultaneously from the primary visual cortex (V1) and higher visual areas (HVAs) while mice learned multiple tasks, as well as during unrewarded exposure to the same stimuli. Similar to previous studies, we found that neural changes in task mice were correlated with their behavioural learning. However, the neural changes were mostly replicated in mice with unrewarded exposure, suggesting that the changes were in fact due to unsupervised learning. The neural plasticity was highest in the medial HVAs and obeyed visual, rather than spatial, learning rules. In task mice only, we found a ramping reward-prediction signal in anterior HVAs, potentially involved in supervised learning. Our neural results predict that unsupervised learning may accelerate subsequent task learning, a prediction that we validated with behavioural experiments.

Preprint: https://www.biorxiv.org/content/early/2024/02/27/2024.02.25.581990

All brain functions in animals rely upon neuronal connectivity that is established during early development. Although the activity-dependent mechanisms are deemed important for brain development and adult synaptic plasticity, the precise cellular and molecular mechanisms remain however, largely unknown. This lack of fundamental knowledge regarding developmental neuronal assembly owes its existence to the complexity of the mammalian brain as cell-cell interactions between individual neurons cannot be investigated directly. Here, we used individually identified synaptic partners from Lymnaea stagnalis to interrogate the role of neuronal activity patterns over an extended time period during various growth time points and synaptogenesis. Using intracellular recordings, microelectrode arrays, and time-lapse imaging, we identified unique patterns of activity throughout neurite outgrowth and synapse formation. Perturbation of voltage-gated Ca channels compromised neuronal growth patterns which also invoked a protein kinase A mediated pathway. Our findings underscore the importance of unique activity patterns in regulating neuronal growth, neurite branching, and synapse formation, and identify the underlying cellular and molecular mechanisms.

Modern algorithms for biological segmentation can match inter-human agreement in annotation quality. This however is not a performance bound: a hypothetical human-consensus segmentation could reduce error rates in half. To obtain a model that generalizes better we adapted the pretrained transformer backbone of a foundation model (SAM) to the Cellpose framework. The resulting Cellpose-SAM model substantially outperforms inter-human agreement and approaches the human-consensus bound. We increase generalization performance further by making the model robust to channel shuffling, cell size, shot noise, downsampling, isotropic and anisotropic blur. The new model can be readily adopted into the Cellpose ecosystem which includes finetuning, human-in-the-loop training, image restoration and 3D segmentation approaches. These properties establish Cellpose-SAM as a foundation model for biological segmentation.

Circulating tumor cells (CTCs) are critical biomarkers for predicting therapy response and survival in breast cancer patients. Multicellular CTC clusters exhibit enhanced metastatic potential, yet their detection and characterization are constrained by low frequency in blood samples and reliance on labor-intensive manual analysis. Advancing these methods could significantly improve prognostic evaluation and therapeutic strategies.Leveraging FDA-approved CellSearch technology and single-cell sequencing, we analyzed 2, 853 blood specimens, longitudinally collected from 1358 patients with advanced cancer (breast, prostate, etc) and other diseases. Integrating machine learning and deep learning tools, we developed a novel CTCpose platform to automate detection and analysis of CTCs, immune cells, and their interactions. Using artificial intelligence (AI)-driven image analysis, we extracted over 270 cellular and nuclear features including intensity, morphometry, fourier shape, gradient/edge, and haralick of cytokeratin, CD45, and DAPI expression patterns, enabling precise characterization of CTCs, white blood cells (WBCs), CTC clusters, and their interactions with immune cells (WBCs).The CTCpose platform enabled automated identification of CTCs, WBCs, homotypic CTC clusters, heterogenous CTC-WBC clusters, and immune cell clusters, providing comprehensive insights into cell morphology, biomarker expression, and spatial organization. These features correlated with patient survival, disease progression, and treatment response. Our findings highlight the clinical significance of CTC-immune cell interactions and dynamic alterations of CTCs (singles and clusters) and underscore their potential in stratifying patients into distinct risk categories.This study demonstrates the transformative potential of deep learning in overcoming limitations of traditional CTC detection methods and integrating imaging data with large cohorts of patient data. By automating and enhancing the analysis of CTC-immune cell interactions, we present a robust framework for developing predictive models with direct clinical relevance. This work opens avenues for personalized treatment strategies, underscoring the impact of AI in advancing precision oncology.Yuanfei Sun, Joshua R. Squires, Andrew Hoffmann, Youbin Zhang, Allegra Minor, Anmol Singh, David Scholten, Chengsheng Mao, Yuan Luo, Deyu Fang, William J. Gradishar, Massimo Cristofanilli, Carsen Stringer, Huiping Liu. Deep learning enables automated detection of circulating tumor cell-immune cell interactions with prognostic insights in cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 2420.

Cognitive maps confer animals with flexible intelligence by representing spatial, temporal and abstract relationships that can be used to shape thought, planning and behaviour. Cognitive maps have been observed in the hippocampus1, but their algorithmic form and learning mechanisms remain obscure. Here we used large-scale, longitudinal two-photon calcium imaging to record activity from thousands of neurons in the CA1 region of the hippocampus while mice learned to efficiently collect rewards from two subtly different linear tracks in virtual reality. Throughout learning, both animal behaviour and hippocampal neural activity progressed through multiple stages, gradually revealing improved task representation that mirrored improved behavioural efficiency. The learning process involved progressive decorrelations in initially similar hippocampal neural activity within and across tracks, ultimately resulting in orthogonalized representations resembling a state machine capturing the inherent structure of the task. This decorrelation process was driven by individual neurons acquiring task-state-specific responses (that is, 'state cells'). Although various standard artificial neural networks did not naturally capture these dynamics, the clone-structured causal graph, a hidden Markov model variant, uniquely reproduced both the final orthogonalized states and the learning trajectory seen in animals. The observed cellular and population dynamics constrain the mechanisms underlying cognitive map formation in the hippocampus, pointing to hidden state inference as a fundamental computational principle, with implications for both biological and artificial intelligence.

Limited color channels in fluorescence microscopy have long constrained spatial analysis in biological specimens. Here, we introduce cycle Hybridization Chain Reaction (HCR), a method that integrates multicycle DNA barcoding with HCR to overcome this limitation. cycleHCR enables highly multiplexed imaging of RNA and proteins using a unified barcode system. Whole-embryo transcriptomics imaging achieved precise three-dimensional gene expression and cell fate mapping across a specimen depth of ~310 μm. When combined with expansion microscopy, cycleHCR revealed an intricate network of 10 subcellular structures in mouse embryonic fibroblasts. In mouse hippocampal slices, multiplex RNA and protein imaging uncovered complex gene expression gradients and cell-type-specific nuclear structural variations. cycleHCR provides a quantitative framework for elucidating spatial regulation in deep tissue contexts for research and potentially diagnostic applications.

bioRxiv preprint: 10.1101/2024.05.17.594641

Generalist methods for cellular segmentation have good out-of-the-box performance on a variety of image types; however, existing methods struggle for images that are degraded by noise, blurring or undersampling, all of which are common in microscopy. We focused the development of Cellpose3 on addressing these cases and here we demonstrate substantial out-of-the-box gains in segmentation and image quality for noisy, blurry and undersampled images. Unlike previous approaches that train models to restore pixel values, we trained Cellpose3 to output images that are well segmented by a generalist segmentation model, while maintaining perceptual similarity to the target images. Furthermore, we trained the restoration models on a large, varied collection of datasets, thus ensuring good generalization to user images. We provide these tools as 'one-click' buttons inside the graphical interface of Cellpose as well as in the Cellpose API.

Artificial neural networks learn faster if they are initialized well. Good initializations can generate high-dimensional macroscopic dynamics with long timescales. It is not known if biological neural networks have similar properties. Here we show that the eigenvalue spectrum and dynamical properties of large-scale neural recordings in mice (two-photon and electrophysiology) are similar to those produced by linear dynamics governed by a random symmetric matrix that is critically normalized. An exception was hippocampal area CA1: population activity in this area resembled an efficient, uncorrelated neural code, which may be optimized for information storage capacity. Global emergent activity modes persisted in simulations with sparse, clustered or spatial connectivity. We hypothesize that the spontaneous neural activity reflects a critical initialization of whole-brain neural circuits that is optimized for learning time-dependent tasks.

Simultaneous recordings from hundreds or thousands of neurons are becoming routine because of innovations in instrumentation, molecular tools, and data processing software. Such recordings can be analyzed with data science methods, but it is not immediately clear what methods to use or how to adapt them for neuroscience applications. We review, categorize, and illustrate diverse analysis methods for neural population recordings and describe how these methods have been used to make progress on longstanding questions in neuroscience. We review a variety of approaches, ranging from the mathematically simple to the complex, from exploratory to hypothesis-driven, and from recently developed to more established methods. We also illustrate some of the common statistical pitfalls in analyzing large-scale neural data.