Sensory-guided decisions are the result of sensorimotor transformations across many brain areas. Recent studies have localized the motor- and decision-related components of these transformations using brain-wide neural recordings. It has been more difficult to localize sensory computations in the same way. Here we developed a new approach for linking sensory computations to behavior by training mice to discriminate between two stimuli and testing their responses with new stimuli. In separate animals, we calculated the similarity of neural representations between train and test stimuli, using recordings of up to 73,000 simultaneously-recorded neurons from 9 primary and higher-order visual areas (HVAs) across layers 2 and 3. We found that neural discrimination on test but not train images correlated with behavioral discrimination, and this relation required prior visual experience as it was not present in dark-reared mice. The link between neural and behavioral performance was highest in the medial HVAs, suggesting this region is a critical component of sensory transformations and generalization.

Neural computations are implemented by distributed neural populations that often span multiple brain areas. Causal photo-activation experiments done simultaneously with neural recordings can greatly improve our understanding of these computations, but such methods are typically limited to small subsets of neurons in restricted fields of view. Here we describe a new system called raster photostimulation for photo-activating and recording thousands of neurons, over a short 300 ms time window and over a large 5 mm field-of-view on a two-photon mesoscope. The photo-activation is precisely matched to the neural recording configuration, as it uses the same optical path, although with a different laser that is independently gated. We demonstrate pixel-level precision, frame-by-frame mask updating, and single-frame photostimulation of thousands of neurons. While this method lacks the precise temporal control of alternative methods, it compensates with ease-of-use, spatial precision, cost of implementation and by pushing the limits on the number of near-simultaneously stimulated neurons.

The brain’s capabilities rely on both the molecular properties of individual cells and their interactions across brain-wide networks. However, relating gene expression to activity in individual neurons across the entire brain remains elusive. Here we developed an experimental-computational platform, WARP, for whole-brain imaging of neuronal activity during behavior, expansion-assisted spatial transcriptomics, and cellular-level registration of these two modalities. Through joint analysis of whole-brain neuronal activity during multiple behaviors, cellular gene expression, and anatomy, we identified functions of molecularly defined populations — including luminance coding in a cckb-pou4f2 midbrain population and task-structured activity in pvalb7-eomesa hippocampal-like neurons — and defined over 2,000 other function-gene-anatomy subpopulations. Analysis of this unprecedented multimodal dataset also revealed that most gene-matched neurons showed stronger activity correlations, highlighting a brain-wide role for gene expression in functional organization. WARP establishes a foundational platform and open-access dataset for cross-experiment discovery, high-throughput function-to-gene mapping, unification of cell biology and systems neuroscience, and scalable circuit modeling at the whole-brain scale.

Neural recordings using optical methods have improved dramatically. For example, we demonstrate here recordings of over 100,000 neurons from the mouse cortex obtained with a standard commercial microscope. To process such large datasets, we developed Suite2p, a collection of efficient algorithms for motion correction, cell detection, activity extraction and quality control. We also developed new approaches to benchmark performance on these tasks. Our GPU-accelerated non-rigid motion correction substantially outperforms alternative methods, while running over five times faster. For cell detection, Suite2p outperforms the CNMF algorithm in Caiman and Fiola, finding more cells and producing fewer false positives, while running in a fraction of the time. We also introduce quality control steps for users to evaluate performance on their own data, while offering alternative algorithms for specialized types of recordings such as those from one-photon and voltage imaging.

fMRI signals were traditionally seen as slow and sampled in the order of seconds, but recent technological advances have enabled much faster sampling rates. We hypothesized that high-frequency fMRI signals can capture spontaneous neural activity that index brain states. Using fast fMRI (TR=378ms) and simultaneous EEG in 27 humans drifting between sleep and wakefulness, we found that fMRI spectral power increased during NREM sleep (compared to wakefulness) across several frequency ranges as fast as 1Hz. This fast fMRI power was correlated with canonical arousal-linked EEG rhythms (alpha and delta), with spatiotemporal correlation patterns for each rhythm reflecting a combination of shared arousal dynamics and rhythm-specific neural signatures. Using machine learning, we found that alpha and delta EEG rhythms can be decoded from fast fMRI signals, in subjects held-out from the training set, showing that fMRI as fast as 0.9Hz (alpha) and 0.7Hz (delta) contains reliable neurally-coupled information that generalizes across individuals. Finally, we demonstrate that this fast fMRI acquisition allows for EEG rhythms to be decoded from 3.8s windows of fMRI data. These results reveal that high-frequency fMRI signals are coupled to dynamically varying brain states, and that fast fMRI sampling allows for more temporally precise quantification of spontaneous neural activity than previously thought possible.

The brain exhibits rich oscillatory dynamics that play critical roles in vigilance and cognition, such as the neural rhythms that define sleep. These rhythms continuously fluctuate, signaling major changes in vigilance, but the widespread brain dynamics underlying these oscillations are difficult to investigate. Using simultaneous EEG and fast fMRI in humans who fell asleep inside the scanner, we developed a machine learning approach to investigate which fMRI regions and networks predict fluctuations in neural rhythms. We demonstrated that the rise and fall of alpha (8-12 Hz) and delta (1-4 Hz) power-two canonical EEG bands critically involved with cognition and vigilance-can be predicted from fMRI data in subjects that were not present in the training set. This approach also identified predictive information in individual brain regions across the cortex and subcortex. Finally, we developed an approach to identify shared and unique predictive information, and found that information about alpha rhythms was highly separable in two networks linked to arousal and visual systems. Conversely, delta rhythms were diffusely represented on a large spatial scale primarily across the cortex. These results demonstrate that EEG rhythms can be predicted from fMRI data, identify large-scale network patterns that underlie alpha and delta rhythms, and establish a novel framework for investigating multimodal brain dynamics.

Predictive coding is a theoretical framework that can explain how animals build internal models of their sensory environments by predicting sensory inputs. Predictive coding may capture either spatial or temporal relationships between sensory objects. While the original theory by Rao and Ballard, 1999 described spatial predictive coding, much of the recent experimental data has been interpreted as evidence for temporal predictive coding. Here we directly tested whether the “mismatch” neural responses in sensory cortex are due to a spatial or a temporal internal model. We adopted two common paradigms to study predictive coding: one based on virtual-reality and one based on static images. After training mice with repeated visual stimulation for several days, we performed multiple manipulations, including: 1) we introduced a novel stimulus, 2) we replaced a stimulus with a novel gray wall, 3) we duplicated a trained stimulus, or 4) we altered the order of the stimuli. The first two manipulations induced a substantial mismatch response in neural populations of up to 20,000 neurons recorded across primary and higher-order visual cortex, while the third and fourth ones did not. Thus, a mismatch response only occurred if a new spatial – not temporal – pattern was introduced.

Spatial multiomic profiling has been transforming the understanding of local tumor ecosystems. Yet, the spatial analyses of tumor-immune interactions at systemic levels, such as in liquid biopsies, are challenging. Within the last 10 years, we have longitudinally collected nearly 3,000 patient blood samples for multiplexing imaging of circulating tumor cells (CTCs) and their interactions with white blood cells (WBCs). Multicellular CTC clusters exhibit enhanced metastatic potential. The detection of CTCs and characterization of tumor immune ecosystems are constrained by (1) low frequency of CTCs in blood samples; (2) specific lineages of immune cells are not recognized by limited channels of current imaging methods, (3) reliance on labor-intensive manual analysis slows down the discovery of biomarkers for predicting therapy response and survival in cancer patients. We hypothesize that an AI-powered platform will accelerate the lineage and spatial characterization of tumor immune ecosystems for prognostic evaluations.

Artificial neural networks (ANNs) have been shown to predict neural responses in primary visual cortex (V1) better than classical models. However, this performance often comes at the expense of simplicity and interpretability. Here we introduce a new class of simplified ANN models that can predict over 70% of the response variance of V1 neurons. To achieve this high performance, we first recorded a new dataset of over 29,000 neurons responding to up to 65,000 natural image presentations in mouse V1. We found that ANN models required only two convolutional layers for good performance, with a relatively small first layer. We further found that we could make the second layer small without loss of performance, by fitting individual "minimodels" to each neuron. Similar simplifications applied for models of monkey V1 neurons. We show that the minimodels can be used to gain insight into how stimulus invariance arises in biological neurons.

Preprint: https://www.biorxiv.org/content/early/2024/07/02/2024.06.30.601394

Representation learning in neural networks may be implemented with supervised or unsupervised algorithms, distinguished by the availability of instruction. In the sensory cortex, perceptual learning drives neural plasticity1-13, but it is not known whether this is due to supervised or unsupervised learning. Here we recorded populations of up to 90,000 neurons simultaneously from the primary visual cortex (V1) and higher visual areas (HVAs) while mice learned multiple tasks, as well as during unrewarded exposure to the same stimuli. Similar to previous studies, we found that neural changes in task mice were correlated with their behavioural learning. However, the neural changes were mostly replicated in mice with unrewarded exposure, suggesting that the changes were in fact due to unsupervised learning. The neural plasticity was highest in the medial HVAs and obeyed visual, rather than spatial, learning rules. In task mice only, we found a ramping reward-prediction signal in anterior HVAs, potentially involved in supervised learning. Our neural results predict that unsupervised learning may accelerate subsequent task learning, a prediction that we validated with behavioural experiments.

Preprint: https://www.biorxiv.org/content/early/2024/02/27/2024.02.25.581990