Two-photon calcium imaging enables measurements of neural activity from individual neurons and their subcellular processes in awake animals. Axial brain motion can change measured fluorescence independently of neural activity and move small structures out of the imaging plane, confounding two-photon calcium measurements in behaving animals. We developed two complementary post-acquisition methods, selected according to whether the imaged structure remains detectable across the encountered depth range. When it does, z-stack correction estimates imaging depth from a separately acquired reference z-stack and uses an axial fluorescence profile specific to each neural structure to rescale fluorescence to a common reference depth. When a structure can leave a single imaging plane, volume-based z-registration reconstructs a common-depth movie from repeated acquisitions of closely spaced planes. In awake mice, both methods tracked experimentally imposed axial displacements within the sampled range, with depth-estimation root mean squared errors (RMSEs) below 1 μm for z-stack correction and 0.69-1.03 μm for volume-based z-registration. Correction changed fluorescence substantially for a subset of somata, with larger effects in axonal boutons. Both methods are implemented in suite2p-in-depth, a Python package that integrates with Suite2p and provides diagnostics for depth estimation, correction quality and data reliability. Together, these methods enable axial-motion artifacts to be assessed and corrected within established Suite2p workflows.

Circulating tumor cells (CTCs) and immune cells form dynamic multicellular ecosystems in blood, but their spatial organization and clinical relevance have not been systematically characterized. We developed the Cell and Cluster Identification Program (CCIP), an artificial intelligence–based framework that analyzes routine multiplex immunofluorescence blood scans to segment cells, identify CTCs and five immune lineages with high accuracy, and quantify multicellular clusters and tumor–immune interactions. Applying CCIP to 2,693 blood scans from 1,399 patients, we profiled over 60 million cells (>7 million multi-cell clusters) and linked imaging-derived features to patient outcomes. Correlated with circulating-tumor DNA mutation burdens, a 14-feature image model predicted overall survival in breast cancer, outperformed clinicopathologic variables and CTC enumeration, and generalized to prostate cancer. Prognostic imaging signatures were also associated with therapy response-related progression-free survival as well as with single-cell RNA sequencing–derived immune suppression states, connecting circulating tumor–immune architecture with systemic immune dysfunction.Competing Interest StatementThe authors have declared no competing interest.The data supporting the findings of this study are available in the Supporting Data Values file. Human scRNA-seq datasets generated in this study are available at GSE294399(19): https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE294399. The entire set of scanned immuno-fluorescence images of patient blood and pre-calculated image features and type prediction for individual cells, as well as cell-cluster features can be downloaded through Globus upon requests: https://app.globus.org/file-manager?origin_id=75af3eb6-b480-4f19-882c-ed079225b0ec&origin_path=%2F.National Institutes of Health, https://ror.org/01cwqze88, R01CA245699, R01AI167272, R01CA298232, T32 CA009560Chan Zuckerberg Biohub Chicago, https://ror.org/014nxkk19American Cancer Society, CSCC-Team-23-980420-01-CSCCProstate Cancer Foundation, Challenge Award 2017CHAL2008NIDDK training grant, U2CDK129917, TL1DK132769Lynn Sage Breast Cancer Research FoundationNorthwestern University, Pharmacology start-up grant

Sensory-guided decisions are the result of sensorimotor transformations across many brain areas. Recent studies have localized the motor- and decision-related components of these transformations using brain-wide neural recordings. It has been more difficult to localize sensory computations in the same way. Here we developed a new approach for linking sensory computations to behavior by training mice to discriminate between two stimuli and testing their responses with new stimuli. In separate animals, we calculated the similarity of neural representations between train and test stimuli, using recordings of up to 73,000 simultaneously-recorded neurons from 9 primary and higher-order visual areas (HVAs) across layers 2 and 3. We found that neural discrimination on test but not train images correlated with behavioral discrimination, and this relation required prior visual experience as it was not present in dark-reared mice. The link between neural and behavioral performance was highest in the medial HVAs, suggesting this region is a critical component of sensory transformations and generalization.

Neural computations are implemented by distributed neural populations that often span multiple brain areas. Causal photo-activation experiments done simultaneously with neural recordings can greatly improve our understanding of these computations, but such methods are typically limited to small subsets of neurons in restricted fields of view. Here we describe a new system called raster photostimulation for photo-activating and recording thousands of neurons, over a short 300 ms time window and over a large 5 mm field-of-view on a two-photon mesoscope. The photo-activation is precisely matched to the neural recording configuration, as it uses the same optical path, although with a different laser that is independently gated. We demonstrate pixel-level precision, frame-by-frame mask updating, and single-frame photostimulation of thousands of neurons. While this method lacks the precise temporal control of alternative methods, it compensates with ease-of-use, spatial precision, cost of implementation and by pushing the limits on the number of near-simultaneously stimulated neurons.

The brain’s capabilities rely on both the molecular properties of individual cells and their interactions across brain-wide networks. However, relating gene expression to activity in individual neurons across the entire brain remains elusive. Here we developed an experimental-computational platform, WARP, for whole-brain imaging of neuronal activity during behavior, expansion-assisted spatial transcriptomics, and cellular-level registration of these two modalities. Through joint analysis of whole-brain neuronal activity during multiple behaviors, cellular gene expression, and anatomy, we identified functions of molecularly defined populations — including luminance coding in a cckb-pou4f2 midbrain population and task-structured activity in pvalb7-eomesa hippocampal-like neurons — and defined over 2,000 other function-gene-anatomy subpopulations. Analysis of this unprecedented multimodal dataset also revealed that most gene-matched neurons showed stronger activity correlations, highlighting a brain-wide role for gene expression in functional organization. WARP establishes a foundational platform and open-access dataset for cross-experiment discovery, high-throughput function-to-gene mapping, unification of cell biology and systems neuroscience, and scalable circuit modeling at the whole-brain scale.

Neural recordings using optical methods have improved dramatically. For example, we demonstrate here recordings of over 100,000 neurons from the mouse cortex obtained with a standard commercial microscope. To process such large datasets, we developed Suite2p, a collection of efficient algorithms for motion correction, cell detection, activity extraction and quality control. We also developed new approaches to benchmark performance on these tasks. Our GPU-accelerated non-rigid motion correction substantially outperforms alternative methods, while running over five times faster. For cell detection, Suite2p outperforms the CNMF algorithm in Caiman and Fiola, finding more cells and producing fewer false positives, while running in a fraction of the time. We also introduce quality control steps for users to evaluate performance on their own data, while offering alternative algorithms for specialized types of recordings such as those from one-photon and voltage imaging.

Neural activity in mouse primary visual cortex (V1) correlates strongly with orofacial movements. Such strong modulation has not been found in the primate visual cortex during eye fixation [1], which led to the suggestion that the modulation may primarily depend on eye movements in both species [2]. Here we examined the influence of eye movements on neural activity in mouse visual cortex both in complete darkness and in the presence of different types of visual input. In all cases, we found that eye movements explain a smaller fraction of neural activity variance compared to orofacial behaviors. Additionally, we found that eye movements were correlated to orofacial movements, such as whisking and sniffing, and thus may be indirectly correlated to neural activity. These results further emphasize the impact of movement signals on mouse visual cortex during free viewing behavior.

fMRI signals were traditionally seen as slow and sampled in the order of seconds, but recent technological advances have enabled much faster sampling rates. We hypothesized that high-frequency fMRI signals can capture spontaneous neural activity that index brain states. Using fast fMRI (TR=378ms) and simultaneous EEG in 27 humans drifting between sleep and wakefulness, we found that fMRI spectral power increased during NREM sleep (compared to wakefulness) across several frequency ranges as fast as 1Hz. This fast fMRI power was correlated with canonical arousal-linked EEG rhythms (alpha and delta), with spatiotemporal correlation patterns for each rhythm reflecting a combination of shared arousal dynamics and rhythm-specific neural signatures. Using machine learning, we found that alpha and delta EEG rhythms can be decoded from fast fMRI signals, in subjects held-out from the training set, showing that fMRI as fast as 0.9Hz (alpha) and 0.7Hz (delta) contains reliable neurally-coupled information that generalizes across individuals. Finally, we demonstrate that this fast fMRI acquisition allows for EEG rhythms to be decoded from 3.8s windows of fMRI data. These results reveal that high-frequency fMRI signals are coupled to dynamically varying brain states, and that fast fMRI sampling allows for more temporally precise quantification of spontaneous neural activity than previously thought possible.

The brain exhibits rich oscillatory dynamics that play critical roles in vigilance and cognition, such as the neural rhythms that define sleep. These rhythms continuously fluctuate, signaling major changes in vigilance, but the widespread brain dynamics underlying these oscillations are difficult to investigate. Using simultaneous EEG and fast fMRI in humans who fell asleep inside the scanner, we developed a machine learning approach to investigate which fMRI regions and networks predict fluctuations in neural rhythms. We demonstrated that the rise and fall of alpha (8-12 Hz) and delta (1-4 Hz) power-two canonical EEG bands critically involved with cognition and vigilance-can be predicted from fMRI data in subjects that were not present in the training set. This approach also identified predictive information in individual brain regions across the cortex and subcortex. Finally, we developed an approach to identify shared and unique predictive information, and found that information about alpha rhythms was highly separable in two networks linked to arousal and visual systems. Conversely, delta rhythms were diffusely represented on a large spatial scale primarily across the cortex. These results demonstrate that EEG rhythms can be predicted from fMRI data, identify large-scale network patterns that underlie alpha and delta rhythms, and establish a novel framework for investigating multimodal brain dynamics.

Predictive coding is a theoretical framework that can explain how animals build internal models of their sensory environments by predicting sensory inputs. Predictive coding may capture either spatial or temporal relationships between sensory objects. While the original theory by Rao and Ballard, 1999 described spatial predictive coding, much of the recent experimental data has been interpreted as evidence for temporal predictive coding. Here we directly tested whether the “mismatch” neural responses in sensory cortex are due to a spatial or a temporal internal model. We adopted two common paradigms to study predictive coding: one based on virtual-reality and one based on static images. After training mice with repeated visual stimulation for several days, we performed multiple manipulations, including: 1) we introduced a novel stimulus, 2) we replaced a stimulus with a novel gray wall, 3) we duplicated a trained stimulus, or 4) we altered the order of the stimuli. The first two manipulations induced a substantial mismatch response in neural populations of up to 20,000 neurons recorded across primary and higher-order visual cortex, while the third and fourth ones did not. Thus, a mismatch response only occurred if a new spatial – not temporal – pattern was introduced.