Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction; however, so far there has been little consideration of how direct policy learning might inform our understanding. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning.

Animals learn both whether and when a reward will occur. Neural models of timing posit that animals learn the mean time until reward perturbed by a fixed relative uncertainty. Nonetheless, animals can learn to perform actions for reward even in highly variable natural environments. Optimal inference in the presence of variable information requires probabilistic models, yet it is unclear whether animals can infer such models for reward timing. Here, we develop a behavioral paradigm in which optimal performance required knowledge of the distribution from which reward delays were chosen. We found that mice were able to accurately adjust their behavior to the SD of the reward delay distribution. Importantly, mice were able to flexibly adjust the amount of prior information used for inference according to the moment-by-moment demands of the task. The ability to infer probabilistic models for timing may allow mice to adapt to complex and dynamic natural environments.

The ability to image neurons anywhere in the mammalian brain is a major goal of optical microscopy. Here we describe a minimally invasive microendoscopy system for studying the morphology and function of neurons at depth. Utilizing a guide cannula with an ultrathin wall, we demonstrated in vivo two-photon fluorescence imaging of deeply buried nuclei such as the striatum (2.5 mm depth), substantia nigra (4.4 mm depth) and lateral hypothalamus (5.0 mm depth) in mouse brain. We reported, for the first time, the observation of neuronal activity with subcellular resolution in the lateral hypothalamus and substantia nigra of head-fixed awake mice.

The interaction of descending neocortical outputs and subcortical premotor circuits is critical for shaping skilled movements. Two broad classes of motor cortical output projection neurons provide input to many subcortical motor areas: pyramidal tract (PT) neurons, which project throughout the neuraxis, and intratelencephalic (IT) neurons, which project within the cortex and subcortical striatum. It is unclear whether these classes are functionally in series or whether each class carries distinct components of descending motor control signals. Here, we combine large-scale neural recordings across all layers of motor cortex with cell type-specific perturbations to study cortically dependent mouse motor behaviors: kinematically variable manipulation of a joystick and a kinematically precise reach-to-grasp. We find that striatum-projecting IT neuron activity preferentially represents amplitude, whereas pons-projecting PT neurons preferentially represent the variable direction of forelimb movements. Thus, separable components of descending motor cortical commands are distributed across motor cortical projection cell classes.

Midbrain dopaminergic (DA) neurons are thought to guide learning via phasic elevations of firing in response to reward predicting stimuli. The mechanism for these signals remains unclear. Using extracellular recording during associative learning, we found that inhibitory neurons in the ventral midbrain of mice responded to salient auditory stimuli with a burst of activity that occurred before the onset of the phasic response of DA neurons. This population of inhibitory neurons exhibited enhanced responses during extinction and was anticorrelated with the phasic response of simultaneously recorded DA neurons. Optogenetic stimulation revealed that this population was, in part, derived from inhibitory projection neurons of the substantia nigra that provide a robust monosynaptic inhibition of DA neurons. Thus, our results elaborate on the dynamic upstream circuits that shape the phasic activity of DA neurons and suggest that the inhibitory microcircuit of the midbrain is critical for new learning in extinction.

Measuring the dynamics of neural processing across time scales requires following the spiking of thousands of individual neurons over milliseconds and months. To address this need, we introduce the Neuropixels 2.0 probe together with newly designed analysis algorithms. The probe has more than 5000 sites and is miniaturized to facilitate chronic implants in small mammals and recording during unrestrained behavior. High-quality recordings over long time scales were reliably obtained in mice and rats in six laboratories. Improved site density and arrangement combined with newly created data processing methods enable automatic post hoc correction for brain movements, allowing recording from the same neurons for more than 2 months. These probes and algorithms enable stable recordings from thousands of sites during free behavior, even in small animals such as mice.

For goal-directed behaviour it is critical that we can both select the appropriate action and learn to modify the underlying movements (for example, the pitch of a note or velocity of a reach) to improve outcomes. The basal ganglia are a critical nexus where circuits necessary for the production of behaviour, such as the neocortex and thalamus, are integrated with reward signalling to reinforce successful, purposive actions. The dorsal striatum, a major input structure of basal ganglia, is composed of two opponent pathways, direct and indirect, thought to select actions that elicit positive outcomes and suppress actions that do not, respectively. Activity-dependent plasticity modulated by reward is thought to be sufficient for selecting actions in the striatum. Although perturbations of basal ganglia function produce profound changes in movement, it remains unknown whether activity-dependent plasticity is sufficient to produce learned changes in movement kinematics, such as velocity. Here we use cell-type-specific stimulation in mice delivered in closed loop during movement to demonstrate that activity in either the direct or indirect pathway is sufficient to produce specific and sustained increases or decreases in velocity, without affecting action selection or motivation. These behavioural changes were a form of learning that accumulated over trials, persisted after the cessation of stimulation, and were abolished in the presence of dopamine antagonists. Our results reveal that the direct and indirect pathways can each bidirectionally control movement velocity, demonstrating unprecedented specificity and flexibility in the control of volition by the basal ganglia.

Compared to the dorsal hippocampus, relatively few studies have characterized neuronal responses in the ventral hippocampus. In particular, it is unclear whether and how cells in the ventral region represent space and/or respond to contextual changes. We recorded from dorsal and ventral CA1 neurons in freely moving mice exposed to manipulations of visuospatial and olfactory contexts. We found that ventral cells respond to alterations of the visuospatial environment such as exposure to novel local cues, cue rotations, and contextual expansion in similar ways to dorsal cells, with the exception of cue rotations. Furthermore, we found that ventral cells responded to odors much more strongly than dorsal cells, particularly to odors of high valence. Similar to earlier studies recording from the ventral hippocampus in CA3, we also found increased scaling of place cell field size along the longitudinal hippocampal axis. Although the increase in place field size observed toward the ventral pole has previously been taken to suggest a decrease in spatial information coded by ventral place cells, we hypothesized that a change in spatial scaling could instead signal a shift in representational coding that preserves the resolution of spatial information. To explore this possibility, we examined population activity using principal component analysis (PCA) and neural location reconstruction techniques. Our results suggest that ventral populations encode a distributed representation of space, and that the resolution of spatial information at the population level is comparable to that of dorsal populations of similar size. Finally, through the use of neural network modeling, we suggest that the redundancy in spatial representation along the longitudinal hippocampal axis may allow the hippocampus to overcome the conflict between memory interference and generalization inherent in neural network memory. Our results suggest that ventral population activity is well suited for generalization across locations and contexts. © 2014 Wiley Periodicals, Inc.

Neuronal cell types are the nodes of neural circuits that determine the flow of information within the brain. Neuronal morphology, especially the shape of the axonal arbor, provides an essential descriptor of cell type and reveals how individual neurons route their output across the brain. Despite the importance of morphology, few projection neurons in the mouse brain have been reconstructed in their entirety. Here we present a robust and efficient platform for imaging and reconstructing complete neuronal morphologies, including axonal arbors that span substantial portions of the brain. We used this platform to reconstruct more than 1,000 projection neurons in the motor cortex, thalamus, subiculum, and hypothalamus. Together, the reconstructed neurons constitute more than 85 meters of axonal length and are available in a searchable online database. Axonal shapes revealed previously unknown subtypes of projection neurons and suggest organizational principles of long-range connectivity.

A number of recent studies have provided compelling demonstrations that both mice and rats can be trained to perform a variety of behavioral tasks while restrained by mechanical elements mounted to the skull. The independent development of this technique by a number of laboratories has led to diverse solutions. We found that these solutions often used expensive materials and impeded future development and modification in the absence of engineering support. In order to address these issues, here we report on the development of a flexible single hardware design for electrophysiology and imaging both in brain tissue in vitro. Our hardware facilitates the rapid conversion of a single preparation between physiology and imaging system and the conversion of a given system between preparations. In addition, our use of rapid prototyping machines ("3D printers") allows for the deployment of new designs within a day. Here, we present specifications for design and manufacturing as well as some data from our lab demonstrating the suitability of the design for physiology in behaving animals and imaging in vitro and in vivo.