Persistent neural activity that outlasts an initial stimulus is thought to provide a mechanism for the transient storage of memory. In this issue of Neuron, Fransén et al. identify important principles for a cell-autonomous mechanism of graded persistent firing using an elegant combination of experimental and computational approaches.

Ligand-gated ion channels involved in the modulation of synaptic strength are the AMPA, kainate, and NMDA glutamate receptors. Small molecules that potentiate AMPA receptor currents relieve cognitive deficits caused by neurodegenerative diseases such as Alzheimer’s disease and show promise in the treatment of depression. Previously, there has been limited understanding of the molecular mechanism of action for AMPA receptor potentiators. Here we present cocrystal structures of the glutamate receptor GluR2 S1S2 ligand-binding domain in complex with aniracetam [1-(4-methoxybenzoyl)-2-pyrrolidinone] or CX614 (pyrrolidino-1,3-oxazino benzo-1,4-dioxan-10-one), two AMPA receptor potentiators that preferentially slow AMPA receptor deactivation. Both potentiators bind within the dimer interface of the nondesensitized receptor at a common site located on the twofold axis of molecular symmetry. Importantly, the potentiator binding site is adjacent to the "hinge" in the ligand-binding core "clamshell" that undergoes conformational rearrangement after glutamate binding. Using rapid solution exchange, patch-clamp electrophysiology experiments, we show that point mutations of residues that interact with potentiators in the cocrystal disrupt potentiator function. We suggest that the potentiators slow deactivation by stabilizing the clamshell in its closed-cleft, glutamate-bound conformation.

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioral policies and indirect learning via value functions. Policy learning and value learning employ distinct algorithms that optimize behavioral performance and reward prediction, respectively. In animals, behavioral learning and the role of mesolimbic dopamine signaling have been extensively evaluated with respect to reward prediction; however, to date there has been little consideration of how direct policy learning might inform our understanding. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioral policies evolve as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioral policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically-calibrated manipulations of mesolimbic dopamine produced multiple effects inconsistent with value learning but predicted by a neural network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioral policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioral policies, expanding the explanatory power of reinforcement learning models for animal learning.

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction; however, so far there has been little consideration of how direct policy learning might inform our understanding. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning.

Animals learn both whether and when a reward will occur. Neural models of timing posit that animals learn the mean time until reward perturbed by a fixed relative uncertainty. Nonetheless, animals can learn to perform actions for reward even in highly variable natural environments. Optimal inference in the presence of variable information requires probabilistic models, yet it is unclear whether animals can infer such models for reward timing. Here, we develop a behavioral paradigm in which optimal performance required knowledge of the distribution from which reward delays were chosen. We found that mice were able to accurately adjust their behavior to the SD of the reward delay distribution. Importantly, mice were able to flexibly adjust the amount of prior information used for inference according to the moment-by-moment demands of the task. The ability to infer probabilistic models for timing may allow mice to adapt to complex and dynamic natural environments.

The ability to image neurons anywhere in the mammalian brain is a major goal of optical microscopy. Here we describe a minimally invasive microendoscopy system for studying the morphology and function of neurons at depth. Utilizing a guide cannula with an ultrathin wall, we demonstrated in vivo two-photon fluorescence imaging of deeply buried nuclei such as the striatum (2.5 mm depth), substantia nigra (4.4 mm depth) and lateral hypothalamus (5.0 mm depth) in mouse brain. We reported, for the first time, the observation of neuronal activity with subcellular resolution in the lateral hypothalamus and substantia nigra of head-fixed awake mice.

The interaction of descending neocortical outputs and subcortical premotor circuits is critical for shaping skilled movements. Two broad classes of motor cortical output projection neurons provide input to many subcortical motor areas: pyramidal tract (PT) neurons, which project throughout the neuraxis, and intratelencephalic (IT) neurons, which project within the cortex and subcortical striatum. It is unclear whether these classes are functionally in series or whether each class carries distinct components of descending motor control signals. Here, we combine large-scale neural recordings across all layers of motor cortex with cell type-specific perturbations to study cortically dependent mouse motor behaviors: kinematically variable manipulation of a joystick and a kinematically precise reach-to-grasp. We find that striatum-projecting IT neuron activity preferentially represents amplitude, whereas pons-projecting PT neurons preferentially represent the variable direction of forelimb movements. Thus, separable components of descending motor cortical commands are distributed across motor cortical projection cell classes.

Midbrain dopaminergic (DA) neurons are thought to guide learning via phasic elevations of firing in response to reward predicting stimuli. The mechanism for these signals remains unclear. Using extracellular recording during associative learning, we found that inhibitory neurons in the ventral midbrain of mice responded to salient auditory stimuli with a burst of activity that occurred before the onset of the phasic response of DA neurons. This population of inhibitory neurons exhibited enhanced responses during extinction and was anticorrelated with the phasic response of simultaneously recorded DA neurons. Optogenetic stimulation revealed that this population was, in part, derived from inhibitory projection neurons of the substantia nigra that provide a robust monosynaptic inhibition of DA neurons. Thus, our results elaborate on the dynamic upstream circuits that shape the phasic activity of DA neurons and suggest that the inhibitory microcircuit of the midbrain is critical for new learning in extinction.

Measuring the dynamics of neural processing across time scales requires following the spiking of thousands of individual neurons over milliseconds and months. To address this need, we introduce the Neuropixels 2.0 probe together with newly designed analysis algorithms. The probe has more than 5000 sites and is miniaturized to facilitate chronic implants in small mammals and recording during unrestrained behavior. High-quality recordings over long time scales were reliably obtained in mice and rats in six laboratories. Improved site density and arrangement combined with newly created data processing methods enable automatic post hoc correction for brain movements, allowing recording from the same neurons for more than 2 months. These probes and algorithms enable stable recordings from thousands of sites during free behavior, even in small animals such as mice.

For goal-directed behaviour it is critical that we can both select the appropriate action and learn to modify the underlying movements (for example, the pitch of a note or velocity of a reach) to improve outcomes. The basal ganglia are a critical nexus where circuits necessary for the production of behaviour, such as the neocortex and thalamus, are integrated with reward signalling to reinforce successful, purposive actions. The dorsal striatum, a major input structure of basal ganglia, is composed of two opponent pathways, direct and indirect, thought to select actions that elicit positive outcomes and suppress actions that do not, respectively. Activity-dependent plasticity modulated by reward is thought to be sufficient for selecting actions in the striatum. Although perturbations of basal ganglia function produce profound changes in movement, it remains unknown whether activity-dependent plasticity is sufficient to produce learned changes in movement kinematics, such as velocity. Here we use cell-type-specific stimulation in mice delivered in closed loop during movement to demonstrate that activity in either the direct or indirect pathway is sufficient to produce specific and sustained increases or decreases in velocity, without affecting action selection or motivation. These behavioural changes were a form of learning that accumulated over trials, persisted after the cessation of stimulation, and were abolished in the presence of dopamine antagonists. Our results reveal that the direct and indirect pathways can each bidirectionally control movement velocity, demonstrating unprecedented specificity and flexibility in the control of volition by the basal ganglia.