Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction; however, so far there has been little consideration of how direct policy learning might inform our understanding. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning.

Calcium imaging is commonly used to measure the neural activity of large groups of neurons in mice. Genetically encoded calcium indicators (GECIs) can be delivered for this purpose using non-invasive genetic methods. Compared to viral gene transfer, transgenic targeting of GECIs provides stable long-term expression and obviates the need for invasive viral injections. Transgenic mice expressing the green GECI GCaMP6 are already widely used. Here we present the generation and characterization of transgenic mice expressing the sensitive red GECI jRGECO1a, driven by the Thy1 promoter. Four transgenic lines with different expression patterns showed sufficiently high expression for cellular in vivo imaging. We used two-photon microscopy to characterize visual responses of individual neurons in the visual cortex in vivo. The signal-to-noise ratio in transgenic mice was comparable to, or better than, mice transduced with adeno-associated virus. In addition, we show that Thy1-jRGECO1a transgenic mice are useful for transcranial population imaging and functional mapping using widefield fluorescence microscopy. We also demonstrate imaging of visual responses in retinal ganglion cells in vitro. Thy1-jRGECO1a transgenic mice are therefore a useful addition to the toolbox for imaging activity in intact neural networks.

Genetically encoded calcium indicators (GECIs) allow measurement of activity in large populations of neurons and in small neuronal compartments, over times of milliseconds to months. Although GFP-based GECIs are widely used for in vivo neurophysiology, GECIs with red-shifted excitation and emission spectra have advantages for in vivo imaging because of reduced scattering and absorption in tissue, and a consequent reduction in phototoxicity. However, current red GECIs are inferior to the state-of-the-art GFP-based GCaMP6 indicators for detecting and quantifying neural activity. Here we present improved red GECIs based on mRuby (jRCaMP1a, b) and mApple (jRGECO1a), with sensitivity comparable to GCaMP6. We characterized the performance of the new red GECIs in cultured neurons and in mouse, Drosophila, zebrafish and C. elegans in vivo. Red GECIs facilitate deep-tissue imaging, dual-color imaging together with GFP-based reporters, and the use of optogenetics in combination with calcium imaging.