Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction; however, so far there has been little consideration of how direct policy learning might inform our understanding. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning.

To accurately track self-location, animals need to integrate their movements through space. In amniotes, representations of self-location have been found in regions such as the hippocampus. It is unknown whether more ancient brain regions contain such representations and by which pathways they may drive locomotion. Fish displaced by water currents must prevent uncontrolled drift to potentially dangerous areas. We found that larval zebrafish track such movements and can later swim back to their earlier location. Whole-brain functional imaging revealed the circuit enabling this process of positional homeostasis. Position-encoding brainstem neurons integrate optic flow, then bias future swimming to correct for past displacements by modulating inferior olive and cerebellar activity. Manipulation of position-encoding or olivary neurons abolished positional homeostasis or evoked behavior as if animals had experienced positional shifts. These results reveal a multiregional hindbrain circuit in vertebrates for optic flow integration, memory of self-location, and its neural pathway to behavior.Competing Interest StatementThe authors have declared no competing interest.

The ability to measure synaptic connectivity and properties is essential for understanding neuronal circuits. However, existing methods that allow such measurements at cellular resolution are laborious and technically demanding. Here, we describe a system that allows such measurements in a high-throughput way by combining two-photon optogenetics and volumetric Ca2+ imaging with whole-cell recording. We reveal a circuit motif for generating fast undulatory locomotion in zebrafish.

Female behavior changes profoundly after mating. In Drosophila, the mechanisms underlying the long-term changes led by seminal products have been extensively studied. However, the effect of the sensory component of copulation on the female's internal state and behavior remains elusive. We pursued this question by dissociating the effect of coital sensory inputs from those of male ejaculate. We found that the sensory inputs of copulation cause a reduction of post-coital receptivity in females, referred to as the "copulation effect." We identified three layers of a neural circuit underlying this phenomenon. Abdominal neurons expressing the mechanosensory channel Piezo convey the signal of copulation to female-specific ascending neurons, LSANs, in the ventral nerve cord. LSANs relay this information to neurons expressing myoinhibitory peptides in the brain. We hereby provide a neural mechanism by which the experience of copulation facilitates females encoding their mating status, thus adjusting behavior to optimize reproduction.

BACKGROUND: Genetically encoded calcium ion (Ca2+) indicators (GECIs) are indispensable tools for measuring Ca2+ dynamics and neuronal activities in vitro and in vivo. Red fluorescent protein (RFP)-based GECIs have inherent advantages relative to green fluorescent protein-based GECIs due to the longer wavelength light used for excitation. Longer wavelength light is associated with decreased phototoxicity and deeper penetration through tissue. Red GECI can also enable multicolor visualization with blue- or cyan-excitable fluorophores.

RESULTS: Here we report the development, structure, and validation of a new RFP-based GECI, K-GECO1, based on a circularly permutated RFP derived from the sea anemone Entacmaea quadricolor. We have characterized the performance of K-GECO1 in cultured HeLa cells, dissociated neurons, stem-cell-derived cardiomyocytes, organotypic brain slices, zebrafish spinal cord in vivo, and mouse brain in vivo.

CONCLUSION: K-GECO1 is the archetype of a new lineage of GECIs based on the RFP eqFP578 scaffold. It offers high sensitivity and fast kinetics, similar or better than those of current state-of-the-art indicators, with diminished lysosomal accumulation and minimal blue-light photoactivation. Further refinements of the K-GECO1 lineage could lead to further improved variants with overall performance that exceeds that of the most highly optimized red GECIs.