Recent success in training artificial agents and robots derives from a combination of direct learning of behavioral policies and indirect learning via value functions. Policy learning and value learning employ distinct algorithms that optimize behavioral performance and reward prediction, respectively. In animals, behavioral learning and the role of mesolimbic dopamine signaling have been extensively evaluated with respect to reward prediction; however, to date there has been little consideration of how direct policy learning might inform our understanding. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioral policies evolve as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioral policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically-calibrated manipulations of mesolimbic dopamine produced multiple effects inconsistent with value learning but predicted by a neural network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioral policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioral policies, expanding the explanatory power of reinforcement learning models for animal learning.

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions. Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction; however, so far there has been little consideration of how direct policy learning might inform our understanding. Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning.

Pioneer transcription factors (PTFs) possess the unique capability to access closed chromatin regions and initiate cell fate changes, yet the underlying mechanisms remain elusive. Here, we characterized the single-molecule dynamics of PTFs targeting chromatin in living cells, revealing a notable 'confined target search' mechanism. PTFs such as FOXA1, FOXA2, SOX2, OCT4 and KLF4 sampled chromatin more frequently than non-PTF MYC, alternating between fast free diffusion in the nucleus and slower confined diffusion within mesoscale zones. Super-resolved microscopy showed closed chromatin organized as mesoscale nucleosome-dense domains, confining FOXA2 diffusion locally and enriching its binding. We pinpointed specific histone-interacting disordered regions, distinct from DNA-binding domains, crucial for confined target search kinetics and pioneer activity within closed chromatin. Fusion to other factors enhanced pioneer activity. Kinetic simulations suggested that transient confinement could increase target association rate by shortening search time and binding repeatedly. Our findings illuminate how PTFs recognize and exploit closed chromatin organization to access targets, revealing a pivotal aspect of gene regulation.

Seconds-scale network states, affecting many neurons within a network, modulate neural activity by complementing fast integration of neuron-specific inputs that arrive in the milliseconds before spiking. Non-rhythmic subthreshold dynamics at intermediate timescales, however, are less well-characterized. We found, using automated whole cell patch clamping in vivo, that spikes recorded in CA1 and barrel cortex in awake mice are often preceded not only by monotonic voltage rises lasting milliseconds, but also by more gradual (lasting 10s-100s of ms) depolarizations. The latter exert a gating function on spiking, in a fashion that depends on the gradual rise duration: the probability of spiking was higher for longer gradual rises, even controlling for the amplitude of the gradual rises. Barrel cortex double-autopatch recordings show that gradual rises are shared across some but not all neurons. The gradual rises may represent a new kind of state, intermediate both in timescale and in proportion of neurons participating, which gates a neuron's ability to respond to subsequent inputs.

The Drosophila Methoprene-tolerant (Met) and Germ cell-expressed (Gce) bHLH-PAS transcription factors are products of two paralogous genes. Both proteins potentially mediate the effect of juvenile hormone (JH) as candidate JH receptors. Here we report that Met and Gce are partially redundant in transducing JH action. Both Met and gce null single mutants are fully viable, but the Met gce double mutant, Met(27) gce(2.5k), dies during the larval-pupal transition. Precocious and enhanced caspase-dependent programmed cell death (PCD) appears in fat body cells of Met(27) gce(2.5k) during the early larval stages. Expression of Kr-h1, a JH response gene that inhibits 20-hydroxyecdysone (20E)-induced broad (br) expression, is abolished in Met(27) gce(2.5k) during larval molts. Consequently, expression of br occurs precociously in Met(27) gce(2.5k), which may cause precocious caspase-dependent PCD during the early larval stages. Defective phenotypes and gene expression changes in Met(27) gce(2.5k) double mutants are similar to those found in JH-deficient animals. Importantly, exogenous application of JH agonists rescued the JH-deficient animals but not the Met(27) gce(2.5k) mutants. Our data suggest a model in which Drosophila Met and Gce redundantly transduce JH action to prevent 20E-induced caspase-dependent PCD during larval molts by induction of Kr-h1 expression and inhibition of br expression.

Animals can learn general task structures and use them to solve new problems with novel sensory specifics. This capacity of ‘learning to learn’, or meta-learning, is difficult to achieve in artificial systems, and the mechanisms by which it is achieved in animals are unknown. As a step toward enabling mechanistic studies, we developed a behavioral paradigm that demonstrates meta-learning in head-fixed mice. We trained mice to perform a two-alternative forced-choice task in virtual reality (VR), and successively changed the visual cues that signaled reward location. Mice showed increased learning speed in both cue generalization and serial reversal tasks. During reversal learning, behavior exhibited sharp transitions, with the transition occurring earlier in each successive reversal. Analysis of motor patterns revealed that animals utilized similar motor programs to execute the same actions in response to different cues but modified the motor programs during reversal learning. Our study demonstrates that mice can perform meta-learning tasks in VR, thus opening up opportunities for future mechanistic studies.

Mitochondria-derived reactive oxygen species (mROS) are required for the survival, proliferation, and metastasis of cancer cells. The mechanism by which mitochondrial metabolism regulates mROS levels to support cancer cells is not fully understood. To address this, we conducted a metabolism-focused CRISPR-Cas9 genetic screen and uncovered that loss of genes encoding subunits of mitochondrial complex I was deleterious in the presence of the mitochondria-targeted antioxidant mito-vitamin E (MVE). Genetic or pharmacologic inhibition of mitochondrial complex I in combination with the mitochondria-targeted antioxidants, MVE or MitoTEMPO, induced a robust integrated stress response (ISR) and markedly diminished cell survival and proliferation in vitro. This was not observed following inhibition of mitochondrial complex III. Administration of MitoTEMPO in combination with the mitochondrial complex I inhibitor phenformin decreased the leukemic burden in a mouse model of T cell acute lymphoblastic leukemia. Thus, mitochondrial complex I is a dominant metabolic determinant of mROS-dependent cellular fitness.

The thirteen nuclear cleavages that give rise to the Drosophila blastoderm are some of the fastest known cell cycles. Surprisingly, the fertilized egg is provided with at most one-third of the dNTPs needed to complete the thirteen rounds of DNA replication. The rest must be synthesized by the embryo, concurrent with cleavage divisions. What is the reason for the limited supply of DNA building blocks? We propose that frugal control of dNTP synthesis contributes to the well-characterized deceleration of the cleavage cycles and is needed for robust accumulation of zygotic gene products. In support of this model, we demonstrate that when the levels of dNTPs are abnormally high, nuclear cleavages fail to sufficiently decelerate, the levels of zygotic transcription are dramatically reduced, and the embryo catastrophically fails early in gastrulation. Our work reveals a direct connection between metabolism, the cell cycle, and zygotic transcription.

A specialist neuron uses an intriguing process to help control the body's response to hunger. A lipid pathway involving the breakdown of cellular components regulates the expression of a neuropeptide that affects feeding and body weight.