Animals are often bombarded with visual information and must prioritize specific visual features based on their current needs. The neuronal circuits that detect and relay visual features have been well studied. Much less is known about how an animal adjusts its visual attention as its goals or environmental conditions change. During social behaviours, flies need to focus on nearby flies. Here we study how the flow of visual information is altered when female Drosophila enter an aggressive state. From the connectome, we identify three state-dependent circuit motifs poised to modify the response of an aggressive female to fly-sized visual objects: convergence of excitatory inputs from neurons conveying select visual features and internal state; dendritic disinhibition of select visual feature detectors; and a switch that toggles between two visual feature detectors. Using cell-type-specific genetic tools, together with behavioural and neurophysiological analyses, we show that each of these circuit motifs is used during female aggression. We reveal that features of this same switch operate in male Drosophila during courtship pursuit, suggesting that disparate social behaviours may share circuit mechanisms. Our study provides a compelling example of using the connectome to infer circuit mechanisms that underlie dynamic processing of sensory signals.

In the fruit fly, Drosophila melanogaster, connectome data and genetic tools provide a unique opportunity to study complex behaviors including navigation, mating, aggression, and grooming in an organism with a tractable nervous system of 140,000 neurons. Here we present the Fly Disco, a flexible system for high quality video collection, optogenetic manipulation, and fine-grained behavioral analysis of freely walking and socializing fruit fly groups. The data collection hardware and software automates the collection of videos synced to programmable optogenetic stimuli. Key pipeline features include behavioral analysis based on trajectories of 21 keypoints and optogenetic-specific summary statistics and data visualization. We created the multifly dataset for pose estimation that includes 9701 examples enriched in complex behaviors. All hardware designs, software, and the multifly dataset are freely available.

Motor neurons are the final common pathway through which the brain controls movement of the body, forming the basic elements from which all movement is composed. Yet how a single motor neuron contributes to control during natural movement remains unclear. Here we anatomically and functionally characterize the individual roles of the motor neurons that control head movement in the fly, Drosophila melanogaster. Counterintuitively, we find that activity in a single motor neuron rotates the head in different directions, depending on the starting posture of the head, such that the head converges towards a pose determined by the identity of the stimulated motor neuron. A feedback model predicts that this convergent behaviour results from motor neuron drive interacting with proprioceptive feedback. We identify and genetically suppress a single class of proprioceptive neuron that changes the motor neuron-induced convergence as predicted by the feedback model. These data suggest a framework for how the brain controls movements: instead of directly generating movement in a given direction by activating a fixed set of motor neurons, the brain controls movements by adding bias to a continuing proprioceptive-motor loop.

Animals rely on visual motion for navigating the world, and research in flies has clarified how neural circuits extract information from moving visual scenes. However, the major pathways connecting these patterns of optic flow to behavior remain poorly understood. Using a high-throughput quantitative assay of visually guided behaviors and genetic neuronal silencing, we discovered a region in Drosophila’s protocerebrum critical for visual motion following. We used neuronal silencing, calcium imaging, and optogenetics to identify a single cell type, LPC1, that innervates this region, detects translational optic flow, and plays a key role in regulating forward walking. Moreover, the population of LPC1s can estimate the travelling direction, such as when gaze direction diverges from body heading. By linking specific cell types and their visual computations to specific behaviors, our findings establish a foundation for understanding how the nervous system uses vision to guide navigation.

Persistent internal states are important for maintaining survival-promoting behaviors, such as aggression. In female Drosophila melanogaster, we have previously shown that individually activating either aIPg or pC1d cell types can induce aggression. Here we investigate further the individual roles of these cholinergic, sexually dimorphic cell types, and the reciprocal connections between them, in generating a persistent aggressive internal state. We find that a brief 30-second optogenetic stimulation of aIPg neurons was sufficient to promote an aggressive internal state lasting at least 10 minutes, whereas similar stimulation of pC1d neurons did not. While we previously showed that stimulation of pC1e alone does not evoke aggression, persistent behavior could be promoted through simultaneous stimulation of pC1d and pC1e, suggesting an unexpected synergy of these cell types in establishing a persistent aggressive state. Neither aIPg nor pC1d show persistent neuronal activity themselves, implying that the persistent internal state is maintained by other mechanisms. Moreover, inactivation of pC1d did not significantly reduce aIPg-evoked persistent aggression arguing that the aggressive state did not depend on pC1d-aIPg recurrent connectivity. Our results suggest the need for alternative models to explain persistent female aggression.

To control reaching, the nervous system must generate large changes in muscle activation to drive the limb toward the target, and must also make smaller adjustments for precise and accurate behavior. Motor cortex controls the arm through projections to diverse targets across the central nervous system, but it has been challenging to identify the roles of cortical projections to specific targets. Here, we selectively disrupt cortico-cerebellar communication in the mouse by optogenetically stimulating the pontine nuclei in a cued reaching task. This perturbation did not typically block movement initiation, but degraded the precision, accuracy, duration, or success rate of the movement. Correspondingly, cerebellar and cortical activity during movement were largely preserved, but differences in hand velocity between control and stimulation conditions predicted from neural activity were correlated with observed velocity differences. These results suggest that while the total output of motor cortex drives reaching, the cortico-cerebellar loop makes small adjustments that contribute to the successful execution of this dexterous movement.

The Importance Weighted Auto Encoder (IWAE) objective has been shown to improve the training of generative models over the standard Variational Auto Encoder (VAE) objective. Here, we derive importance weighted extensions to Adversarial Variational Bayes (AVB) and Adversarial Autoencoder (AAE). These latent variable models use implicitly defined inference networks whose approximate posterior density qφ(z|x) cannot be directly evaluated, an essential ingredient for importance weighting. We show improved training and inference in latent variable models with our adversarially trained importance weighting method, and derive new theoretical connections between adversarial generative model training criteria and marginal likelihood based methods. We apply these methods to the important problem of inferring spiking neural activity from calcium imaging data, a challenging posterior inference problem in neuroscience, and show that posterior samples from the adversarial methods outperform factorized posteriors used in VAEs.

Aggressive social interactions are used to compete for limited resources and are regulated by complex sensory cues and the organism's internal state. While both sexes exhibit aggression, its neuronal underpinnings are understudied in females. Here, we identify a population of sexually dimorphic aIPg neurons in the adult central brain whose optogenetic activation increased, and genetic inactivation reduced, female aggression. Analysis of GAL4 lines identified in an unbiased screen for increased female chasing behavior revealed the involvement of another sexually dimorphic neuron, pC1d, and implicated aIPg and pC1d neurons as core nodes regulating female aggression. Connectomic analysis demonstrated that aIPg neurons and pC1d are interconnected and suggest that aIPg neurons may exert part of their effect by gating the flow of visual information to descending neurons. Our work reveals important regulatory components of the neuronal circuitry that underlies female aggressive social interactions and provides tools for their manipulation.

In this work, we address the problem of precisely localizing key frames of an action, for example, the precise time that a pitcher releases a baseball, or the precise time that a crowd begins to applaud. Key frame localization is a largely overlooked and important action-recognition problem, for example in the field of neuroscience, in which we would like to understand the neural activity that produces the start of a bout of an action. To address this problem, we introduce a novel structured loss function that properly weights the types of errors that matter in such applications: it more heavily penalizes extra and missed action start detections over small misalignments. Our structured loss is based on the best matching between predicted and labeled action starts. We train recurrent neural networks (RNNs) to minimize differentiable approximations of this loss. To evaluate these methods, we introduce the Mouse Reach Dataset, a large, annotated video dataset of mice performing a sequence of actions. The dataset was collected and labeled by experts for the purpose of neuroscience research. On this dataset, we demonstrate that our method outperforms related approaches and baseline methods using an unstructured loss.

The motor cortex controls skilled arm movement by sending temporal patterns of activity to lower motor centres. Local cortical dynamics are thought to shape these patterns throughout movement execution. External inputs have been implicated in setting the initial state of the motor cortex, but they may also have a pattern-generating role. Here we dissect the contribution of local dynamics and inputs to cortical pattern generation during a prehension task in mice. Perturbing cortex to an aberrant state prevented movement initiation, but after the perturbation was released, cortex either bypassed the normal initial state and immediately generated the pattern that controls reaching or failed to generate this pattern. The difference in these two outcomes was probably a result of external inputs. We directly investigated the role of inputs by inactivating the thalamus; this perturbed cortical activity and disrupted limb kinematics at any stage of the movement. Activation of thalamocortical axon terminals at different frequencies disrupted cortical activity and arm movement in a graded manner. Simultaneous recordings revealed that both thalamic activity and the current state of cortex predicted changes in cortical activity. Thus, the pattern generator for dexterous arm movement is distributed across multiple, strongly interacting brain regions.