A general method to recognize and track unmarked animals within a population will enable new studies of social behavior and individuality.

Hb9 is a homeodomain-containing transcription factor that acts in combination with Nkx6, Lim3, and Tail-up (Islet) to guide the stereotyped differentiation, connectivity, and function of a subset of neurons in Drosophila. The role of Hb9 in directing neuronal differentiation is well documented, but the lineage of Hb9(+) neurons is only partly characterized, its regulation is poorly understood, and most of the downstream genes through which it acts remain at large. Here, we complete the lineage tracing of all embryonic Hb9(+) neurons (to eight neuronal lineages) and provide evidence that hb9, lim3, and tail-up are coordinately regulated by a common set of upstream factors. Through the parallel use of micro-array gene expression profiling and the Dam-ID method, we searched for Hb9-regulated genes, uncovering transcription factors as the most over-represented class of genes regulated by Hb9 (and Nkx6) in the CNS. By a nearly ten-to-one ratio, Hb9 represses rather than activates transcription factors, highlighting transcriptional repression of other transcription factors as a core mechanism by which Hb9 governs neuronal determination. From the small set of genes activated by Hb9, we characterized the expression and function of two - fd59a/foxd, which encodes a transcription factor, and Nitric oxide synthase. Under standard lab conditions, both genes are dispensable for Drosophila development, but Nos appears to inhibit hyper-active behavior and fd59a appears to act in octopaminergic neurons to control egg-laying behavior. Together our data clarify the mechanisms through which Hb9 governs neuronal specification and differentiation and provide an initial characterization of the expression and function of Nos and fd59a in the Drosophila CNS.

We present a machine learning–based system for automatically computing interpretable, quantitative measures of animal behavior. Through our interactive system, users encode their intuition about behavior by annotating a small set of video frames. These manual labels are converted into classifiers that can automatically annotate behaviors in screen-scale data sets. Our general-purpose system can create a variety of accurate individual and social behavior classifiers for different organisms, including mice and adult and larval Drosophila.

An important role of visual systems is to detect nearby predators, prey, and potential mates [1], which may be distinguished in part by their motion. When an animal is at rest, an object moving in any direction may easily be detected by motion-sensitive visual circuits [2, 3]. During locomotion, however, this strategy is compromised because the observer must detect a moving object within the pattern of optic flow created by its own motion through the stationary background. However, objects that move creating back-to-front (regressive) motion may be unambiguously distinguished from stationary objects because forward locomotion creates only front-to-back (progressive) optic flow. Thus, moving animals should exhibit an enhanced sensitivity to regressively moving objects. We explicitly tested this hypothesis by constructing a simple fly-sized robot that was programmed to interact with a real fly. Our measurements indicate that whereas walking female flies freeze in response to a regressively moving object, they ignore a progressively moving one. Regressive motion salience also explains observations of behaviors exhibited by pairs of walking flies. Because the assumptions underlying the regressive motion salience hypothesis are general, we suspect that the behavior we have observed in Drosophila may be widespread among eyed, motile organisms.

Automated tracking of animal movement allows analyses that would not otherwise be possible by providing great quantities of data. The additional capability of tracking in real time–with minimal latency–opens up the experimental possibility of manipulating sensory feedback, thus allowing detailed explorations of the neural basis for control of behaviour. Here, we describe a system capable of tracking the three-dimensional position and body orientation of animals such as flies and birds. The system operates with less than 40 ms latency and can track multiple animals simultaneously. To achieve these results, a multi-target tracking algorithm was developed based on the extended Kalman filter and the nearest neighbour standard filter data association algorithm. In one implementation, an 11-camera system is capable of tracking three flies simultaneously at 60 frames per second using a gigabit network of nine standard Intel Pentium 4 and Core 2 Duo computers. This manuscript presents the rationale and details of the algorithms employed and shows three implementations of the system. An experiment was performed using the tracking system to measure the effect of visual contrast on the flight speed of Drosophila melanogaster. At low contrasts, speed is more variable and faster on average than at high contrasts. Thus, the system is already a useful tool to study the neurobiology and behaviour of freely flying animals. If combined with other techniques, such as ’virtual reality’-type computer graphics or genetic manipulation, the tracking system would offer a powerful new way to investigate the biology of flying animals.

We present a camera-based method for automatically quantifying the individual and social behaviors of fruit flies, Drosophila melanogaster, interacting in a planar arena. Our system includes machine-vision algorithms that accurately track many individuals without swapping identities and classification algorithms that detect behaviors. The data may be represented as an ethogram that plots the time course of behaviors exhibited by each fly or as a vector that concisely captures the statistical properties of all behaviors displayed in a given period. We found that behavioral differences between individuals were consistent over time and were sufficient to accurately predict gender and genotype. In addition, we found that the relative positions of flies during social interactions vary according to gender, genotype and social environment. We expect that our software, which permits high-throughput screening, will complement existing molecular methods available in Drosophila, facilitating new investigations into the genetic and cellular basis of behavior.

We present a particle filtering algorithm for robustly tracking the contours of multiple deformable objects through severe occlusions. Our algorithm combines a multiple blob tracker with a contour tracker in a manner that keeps the required number of samples small. This is a natural combination because both algorithms have complementary strengths. The multiple blob tracker uses a natural multi-target model and searches a smaller and simpler space. On the other hand, contour tracking gives more fine-tuned results and relies on cues that are available during severe occlusions. Our choice of combination of these two algorithms accentuates the advantages of each. We demonstrate good performance on challenging video of three identical mice that contains multiple instances of severe occlusion.