Object detection and classification are key tasks in computer vision that can facilitate high-throughput image analysis of microscopy data. We present a set of local image descriptors for three-dimensional (3D) microscopy datasets inspired by the well-known Haar wavelet framework. We add orientation, illumination and scale information by assuming that the neighborhood surrounding points of interests in the image can be described with ellipsoids, and we increase discriminative power by incorporating edge and shape information into the features. The calculation of the local image descriptors is implemented in a Graphics Processing Unit (GPU) in order to reduce computation time to 1 millisecond per object of interest. We present results for cell division detection in 3D time-lapse fluorescence microscopy with 97.6% accuracy.

A crucial issue in studies of morphogen gradients relates to their range: the distance over which they can act as direct regulators of cell signaling, gene expression and cell differentiation. To address this, we present a straightforward statistical framework that can be used in multiple developmental systems. We illustrate the developed approach by providing a point estimate and confidence interval for the spatial range of the graded distribution of nuclear Dorsal, a transcription factor that controls the dorsoventral pattern of the Drosophila embryo.

Pushing the frontier of fluorescence microscopy requires the design of enhanced fluorophores with finely tuned properties. We recently discovered that incorporation of four-membered azetidine rings into classic fluorophore structures elicits substantial increases in brightness and photostability, resulting in the Janelia Fluor (JF) series of dyes. We refined and extended this strategy, finding that incorporation of 3-substituted azetidine groups allows rational tuning of the spectral and chemical properties of rhodamine dyes with unprecedented precision. This strategy allowed us to establish principles for fine-tuning the properties of fluorophores and to develop a palette of new fluorescent and fluorogenic labels with excitation ranging from blue to the far-red. Our results demonstrate the versatility of these new dyes in cells, tissues and animals.

We describe the implementation and use of an adaptive imaging framework for optimizing spatial resolution and signal strength in a light-sheet microscope. The framework, termed AutoPilot, comprises hardware and software modules for automatically measuring and compensating for mismatches between light-sheet and detection focal planes in living specimens. Our protocol enables researchers to introduce adaptive imaging capabilities in an existing light-sheet microscope or use our SiMView microscope blueprint to set up a new adaptive multiview light-sheet microscope. The protocol describes (i) the mechano-optical implementation of the adaptive imaging hardware, including technical drawings for all custom microscope components; (ii) the algorithms and software library for automated adaptive imaging, including the pseudocode and annotated source code for all software modules; and (iii) the execution of adaptive imaging experiments, as well as the configuration and practical use of the AutoPilot framework. Setup of the adaptive imaging hardware and software takes 1-2 weeks each. Previous experience with light-sheet microscopy and some familiarity with software engineering and building of optical instruments are recommended. Successful implementation of the protocol recovers near diffraction-limited performance in many parts of typical multicellular organisms studied with light-sheet microscopy, such as fruit fly and zebrafish embryos, for which resolution and signal strength are improved two- to fivefold.

Mechanics plays a key role in the development of higher organisms. However, understanding this relationship is complicated by the difficulty of modeling the link between local forces generated at the subcellular level and deformations observed at the tissue and whole-embryo levels. Here we propose an approach first developed for lipid bilayers and cell membranes, in which force-generation by cytoskeletal elements enters a continuum mechanics formulation for the full system in the form of local changes in preferred curvature. This allows us to express and solve the system using only tissue strains. Locations of preferred curvature are simply related to products of gene expression. A solution, in that context, means relaxing the system’s mechanical energy to yield global morphogenetic predictions that accommodate a tendency toward the local preferred curvature, without a need to explicitly model force-generation mechanisms at the molecular level. Our computational framework, which we call SPHARM-MECH, extends a 3D spherical harmonics parameterization known as SPHARM to combine this level of abstraction with a sparse shape representation. The integration of these two principles allows computer simulations to be performed in three dimensions on highly complex shapes, gene expression patterns, and mechanical constraints. We demonstrate our approach by modeling mesoderm invagination in the fruit-fly embryo, where local forces generated by the acto-myosin meshwork in the region of the future mesoderm lead to formation of a ventral tissue fold. The process is accompanied by substantial changes in cell shape and long-range cell movements. Applying SPHARM-MECH to whole-embryo live imaging data acquired with light-sheet microscopy reveals significant correlation between calculated and observed tissue movements. Our analysis predicts the observed cell shape anisotropy on the ventral side of the embryo and suggests an active mechanical role of mesoderm invagination in supporting the onset of germ-band extension.

Optimal image quality in light-sheet microscopy requires a perfect overlap between the illuminating light sheet and the focal plane of the detection objective. However, mismatches between the light-sheet and detection planes are common owing to the spatiotemporally varying optical properties of living specimens. Here we present the AutoPilot framework, an automated method for spatiotemporally adaptive imaging that integrates (i) a multi-view light-sheet microscope capable of digitally translating and rotating light-sheet and detection planes in three dimensions and (ii) a computational method that continuously optimizes spatial resolution across the specimen volume in real time. We demonstrate long-term adaptive imaging of entire developing zebrafish (Danio rerio) and Drosophila melanogaster embryos and perform adaptive whole-brain functional imaging in larval zebrafish. Our method improves spatial resolution and signal strength two to five-fold, recovers cellular and sub-cellular structures in many regions that are not resolved by non-adaptive imaging, adapts to spatiotemporal dynamics of genetically encoded fluorescent markers and robustly optimizes imaging performance during large-scale morphogenetic changes in living organisms.

We present a method to automatically identify and track nuclei in time-lapse microscopy recordings of entire developing embryos. The method combines deep learning and global optimization. On a mouse dataset, it reconstructs 75.8% of cell lineages spanning 1 h, as compared to 31.8% for the competing method. Our approach improves understanding of where and when cell fate decisions are made in developing embryos, tissues, and organs.

We present a method to automatically identify and track nuclei in time-lapse microscopy recordings of entire developing embryos. The method combines deep learning and global optimization. On a mouse dataset, it reconstructs 75.8% of cell lineages spanning 1 h, as compared to 31.8% for the competing method. Our approach improves understanding of where and when cell fate decisions are made in developing embryos, tissues, and organs.

Light-sheet imaging of cleared and expanded samples creates terabyte-sized datasets that consist of many unaligned three-dimensional image tiles, which must be reconstructed before analysis. We developed the BigStitcher software to address this challenge. BigStitcher enables interactive visualization, fast and precise alignment, spatially resolved quality estimation, real-time fusion and deconvolution of dual-illumination, multitile, multiview datasets. The software also compensates for optical effects, thereby improving accuracy and enabling subsequent biological analysis.

Whole-brain imaging allows for comprehensive functional mapping of distributed neural pathways, but neuronal perturbation experiments are usually limited to targeting predefined regions or genetically identifiable cell types. To complement whole-brain measures of activity with brain-wide manipulations for testing causal interactions, we introduce a system that uses measuredactivity patterns to guide optical perturbations of any subset of neurons in the same fictively behaving larval zebrafish. First, a light-sheet microscope collects whole-brain data that are rapidly analyzed by a distributed computing system to generate functional brain maps. On the basis of these maps, the experimenter can then optically ablate neurons and image activity changes across the brain. We applied this method to characterize contributions of behaviorally tuned populations to the optomotor response. We extended the system to optogenetically stimulate arbitrary subsets of neurons during whole-brain imaging. These open-source methods enable delineating the contributions of neurons to brain-wide circuit dynamics and behavior in individual animals.