In order to study anatomy of organisms with high-resolution there is an increasing demand to image large specimen in three dimensions (3D). Confocal microscopy is able to produce high-resolution 3D images, but these are limited by its relatively small field of view compared to the size of large biological specimens. To overcome this drawback, motorized stages moving the sample are used to create a tiled scan of the whole specimen. The physical coordinates provided by the microscope stage are not precise enough to allow reconstruction (”Stitching”) of the whole image from individual image stacks.
We developed an algorithm, as well as an ImageJ plug-in, based on the Fourier Shift Theorem that computes all possible translations (x, y, z) between two 3D images at once, yielding the best overlap in terms of the cross correlation measure. Apart from the obvious gain in computation time it has the advantage that it cannot be trapped in local minima as it simply computes all possible solutions. Computing the overlap between two adjacent image stacks is fast (12 seconds for two 512x512x89 images on a Intel ® Core2Duo with 2.2GHz) making it suitable for real time use, i.e. computing the output image during acquisition of the individual image stacks.
To compensate the possible shading- and brightness differences we apply a smooth linear intensity transition between the overlapping stacks. Additionally we extended the to generic 3D registration using gradient based rotation detection on top of the phase correlation method. We demonstrate the performance of our 3D stitching plug-in on several tiled confocal images and show an example of its application for 3D registration.

Fiji is a distribution of the popular open-source software ImageJ focused on biological-image analysis. Fiji uses modern software engineering practices to combine powerful software libraries with a broad range of scripting languages to enable rapid prototyping of image-processing algorithms. Fiji facilitates the transformation of new algorithms into ImageJ plugins that can be shared with end users through an integrated update system. We propose Fiji as a platform for productive collaboration between computer science and biology research communities.

MOTIVATION: Modern anatomical and developmental studies often require high-resolution imaging of large specimens in three dimensions (3D). Confocal microscopy produces high-resolution 3D images, but is limited by a relatively small field of view compared with the size of large biological specimens. Therefore, motorized stages that move the sample are used to create a tiled scan of the whole specimen. The physical coordinates provided by the microscope stage are not precise enough to allow direct reconstruction (Stitching) of the whole image from individual image stacks.

RESULTS: To optimally stitch a large collection of 3D confocal images, we developed a method that, based on the Fourier Shift Theorem, computes all possible translations between pairs of 3D images, yielding the best overlap in terms of the cross-correlation measure and subsequently finds the globally optimal configuration of the whole group of 3D images. This method avoids the propagation of errors by consecutive registration steps. Additionally, to compensate the brightness differences between tiles, we apply a smooth, non-linear intensity transition between the overlapping images. Our stitching approach is fast, works on 2D and 3D images, and for small image sets does not require prior knowledge about the tile configuration.

AVAILABILITY: The implementation of this method is available as an ImageJ plugin distributed as a part of the Fiji project (Fiji is just ImageJ: http://pacific.mpi-cbg.de/).

Graph neural networks trained to predict observable dynamics can be used to decompose the temporal activity of complex heterogeneous systems into simple, interpretable representations. Here we apply this framework to simulated neural assemblies with thousands of neurons and demonstrate that it can jointly reveal the connectivity matrix, the neuron types, the signaling functions, and in some cases hidden external stimuli. In contrast to existing machine learning approaches such as recurrent neural networks and transformers, which emphasize predictive accuracy but offer limited interpretability, our method provides both reliable forecasts of neural activity and interpretable decomposition of the mechanisms governing large neural assemblies.

The Drosophila brain is formed by an invariant set of lineages, each of which is derived from a unique neural stem cell (neuroblast) and forms a genetic and structural unit of the brain. The task of reconstructing brain circuitry at the level of individual neurons can be made significantly easier by assigning neurons to their respective lineages. In this article we address the automation of neuron and lineage identification. We focused on the Drosophila brain lineages at the larval stage when they form easily recognizable secondary axon tracts (SATs) that were previously partially characterized. We now generated an annotated digital database containing all lineage tracts reconstructed from five registered wild-type brains, at higher resolution and including some that were previously not characterized. We developed a method for SAT structural comparisons based on a dynamic programming approach akin to nucleotide sequence alignment and a machine learning classifier trained on the annotated database of reference SATs. We quantified the stereotypy of SATs by measuring the residual variability of aligned wild-type SATs. Next, we used our method for the identification of SATs within wild-type larval brains, and found it highly accurate (93-99%). The method proved highly robust for the identification of lineages in mutant brains and in brains that differed in developmental time or labeling. We describe for the first time an algorithm that quantifies neuronal projection stereotypy in the Drosophila brain and use the algorithm for automatic neuron and lineage recognition.

MOTIVATION: Serial section microscopy is an established method for detailed anatomy reconstruction of biological specimen. During the last decade, high resolution electron microscopy (EM) of serial sections has become the de-facto standard for reconstruction of neural connectivity at ever increasing scales (EM connectomics). In serial section microscopy, the axial dimension of the volume is sampled by physically removing thin sections from the embedded specimen and subsequently imaging either the block-face or the section series. This process has limited precision leading to inhomogeneous non-planar sampling of the axial dimension of the volume which, in turn, results in distorted image volumes. This includes that section series may be collected and imaged in unknown order.

RESULTS: We developed methods to identify and correct these distortions through image-based signal analysis without any additional physical apparatus or measurements. We demonstrate the efficacy of our methods in proof of principle experiments and application to real world problems.

AVAILABILITY AND IMPLEMENTATION: We made our work available as libraries for the ImageJ distribution Fiji and for deployment in a high performance parallel computing environment. Our sources are open and available at http://github.com/saalfeldlab/section-sort, http://github.com/saalfeldlab/z-spacing and http://github.com/saalfeldlab/z-spacing-spark CONTACT: : saalfelds@janelia.hhmi.orgSupplementary information: Supplementary data are available at Bioinformatics online.

SUMMARY: ImgLib2 is an open-source Java library for n-dimensional data representation and manipulation with focus on image processing. It aims at minimizing code duplication by cleanly separating pixel-algebra, data access and data representation in memory. Algorithms can be implemented for classes of pixel types and generic access patterns by which they become independent of the specific dimensionality, pixel type and data representation. ImgLib2 illustrates that an elegant high-level programming interface can be achieved without sacrificing performance. It provides efficient implementations of common data types, storage layouts and algorithms. It is the data model underlying ImageJ2, the KNIME Image Processing toolbox and an increasing number of Fiji-Plugins.

AVAILABILITY: ImgLib2 is licensed under BSD. Documentation and source code are available at http://imglib2.net and in a public repository at https://github.com/imagej/imglib.

SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics Online.

CONTACT: saalfeld@mpi-cbg.de

The point spread function (PSF) is fundamental to any type of microscopy, most importantly so for single-molecule localization techniques, where the exact PSF shape is crucial for precise molecule localization at the nanoscale. Optical aberrations and fixed fluorophore dipoles often result in non-isotropic and distorted PSFs, impairing and biasing conventional fitting approaches. Further, PSF shapes are deliberately modified in PSF engineering approaches for providing improved sensitivity, e.g., for 3D localization or determination of dipole orientation. As this can lead to highly complex PSF shapes, a tool for visualizing expected PSFs would facilitate the interpretation of obtained data and the design of experimental approaches. To this end, we introduce a comprehensive and accessible computer application that allows for the simulation of realistic PSFs based on the full vectorial PSF model. Our tool incorporates a wide range of microscope and fluorophore parameters, including orientationally constrained fluorophores, as well as custom aberrations, transmission and phase masks, thus enabling an accurate representation of various imaging conditions. An additional feature is the simulation of crowded molecular environments with overlapping PSFs. Further, our app directly provides the Cramér–Rao bound for assessing the best achievable localization precision under given conditions. Finally, our software allows for the fitting of custom aberrations directly from experimental data, as well as the generation of a large dataset with randomized simulation parameters, effectively bridging the gap between simulated and experimental scenarios, and enhancing experimental design and result validation.

The purpose of ImgLib, a Generic Java Image Processing Library, is to provide an abstract framework enabling Java developers to design and implement data processing algorithms without having to consider dimensionality, type of data (e. g. byte, float, complex float), or strategies for data access (e. g. linear arrays, cells, paged cells). This kind of programming has significant advantages over the classical way. An algorithm written once for a certain class of Type will potentially run on any compatible Type, even if it does not exist yet. Same applies for data access strategies and the number of dimensions.
We achieve this abstraction by accessing data through Iterators and Type interfaces. Iterators guarantee e fficient traversal through pixels depending on whether random coordinate access is required or just all pixels have to be visited once, whether real or integer coordinates are accessed, whether coordinates outside of image boundaries are accessed or not. Type interfaces define the supported operators on pixel values (like basic algebra) and hide the underlying basic type from algorithm implementation.