MOTIVATION: Modern anatomical and developmental studies often require high-resolution imaging of large specimens in three dimensions (3D). Confocal microscopy produces high-resolution 3D images, but is limited by a relatively small field of view compared with the size of large biological specimens. Therefore, motorized stages that move the sample are used to create a tiled scan of the whole specimen. The physical coordinates provided by the microscope stage are not precise enough to allow direct reconstruction (Stitching) of the whole image from individual image stacks.

RESULTS: To optimally stitch a large collection of 3D confocal images, we developed a method that, based on the Fourier Shift Theorem, computes all possible translations between pairs of 3D images, yielding the best overlap in terms of the cross-correlation measure and subsequently finds the globally optimal configuration of the whole group of 3D images. This method avoids the propagation of errors by consecutive registration steps. Additionally, to compensate the brightness differences between tiles, we apply a smooth, non-linear intensity transition between the overlapping images. Our stitching approach is fast, works on 2D and 3D images, and for small image sets does not require prior knowledge about the tile configuration.

AVAILABILITY: The implementation of this method is available as an ImageJ plugin distributed as a part of the Fiji project (Fiji is just ImageJ: http://pacific.mpi-cbg.de/).

The Drosophila brain is formed by an invariant set of lineages, each of which is derived from a unique neural stem cell (neuroblast) and forms a genetic and structural unit of the brain. The task of reconstructing brain circuitry at the level of individual neurons can be made significantly easier by assigning neurons to their respective lineages. In this article we address the automation of neuron and lineage identification. We focused on the Drosophila brain lineages at the larval stage when they form easily recognizable secondary axon tracts (SATs) that were previously partially characterized. We now generated an annotated digital database containing all lineage tracts reconstructed from five registered wild-type brains, at higher resolution and including some that were previously not characterized. We developed a method for SAT structural comparisons based on a dynamic programming approach akin to nucleotide sequence alignment and a machine learning classifier trained on the annotated database of reference SATs. We quantified the stereotypy of SATs by measuring the residual variability of aligned wild-type SATs. Next, we used our method for the identification of SATs within wild-type larval brains, and found it highly accurate (93-99%). The method proved highly robust for the identification of lineages in mutant brains and in brains that differed in developmental time or labeling. We describe for the first time an algorithm that quantifies neuronal projection stereotypy in the Drosophila brain and use the algorithm for automatic neuron and lineage recognition.

MOTIVATION: Serial section microscopy is an established method for detailed anatomy reconstruction of biological specimen. During the last decade, high resolution electron microscopy (EM) of serial sections has become the de-facto standard for reconstruction of neural connectivity at ever increasing scales (EM connectomics). In serial section microscopy, the axial dimension of the volume is sampled by physically removing thin sections from the embedded specimen and subsequently imaging either the block-face or the section series. This process has limited precision leading to inhomogeneous non-planar sampling of the axial dimension of the volume which, in turn, results in distorted image volumes. This includes that section series may be collected and imaged in unknown order.

RESULTS: We developed methods to identify and correct these distortions through image-based signal analysis without any additional physical apparatus or measurements. We demonstrate the efficacy of our methods in proof of principle experiments and application to real world problems.

AVAILABILITY AND IMPLEMENTATION: We made our work available as libraries for the ImageJ distribution Fiji and for deployment in a high performance parallel computing environment. Our sources are open and available at http://github.com/saalfeldlab/section-sort, http://github.com/saalfeldlab/z-spacing and http://github.com/saalfeldlab/z-spacing-spark CONTACT: : saalfelds@janelia.hhmi.orgSupplementary information: Supplementary data are available at Bioinformatics online.

SUMMARY: ImgLib2 is an open-source Java library for n-dimensional data representation and manipulation with focus on image processing. It aims at minimizing code duplication by cleanly separating pixel-algebra, data access and data representation in memory. Algorithms can be implemented for classes of pixel types and generic access patterns by which they become independent of the specific dimensionality, pixel type and data representation. ImgLib2 illustrates that an elegant high-level programming interface can be achieved without sacrificing performance. It provides efficient implementations of common data types, storage layouts and algorithms. It is the data model underlying ImageJ2, the KNIME Image Processing toolbox and an increasing number of Fiji-Plugins.

AVAILABILITY: ImgLib2 is licensed under BSD. Documentation and source code are available at http://imglib2.net and in a public repository at https://github.com/imagej/imglib.

SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics Online.

CONTACT: saalfeld@mpi-cbg.de

The point spread function (PSF) is fundamental to any type of microscopy, most importantly so for single-molecule localization techniques, where the exact PSF shape is crucial for precise molecule localization at the nanoscale. Optical aberrations and fixed fluorophore dipoles often result in non-isotropic and distorted PSFs, impairing and biasing conventional fitting approaches. Further, PSF shapes are deliberately modified in PSF engineering approaches for providing improved sensitivity, e.g., for 3D localization or determination of dipole orientation. As this can lead to highly complex PSF shapes, a tool for visualizing expected PSFs would facilitate the interpretation of obtained data and the design of experimental approaches. To this end, we introduce a comprehensive and accessible computer application that allows for the simulation of realistic PSFs based on the full vectorial PSF model. Our tool incorporates a wide range of microscope and fluorophore parameters, including orientationally constrained fluorophores, as well as custom aberrations, transmission and phase masks, thus enabling an accurate representation of various imaging conditions. An additional feature is the simulation of crowded molecular environments with overlapping PSFs. Further, our app directly provides the Cramér–Rao bound for assessing the best achievable localization precision under given conditions. Finally, our software allows for the fitting of custom aberrations directly from experimental data, as well as the generation of a large dataset with randomized simulation parameters, effectively bridging the gap between simulated and experimental scenarios, and enhancing experimental design and result validation.

The purpose of ImgLib, a Generic Java Image Processing Library, is to provide an abstract framework enabling Java developers to design and implement data processing algorithms without having to consider dimensionality, type of data (e. g. byte, float, complex float), or strategies for data access (e. g. linear arrays, cells, paged cells). This kind of programming has significant advantages over the classical way. An algorithm written once for a certain class of Type will potentially run on any compatible Type, even if it does not exist yet. Same applies for data access strategies and the number of dimensions.
We achieve this abstraction by accessing data through Iterators and Type interfaces. Iterators guarantee e fficient traversal through pixels depending on whether random coordinate access is required or just all pixels have to be visited once, whether real or integer coordinates are accessed, whether coordinates outside of image boundaries are accessed or not. Type interfaces define the supported operators on pixel values (like basic algebra) and hide the underlying basic type from algorithm implementation.

Large electron microscopy image datasets for connectomics are typically composed of thousands to millions of partially overlapping two-dimensional images (tiles), which must be registered into a coherent volume prior to further analysis. A common registration strategy is to find matching features between neighboring and overlapping image pairs, followed by a numerical estimation of optimal image deformation using a so-called solver program. 
Existing solvers are inadequate for large data volumes, and inefficient for small-scale image registration. 
In this work, an efficient and accurate matrix-based solver method is presented. A linear system is constructed that combines minimization of feature-pair square distances with explicit constraints in a regularization term. In absence of reliable priors for regularization, we show how to construct a rigid-model approximation to use as prior. The linear system is solved using available computer programs, whose performance on typical registration tasks we briefly compare, and to which future scale-up is delegated. Our method is applied to the joint alignment of 2.67 million images, with more than 200 million point-pairs and has been used for successfully aligning the first full adult fruit fly brain.

We present a method combining affinity prediction with region agglomeration, which improves significantly upon the state of the art of neuron segmentation from electron microscopy (EM) in accuracy and scalability. Our method consists of a 3D U-net, trained to predict affinities between voxels, followed by iterative region agglomeration. We train using a structured loss based on MALIS, encouraging topologically correct segmentations obtained from affinity thresholding. Our extension consists of two parts: First, we present a quasi-linear method to compute the loss gradient, improving over the original quadratic algorithm. Second, we compute the gradient in two separate passes to avoid spurious gradient contributions in early training stages. Our predictions are accurate enough that simple learning-free percentile-based agglomeration outperforms more involved methods used earlier on inferior predictions. We present results on three diverse EM datasets, achieving relative improvements over previous results of 27%, 15%, and 250%. Our findings suggest that a single method can be applied to both nearly isotropic block-face EM data and anisotropic serial sectioned EM data. The runtime of our method scales linearly with the size of the volume and achieves a throughput of ~2.6 seconds per megavoxel, qualifying our method for the processing of very large datasets.