Publications

Sample-induced aberrations and optical imperfections limit the resolution of fluorescence microscopy. Phase diversity is a powerful technique that leverages complementary phase information in sequentially acquired images with deliberately introduced aberrations--the phase diversities--to enable phase and object reconstruction and restore diffraction-limited resolution. These phase diversities are typically introduced into the optical path via a deformable mirror. Existing phase-diversity-based methods are limited to Zernike modes, require large numbers of diversity images, or depend on accurate mirror calibration--which are all suboptimal. We present DeepPD, a deep learning-based framework that combines neural representations of the object and wavefront with a learned model of the deformable mirror to jointly estimate both object and phase from only five images. DeepPD improves robustness and reconstruction quality over previous approaches, even under severe aberrations. We demonstrate its performance on calibration targets and biological samples, including immunolabeled myosin in fixed PtK2 cells.

Modern microscopy methods incorporate computational modeling of optical systems as an integral part of the imaging process, either to solve inverse problems or enable optimization of the optical system design. These methods often depend on differentiable simulations of optical systems, yet no standardized framework exists—forcing computational optics researchers to repeatedly and independently implement simulations that are prone to errors, difficult to reuse in other applications, and often computationally suboptimal. These common problems limit the potential impact of computational optics as a field. We present Chromatix: an open-source, GPU-accelerated differentiable wave optics library. Chromatix builds on JAX to enable fast simulation of diverse optical systems and inverse problem solving, scaling these simulations from single-CPU laptops to multi-GPU servers. The library implements various optical elements (e.g., lenses, polarizers and spatial light modulators) and multiple light propagation models (e.g., Fresnel approximation, angular spectrum and off-axis propagation) that can be flexibly combined to model various computational optics applications such as snapshot microscopy, holography, and phase retrieval of multiple scattering samples. These simulations can be automatically parallelized to scale across multiple GPUs with a single-line change to the modeling code, enabling simulation and optimization of previously impractical optical system designs. We demonstrate Chromatix’s capacity to substantially accelerate optics simulation and optimization on existing methods in computational optics, speeding up optical simulation and optimization from 2-6× on a single GPU to up to 22× on 8 GPUs (depending on the particular system being modeled) compared to the original implementations. Chromatix establishes a standard for wave optics simulations, democratizing access to and expanding the design space of computational optics.i

Fluorescence microscopy is an invaluable tool in biology, yet its performance is compromised when the wavefront of light is distorted due to optical imperfections or the refractile nature of the sample. Such optical aberrations can dramatically lower the information content of images by degrading image contrast, resolution, and signal. Adaptive optics (AO) methods can sense and subsequently cancel the aberrated wavefront, but are too complex, inefficient, slow, or expensive for routine adoption by most labs. Here we introduce a rapid, sensitive, and robust wavefront sensing scheme based on phase diversity, a method successfully deployed in astronomy but underused in microscopy. Our method enables accurate wavefront sensing to less than λ/35 root mean square (RMS) error with few measurements, and AO with no additional hardware besides a corrective element. After validating the method with simulations, we demonstrate calibration of a deformable mirror > 100-fold faster than comparable methods (corresponding to wavefront sensing on the ~100 ms scale), and sensing and subsequent correction of severe aberrations (RMS wavefront distortion exceeding λ/2), restoring diffraction-limited imaging on extended biological samples.

Natural physical, chemical, and biological dynamical systems are often complex, with heterogeneous components interacting in diverse ways. We show that graph neural networks can be designed to jointly learn the interaction rules and the structure of the heterogeneity from data alone. The learned latent structure and dynamics can be used to virtually decompose the complex system which is necessary to parameterize and infer the underlying governing equations. We tested the approach with simulation experiments of moving particles and vector fields that interact with each other. While our current aim is to better understand and validate the approach with simulated data, we anticipate it to become a generally applicable tool to uncover the governing rules underlying complex dynamics observed in nature.

The point spread function (PSF) is fundamental to any type of microscopy, most importantly so for single-molecule localization techniques, where the exact PSF shape is crucial for precise molecule localization at the nanoscale. Optical aberrations and fixed fluorophore dipoles often result in non-isotropic and distorted PSFs, impairing and biasing conventional fitting approaches. Further, PSF shapes are deliberately modified in PSF engineering approaches for providing improved sensitivity, e.g., for 3D localization or determination of dipole orientation. As this can lead to highly complex PSF shapes, a tool for visualizing expected PSFs would facilitate the interpretation of obtained data and the design of experimental approaches. To this end, we introduce a comprehensive and accessible computer application that allows for the simulation of realistic PSFs based on the full vectorial PSF model. Our tool incorporates a wide range of microscope and fluorophore parameters, including orientationally constrained fluorophores, as well as custom aberrations, transmission and phase masks, thus enabling an accurate representation of various imaging conditions. An additional feature is the simulation of crowded molecular environments with overlapping PSFs. Further, our app directly provides the Cramér–Rao bound for assessing the best achievable localization precision under given conditions. Finally, our software allows for the fitting of custom aberrations directly from experimental data, as well as the generation of a large dataset with randomized simulation parameters, effectively bridging the gap between simulated and experimental scenarios, and enhancing experimental design and result validation.

Single molecule localization microscopy relies on the precise quantification of the position of single dye emitters in a sample. This precision is improved by the number of photons that can be detected from each molecule. Particularly recording at cryogenic temperatures dramatically reduces photobleaching and would, hence, in principle, allow the user to massively increase the illumination time to several seconds. The downside of long illuminations, however, would be image blur due to inevitable jitter or drift occurring during the illuminations, which deteriorates the localization precision. In this paper, we theoretically demonstrate that a parallel recording of the fiducial marker beads together with a fitting approach accounting for the full drift trajectory allows for largely eliminating drift effects for drift magnitudes of several hundred nanometers per frame. We showcase the method for linear and diffusional drift as well as oscillations, assuming fixed dipole orientations during each illumination.