Deep neural networks have been applied to improve the image quality of fluorescence microscopy imaging. Previous methods are based on convolutional neural networks (CNNs) which generally require more time-consuming training of separate models for each new imaging experiment, impairing the applicability and generalization. Once the model is trained (typically with tens to hundreds of image pairs) it can then be used to enhance new images that are like the training data. In this study, we proposed a novel imaging-transformer based model, Convolutional Neural Network Transformer (CNNT), to outperform the CNN networks for image denoising. In our scheme we have trained a single CNNT based backbone model from pairwise high-low SNR images for one type of fluorescence microscope (instance structured illumination, iSim). Fast adaption to new applications was achieved by fine-tuning the backbone on only 5-10 sample pairs per new experiment. Results show the CNNT backbone and fine-tuning scheme significantly reduces the training time and improves the image quality, outperformed training separate models using CNN approaches such as - RCAN and Noise2Fast. Here we show three examples of the efficacy of this approach on denoising wide-field, two-photon and confocal fluorescence data. In the confocal experiment, which is a 5 by 5 tiled acquisition, the fine-tuned CNNT model reduces the scan time form one hour to eight minutes, with improved quality.

Deep neural networks can improve the quality of fluorescence microscopy images. Previous methods, based on Convolutional Neural Networks (CNNs), require time-consuming training of individual models for each experiment, impairing their applicability and generalization. In this study, we propose a novel imaging-transformer based model, Convolutional Neural Network Transformer (CNNT), that outperforms CNN based networks for image denoising. We train a general CNNT based backbone model from pairwise high-low Signal-to-Noise Ratio (SNR) image volumes, gathered from a single type of fluorescence microscope, an instant Structured Illumination Microscope. Fast adaptation to new microscopes is achieved by fine-tuning the backbone on only 5-10 image volume pairs per new experiment. Results show that the CNNT backbone and fine-tuning scheme significantly reduces training time and improves image quality, outperforming models trained using only CNNs such as 3D-RCAN and Noise2Fast. We show three examples of efficacy of this approach in wide-field, two-photon, and confocal fluorescence microscopy.

The sense of direction is critical for survival in changing environments and relies on flexibly integrating self-motion signals with external sensory cues. While the anatomical substrates involved in head direction (HD) coding are well known, the mechanisms by which visual information updates HD representations remain poorly understood. Retrosplenial cortex (RSC) plays a key role in forming coherent representations of space in mammals and it encodes a variety of navigational variables, including HD. Here, we use simultaneous two-area tetrode recording to show that RSC HD representation is nearly synchronous with that of the anterodorsal nucleus of thalamus (ADn), the obligatory thalamic relay of HD to cortex, during rotation of a prominent visual cue. Moreover, coordination of HD representations in the two regions is maintained during darkness. We further show that anatomical and functional connectivity are consistent with a strong feedforward drive of HD information from ADn to RSC, with anatomically restricted corticothalamic feedback. Together, our results indicate a concerted global HD reference update across cortex and thalamus.

Endoplasmic reticulum exit sites (ERESs) are tubular outgrowths of endoplasmic reticulum that serve as the earliest station for protein sorting and export into the secretory pathway. How these structures respond to different cellular conditions remains unclear. Here, we report that ERESs undergo lysosome-dependent microautophagy when Ca is released by lysosomes in response to nutrient stressors such as mTOR inhibition or amino acid starvation in mammalian cells. Targeting and uptake of ERESs into lysosomes were observed by super-resolution live-cell imaging and focus ion beam scanning electron microscopy (FIB-SEM). The mechanism was ESCRT dependent and required ubiquitinated SEC31, ALG2, and ALIX, with a knockout of ALG2 or function-blocking mutations of ALIX preventing engulfment of ERESs by lysosomes. In vitro, reconstitution of the pathway was possible using lysosomal lipid-mimicking giant unilamellar vesicles and purified recombinant components. Together, these findings demonstrate a pathway of lysosome-dependent ERES microautophagy mediated by COPII, ALG2, and ESCRTS induced by nutrient stress.

Breakthrough technologies for monitoring and manipulating single-neuron activity provide unprecedented opportunities for whole-brain neuroscience in larval zebrafish1–9. Understanding the neural mechanisms of visually guided behavior also requires precise stimulus control, but little prior research has accounted for physical distortions that result from refraction and reflection at an air-water interface that usually separates the projected stimulus from the fish10–12. Here we provide a computational tool that transforms between projected and received stimuli in order to detect and control these distortions. The tool considers the most commonly encountered interface geometry, and we show that this and other common configurations produce stereotyped distortions. By correcting these distortions, we reduced discrepancies in the literature concerning stimuli that evoke escape behavior13,14, and we expect this tool will help reconcile other confusing aspects of the literature. This tool also aids experimental design, and we illustrate the dangers that uncorrected stimuli pose to receptive field mapping experiments.

Diverse traits often covary between species [1-3]. The possibility that a single mutation could contribute to the evolution of several characters between species [3] is rarely investigated as relatively few cases are dissected at the nucleotide level. Drosophila santomea has evolved additional sex comb sensory teeth on its legs and has lost two sensory bristles on its genitalia. We present evidence that a single nucleotide substitution in an enhancer of the scute gene contributes to both changes. The mutation alters a binding site for the Hox protein Abdominal-B in the developing genitalia, leading to bristle loss, and for another factor in the developing leg, leading to bristle gain. Our study suggests that morphological evolution between species can occur through a single nucleotide change affecting several sexually dimorphic traits. VIDEO ABSTRACT.

Microscopic images of specific proteins in their cellular context yield important insights into biological processes and cellular architecture. The advent of superresolution optical microscopy techniques provides the possibility to augment EM with nanometer-resolution fluorescence microscopy to access the precise location of proteins in the context of cellular ultrastructure. Unfortunately, efforts to combine superresolution fluorescence and EM have been stymied by the divergent and incompatible sample preparation protocols of the two methods. Here, we describe a protocol that preserves both the delicate photoactivatable fluorescent protein labels essential for superresolution microscopy and the fine ultrastructural context of EM. This preparation enables direct 3D imaging in 500- to 750-nm sections with interferometric photoactivatable localization microscopy followed by scanning EM images generated by focused ion beam ablation. We use this process to "colorize" detailed EM images of the mitochondrion with the position of labeled proteins. The approach presented here has provided a new level of definition of the in vivo nature of organization of mitochondrial nucleoids, and we expect this straightforward method to be applicable to many other biological questions that can be answered by direct imaging.

The ability to localize proteins precisely within subcellular space is crucial to understanding the functioning of biological systems. Recently, we described a protocol that correlates a precise map of fluorescent fusion proteins localized using three-dimensional super-resolution optical microscopy with the fine ultrastructural context of three-dimensional electron micrographs. While it achieved the difficult simultaneous objectives of high photoactivated fluorophore preservation and ultrastructure preservation, it required a super-resolution optical and specialized electron microscope that is not available to many researchers. We present here a faster and more practical protocol with the advantage of a simpler two-dimensional optical (Photoactivated Localization Microscopy (PALM)) and scanning electron microscope (SEM) system that retains the often mutually exclusive attributes of fluorophore preservation and ultrastructure preservation. As before, cryosections were prepared using the Tokuyasu protocol, but the staining protocol was modified to be amenable for use in a standard SEM without the need for focused ion beam ablation. We show the versatility of this technique by labeling different cellular compartments and structures including mitochondrial nucleoids, peroxisomes, and the nuclear lamina. We also demonstrate simultaneous two-color PALM imaging with correlated electron micrographs. Lastly, this technique can be used with small-molecule dyes as demonstrated with actin labeling using phalloidin conjugated to a caged dye. By retaining the dense protein labeling expected for super-resolution microscopy combined with ultrastructural preservation, simplifying the tools required for correlative microscopy, and expanding the number of useful labels we expect this method to be accessible and valuable to a wide variety of researchers.

In the nucleus, biological processes are driven by proteins that diffuse through and bind to a meshwork of nucleic acid polymers. To better understand this interplay, we present an imaging platform to simultaneously visualize single protein dynamics together with the local chromatin environment in live cells. Together with super-resolution imaging, new fluorescent probes, and biophysical modeling, we demonstrate that nucleosomes display differential diffusion and packing arrangements as chromatin density increases whereas the viscoelastic properties and accessibility of the interchromatin space remain constant. Perturbing nuclear functions impacts nucleosome diffusive properties in a manner that is dependent both on local chromatin density and on relative location within the nucleus. Our results support a model wherein transcription locally stabilizes nucleosomes while simultaneously allowing for the free exchange of nuclear proteins. Additionally, they reveal that nuclear heterogeneity arises from both active and passive processes and highlight the need to account for different organizational principles when modeling different chromatin environments.

We combine super-resolution localization fluorescence microscopy with transmission electron microscopy of metal replicas to locate proteins on the landscape of the cellular plasma membrane at the nanoscale. We validate robust correlation on the scale of 20 nm by imaging endogenous clathrin (in two and three dimensions) and apply the method to find the previously unknown three-dimensional position of the endocytic protein epsin on clathrin-coated structures at the plasma membrane.