At Janelia Farm, the Fly EM Project is developing technology that will enable us to create a map of every neuron and synapse in the Drosophila nervous system, using novel approaches to electron microscopy (EM) as the foundation. In the same way that the fly genome paved the way for larger projects, including sequencing the human genome, Fly EM may ultimately contribute to our understanding of the human brain by establishing a fly ‘connectome’ – a map that shows how all neurons in the fly brain are connected to each other. We began our entry into EM reconstruction with the fly’s adult visual system, where much is known about cell types from previous EM and histological studies2-5, as well as ongoing studies in the Fly Light Project. In addition to establishing and publishing a fly connectome, Fly EM will make technology and methodology available that is needed to perform large-scale EM reconstructions.
The goals of Fly EM are:
To date, serial section transmission electron microscopy (ssTEM) is the only technique to provide a proof-of-concept "connectome" by reconstructing the entire nervous system of C. elegans1, a process that took more than twelve years. (We believe that FIB-SEM is now emerging as an imaging technique that often has superior properties compared to ssTEM.) To scale up from 302 neurons in the worm to an estimated 100,000 neurons comprising the adult fly brain requires electron microscopy datasets many orders of magnitude larger and automated image processing. To meet these requirements, the Fly EM project has assembled a team of talented individuals with diverse expertise, including computer science, electrical engineering, theoretical physics, Drosophila neuroanatomy, and, of course, electron microscopy. Using the Drosophila Genome Project as its model, the Fly EM project provides the organizational and operational expertise to establish and run a large-scale pipeline for EM reconstruction. Pipelining this complex process allows Fly EM to pursue the ambitious goal of reconstructing an entire fly nervous system.
FlyEM has explored multiple strategies for large-scale imaging, in particular, serial section TEM and FIB-SEM.
To reconstruct large regions of Drosophila nervous tissue, we optimized specimen preparation (eg. high-pressure freezing followed by freeze-substitution) for automated image processing, as well as serial sectioning. The fly is amenable to large-scale EM imaging, in part, because the larval and adult nervous systems are within the range of scales suitable for serial sectioning (10^3 – 10^6 µm3). To date, we have collected over 3000 serial sections through the lamina and medulla and over 5000 serial sections through the entire L1 CNS with conventional ultramicrotomy. Imaging these sections at a resolution sufficient to detect synapses requires hundreds of thousands of images, a time consuming process without automation. We improved imaging throughput by customizing the Leginon Automated TEM Image Acquisition System (developed by NanoImaging Services, Inc. in La Jolla, CA).
Before 2012, our group acquired datasets primarily using ssTEM with anisotropic resolution (e.g., 3 x 3 x 50 nm). The relatively poor resolution in the Z direction limited our ability to trace very fine processes and the effectiveness of automatic image segmentation. Therefore, we are now refining and improving techniques using FIB-SEM (spearheaded by Harald Hess's Lab) to produce our datasets. While FIB-SEM technology has a smaller field of view when compared to ssTEM, improvements to scalability have allowed our group to generate datasets encompassing more than seven complete columns of medulla data with isotropic resolution of 10 x 10 x 10 nm and minimal imaging artifacts.
Employing methods from electrical engineering and computer vision, the Fly EM team lead by Chklovskii and Scheffer labs developed a semi-automated pipeline for serial section TEM reconstruction (Chklovskii et al. 2010), which consists of three main image processing tasks: registration, segmentation, and linkage. Briefly, registration involves horizontal alignment of images to produce a “panorama” of each EM section, and vertical alignment of each section to produce an image stack. Segmentation partitions the stack into voxels, defining profiles that belong to neurons. Linkage is the final process by which voxels belonging to putative neurons are linked across adjacent sections. Despite our ability to automate image processing and segment large datasets, the resulting reconstruction contains errors, which must be corrected in a proofreading step. We have adapted this procedure to now work with isotropic FIB-SEM datasets. While some of these techniques remain similar in spirit, there is no need for an explicit linkage step since segmentation that was previously applied separately on each section in 2D can now be applied in 3D.
The last step of the EM reconstruction pipeline is proofreading and synapse annotation, which is performed using custom written visualization and editing software called Raveler. Raveler includes tools for correcting computer segmentation errors (such as erroneously assigned voxels), annotating location of synapses and highlighting other areas of interest. To maintain performance while proofreading large datasets (> 2-3 terabytes), Raveler splits images into tiles that are dynamically loaded like Google Maps.
At the core of the proofreading process is a team of 9 proofreaders (see Gallery) who participate in a training curriculum that includes (1) proofreader training datasets, (2) proofreader instruction manual, (3) comparison software for identifying discrepancies between proofread datasets, and (4) automated tracking system for monitoring proofreading rate, as well as cataloging proofread datasets. At present, our proofreading team is comprised of 6 full-time and 2 part-time proofreaders.
Our proofreading workflow is divided by objectives. The first objective is to pull out cell shapes and identify cells of interest. The second objective is to identify and trace post-synaptic dendrites back to identified cells. To accomplish the first objective, ~ 80% of the volume must be proofread. Neuroanatomists identify relevant cell shapes and annotate the location of pre- and post-synaptic sites using tools in Raveler. To accomplish the second objective, proofreaders link the annotated post-synaptic sites to cells of interest and the final connectome is produced.
As we transition to FIB-SEM datasets, our algorithms are more capable of automatically segmenting the volume and restricting the parts that must be examined by proofreaders (focused proofreading). For example, we determine the significance and probability of potential errors in the segmentation and have proofreaders avoid areas where we are either confident of correctness or where an error minimally impacts the topology/reconstruction (Plaza, Scheffer, Saunder 2012).
- Dense reconstruction in a TEM dataset of cell shapes and connectome in single medulla column
- Sparse tracing of major pathways believed to contribute to the elementary motion detector in 19 medulla columns of a TEM dataset
- Reconstruction of single lamina cartridge as a proof-of-principle for our pipeline
- FIB-SEM imaging technology
- STEM technology
- Image stack of seven columns of medulla data using FIB-SEM
- Image stack of lamina cartridge and larval neuropil using FIB-SEM as proof-of-principle
- Complete series of serial sections through L1 CNS
- Algorithms for automatic segmentation and registration of EM images (2D and 3D)
- Raveler software for editing computer segmentation and synapse annotation
- Establishing proofreading team at Janelia
- Establishing off-site proofreading effort at Dalhousie University
FlyEM maintains much of its software in the open-source repository GitHub: http://janelia-flyem.github.com. We will provide information on official release versions of these packages when it reaches reasonable maturity.
Plaza, S., Scheffer, L., and Saunders, M. (2012). Minimizing Manual Image Segmentation Turn-Around Time for Neuronal Reconstruction by Embracing Uncertainty. PLOS ONE.
Hu,T., Nunez-Iglesias, J., Vitaladevuni, S., Scheffer, L., Xu, S., Bolorizadeh, M., Hess, H., Fetter, R., and Chklovskii, D. (2012) Super-resolution using Sparse Representations over Learned Dictionaries: Reconstruction of Brain Structure using Electron Microscopy. CoRR abs/1210.0564.
Chklovskii, D.B., Vitaladevuni, S., and Scheffer, L.K. (2010). Semi-automated reconstruction of neural circuits using electron microscopy. Current opinion in neurobiology 20, 667-675.
Veeraraghava, A., Genkin A., Vitaladevuni, S., Scheffer, L., Xu, S., Hess, H., Fetter, R., Cantoni, M., Knott, G., and Chklovskii, D. (2010). Increasing depth resolution of electron microscopy of neural circuits using sparse tomographic reconstruction. CVPR: 1767-1774.
Vitaladevuni, S.N., and Basri, R. (2010). Co-clustering of image segments using convex optimization applied to EM neuronal reconstruction. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on (IEEE), pp. 2203-2210.
Fly EM will generally pursue an open policy with our datasets, software, and algorithms after relevant publications. When an EM reconstruction is published, the derived connectome and reconstructed neuronal skeletons will be made available online. The raw data and annotatations will be made available upon request as logistics dictate. To encourage further collaboration and scientific discovery, a small fraction of our raw data and corresponding segmentation will be made available independent of publication. Our goal is to enable others who wish to approach the many algorithmic challenges, but who do not have access to an EM facility, to have the data they need to support methods development, as well as our results to use as a benchmark. Fly EM emphasizes publication of supporting techniques and software approaches before major EM reconstruction releases to encourage rapid feedback from the community and adoption of our strategies.
Team Members Groups