We propose a method to facilitate exploration and analysis of new large data sets. In particular, we give an unsupervised deep learning approach to learning a latent representation that captures semantic similarity in the data set. The core idea is to use data augmentations that preserve semantic meaning to generate synthetic examples of elements whose feature representations should be close to one another.
We demonstrate the utility of our method applied to nano-scale electron microscopy data, where even relatively small portions of animal brains can require terabytes of image data. Although supervised methods can be used to predict and identify known patterns of interest, the scale of the data makes it difficult to mine and analyze patterns that are not known a priori. We show the ability of our learned representation to enable query by example, so that if a scientist notices an interesting pattern in the data, they can be presented with other locations with matching patterns. We also demonstrate that clustering of data in the learned space correlates with biologically-meaningful distinctions. Finally, we introduce a visualization tool and software ecosystem to facilitate user-friendly interactive analysis and uncover interesting biological patterns. In short, our work opens possible new avenues in understanding of and discovery in large data sets, arising in domains such as EM analysis.

How pulsed contractile dynamics drive the remodeling of cell and tissue topologies in epithelial sheets has been a key question in development and disease. Due to constraints in imaging and analysis technologies, studies that have described the in vivo mechanisms underlying changes in cell and neighbor relationships have largely been confined to analyses of planar apical regions. Thus, how the volumetric nature of epithelial cells affects force propagation and remodeling of the cell surface in three dimensions, including especially the apical-basal axis, is unclear. Here, we perform lattice light sheet microscopy (LLSM)-based analysis to determine how far and fast forces propagate across different apical-basal layers, as well as where topological changes initiate from in a columnar epithelium. These datasets are highly time- and depth-resolved and reveal that topology-changing forces are spatially entangled, with contractile force generation occurring across the observed apical-basal axis in a pulsed fashion, while the conservation of cell volumes constrains instantaneous cell deformations. Leading layer behaviors occur opportunistically in response to favorable phasic conditions, with lagging layers "zippering" to catch up as new contractile pulses propel further changes in cell topologies. These results argue against specific zones of topological initiation and demonstrate the importance of systematic 4D-based analysis in understanding how forces and deformations in cell dimensions propagate in a three-dimensional environment.

Although fluorescence microscopy provides a crucial window into the physiology of living specimens, many biological processes are too fragile, are too small, or occur too rapidly to see clearly with existing tools. We crafted ultrathin light sheets from two-dimensional optical lattices that allowed us to image three-dimensional (3D) dynamics for hundreds of volumes, often at subsecond intervals, at the diffraction limit and beyond. We applied this to systems spanning four orders of magnitude in space and time, including the diffusion of single transcription factor molecules in stem cell spheroids, the dynamic instability of mitotic microtubules, the immunological synapse, neutrophil motility in a 3D matrix, and embryogenesis in Caenorhabditis elegans and Drosophila melanogaster. The results provide a visceral reminder of the beauty and the complexity of living systems.

We rely on movement to explore the environment, for example, by palpating an object. In somatosensory cortex, activity related to movement of digits or whiskers is suppressed, which could facilitate detection of touch. Movement-related suppression is generally assumed to involve corollary discharges. Here we uncovered a thalamocortical mechanism in which cortical fast-spiking interneurons, driven by sensory input, suppress movement-related activity in layer 4 (L4) excitatory neurons. In mice locating objects with their whiskers, neurons in the ventral posteromedial nucleus (VPM) fired in response to touch and whisker movement. Cortical L4 fast-spiking interneurons inherited these responses from VPM. In contrast, L4 excitatory neurons responded mainly to touch. Optogenetic experiments revealed that fast-spiking interneurons reduced movement-related spiking in excitatory neurons, enhancing selectivity for touch-related information during active tactile sensation. These observations suggest a fundamental computation performed by the thalamocortical circuit to accentuate salient tactile information.

Layer 6b (L6b), the deepest neocortical layer, projects to cortical targets and higher-order thalamus and is the only layer responsive to the wake-promoting neuropeptide orexin/hypocretin. These characteristics suggest that L6b can strongly modulate brain state, but projections to L6b and their influence remain unknown. Here, we examine the inputs to L6b ex vivo in the mouse primary somatosensory cortex with rabies-based retrograde tracing and channelrhodopsin-assisted circuit mapping in brain slices. We find that L6b receives its strongest excitatory input from intracortical long-range projection neurons, including those in the contralateral hemisphere. In contrast, local intracortical input and thalamocortical input were significantly weaker. Moreover, our data suggest that L6b receives far less thalamocortical input than other cortical layers. L6b was most strongly inhibited by PV and SST interneurons. This study shows that L6b integrates long-range intracortical information and is not part of the traditional thalamocortical loop.

Naïve Bayes Nearest Neighbour (NBNN) is a simple and effective framework which addresses many of the pitfalls of K-Nearest Neighbour (KNN) classification. It has yielded competitive results on several computer vision benchmarks. Its central tenet is that during NN search, a query is not compared to every example in a database, ignoring class information. Instead, NN searches are performed within each class, generating a score per class. A key problem with NN techniques, including NBNN, is that they fail when the data representation does not capture perceptual (e.g. class-based) similarity. NBNN circumvents this by using independent engineered descriptors (e.g. SIFT). To extend its applicability outside of image-based domains, we propose to learn a metric which captures perceptual similarity. Similar to how Neighbourhood Components Analysis optimizes a differentiable form of KNN classification, we propose 'Class Conditional' metric learning (CCML), which optimizes a soft form of the NBNN selection rule. Typical metric learning algorithms learn either a global or local metric. However, our proposed method can be adjusted to a particular level of locality by tuning a single parameter. An empirical evaluation on classification and retrieval tasks demonstrates that our proposed method clearly outperforms existing learned distance metrics across a variety of image and non-image datasets.

We determined how learning modifies neural representations in primary visual cortex (V1) during acquisition of a visually guided behavioral task. We imaged the activity of the same layer 2/3 neuronal populations as mice learned to discriminate two visual patterns while running through a virtual corridor, where one pattern was rewarded. Improvements in behavioral performance were closely associated with increasingly distinguishable population-level representations of task-relevant stimuli, as a result of stabilization of existing and recruitment of new neurons selective for these stimuli. These effects correlated with the appearance of multiple task-dependent signals during learning: those that increased neuronal selectivity across the population when expert animals engaged in the task, and those reflecting anticipation or behavioral choices specifically in neuronal subsets preferring the rewarded stimulus. Therefore, learning engages diverse mechanisms that modify sensory and non-sensory representations in V1 to adjust its processing to task requirements and the behavioral relevance of visual stimuli.

Animals infer when and where a reward is available from experience with informative sensory stimuli and their own actions. In vertebrates, this is thought to depend upon the release of dopamine from midbrain dopaminergic neurons. Studies of the role of dopamine have focused almost exclusively on their encoding of informative sensory stimuli; however, many dopaminergic neurons are active just prior to movement initiation, even in the absence of sensory stimuli. How should current frameworks for understanding the role of dopamine incorporate these observations? To address this question, we review recent anatomical and functional evidence for action-related dopamine signaling. We conclude by proposing a framework in which dopaminergic neurons encode subjective signals of action initiation to solve an internal credit assignment problem.

Single-beam scanning electron microscopes (SEM) are widely used to acquire massive datasets for biomedical study, material analysis, and fabrication inspection. Datasets are typically acquired with uniform acquisition: applying the electron beam with the same power and duration to all image pixels, even if there is great variety in the pixels' importance for eventual use. Many SEMs are now able to move the beam to any pixel in the field of view without delay, enabling them, in principle, to invest their time budget more effectively with non-uniform imaging.