Pretrained neural network models for biological segmentation can provide good out-of-the-box results for many image types. However, such models do not allow users to adapt the segmentation style to their specific needs and can perform suboptimally for test images that are very different from the training images. Here we introduce Cellpose 2.0, a new package that includes an ensemble of diverse pretrained models as well as a human-in-the-loop pipeline for rapid prototyping of new custom models. We show that models pretrained on the Cellpose dataset can be fine-tuned with only 500-1,000 user-annotated regions of interest (ROI) to perform nearly as well as models trained on entire datasets with up to 200,000 ROI. A human-in-the-loop approach further reduced the required user annotation to 100-200 ROI, while maintaining high-quality segmentations. We provide software tools such as an annotation graphical user interface, a model zoo and a human-in-the-loop pipeline to facilitate the adoption of Cellpose 2.0.

Modern algorithms for biological segmentation can match inter-human agreement in annotation quality. This however is not a performance bound: a hypothetical human-consensus segmentation could reduce error rates in half. To obtain a model that generalizes better we adapted the pretrained transformer backbone of a foundation model (SAM) to the Cellpose framework. The resulting Cellpose-SAM model substantially outperforms inter-human agreement and approaches the human-consensus bound. We increase generalization performance further by making the model robust to channel shuffling, cell size, shot noise, downsampling, isotropic and anisotropic blur. The new model can be readily adopted into the Cellpose ecosystem which includes finetuning, human-in-the-loop training, image restoration and 3D segmentation approaches. These properties establish Cellpose-SAM as a foundation model for biological segmentation.

Generalist methods for cellular segmentation have good out-of-the-box performance on a variety of image types; however, existing methods struggle for images that are degraded by noise, blurring or undersampling, all of which are common in microscopy. We focused the development of Cellpose3 on addressing these cases and here we demonstrate substantial out-of-the-box gains in segmentation and image quality for noisy, blurry and undersampled images. Unlike previous approaches that train models to restore pixel values, we trained Cellpose3 to output images that are well segmented by a generalist segmentation model, while maintaining perceptual similarity to the target images. Furthermore, we trained the restoration models on a large, varied collection of datasets, thus ensuring good generalization to user images. We provide these tools as 'one-click' buttons inside the graphical interface of Cellpose as well as in the Cellpose API.

Many biological applications require the segmentation of cell bodies, membranes and nuclei from microscopy images. Deep learning has enabled great progress on this problem, but current methods are specialized for images that have large training datasets. Here we introduce a generalist, deep learning-based segmentation algorithm called Cellpose, which can very precisely segment a wide range of image types out-of-the-box and does not require model retraining or parameter adjustments. We trained Cellpose on a new dataset of highly-varied images of cells, containing over 70,000 segmented objects. To support community contributions to the training data, we developed software for manual labelling and for curation of the automated results, with optional direct upload to our data repository. Periodically retraining the model on the community-contributed data will ensure that Cellpose improves constantly.

Many biological applications require the segmentation of cell bodies, membranes and nuclei from microscopy images. Deep learning has enabled great progress on this problem, but current methods are specialized for images that have large training datasets. Here we introduce a generalist, deep learning-based segmentation method called Cellpose, which can precisely segment cells from a wide range of image types and does not require model retraining or parameter adjustments. Cellpose was trained on a new dataset of highly varied images of cells, containing over 70,000 segmented objects. We also demonstrate a three-dimensional (3D) extension of Cellpose that reuses the two-dimensional (2D) model and does not require 3D-labeled data. To support community contributions to the training data, we developed software for manual labeling and for curation of the automated results. Periodically retraining the model on the community-contributed data will ensure that Cellpose improves constantly.

BACKGROUND: Male-specific products of the fruitless (fru) gene control the development and function of neuronal circuits that underlie male-specific behaviors in Drosophila, including courtship. Alternative splicing generates at least three distinct Fru isoforms, each containing a different zinc-finger domain. Here, we examine the expression and function of each of these isoforms. RESULTS: We show that most fru(+) cells express all three isoforms, yet each isoform has a distinct function in the elaboration of sexually dimorphic circuitry and behavior. The strongest impairment in courtship behavior is observed in fru(C) mutants, which fail to copulate, lack sine song, and do not generate courtship song in the absence of visual stimuli. Cellular dimorphisms in the fru circuit are dependent on Fru(C) rather than other single Fru isoforms. Removal of Fru(C) from the neuronal classes vAB3 or aSP4 leads to cell-autonomous feminization of arborizations and loss of courtship in the dark. CONCLUSIONS: These data map specific aspects of courtship behavior to the level of single fru isoforms and fru(+) cell types-an important step toward elucidating the chain of causality from gene to circuit to behavior.

Neural circuit assembly features simultaneous targeting of numerous neuronal processes from constituent neuron types, yet the dynamics is poorly understood. Here, we use the Drosophila olfactory circuit to investigate dynamic cellular processes by which olfactory receptor neurons (ORNs) target axons precisely to specific glomeruli in the ipsi- and contralateral antennal lobes. Time-lapse imaging of individual axons from 30 ORN types revealed a rich diversity in extension speed, innervation timing, and ipsilateral branch locations and identified that ipsilateral targeting occurs via stabilization of transient interstitial branches. Fast imaging using adaptive optics-corrected lattice light-sheet microscopy showed that upon approaching target, many ORN types exhibiting "exploring branches" consisted of parallel microtubule-based terminal branches emanating from an F-actin-rich hub. Antennal nerve ablations uncovered essential roles for bilateral axons in contralateral target selection and for ORN axons to facilitate dendritic refinement of postsynaptic partner neurons. Altogether, these observations provide cellular bases for wiring specificity establishment.

The transcriptional response of β-actin to extra-cellular stimuli is a paradigm for transcription factor complex assembly and regulation. Serum induction leads to a precisely timed pulse of β-actin transcription in the cell population. Actin protein is proposed to be involved in this response, but it is not known whether cellular actin levels affect nuclear β-actin transcription. We perturbed the levels of key signaling factors and examined the effect on the induced transcriptional pulse by following endogenous β-actin alleles in single living cells. Lowering serum response factor (SRF) protein levels leads to loss of pulse integrity, whereas reducing actin protein levels reveals positive feedback regulation, resulting in elevated gene activation and a prolonged transcriptional response. Thus, transcriptional pulse fidelity requires regulated amounts of signaling proteins, and perturbations in factor levels eliminate the physiological response, resulting in either tuning down or exaggeration of the transcriptional pulse.

The cyanobacterial culture HT-58-2, composed of a filamentous cyanobacterium and accompanying community bacteria, produces chlorophyll a as well as the tetrapyrrole macrocycles known as tolyporphins. Almost all known tolyporphins (A-M except K) contain a dioxobacteriochlorin chromophore and exhibit an absorption spectrum somewhat similar to that of chlorophyll a. Here, hyperspectral confocal fluorescence microscopy was employed to noninvasively probe the locale of tolyporphins within live cells under various growth conditions (media, illumination, culture age). Cultures grown in nitrate-depleted media (BG-11 vs. nitrate-rich, BG-11) are known to increase the production of tolyporphins by orders of magnitude (rivaling that of chlorophyll a) over a period of 30-45 days. Multivariate curve resolution (MCR) was applied to an image set containing images from each condition to obtain pure component spectra of the endogenous pigments. The relative abundances of these components were then calculated for individual pixels in each image in the entire set, and 3D-volume renderings were obtained. At 30 days in media with or without nitrate, the chlorophyll a and phycobilisomes (combined phycocyanin and phycobilin components) co-localize in the filament outer cytoplasmic region. Tolyporphins localize in a distinct peripheral pattern in cells grown in BG-11 versus a diffuse pattern (mimicking the chlorophyll a localization) upon growth in BG-11. In BG-11, distinct puncta of tolyporphins were commonly found at the septa between cells and at the end of filaments. This work quantifies the relative abundance and envelope localization of tolyporphins in single cells, and illustrates the ability to identify novel tetrapyrroles in the presence of chlorophyll a in a photosynthetic microorganism within a non-axenic culture.