SummarySegmentation of single cells in microscopy images is one of the major challenges in computational biology. It is the first step of most bioimage analysis tasks, and essential to create training sets for more advanced deep learning approaches. Here, we propose 3D-Cell-Annotator to solve this task using 3D active surfaces together with shape descriptors as prior information in a semi-automated fashion. The software uses the convenient 3D interface of the widely used Medical Imaging Interaction Toolkit (MITK). Results on 3D biological structures (e.g. spheroids, organoids and embryos) show that the precision of the segmentation reaches the level of a human expert.Availability and implementation3D-Cell-Annotator is implemented in CUDA/C++ as a patch for the segmentation module of MITK. The 3D-Cell-Annotator enabled MITK distribution can be downloaded at: www.3D-cell-annotator.org. It works under Windows 64-bit systems and recent Linux distributions even on a consumer level laptop with a CUDA-enabled video card using recent NVIDIA drivers.Supplementary information Supplementary data are available at Bioinformatics online.
Multicellular 3D biological models, the so-called ‘-oids’ (e.g. spheroids and organoids) are increasingly used as cellular models for anti-cancer drug screening and toxicology studies, since they represent physiological proxies of human tissues and can replace animal models (Zanoni et al., 2016). Light-Sheet Fluorescence Microscopy (LSFM), confocal and multiphoton systems allow an in-depth observation of tissues in the size range of a few hundred microns. Despite the exponentially growing popularity of 3D models, few tools are available to analyse large aggregates at a single-cell level (Carragher et al., 2018). In this work, we are focussing on the 3D nuclei segmentation problem as one of the most fundamental tasks of bioimage analysis, and the starting point of further phenotype-based statistics. Recently, it has been shown that deep learning-based systems highly outperform classical image processing methods in 2D nuclei segmentation (Hollandi et al., 2019). However, these methods need accurate and large training sets, e.g. 3D-annotated spheroids. Creating such ground truth datasets in 2D is mostly straightforward by drawing the contours of each cell on a 2D canvas. Similarly, the obvious extension to 3D would involve the annotation of each slice of the volume data. However, it is quite evident that this oversimplified method is not only time-consuming, but also leads to discontinuous object surfaces, while the quality of segmentation strongly depends on the chosen plane. To overcome these problems, we designed 3D-Cell-Annotator (Fig. 1a). It provides an alternative for precise outlining of 3D shapes using a special type of active surface model (Molnar et al., 2017).
Instead of the classical slice-by-slice manual annotation approach, to obtain accurate single-cell annotation we propose a 3D active surface-based solution with shape priors called 3D selective segmentation (Molnar et al., 2017). Because of the pure 3D nature of our method, the spatial dependencies across all dimensions are considered by the algorithm, thus most of the time-consuming hand-drawing work may be eliminated. A 3D-Cell-Annotator is distributed as a module of the widely used Medical Imaging Interaction Toolkit (MITK, Nolden et al., 2013). Active surface models are computationally complex and expensive; therefore, our model was targeted to Graphic Processing Unit, implemented in the NVidia CUDA framework to provide a speed increase of several orders of magnitude compared with classical CPU implementations. Annotation can be provided cell-by-cell by placing initial seedpoints. A fully automated batch segmentation mode is also available. Although the general active surface algorithm may output clusters of objects when multiple cells share boundaries, the proposed selective active surface applies forces to fulfil shape descriptor values provided by the user. Mathematical foundations are explained in Supplementary Material S1. Flow-chart of the segmentation approach is reported in Supplementary Material S2.
To evaluate 3D-Cell-Annotator we used (i) a confocal dataset of 77 z-stacks, each containing one single cell (Poulet et al., 2015; Fig. 1b); (ii) an LSFM dataset used in Gole et al. 2016, representing a low intensity contrast cancer multicellular spheroid composed of 52 cells (Fig. 1c); and (iii) a mouse embryo dataset containing 56 cells acquired by a confocal microscope (Saiz et al., 2016; Fig. 1d). All the used datasets are publicly available (Supplementary Material S3). We computed the Jaccard Index (Supplementary Material S4) for the segmentations obtained by 3D-Cell-Annotator compared with other tools (Supplementary Material S5), as well as to manual segmentations executed by expert annotators familiar with 3D microscopy images and 3D-Cell-Annotator. All the details of the performed experiments are reported in Supplementary Material S6. The obtained results (Supplementary Materials S7–S9) prove that 3D-Cell-Annotator offers an accuracy level reaching that of a human expert. In addition, besides outperforming state-of-the-art freely available tools, it decreases the time of annotation by 3-fold.
3D-Cell-Annotator provides a user friendly and precise solution for segmenting single cells in 3D cell cultures imaged with confocal, multiphoton or LSFM, even for large datasets with touching cells and suboptimal imaging conditions, like in the case of spheroids, organoids and embryos. Reaching the accuracy of an expert human annotator, 3D-Cell-Annotator is an optimal solution for generating training tests for more advanced machine learning approaches. Further improvements will include the implementation of different approaches for cell splitting. Source code, user manual, video tutorials and all the masks discussed in this work are available at: www.3D-cell-annotator.org.
We thank Laurent Gole (Institute of Molecular and Cell Biology, A*STAR, Singapore), Anna-Katerina Hadjantonakis and Nestor Saiz (Sloan Kettering Institute, New York, USA) for material and information shared about 3D-cell-segmentation tools; Máté Görbe and Gabriella Tick (BRC, Hungary) for creating the website and videos; Dora Bokor, PharmD and Réka Kresák (BRC, Hungary) for proofreading the article and the Supplementary Materials.
This work was supported by the LENDULET-BIOMAG Grant [2018-342] and from the European Regional Development Funds [GINOP-2.3.2-15-2016-00026 and GINOP-2.3.2-15-2016-00037] to E.T., T.T., M.K., A.D., J.M., F.P. and H.P.; the Union for International Cancer Control (UICC) for a granted UICC Technical Fellowship [ref: UICC-TF/19/640197] to F.P.
Conflict of Interest: none declared.