Bioinformatics
Oxford University Press
image
3D-Cell-Annotator: an open-source active surface tool for single-cell segmentation in 3D microscopy images
Volume: 36, Issue: 9
DOI 10.1093/bioinformatics/btaa029
  • XML   
  •       
Abstract

SummarySegmentation of single cells in microscopy images is one of the major challenges in computational biology. It is the first step of most bioimage analysis tasks, and essential to create training sets for more advanced deep learning approaches. Here, we propose 3D-Cell-Annotator to solve this task using 3D active surfaces together with shape descriptors as prior information in a semi-automated fashion. The software uses the convenient 3D interface of the widely used Medical Imaging Interaction Toolkit (MITK). Results on 3D biological structures (e.g. spheroids, organoids and embryos) show that the precision of the segmentation reaches the level of a human expert.Availability and implementation3D-Cell-Annotator is implemented in CUDA/C++ as a patch for the segmentation module of MITK. The 3D-Cell-Annotator enabled MITK distribution can be downloaded at: www.3D-cell-annotator.org. It works under Windows 64-bit systems and recent Linux distributions even on a consumer level laptop with a CUDA-enabled video card using recent NVIDIA drivers.Supplementary information Supplementary data are available at Bioinformatics online.

Tasnadi, Toth, Kovacs, Diosdi, Pampaloni, Molnar, Piccinini, Horvath, and Valencia: 3D-Cell-Annotator: an open-source active surface tool for single-cell segmentation in 3D microscopy images

1 Introduction

Multicellular 3D biological models, the so-called ‘-oids’ (e.g. spheroids and organoids) are increasingly used as cellular models for anti-cancer drug screening and toxicology studies, since they represent physiological proxies of human tissues and can replace animal models (Zanoni et al., 2016). Light-Sheet Fluorescence Microscopy (LSFM), confocal and multiphoton systems allow an in-depth observation of tissues in the size range of a few hundred microns. Despite the exponentially growing popularity of 3D models, few tools are available to analyse large aggregates at a single-cell level (Carragher et al., 2018). In this work, we are focussing on the 3D nuclei segmentation problem as one of the most fundamental tasks of bioimage analysis, and the starting point of further phenotype-based statistics. Recently, it has been shown that deep learning-based systems highly outperform classical image processing methods in 2D nuclei segmentation (Hollandi et al., 2019). However, these methods need accurate and large training sets, e.g. 3D-annotated spheroids. Creating such ground truth datasets in 2D is mostly straightforward by drawing the contours of each cell on a 2D canvas. Similarly, the obvious extension to 3D would involve the annotation of each slice of the volume data. However, it is quite evident that this oversimplified method is not only time-consuming, but also leads to discontinuous object surfaces, while the quality of segmentation strongly depends on the chosen plane. To overcome these problems, we designed 3D-Cell-Annotator (Fig. 1a). It provides an alternative for precise outlining of 3D shapes using a special type of active surface model (Molnar et al., 2017).

(a) 3D-Cell-Annotator graphical user interface. (b) Confocal single-cell dataset annotated by three experts. Despite the fact that the annotators are all experts in the field, the obtained segmentations slightly differ (green contours). However, those obtained by the proposed software does not vary significantly (red contour). (c) Segmentations of a cancer-derived multicellular spheroid, imaged with an LSFM at a single-cell level. (d) 3D-Cell-Annotator can be used to extract cells with a special phenotype as shown in the magnified cell (red). (Color version of this figure is available at Bioinformatics online.)
Fig. 1.
(a) 3D-Cell-Annotator graphical user interface. (b) Confocal single-cell dataset annotated by three experts. Despite the fact that the annotators are all experts in the field, the obtained segmentations slightly differ (green contours). However, those obtained by the proposed software does not vary significantly (red contour). (c) Segmentations of a cancer-derived multicellular spheroid, imaged with an LSFM at a single-cell level. (d) 3D-Cell-Annotator can be used to extract cells with a special phenotype as shown in the magnified cell (red). (Color version of this figure is available at Bioinformatics online.)

2 The proposed software

Instead of the classical slice-by-slice manual annotation approach, to obtain accurate single-cell annotation we propose a 3D active surface-based solution with shape priors called 3D selective segmentation (Molnar et al., 2017). Because of the pure 3D nature of our method, the spatial dependencies across all dimensions are considered by the algorithm, thus most of the time-consuming hand-drawing work may be eliminated. A 3D-Cell-Annotator is distributed as a module of the widely used Medical Imaging Interaction Toolkit (MITK, Nolden et al., 2013). Active surface models are computationally complex and expensive; therefore, our model was targeted to Graphic Processing Unit, implemented in the NVidia CUDA framework to provide a speed increase of several orders of magnitude compared with classical CPU implementations. Annotation can be provided cell-by-cell by placing initial seedpoints. A fully automated batch segmentation mode is also available. Although the general active surface algorithm may output clusters of objects when multiple cells share boundaries, the proposed selective active surface applies forces to fulfil shape descriptor values provided by the user. Mathematical foundations are explained in Supplementary Material S1. Flow-chart of the segmentation approach is reported in Supplementary Material S2.

3 Results

To evaluate 3D-Cell-Annotator we used (i) a confocal dataset of 77 z-stacks, each containing one single cell (Poulet et al., 2015; Fig. 1b); (ii) an LSFM dataset used in Gole et al. 2016, representing a low intensity contrast cancer multicellular spheroid composed of 52 cells (Fig. 1c); and (iii) a mouse embryo dataset containing 56 cells acquired by a confocal microscope (Saiz et al., 2016; Fig. 1d). All the used datasets are publicly available (Supplementary Material S3). We computed the Jaccard Index (Supplementary Material S4) for the segmentations obtained by 3D-Cell-Annotator compared with other tools (Supplementary Material S5), as well as to manual segmentations executed by expert annotators familiar with 3D microscopy images and 3D-Cell-Annotator. All the details of the performed experiments are reported in Supplementary Material S6. The obtained results (Supplementary Materials S7–S9) prove that 3D-Cell-Annotator offers an accuracy level reaching that of a human expert. In addition, besides outperforming state-of-the-art freely available tools, it decreases the time of annotation by 3-fold.

4 Conclusions

3D-Cell-Annotator provides a user friendly and precise solution for segmenting single cells in 3D cell cultures imaged with confocal, multiphoton or LSFM, even for large datasets with touching cells and suboptimal imaging conditions, like in the case of spheroids, organoids and embryos. Reaching the accuracy of an expert human annotator, 3D-Cell-Annotator is an optimal solution for generating training tests for more advanced machine learning approaches. Further improvements will include the implementation of different approaches for cell splitting. Source code, user manual, video tutorials and all the masks discussed in this work are available at: www.3D-cell-annotator.org.

Acknowledgements

We thank Laurent Gole (Institute of Molecular and Cell Biology, A*STAR, Singapore), Anna-Katerina Hadjantonakis and Nestor Saiz (Sloan Kettering Institute, New York, USA) for material and information shared about 3D-cell-segmentation tools; Máté Görbe and Gabriella Tick (BRC, Hungary) for creating the website and videos; Dora Bokor, PharmD and Réka Kresák (BRC, Hungary) for proofreading the article and the Supplementary Materials.

Funding

This work was supported by the LENDULET-BIOMAG Grant [2018-342] and from the European Regional Development Funds [GINOP-2.3.2-15-2016-00026 and GINOP-2.3.2-15-2016-00037] to E.T., T.T., M.K., A.D., J.M., F.P. and H.P.; the Union for International Cancer Control (UICC) for a granted UICC Technical Fellowship [ref: UICC-TF/19/640197] to F.P.

Conflict of Interest: none declared.

References

1 

Carragher N.et al (2018) . Concerns, challenges and promises of high-content analysis of 3D cellular models. Nat. Rev. Drug Discov., 17, , pp.606.

2 

Gole L.et al (2016) . OpenSegSPIM: a user-friendly segmentation tool for SPIM data. Bioinformatics, 32, , pp.2075–2077.

3 

Hollandi R.et al (2019) . A deep learning framework for nucleus segmentation using image style transfer. https://www.biorxiv.org/content/10.1101/580605v1 (25 January 2020, date last accessed).

4 

Molnar J.et al (2017). Active surfaces for selective object segmentation in 3D. In: IEEE International Conference on Digital Image Computing: Techniques and Applications (DICTA), 29 November - 1 December 2017, pp. 1–7. IEEE, Sydney, NSW, Australia.

5 

Nolden M.et al (2013) . The Medical Imaging Interaction Toolkit: challenges and advances. Int. J. Comput. Assist. Radiol. Surg., 8, , pp.607–620.

6 

Poulet A.et al (2015) . NucleusJ: an ImageJ plugin for quantifying 3D images of interphase nuclei. Bioinformatics, 31, , pp.1144–1146.

7 

Saiz N.et al (2016) . Quantitative analysis of protein expression to study lineage specification in mouse preimplantation embryos. J. Vis. Exp., 108, , pp.e53654.

8 

Zanoni M.et al (2016) . 3D tumor spheroid models for in vitro therapeutic screening: a systematic approach to enhance the biological relevance of data obtained. Sci. Rep., 6, , pp.19103.

Supplementary materials
  • btaa029_supplementary_data.pdf info     save_alt  
  • btaa029_supplementary_data.pdf info     save_alt  
https://www.researchpad.co/tools/openurl?pubtype=article&doi=10.1093/bioinformatics/btaa029&title=3D-Cell-Annotator: an open-source active surface tool for single-cell segmentation in 3D microscopy images&author=Ervin A Tasnadi,Timea Toth,Maria Kovacs,Akos Diosdi,Francesco Pampaloni,Jozsef Molnar,Filippo Piccinini,Peter Horvath,Alfonso Valencia,&keyword=&subject=Applications Notes,Bioimage Informatics,