Please enter verification code
Quantitative Spatial Analysis on Whole Slide Images Using U-Net
Computational Biology and Bioinformatics
Volume 8, Issue 2, December 2020, Pages: 90-96
Received: Oct. 22, 2020; Accepted: Nov. 18, 2020; Published: Dec. 4, 2020
Views 90      Downloads 65
Sanghoon Lee, Department of Computer Sciences and Electrical Engineering, Marshall University, Huntington, United States of America
Yanjun Zhao, Department of Computer Science, Troy University, Troy, United States of America
Mohamed Masoud, Department of Neurology, Emory University, Atlanta, United States of America
Saeid Belkasim, Department of Computer Science, Georgia State University, Atlanta, United States of America
Article Tools
Follow on us
Advances in whole slide imaging technology have promoted a high use of digital slide images and generated a large volume of image data that is reliable and useful in determining treatment outcome. Recent technologies closely related to machine learning and deep learning algorithms have contributed to the success of digital histopathology by analyzing the digitized slide images providing quantitative information that are useful for faster turnaround times and effective treatment for the patient. The digital histopathological image analysis has received much attention due to its capability of mitigating the problem of the hand-crafted features. Features directly learned from raw data are trainable within the deep learning procedure and can be used for the histopathology image classification task. However, understanding the spatial context of cancer cells is still a challenging issue because of the heterogeneity of the tumor microenvironment which varies greatly, preventing successful diagnosis and leads to inappropriate therapeutic approaches for cancer patients. In this paper, we present a spatial analysis method for tumor microenvironment analysis using the U-Net architecture, a semantic segmentation deep-learning model, for a better understanding of the spatial relations between tissue types. We demonstrate the effectiveness of the U-Net architecture using a dataset created by an international crowdsourcing study. Moreover, we show that the quantitative estimates can be derived from the univariate spatial analysis.
Cancer, Whole Slide Images, Spatial Analysis, U-Net, Machine Learning
To cite this article
Sanghoon Lee, Yanjun Zhao, Mohamed Masoud, Saeid Belkasim, Quantitative Spatial Analysis on Whole Slide Images Using U-Net, Computational Biology and Bioinformatics. Vol. 8, No. 2, 2020, pp. 90-96. doi: 10.11648/j.cbb.20200802.18
Copyright © 2020 Authors retain the copyright of this article.
This article is an open access article distributed under the Creative Commons Attribution License ( which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Pantanowitz, L., Sinard, J. H., Henricks, W. H., Fatheree, L. A., Carter, A. B., Contis, L., Beckwith, B. A., Evans, A. J., Lal, A. and Parwani, A. V., 2013. Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center. Archives of Pathology and Laboratory Medicine, 137 (12), pp. 1710-1722.
Abels, E. and Pantanowitz, L., 2017. Current state of the regulatory trajectory for whole slide imaging devices in the USA. Journal of pathology informatics, 8.
Komura, D. and Ishikawa, S., 2018. Machine learning methods for histopathological image analysis. Computational and structural biotechnology journal, 16, pp. 34-42.
Lee, S., Amgad, M., Chittajallu, D. R., McCormick, M., Pollack, B. P., Elfandy, H., Hussein, H., Gutman, D. A. and Cooper, L. A., 2020. HistomicsML2. 0: Fast interactive machine learning for whole slide imaging data. arXiv preprint arXiv: 2001.11547.
Nalisnik, M., Amgad, M., Lee, S., Halani, S. H., Vega, J. E. V., Brat, D. J., Gutman, D. A. and Cooper, L. A., 2017. Interactive phenotyping of large-scale histology imaging data with HistomicsML. Scientific reports, 7 (1), pp. 1-12.
Lee, S., Amgad, M., Masoud, M., Subramanian, R., Gutman, D. and Cooper, L., 2019, November. An Ensemble-based Active Learning for Breast Cancer Classification. In 2019 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) (pp. 2549-2553). IEEE.
Cruz-Roa, A., Gilmore, H., Basavanhally, A., Feldman, M., Ganesan, S., Shih, N. N., Tomaszewski, J., González, F. A. and Madabhushi, A., 2017. Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent. Scientific reports, 7, p. 46450.
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015 May 28; 521 (7553): 436-44. doi: 10.1038/nature14539. PMID: 26017442.
Litjens G, Sánchez CI, Timofeeva N, Hermsen M, Nagtegaal I, Kovacs I, Hulsbergen-van de Kaa C, Bult P, van Ginneken B, van der Laak J. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep. 2016 May 23; 6: 26286. doi: 10.1038/srep26286. PMID: 27212078; PMCID: PMC4876324.
Hou L, Samaras D, Kurc TM, Gao Y, Davis JE, Saltz JH. Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit. 2016 Jun-Jul; 2016: 2424-2433. doi: 10.1109/CVPR.2016.266. PMID: 27795661; PMCID: PMC5085270.
Sun W, Zheng B, Qian W. Automatic feature learning using multichannel ROI based on deep structured algorithms for computerized lung cancer diagnosis. Comput Biol Med. 2017 Oct 1; 89: 530-539. doi: 10.1016/j.compbiomed.2017.04.006. Epub 2017 Apr 13. PMID: 28473055.
Murthy V, Hou L, Samaras D, Kurc TM, Saltz JH. Center-Focusing Multi-task CNN with Injected Features for Classification of Glioma Nuclear Images. IEEE Winter Conf Appl Comput Vis. 2017 Mar; 2017: 834-841. doi: 10.1109/WACV.2017.98. Epub 2017 May 15. PMID: 29881826; PMCID: PMC5988234.
Xu Y, Jia Z, Wang LB, Ai Y, Zhang F, Lai M, Chang EI. Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features. BMC Bioinformatics. 2017 May 26; 18 (1): 281. doi: 10.1186/s12859-017-1685-x. PMID: 28549410; PMCID: PMC5446756.
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Silva, V. W. K., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S. and Fuchs, T. J., 2019. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature medicine, 25 (8), pp. 1301-1309.
Kothari, S., Phan, J. H., Stokes, T. H. and Wang, M. D., 2013. Pathology imaging informatics for quantitative analysis of whole-slide images. Journal of the American Medical Informatics Association, 20 (6), pp. 1099-1108.
Topol, E J., 2019. High-performance medicine: the convergence of human and artificial intelligence. Nature medicine, 25 (1), pp. 44-56.
Vandenberghe, M. E., Scott, M. L., Scorer, P. W., Söderberg, M., Balcerzak, D. and Barker, C., 2017. Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer. Scientific reports, 7, p. 45938.
Joyce JA. Therapeutic targeting of the tumor microenvironment. Cancer cell. 7 (6): 513-20, 2005.
Lorusso G, Ruegg C. The tumor microenvironment and its contribution to tumor evolution toward metastasis. Histochemistry and cell biology. 130 (6): 1091-103, 2008.
Yu KH, Zhang C, Berry GJ, Altman RB, Ré C, Rubin DL, Snyder M. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features. Nat Commun. 2016 Aug 16; 7: 12474. doi: 10.1038/ncomms12474. PMID: 27527408; PMCID: PMC4990706.
Ehteshami Bejnordi B, Balkenhol M, Litjens G, Holland R, Bult P, Karssemeijer N, van der Laak JA. Automated Detection of DCIS in Whole-Slide H&E Stained Breast Histopathology Images. IEEE Trans Med Imaging. 2016 Sep; 35 (9): 2141-2150. doi: 10.1109/TMI.2016.2550620. Epub 2016 Apr 5. PMID: 27076354.
Tomczak, K., Czerwińska, P., and Wiznerowicz, M.: ‘The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge’, Contemp Oncol (Pozn), 2015, 19, (1A), pp. A68-A77.
Amgad, M., Elfandy, H., Hussein, H., Atteya, L. A., Elsebaie, M. A., Abo Elnasr, L. S., Sakr, R. A., Salem, H. S., Ismail, A. F., Saad, A. M. and Ahmed, J., 2019. Structured crowdsourcing enables convolutional segmentation of histology images. Bioinformatics, 35 (18), pp. 3461-3467.
Long, J., Shelhamer, E. and Darrell, T., 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431-3440).
Ronneberger, O., Fischer, P. and Brox, T., 2015, October. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Springer, Cham.
Jana M, Sar N. Modeling of hotspot detection using cluster outlier analysis and Getis-Ord Gi* statistic of educational development in upper-primary level, India. Modeling Earth Systems and Environment. 2 (2): 60, 2016.
Junior GB, Paiva AC, Silva AC, de Oliveira AC. Classification of breast tissues using Getis-Ord statistics and support vector machines. Intelligent Decision Technologies. 3 (4): 197-205, 2009.
Science Publishing Group
1 Rockefeller Plaza,
10th and 11th Floors,
New York, NY 10020
Tel: (001)347-983-5186