Machine and Deep Learning Methods Enable the Accurate and Efficient Segmentation, Grading, Diagnosis and Prognosis of Brain Tumors
Article Information
Mohsen Tabatabaei1, Zeinab Khazaei2, Kamran Tavakol3*, Andrea Tavakol4
1PhD, Health Information Management, Office of Vice Chancellor for Research, Arak University of Medical Sciences, Arak, Iran
2PhD Student, School of Management and Economics, Science & Research Branch, Azad University. Tehran, Iran
3Professor Emeritus, College of Medicine, Howard University. Washington, DC, USA
4College of Information Studies, University of Maryland, College Park, MD, USA
*Corresponding Author: Dr. Kamran Tavakol, Professor Emeritus, College of Medicine, Howard University, Washington, DC, USA
Received: 15 March 2020; Accepted: 02 April 2020; Published: 09 April 2020
Citation: Mohsen Tabatabaei, Zeinab Khazaei, Kamran Tavakol, Andrea Tavakol. Machine and Deep Learning Methods Enable the Accurate and Efficient Segmentation, Grading, Diagnosis and Prognosis of Brain Tumors. Journal of Radiology and Clinical Imaging 3 (2020): 058-063.
Share at FacebookAbstract
Early diagnosis and management of brain tumors are crucial and save patient from adverse effects or even death. Computer science advances have been made such that artificial intelligence (AI) brings enhancement to the classification, segmentation, grading and staging of brain tumors while assisting in the prediction of the diagnosis, treatment, future care and the prognosis, based on the analysis of MRI data. This review is aimed at enhancing the technical awareness of radiology residents, practitioners and physicians in general medicine, neurology and neurosurgery. Also, researchers from computer science, neuroscience and other relevant areas will find this review very helpful. We searched for relevant articles in Google Scholar, Scopus, and Pubmed, using these keywords: artificial intelligence, machine learning, deep learning, brain tumors, magnetic resonance imaging, MRI, and radiological image analyses. Out of 138 articles, 17 were selected, which described various AI algorithms to analyze brain tumor MRI data. This review presents the significant findings and conclusions of the selected articles, and the role of machine and deep learning methods in the classification, segmentation, diagnosis and prognosis of brain tumors. These AI methods enable radiologists to perform accurate, rapid and efficient mapping, classification, segmentation, grading and staging of brain tumors while predicting the diagnosis, treatment, future care and the clinical prognosis of patients with brain tumors.
Keywords
Brain tumors; Artificial intelligence; Magnetic resonance imaging; Machine & deep learning methods; Grading and staging analyses
Brain tumors articles, Artificial intelligence articles, Magnetic resonance imaging articles, Machine & deep learning methods articles, Grading and staging analyses articles
Brain tumors articles Brain tumors Research articles Brain tumors review articles Brain tumors PubMed articles Brain tumors PubMed Central articles Brain tumors 2023 articles Brain tumors 2024 articles Brain tumors Scopus articles Brain tumors impact factor journals Brain tumors Scopus journals Brain tumors PubMed journals Brain tumors medical journals Brain tumors free journals Brain tumors best journals Brain tumors top journals Brain tumors free medical journals Brain tumors famous journals Brain tumors Google Scholar indexed journals Artificial intelligence articles Artificial intelligence Research articles Artificial intelligence review articles Artificial intelligence PubMed articles Artificial intelligence PubMed Central articles Artificial intelligence 2023 articles Artificial intelligence 2024 articles Artificial intelligence Scopus articles Artificial intelligence impact factor journals Artificial intelligence Scopus journals Artificial intelligence PubMed journals Artificial intelligence medical journals Artificial intelligence free journals Artificial intelligence best journals Artificial intelligence top journals Artificial intelligence free medical journals Artificial intelligence famous journals Artificial intelligence Google Scholar indexed journals Magnetic resonance imaging articles Magnetic resonance imaging Research articles Magnetic resonance imaging review articles Magnetic resonance imaging PubMed articles Magnetic resonance imaging PubMed Central articles Magnetic resonance imaging 2023 articles Magnetic resonance imaging 2024 articles Magnetic resonance imaging Scopus articles Magnetic resonance imaging impact factor journals Magnetic resonance imaging Scopus journals Magnetic resonance imaging PubMed journals Magnetic resonance imaging medical journals Magnetic resonance imaging free journals Magnetic resonance imaging best journals Magnetic resonance imaging top journals Magnetic resonance imaging free medical journals Magnetic resonance imaging famous journals Magnetic resonance imaging Google Scholar indexed journals Machine & deep learning methods articles Machine & deep learning methods Research articles Machine & deep learning methods review articles Machine & deep learning methods PubMed articles Machine & deep learning methods PubMed Central articles Machine & deep learning methods 2023 articles Machine & deep learning methods 2024 articles Machine & deep learning methods Scopus articles Machine & deep learning methods impact factor journals Machine & deep learning methods Scopus journals Machine & deep learning methods PubMed journals Machine & deep learning methods medical journals Machine & deep learning methods free journals Machine & deep learning methods best journals Machine & deep learning methods top journals Machine & deep learning methods free medical journals Machine & deep learning methods famous journals Machine & deep learning methods Google Scholar indexed journals Grading and staging analyses articles Grading and staging analyses Research articles Grading and staging analyses review articles Grading and staging analyses PubMed articles Grading and staging analyses PubMed Central articles Grading and staging analyses 2023 articles Grading and staging analyses 2024 articles Grading and staging analyses Scopus articles Grading and staging analyses impact factor journals Grading and staging analyses Scopus journals Grading and staging analyses PubMed journals Grading and staging analyses medical journals Grading and staging analyses free journals Grading and staging analyses best journals Grading and staging analyses top journals Grading and staging analyses free medical journals Grading and staging analyses famous journals Grading and staging analyses Google Scholar indexed journals Convolutional neural network articles Convolutional neural network Research articles Convolutional neural network review articles Convolutional neural network PubMed articles Convolutional neural network PubMed Central articles Convolutional neural network 2023 articles Convolutional neural network 2024 articles Convolutional neural network Scopus articles Convolutional neural network impact factor journals Convolutional neural network Scopus journals Convolutional neural network PubMed journals Convolutional neural network medical journals Convolutional neural network free journals Convolutional neural network best journals Convolutional neural network top journals Convolutional neural network free medical journals Convolutional neural network famous journals Convolutional neural network Google Scholar indexed journals Brain tumor segmentation articles Brain tumor segmentation Research articles Brain tumor segmentation review articles Brain tumor segmentation PubMed articles Brain tumor segmentation PubMed Central articles Brain tumor segmentation 2023 articles Brain tumor segmentation 2024 articles Brain tumor segmentation Scopus articles Brain tumor segmentation impact factor journals Brain tumor segmentation Scopus journals Brain tumor segmentation PubMed journals Brain tumor segmentation medical journals Brain tumor segmentation free journals Brain tumor segmentation best journals Brain tumor segmentation top journals Brain tumor segmentation free medical journals Brain tumor segmentation famous journals Brain tumor segmentation Google Scholar indexed journals Fluid-attenuated inversion recovery articles Fluid-attenuated inversion recovery Research articles Fluid-attenuated inversion recovery review articles Fluid-attenuated inversion recovery PubMed articles Fluid-attenuated inversion recovery PubMed Central articles Fluid-attenuated inversion recovery 2023 articles Fluid-attenuated inversion recovery 2024 articles Fluid-attenuated inversion recovery Scopus articles Fluid-attenuated inversion recovery impact factor journals Fluid-attenuated inversion recovery Scopus journals Fluid-attenuated inversion recovery PubMed journals Fluid-attenuated inversion recovery medical journals Fluid-attenuated inversion recovery free journals Fluid-attenuated inversion recovery best journals Fluid-attenuated inversion recovery top journals Fluid-attenuated inversion recovery free medical journals Fluid-attenuated inversion recovery famous journals Fluid-attenuated inversion recovery Google Scholar indexed journals Computer tomography articles Computer tomography Research articles Computer tomography review articles Computer tomography PubMed articles Computer tomography PubMed Central articles Computer tomography 2023 articles Computer tomography 2024 articles Computer tomography Scopus articles Computer tomography impact factor journals Computer tomography Scopus journals Computer tomography PubMed journals Computer tomography medical journals Computer tomography free journals Computer tomography best journals Computer tomography top journals Computer tomography free medical journals Computer tomography famous journals Computer tomography Google Scholar indexed journals Receiver operating characteristic articles Receiver operating characteristic Research articles Receiver operating characteristic review articles Receiver operating characteristic PubMed articles Receiver operating characteristic PubMed Central articles Receiver operating characteristic 2023 articles Receiver operating characteristic 2024 articles Receiver operating characteristic Scopus articles Receiver operating characteristic impact factor journals Receiver operating characteristic Scopus journals Receiver operating characteristic PubMed journals Receiver operating characteristic medical journals Receiver operating characteristic free journals Receiver operating characteristic best journals Receiver operating characteristic top journals Receiver operating characteristic free medical journals Receiver operating characteristic famous journals Receiver operating characteristic Google Scholar indexed journals
Article Details
Abbreviations:
AI - Artificial intelligence; FLAIR - Fluid-attenuated inversion recovery; BraTS - Brain tumor segmentation; iGTV - Infiltrative gross tumor volume; CNN - Convolutional neural network; MRTA - MRI texture analysis; CNS - Central nervous system; RescueNet - Cyclic unpaired encoder-decoder network for segmentation; CT - Computer tomography; ROC - Receiver operating characteristic; DCSNN - Deep convolutional symmetric neural network; rsFC - Resting-state functional connectivity; DSC - Dice similarity coefficient; VGG - Visual geometry group; DTI - Diffusion tensor imaging; iGTV - Infiltrative gross tumor volume
1. Introduction
Brain tumors occur worldwide and impair the proper function of the central nervous system (CNS) by damaging the tissue and increasing the intracranial pressure, which may lead to death. Gliomas are the 10th most common form of tumors affecting children and adults alike [1]. Fortunately, it is one of the most curable cancers if detected early [1]. Therefore, the early diagnosis of brain tumors is crucial and can save patients from adverse effects or death [1]. Magnetic resonance imaging (MRI) is used to assess tumors and evaluate the treatment success in the clinical practice. Therefore, accurately segmenting the brain tumors from MRI data is the key to clinical diagnosis and treatment planning. The processing of a large quantity of MRI data per patient is time consuming and cannot be achieved in a short time. However, the analysis of data from brain tumor MRI is a demanding research area and helpful to radiologists for rapid and valid analysis of brain tumors [2].
The application of artificial intelligence (AI) has led to novel discoveries across multiple disciplines including medicine [3, 4]. Much credible evidence have emerged in medical literature, suggesting that computer science can bring significant improvements to clinical medicine including radiology. Future collaboration among computer scientists and medical practitioners are critical to maximize the benefits of AI technology in the management of various diseases, especially neurological disorders and brain tumors. In recent years, AI advances have been made that enhance the diagnosis and management of brain tumors [1, 2].
Novel AI approaches, such as machine and deep learning methods, have been developed and continue to evolve for the segmentation, classification and grading of brain tumors. In radiology, these methods provide much insight into the diagnosis, treatment, follow-up and prognosis with the potential to bring significant efficiency to the clinical practice [1-5]. The following review of the most recent articles is an attempt to enhance the technical awareness of not only junior radiology residents and the practitioners, but also physicians in general medicine, neurology and neurosurgery. Also, researchers from computer science, neuroscience and other relevant areas will find this review very helpful.
2. Methods
For the purpose of this review, major search engines, such as Google Scholar, Scopus, Pubmed, and Medline, were searched, using the following keywords: artificial intelligence, machine learning, deep learning, brain tumors, magnetic resonance imaging, MRI, and radiological image analyses. Initially, 138 most recent peer-reviewed, original research and reviews on relevant topics were identified by the authors. After a careful review of the contents, 17 articles were selected [1-17], all of which describing various AI algorithms to analyze datasets from the MRI data of human brain tumors, in addition to the radiologists’ interpretations. This review briefly presents the significant findings and conclusions of the 17 articles, and the role of AI methods in the classification, segmentation, diagnosis and prognosis of brain tumors based on MRI data.
3. Literature Review
The radiology practice is undergoing a new development by taking advantage of some computer science components, such as artificial intelligence [1-5]. Machine and deep learning models are demanding techniques that assist radiologists in the efficient and valid analysis of brain tumor images. The profound task of classifying and segmenting brain tumors can be achieved through various artificial intelligence methods. Specifically, machine and deep learning techniques and algorithms, data mining and hybridization methods have enabled radiologists to obtain efficient segmentation and classification of brain tumors, compared to the interpretation of images from standard MRI [1-5].
The analysis of MRI and computer tomography (CT) data by a 5-year review of the studies conducted between 2014 and 2019 has suggested that most doctors prefer MRI over CT for staging human brain tumors [1]. For this purpose, a combination of deep learning and meta-heuristic methods provide for the crucial task of segmenting and classifying brain tumors [1]. These non-invasive techniques obviate the need for a surgical procedure to achieve similar clinical findings.
4. Machine Learning
4.1 Tumor classification
In 2015, an early machine learning study [6] used four algorithms for the automatic detection and classification of gliomas into benign or malignant group, based on MRI data. It found that a satisfactory area under the receiver operating characteristic (ROC) curves (>0.80) was achieved in three of the algorithms. The study further predicted better classification performance if the data were combined with those from other diagnostic modalities. At the time, this method held the promise of replacing the invasive methods of tumor grading through biopsy and histopathological analysis [6].
4.2 Predicting tumor grade
Later in 2016, a study [7] examined the diagnostic accuracy of grading glioma tumors by assessing the heterogeneity and texture analysis based on preoperative MRI data from 95 patients with low- or high-grade gliomas. The low and high grades were best discriminated with the sensitivity and specificity of 93% and 81%, respectively. This study concluded that assessing heterogeneity of gliomas based on MRI data improved the accuracy of grading and provided for a rapid treatment decision [7].
4.3 Predicting tumor stage
A study conducted in 2019 [2] used convolutional neural network (CNN) algorithm, which serves as a machine learning or a deep learning method. It classifies MRI data from human brain tumors at various stages, compared with those obtained from such methods as Random Forest and K-Nearest Neighbors. The CNN was determined as the best tumor classifying method with an average accuracy of 98% and validation accuracy of 71% [2].
4.4 Prognostic advantages
Another study conducted in 2019, reports that a multivariate machine learning model, i.e., resting-state functional connectivity (rsFC) predicted the severity of aphasia in 126 patients with left-sided low- or high-grade gliomas, affecting the brain’s speech centers [8]. The findings advance our knowledge regarding large-scale language network reorganization in patients with gliomas, and shed light on why the resection of the areas critical to language processing, may not necessarily induce aphasia [8].
5. Deep Learning
Deep learning technique facilitates accurate segmenting, classifying, grading, serially mapping tumor shape and texture, and predicting the survival of patients with brain tumors based on MRI and CT datasets [4, 5]. It further provides for better insight into brain tumors’ nature both qualitatively and quantitatively, and enables enhanced future care and clinical prognosis [5].
5.1 Radiation mapping
A study used diffusion tensor imaging (DTI) scans, a specific deep learning model, to estimate the infiltrative gross tumor volume (iGTV) in 33 patients with brain gliobastomas, compared to healthy standard GTV [9]. The study concluded that free water corrected DTI scans can assist the radiologist to define the infiltrative tumor areas and correctly plan the radiation therapy dosing [9].
5.2 Clinical advantages and challenges
There have been considerable advances in the prediction of molecular biomarkers, and the role of both machine and deep learning methods in the identification of diffuse glioma features, based on patients’ MRI data [10]. Also, there are few examples of molecular markers for other brain tumors [10]. In 2019, a study compared the application of three CNN methods used for the segmentation of the MRI data for 44 patients with brain glioblastoma between two institutions [11]. This study found that the performance of the model decreased considerably when it was applied to data from different institutions compared to those for a single institution. This was evident for the segmentation of the whole tumor and its components. The study concluded that there was a strong effect when data for training CNN models were chosen from multiple institutions, the elucidation of the reasons await further research [11].
5.3 Paired Vs unpaired data
A study published in 2019 [12] developed deep learning algorithms, termed end-to-end network architecture or RescueNet, for the unpaired segmentation of brain tumors from MRI data. It eliminated the need for paired data to train the network and the results were tested on BraTS 2015 & 2017 datasets. This network provided huge advantages and improved the automated algorithms to assist radiologists in analyzing unpaired MRI data [12].
5.4 Limited data
Prior to the above study [12], another work [13] introduced an automated algorithm for brain tumor segmentation. It used a limited MRI data (T1 & FLAIR) per patient, by which the tumor texture and abnormality maps were constructed. In this research, Random Forests classification and dedicated Voxel clustering delineated the tumor compartments, achieving good results for the segmentation of the enhanced brain tissue and the tumor core, and the identification of total abnormal regions [13].
5.5 Influence of data symmetry
Following a non-symmetrical segmentation approach [14] in 2018, another method was developed in 2019, termed deep convolutional symmetrical neural network (DCSNN) [15]. This study was more realistic, accurate and efficient than the former one [14] since it considered the symmetry as an important factor for the correct segmentation of glioma tumors. This method outperformed the complex mapping tasks and resolved the symmetry issue with the non-symmetrical methods [14]. The symmetrical method was validated based on BraTS-2015 database and the results evaluated by dice similarity coefficient (DSC) metrics, with the average DSC being 0.852. This method takes only 10.8 seconds to segment a brain tumor dataset [15].
5.6 Automated segmentation
Another study conducted in 2019 [16] provides a modified deep learning and end-to-end segmentation method, originally derived from CNN. It can automatically generate the segmentation maps slice by slice. The method, validated through BraTS 2015-2017, is currently a competitive, state-of-the-art segmentation approach [16].
5.7 Tumor progression and recurrence
A more recent study in 2019 [17] has examined multiple MRI sequences from the histologically confirmed, high grade gliomas of 55 patients to better identify the tumor progression. The MRI sequence that achieved the highest accuracy was used to develop the deep learning model. The results support the studies that suggest that machine and deep learning approaches are clinically preferred to correctly detect the progression and recurrence of such tumors [17].
Even with proper acquisition of brain tumor images, the accurate and reliable segmentation of brain tumors is a complicated task. Future research on deep learning should be focused on discovering better ways of identifying and understanding the predictive features for the clinical decision making, diagnosis, treatment and prognosis of various brain tumors [10].
6. Conclusions
The exciting role of artificial intelligence in the advancement of radiology practice is rapidly evolving based on the analysis of MRI data. Numerous machine and deep learning methods enable radiologists to perform accurate, rapid and efficient mapping, classification, segmentation, grading and staging of brain tumors while predicting the diagnosis, treatment, future care and prognosis of such cases.
Acknowledgements
The authors are grateful to Dr. Manoochehr Kelarestaghi, Assistant Professor, Kharazmi University, Karaj, Iran for for his excellent review of the manuscript. We appreciate the Office of Vice Chancellor for Research, Arak University of Medical Sciences, Arak, Iran, and the School of Management and Economics, Azad University’s the Science and Research Branch, Tehran, Iran, for their support of this study. Funding was provided by the first and second authors.
Conflict of Interests
The authors had no conflict of interest with any entity during the conduction of this study.
References
- Ravishankar P, Smith DA, Avril S, et al. Uterine carcinosarcoma: a primer for radiologists. Abdominal Radiology 44 (2019): 2874-2885.
- Sagebiel TL, Bhosale PR, Patnana M, et al. Uterine Carcinosarcomas. Seminars in Ultrasound, CT and MRI. 40 (2019): 295-301.
- Denschlag D, Ulrich UA. Uterine carcinosarcomas - Diagnosis and management. Oncology Research and Treatment 41 (2018): 675-679.
- Zhao F, Xu Y, Zhang H, et al. Ultrasonographic findings of uterine carcinosarcoma. Gynecol Obstet Invest 84 (2019): 277-282.
- Hosh M, Antar S, Nazzal A, et al. Uterine Sarcoma: Analysis of 13,089 Cases Based on Surveillance, Epidemiology, and End Results Database. Int J Gynecol Cancer 26 (2016).
- Epstein E, Fischerova D, Valentin L, et al. Ultrasound characteristics of endometrial cancer as defined by International Endometrial Tumor Analysis (IETA) consensus nomenclature: prospective multicenter study. Ultrasound Obstet Gynecol 51 (2018): 818-828.
- Skogen K, Schulz A, Dormagen JB, Ganeshan B, Helseth E, et al. Diagnostic performance of texture analysis on MRI in grading cerebral gliomas. Eur J Radiol 85 (2016): 824-829.
- Yuan B, Zhang N, Yan J, et al. Resting-state functional connectivity predicts individual language impairment of patients with left hemispheric gliomas involving language network. NeuroImage Clin 24 (2019): 102023.
- Peeken JC, Molina-Romero M, Diehl C, Menze BH, Straube C, et al. Deep learning derived tumor infiltration maps for personalized target definition in Glioblastoma radiotherapy. Radiotherapy and Oncology 138 (2019): 166-172.
- Korfiatis P, Erickson B. Deep learning can see the unseeable: Predicting molecular markers from MRI of brain gliomas. Clin Radiol 74 (2019): 367-373.
- AlBadawy EA, Saha A, Mazurowski MA. Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing. Medical Physics 45 (2018): 1150-1158.
- Ghaffari M, Sowmya A, Oliver R. Automated brain tumor Segmentation using multimodal brain scans: A survey based on models submitted to the BraTS 2012-18 challenges. IEEE Reviews in Biomed Eng 13 (2020): 156-168.
- Bonte S, Goethals I, Van Holen R. Machine learning-based brain tumor segmentation on limited data using local texture and abnormality. Computers in biology and medicine 98 (2018): 39-47.
- Hussain S, Anwar SM, Majid M. Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing. 282 (2018): 248-261.
- Chen H, Qin Z, Ding Y, et al. Brain tumor segmentation with deep convolutional symmetric neural network. Neurocomputing (2019).
- Li H, Li A, Wang M. A novel end-to-end brain tumor segmentation method using improved fully convolutional networks. Computers in Biology and Medicine 108 (2019): 150-160.
- Bacchi S, Zerner T, Dongas J, et al. Deep learning in the detection of high-grade glioma recurrence using multiple MRI sequences: A pilot study (2019).
- Bacchi S, Zerner T, Dongas J, et al. Deep learning in the detection of high-grade glioma recurrence using multiple MRI sequences: A pilot study. J Clin Neurosci 70 (2019): 11-13.