ISSN (e): 2250 – 3005 || Volume, 14 || Issue, 4|| Jul. - Aug.

– 2024 ||
International Journal of Computational Engineering Research (IJCER)

A Review on Medical Image Analysis using Deep Learning

Jharana Paikray1, Subhadra Biswal2, Ananya Pritipadma3
1,2,3
Department of Computer Science and Engineering, Einstein Academy of Technology & Management,
Bhubaneswar, India

Abstract—
This survey focuses on the analysis and processing of medical images, including MRI, CT scans, PETs,
ultrasounds, and X-rays, using Artificial Intelligence (AI) and machine learning techniques. Software
frameworks like Nifty Net and Milson are used to study various body parts, and big data datasets like Image Net
are used for analysis.

Keywords—Deep Learning, Medical Image Analysis

I. INTRODUCTION
Various fields of life are using Artificial Intelligence (AI). Medical and healthcare sector is not
untouched by ANN, ML and then ultimately deep learning. CNNs are used for various medical image analysis
purposes. Field of X-ray radiology [1], Ultrasound, CT scan, Medical Resonance Imaging (MRI) scans, positron
emission tomography (PET) are being improved using DNNs and CNNs [2]. Gone are the days, when medical
images had limited datasets. Now a day, big data is being analyzed and evaluated using deep learning
techniques. Many software algorithms and software’s like Nifty Net [3] and MIScnn [4] are being developed
continuously, particularly used for medical image analysis. Imaging techniques like classification, segmentation,
localization, detection and registration are being studied [5]. Medical application areas like neuro, retinal,
pathology, cardiac, breast etc are being studied [6].

II. CONVOLUTIONAL NEURAL NETWORK

CNNs have many layers like input layer, Rectified Linear Unit (RELU) layer, pooling layer and output
layer decide whether image is disease infected or not [5]. Image acquiring, image artifact removal, normalizing
images and image quality improvement is done using different deep learning methods [7]. Supervised and
unsupervised learning techniques have been used for these purposes [5].

Figure.1 CNN layered architecture

III. MEDICAL IMAGE ANALYSIS IN PRESENT DAY

X. Chen et al. (2021) – In this paper, image registration algorithms have been studied. This paper discusses all
deep learning based medical registration since 2013. Various image registration methods in recent times are
studied [8].
Muller & Kramer (2021) - In this paper, open source Python framework MIScnn is discussed for image
segmentation. It is used for deep learning model training, prediction and evaluation. MI Scan is run on Kidney
Tumor segmentation. It is an open source library available at Git repository [4]. Jungo et al. (2021) – Pymia is
an open source Python based package for data handling and evaluation in medical image processing using deep
learning. Data handling is flexible (2D, 3D, full or patch wise) with independence of deep learning framework.

[Link] Open Access Journal Page 264

 A Review on Medical Image Analysis using Deep Learning

Integrating smoothly in the present deep learning frameworks like Tensor Flow and PyTorch. Functionalities
like result calculation and reporting (CSV files, console) and training progress monitoring. Large number of
domainspecific metrics for image segmentation, reconstruction and regression, Alex Net, U-net, GPUs and
increase in data availability enhanced impact of deep learning [9].
Xie et al. (2021) – Computer aided detection/diagnosis has progressed a lot. It is used for lung cancer,
breast cancer, glaucoma and skin cancer. Datasets like Image Net, COCO, ChestXray14 and DeepLesion suffer
from small dataset. Knowledge and experience i.e., domainknowledge of doctors are added in these datasets to
improve them. Additional knowledge is incorporated from natural datasets or other medical datasets. Training
and diagnostic pattern of medical doctors is added to datasets. Lesions are detected using improved datasets
[10].
Ma et al. (2021) – Adversarial attacks/examples can compromise with medical deep learning systems.
Adversarial attacks can be detected easily [11].
Wiestler & Menze (2020) – In this paper, advancement in deep learning have developed neural network
algorithms surpassing human performance in image classification and segmentation. Computer scientists and
neurooncologists are going hand-in-hand for understanding deep learning techniques. Building blocks of
artificial neural network like convolutional neural network are studied. Imaging features like genotype are also
explained. Hurdles in wide use of algorithms are explained [12].
Renard et al. (2020) – In this paper, medical image segmentation tool is used. It is used for clinical
diagnosis and computer assisted surgeries. Accuracy is improved by deep learning algorithms in comparison to
classical segmentation. Deep learning methods are complex in nature. They have high variability so calls for
reproducibility of results. In this literature review, variability and reproducibility for medical images using deep
learning technique have been studied. Mainly, three fields are studied (1) framework of deep learning (2)
variability sources (3) segmentation results evaluation [13].
Smailagic et al. (2020) – O-MedAL, an online deep learning method for medical image analysis is dealt
with. MedAL AL framework is extended. Technologies machine learning and classification are used in
healthcare application areas. Active Learning (AL) is studied in detail on the basis of novelty, accuracy, data
efficiency and computation efficiency [14].
Zhu et al. (2019) – In this paper, overview of deep learning for neuro imaging methods are used.
Radiological clinical applications for classification, risk assessment, segmentation, etc., AI techniques are also
employed for image acquiring, image artifact removal, normalizing images and image quality improvements,
reducing radiation dose and reducing time interval of imaging [7].
Maier et al. (2019) – Reasons for popularity of deep learning are studied. Fundamentals like basic
neuron and other artificial neural networks are studied. Network training and deep learning are also discussed.
Important architectures in deep learning like Auto-encoders, Generative Adversarial networks (GANs),
Google’s Inception network, Ronneburger’s U- net, Resents, Variational networks, RNNs are dealt with.
Advanced deep learning concepts like Data Augmentation, Precision learning, Adversarial examples, Deep
reinforcement learning are also touched upon [15].
Latif et al. (2019) – In this paper, image classification, object detection, pattern recognition, etc. are
surveyed. Deep learning supervised and unsupervised learning techniques have been used for prediction using
standard dataset. Accuracy is increased by extracting meaningful patterns for some particular diseases. This is a
survey kind of paper regarding ML and deep learning methods for medical images [16].

IV. MEDICAL IMAGE ANALYSIS FEW YEARS BACK

Yoon et al. (2019) – In this paper, automated analytical systems as a database system are used for
improving medical images using computers. Construction of large data is done. Deep learning AI architectures
are used for high precision diagnosis of medical images. Labeling and standardization of medical images is
done. Quick and accurate final diagnosis results are observed. Data augmentation is done through rotation, by
left and right flips to increase the data.
Lundervold & Lundervold (2019) - Medical image processing using Deep learning is done especially
considering MRI. ANN, ML then deep learning improved medical data analysis and diagnostics. Focus, in this
paper, is on MRI like MRI acquisition, retrieval, segmentation to disease prediction. Big data, software
frameworks, computing ability, use of neural networks is deeper than previous times. Building blocks of CNN
like convolutional layers, Pooling, Dropout regularization and Batch normalization are discussed. CNN
architectures like Alex Net, VGG, Google Net, Resent, Highway nets, Dense net, ResNext, SEnets, NASNets,
YOLO, GANs, Siamese nets, V-net are compared. Process from image acquisition to image registration is
explained [18].
Budd et al. (2019) – In this paper, role of humans in fully automatic deep learning is found to be
beneficial. In future research, human-in-theloop computing needs to be explored. Key areas discussed are 1)

[Link] Open Access Journal Page 265

 A Review on Medical Image Analysis using Deep Learning

Active Learning, 2) Interpretation and refinement, 3) Practical considerations and 4) Related areas are explored
[19].
Fourcade & Khonsari (2019) – In this paper, the problem investigated is whether algorithms of deep
learning may increase visual diagnosis in medicine. Systematic study of research papers in various medical
arenas like Dermatology, Thoracic, Sinology, O-M-F surgery, Gastroenter logy are studied [20].
Gibson et al. (2018) – Nifty Net which is an open source platform for medical imaging is dealt with. It
accelerates and simplify deep learning solutions. Nifty Net helps in imaging applications like segmentation,
regression, image generation and representation [3].

Figure 2. Artificial Neural Network

Razzak et al. (2018) – In this paper healthcare sector which is using deep learning with good accuracy
is discussed. Quest for deep learning and machine learning is discussed. ANN and deep learning architecture is
studied. Comparison of various architectures is done like DNN, CNN, RNN, Deep Conventional Extreme
Learning Machine (DC-ELM), Deep Boltzmann Machine (DBM), Deep Belief Network (DBN), Deep Auto-
encoder (dA). Applications like diabetic Retinopathy, Histological elements detection, Gastrointestinal (GI)
disease detection, Alzheimer’s and Parkinson’s disease detection is done [21].
Ding gang Shen et al. (2017) – In this paper, deep models like feed forward neural network
unsupervised feature representation learning, stacked auto-encoders, deep belief networks, deep Boltzmann
machine, CNNs are studied. Organ or body part detection, cell detection is done. Deep learning detection and
diagnosis are studied with the help of computer system [30]. Ker at al. (2017) – History of Medical image
processing is studied in this review paper. CNNs, convolution layers, Rectified Linear Unit (RELU) layer, fully
connected layer are studied in case of supervised learning models. Auto-encoders, Restricted Boltzmann
Machines and DBNs, generative adversarial networks are studied in case of unsupervised learning models.
Usage of medical image analysis like classification, localization, detection, segmentation, registration is dealt
with [5].

Figure.3. Flowchart
Bhat (2017) – This survey deals with deep learning for object detection, image classification,
segmentation and registration. Application areas are neuro, retinal, pathology, cardiac and breast [6].
M. Chen et al. (2017) – Computed tomography is used to help disease diagnosis. Computer assisted
diagnosis (CAD) using AI has shown importance in healthcare. Labeled dataset for CT analysis creates a lot of

[Link] Open Access Journal Page 266

 A Review on Medical Image Analysis using Deep Learning

problem. In this paper, Convolutional encoder deep learning framework is used for lung nodules. It requires a
small amount of labeled data for learning of features. Convolutional encoder method can be used for similarity
measurement of lung nodule images [22].
Pawlowski et al. (2017) – In this paper, DLTK (Deep Learning Toolkit) for medical image analysis is
discussed. DLTK is based on Tensor Flow. FCN and U-Net are tested. It is especially created for biomedical
image analysis. U-Net is proved better than FCN except in case of right adrenal gland [23].
Litjens et al. (2017) – Very extensive survey on CNN technique. Use of deep learning for image
classification, detection, segmentation etc. is surveyed. Supervised and unsupervised learning algorithms are
discussed. Deep CNN architectures are studied. Deep learning uses for lesion classification, organ localization,
lesion detection, segmentation and registration are surveyed. Image retrieval, image generation and
enhancement also discussed [24].
De Bruijne (2016) – In this paper, image diagnosis, and disease prognosis and risk assessment using
machine learning are studied. Three main factors discussed are (1) variation in imaging protocol, (2) learning
from weak labels and (3) interpreting and evaluating results [25].
Cho et al. (2015) – In this paper, Convolutional Neural Networks (CNN) are used for natural image
classification systems. It has given very good results. Use of CNN systems to medical image classification is
very promising. But without accuracy goal cannot be achieved. Optimum size of training data set is determined
for high accuracy with low variance [26].
Bar et al. (2015) – In this paper, deep learning techniques are examined for pathology detection in chest
radiograph data. CNNs are used because they can learn mid and high level image representations. CNN is
explored for different pathologies in chest X-ray images. Non- medical learning deep learning technique is used.
Image Net is used to train CNN [1].
Xu et al. (2014) – In this paper, automatic extraction of feature representation is done using deep
learning (DNN). Multiple instance learning (MIL) framework is used for classification training with deep
learning features. Automatic feature learning is better than manual feature. Unsupervised technique can reach
nearly equal performance to fully supervised technique [27].
Brosch & Tam (2013) – In this paper, many times learning of medical images for modeling anatomical
parameters including applications like segmentation & registration. Many times learning method uses layered
networks (deep belief networks DBNs). DBNs are being employed in computer vision because of object
recognition. Efficient training technique for DBNs for training 3D medical images practically [28].
Alison Noble et al. (2011) – For more informative clinical information of medical images like X-ray, computed
tomography, magnetic resonance imaging (MRI), positron.

V. CONCLUSION
R language and even Mat lab can be used for the analysis of various medical images. Deep learning
toolbox can be very useful for further investigation and improvement of medical images. Though different
techniques of Artificial Intelligence, machine learning and deep learning have been extensively used in this
review for the analysis purpose.

REFERENCES
[1]. Y. Bar, I. Diamant, L. Wolf, and H. Greenspan, “Deep learning with non-medical training used for chest pathology identification,”
Med. Imaging 2015 Comput. Diagnosis, vol. 9414, p. 94140V, 2015, doi: 10.1117/12.2083124.
[2]. J. Alison Noble, N. Navab, and H. Becher, “Ultrasonic image analysis and image guided interventions,” Interface Focus, vol. 1, no.
4, pp. 673–685, 2011, doi: 10.1098/rsfs.2011.0025.
[3]. E. Gibson et al., “NiftyNet: a deep-learning platform for medical imaging,” Comput. Methods Programs Biomed., vol. 158, pp.
113–122, 2018, doi: 10.1016/[Link].2018.01.025.
[4]. D. Müller and F. Kramer, “MIScnn: a framework for medical image segmentation with convolutional neural networks and deep
learning,” BMC Med. Imaging, vol. 21, no. 1, pp. 1–11, 2021, doi: 10.1186/s12880-020- 00543-7.
[5]. J. Ker, L. Wang, J. Rao, and T. Lim, “Deep Learning Applications in Medical Image Analysis,” IEEE Access, vol. 6, pp. 9375–
9379, 2017, doi: 10.1109/ACCESS.2017.2788044.
[6]. A. R. Bhat, “Scope of Deep learning in medical image analysis : A survey,” Int. J. Adv. Res. Comput. Eng. Technol., vol. 6, no. 8,
pp. 1218–1223, 2017.
[7]. G. Zhu, B. Jiang, L. Tong, Y. Xie, G. Zaharchuk, and M. Wintermark, “Applications of deep learning to neuroimaging techniques,”
Front. Neurol., vol. 10, no. AUG, pp. 1–13, 2019, doi: 10.3389/fneur.2019.00869.
[8]. X. Chen, A. Diaz-pinto, N. Ravikumar, and A. F. Frangi, “Progress in A. Smailagic et al., “O-MedAL: Online active deep learning
for medical image analysis,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 10, no. 4, pp. 1–15, 2020, doi:
10.1002/widm.1353.
[9]. A. Maier, C. Syben, T. Lasser, and C. Riess, “A gentle introduction to deep learning in medical image processing,” Z. Med. Phys.,
vol. 29, no. 2, pp. 86–101, 2019, doi: 10.1016/[Link].2018.12.003.
[10]. J. Latif, C. Xiao, A. Imran, and S. Tu, “Medical Imaging using Machine Learning and,”2019 2nd Int. Conf. Comput. Math. Eng.
Technol., pp. 1–5, 2019.
[11]. H. J. Yoon, Y. J. Jeong, H. Kang, J. E. Jeong, and D.-Y. Kang, “Medical Image Analysis Using Artificial Intelligence,” Prog. Med.
Phys., vol. 30, no. 2, p. 49, 2019, doi: 10.14316/pmp.2019.30.2.49.

[Link] Open Access Journal Page 267

 A Review on Medical Image Analysis using Deep Learning
[12]. A. S. Lundervold and A. Lundervold, “An overview of deep learning in medical imaging focusing on MRI,” Z. Med. Phys., vol. 29,
no. 2, pp. 102–127, 2019, doi: 10.1016/[Link].2018.11.002.
[13]. S. Budd, E. C. Robinson, and B. Kainz, “Survey on active learning and human-in-the- loop deep learning for medical image
analysis,” arXiv, 2019, doi: 10.1016/[Link].2021.102062.
[14]. A. Fourcade and R. H. Khonsari, “Deep learning in medical image analysis: A third eye for doctors,” J. Stomatol. Oral Maxillofac.
Surg., vol. 120, no. 4, pp. 279–288, 2019, doi: 10.1016/[Link].2019.06.002.
[15]. Biomedical Engineering Progress in Biomedical Engineering OPEN ACCESS,” 2021.
[16]. A. Jungo, O. Scheidegger, M. Reyes, and F. Balsiger, “pymia: A Python package for data handling and evaluation in deep learning-
based medical image analysis,” Comput. Methods Programs Biomed., vol. 198, p. 105796, 2021, doi: 10.1016/[Link].2020.105796.
[17]. X. Xie, J. Niu, X. Liu, Z. Chen, S. Tang, and S. Yu, “A survey on incorporating domain knowledge into deep learning for medical
image analysis,” Med. Image Anal., vol. 69, pp. 1–27, 2021, doi: 10.1016/[Link].2021.101985.

[Link] Open Access Journal Page 268