Applications of Image Segmentation Techniques in Medical Images

Yang-yang Hou

doi:10.4108/eetel.4449

Authors

Yang-yang Hou Jiangsu Second Normal University https://orcid.org/0009-0004-5391-7493

DOI:

https://doi.org/10.4108/eetel.4449

Keywords:

Medical images, Image segmentation, Deep learning, Neural networks

Abstract

Image segmentation is an important research direction in medical image processing tasks, and it is also a challenging task in the field of computer vision. At present, there have been many image segmentation methods, including traditional segmentation methods and deep learning-based segmentation methods. Through the understanding and learning of the current situation in the field of medical image segmentation, this paper systematically combs it. Firstly, it briefly introduces the traditional image segmentation methods such as threshold method, region method and graph cut method, and focuses on the commonly used network architectures based on deep learning such as CNN, FCN, U-Net, SegNet, PSPNet, Mask R-CNN. At the same time, the application in medical image segmentation is expounded. Finally, the challenges and development opportunities of medical image segmentation technology based on deep learning are discussed.

References

R. Wang, T. Lei, R. Cui, B. Zhang, H. Meng, and A. K. Nandi, "Medical image segmentation using deep learning: A survey," IET Image Processing, vol. 16, no. 5, pp. 1243-1267, 2022.

Y. Zhang, "Pathological brain detection in MRI scanning via Hu moment invariants and machine learning," Journal of Experimental & Theoretical Artificial Intelligence, vol. 29, no. 2, pp. 299-312, 2017.

S. Wang, "Pathological Brain Detection by a Novel Image Feature—Fractional Fourier Entropy," Entropy, vol. 17, no. 12, pp. 8278-8296, 2015.

P. Malhotra, S. Gupta, D. Koundal, A. Zaguia, and W. Enbeyle, "Deep Neural Networks for Medical Image Segmentation," Journal of Healthcare Engineering, vol. 2022, p. 9580991, 2022.

A. Qi et al., "Directional mutation and crossover boosted ant colony optimization with application to COVID-19 X-ray image segmentation," Computers in Biology and Medicine, vol. 148, p. 105810, 2022.

L. Abualigah, A. Diabat, P. Sumari, and A. H. Gandomi, A Novel Evolutionary Arithmetic Optimization Algorithm for Multilevel Thresholding Segmentation of COVID-19 CT Images (no. 7). 2021, p. 1155.

L. He and S. Huang, "An efficient krill herd algorithm for color image multilevel thresholding segmentation problem," Applied Soft Computing, vol. 89, p. 106063, 2020.

E. H. Houssein, M. M. Emam, and A. A. Ali, "An efficient multilevel thresholding segmentation method for thermography breast cancer imaging based on improved chimp optimization algorithm," Expert Systems with Applications, vol. 185, p. 115651, 2021.

S. Zhao, D. M. Zhang, and H. W. Huang, "Deep learning–based image instance segmentation for moisture marks of shield tunnel lining," Tunnelling and Underground Space Technology, vol. 95, p. 103156, 2020.

S. L. Bangare, "Classification of optimal brain tissue using dynamic region growing and fuzzy min-max neural network in brain magnetic resonance images," Neuroscience Informatics, vol. 2, no. 3, p. 100019, 2022.

Q. Li, J. Geng, D. Song, W. Nie, P. Saffari, and J. Liu, "Automatic recognition of erosion area on the slope of tailings dam using region growing segmentation algorithm," Arabian Journal of Geosciences, vol. 15, no. 5, p. 438, 2022.

X. Liu, H. Tian, Y. Wang, F. Jiang, and C. Zhang, "Research on Image Segmentation Algorithm and Performance of Power Insulator Based on Adaptive Region Growing," Journal of Electrical Engineering & Technology, vol. 17, no. 6, pp. 3601-3612, 2022.

F. Poux, C. Mattes, Z. Selman, and L. Kobbelt, "Automatic region-growing system for the segmentation of large point clouds," Automation in Construction, vol. 138, p. 104250, 2022.

M. Mushtaq, M. U. Akram, N. S. Alghamdi, J. Fatima, and R. F. Masood, Localization and Edge-Based Segmentation of Lumbar Spine Vertebrae to Identify the Deformities Using Deep Learning Models (no. 4). 2022, p. 1547.

Z. Zhang, H. Fu, H. Dai, J. Shen, Y. Pang, and L. Shao, "ET-Net: A Generic Edge-aTtention Guidance Network for Medical Image Segmentation," in Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, Cham, 2019, pp. 442-450: Springer International Publishing.

Y. Zhang, "Feature Extraction of Brain MRI by Stationary Wavelet Transform and its Applications," Journal of Biological Systems, vol. 18, no. S, pp. 115-132, 2010.

Z. Liu et al., "Liver CT sequence segmentation based with improved U-Net and graph cut," Expert Systems with Applications, vol. 126, pp. 54-63, 2019.

D. Lu et al., "Deep-learning based multiclass retinal fluid segmentation and detection in optical coherence tomography images using a fully convolutional neural network," Medical Image Analysis, vol. 54, pp. 100-110, 2019.

M. N. Reza, I. S. Na, S. W. Baek, and K.-H. Lee, "Rice yield estimation based on K-means clustering with graph-cut segmentation using low-altitude UAV images," Biosystems Engineering, vol. 177, pp. 109-121, 2019.

Y. Su, M. Jiang, L. Gao, X. You, X. Sun, and P. Li, "Graph-Cut-Based Node Embedding for Dimensionality Reduction and Classification of Hyperspectral Remote Sensing Images," in IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium, 2022, pp. 1720-1723.

X. Zhu, Z. Cheng, S. Wang, X. Chen, and G. Lu, "Coronary angiography image segmentation based on PSPNet," Computer Methods and Programs in Biomedicine, vol. 200, p. 105897, 2021.

S. Minaee, Y. Boykov, F. Porikli, A. Plaza, N. Kehtarnavaz, and D. Terzopoulos, "Image Segmentation Using Deep Learning: A Survey," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3523-3542, 2022.

G. Wei, G. Li, J. Zhao, and A. He, Development of a LeNet-5 Gas Identification CNN Structure for Electronic Noses (no. 1). 2019, p. 217.

T. Kattenborn, J. Leitloff, F. Schiefer, and S. Hinz, "Review on Convolutional Neural Networks (CNN) in vegetation remote sensing," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 173, pp. 24-49, 2021.

X. Lei, H. Pan, and X. Huang, "A Dilated CNN Model for Image Classification," IEEE Access, vol. 7, pp. 124087-124095, 2019.

Y. Li, J. Nie, and X. Chao, "Do we really need deep CNN for plant diseases identification?," Computers and Electronics in Agriculture, vol. 178, p. 105803, 2020/11/01/ 2020.

J. Lu, L. Tan, and H. Jiang, Review on Convolutional Neural Network (CNN) Applied to Plant Leaf Disease Classification (no. 8). 2021, p. 707.

Z. Zhang and X. Zhang, "MIDCAN: A multiple input deep convolutional attention network for Covid-19 diagnosis based on chest CT and chest X-ray," Pattern Recognition Letters, vol. 150, pp. 8-16, 2021.

Y. Sun, B. Xue, M. Zhang, and G. G. Yen, "Completely Automated CNN Architecture Design Based on Blocks," IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 4, pp. 1242-1254, 2020.

Y.-D. Zhang, "ANC: Attention Network for COVID-19 Explainable Diagnosis Based on Convolutional Block Attention Module," Computer Modeling in Engineering & Sciences, vol. 127, pp. 1037-1058, 2021.

M. F. Aslan, M. F. Unlersen, K. Sabanci, and A. Durdu, "CNN-based transfer learning–BiLSTM network: A novel approach for COVID-19 infection detection," Applied Soft Computing, vol. 98, p. 106912, 2021.

A. Chavda, J. Dsouza, S. Badgujar, and A. Damani, "Multi-Stage CNN Architecture for Face Mask Detection," in 2021 6th International Conference for Convergence in Technology (I2CT), 2021, pp. 1-8.

Y. Ji, H. Zhang, Z. Zhang, and M. Liu, "CNN-based encoder-decoder networks for salient object detection: A comprehensive review and recent advances," Information Sciences, vol. 546, pp. 835-857, 2021.

C. Tian, Y. Xu, Z. Li, W. Zuo, L. Fei, and H. Liu, "Attention-guided CNN for image denoising," Neural Networks, vol. 124, pp. 117-129, 2020.

Y. Yan, "A Survey of Computer-Aided Tumor Diagnosis Based on Convolutional Neural Network," Biology, vol. 10, no. 11. doi: 10.3390/biology10111084

A. Ben-Cohen, I. Diamant, E. Klang, M. Amitai, and H. Greenspan, "Fully Convolutional Network for Liver Segmentation and Lesions Detection," in Deep Learning and Data Labeling for Medical Applications, 2016, pp. 77-85: Springer International Publishing.

Y. Chen et al., "Automatic Segmentation of Individual Tooth in Dental CBCT Images From Tooth Surface Map by a Multi-Task FCN," IEEE Access, vol. 8, pp. 97296-97309, 2020.

J. Wu, B. Liu, H. Zhang, S. He, and Q. Yang, Fault Detection Based on Fully Convolutional Networks (FCN) (no. 3). 2021, p. 259.

Y.-D. Zhang, "Improving ductal carcinoma in situ classification by convolutional neural network with exponential linear unit and rank-based weighted pooling," Complex & Intelligent Systems, vol. 7, pp. 1295-1310, 2021.

J. Zhang, C. Lu, J. Wang, L. Wang, and X.-G. Yue, Concrete Cracks Detection Based on FCN with Dilated Convolution (no. 13). 2019, p. 2686.

Y. Yuan, M. Chao, and Y. C. Lo, "Automatic Skin Lesion Segmentation Using Deep Fully Convolutional Networks With Jaccard Distance," IEEE Transactions on Medical Imaging, vol. 36, no. 9, pp. 1876-1886, 2017.

A. Dasgupta and S. Singh, "A fully convolutional neural network based structured prediction approach towards the retinal vessel segmentation," in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), 2017, pp. 248-251.

A. A. Ijjeh, S. Ullah, and P. Kudela, "Full wavefield processing by using FCN for delamination detection," Mechanical Systems and Signal Processing, vol. 153, p. 107537, 2021.

O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, 2015, pp. 234-241: Springer International Publishing.

N. Siddique, S. Paheding, C. P. Elkin, and V. Devabhaktuni, "U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications," IEEE Access, vol. 9, pp. 82031-82057, 2021.

H. Wang et al., "Mixed Transformer U-Net for Medical Image Segmentation," in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 2390-2394.

Y.-T. Weng, H.-W. Chan, and T.-Y. Huang, "Automatic Segmentation of Brain Tumor from 3D MR Images Using SegNet, U-Net, and PSP-Net," in Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Cham, 2020, pp. 226-233: Springer International Publishing.

X. Wu, D. Hong, and J. Chanussot, "UIU-Net: U-Net in U-Net for Infrared Small Object Detection," IEEE Transactions on Image Processing, vol. 32, pp. 364-376, 2023.

J. Zhang et al., "LCU-Net: A novel low-cost U-Net for environmental microorganism image segmentation," Pattern Recognition, vol. 115, p. 107885, 2021.

Z. Zhang, C. Wu, S. Coleman, and D. Kerr, "DENSE-INception U-net for medical image segmentation," Computer Methods and Programs in Biomedicine, vol. 192, p. 105395, 2020.

P. Kaur Buttar and M. K. Sachan, "Semantic segmentation of clouds in satellite images based on U-Net++ architecture and attention mechanism," Expert Systems with Applications, vol. 209, p. 118380, 2022.

C. Zhao et al., "Context-aware network fusing transformer and V-Net for semi-supervised segmentation of 3D left atrium," Expert Systems with Applications, vol. 214, p. 119105, 2023.

S. Almotairi, G. Kareem, M. Aouf, B. Almutairi, and M. A.-M. Salem, Liver Tumor Segmentation in CT Scans Using Modified SegNet (no. 5). 2020, p. 1516.

T. Chen et al., "Pavement crack detection and recognition using the architecture of segNet," Journal of Industrial Information Integration, vol. 18, p. 100144, 2020/06/01/ 2020.

N. Yamanakkanavar, J. Y. Choi, and B. Lee, SM-SegNet: A Lightweight Squeeze M-SegNet for Tissue Segmentation in Brain MRI Scans (no. 14). 2022, p. 5148.

Z. Yan et al., "SegNet-based left ventricular MRI segmentation for the diagnosis of cardiac hypertrophy and myocardial infarction," Computer Methods and Programs in Biomedicine, vol. 227, p. 107197, 2022.

Y. Qiao et al., A Diameter Measurement Method of Red Jujubes Trunk Based on Improved PSPNet (no. 8). 2022, p. 1140.

X. Wang, Y. Guo, S. Wang, G. Cheng, X. Wang, and L. He, "Rapid detection of incomplete coal and gangue based on improved PSPNet," Measurement, vol. 201, p. 111646, 2022.

C. Yang, H. Guo, Y. Xie, J. Tian, and X. X. Zhu, "A Method of Image Semantic Segmentation Based on PSPNet

Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation," Mathematical Problems in Engineering, vol. 2022, no. 4, pp. 8958154-59, 2022.

L. Yan et al., "PSP net-based automatic segmentation network model for prostate magnetic resonance imaging," Computer Methods and Programs in Biomedicine, vol. 207, p. 106211, 2021.

P. Bharati and A. Pramanik, "Deep Learning Techniques—R-CNN to Mask R-CNN: A Survey," in Computational Intelligence in Pattern Recognition, Singapore, 2020, pp. 657-668: Springer Singapore.

H. Yu et al., "A multi-stage data augmentation and AD-ResNet-based method for EPB utilization factor prediction," Automation in Construction, vol. 147, p. 104734, 2023.

P. Chu, Z. Li, K. Lammers, R. Lu, and X. Liu, "Deep learning-based apple detection using a suppression mask R-CNN," Pattern Recognition Letters, vol. 147, pp. 206-211, 2021.

M. Machefer, F. Lemarchand, V. Bonnefond, A. Hitchins, and P. Sidiropoulos, Mask R-CNN Refitting Strategy for Plant Counting and Sizing in UAV Imagery (no. 18). 2020, p. 3015.

Y. Tian, G. Yang, Z. Wang, E. Li, and Z. Liang, "Instance segmentation of apple flowers using the improved mask R–CNN model," Biosystems Engineering, vol. 193, pp. 264-278, 2020.

X. Xu et al., Crack Detection and Comparison Study Based on Faster R-CNN and Mask R-CNN (no. 3). 2022, p. 1215.

J. M. J. Valanarasu and V. M. Patel, "UNeXt: MLP-Based Rapid Medical Image Segmentation Network," in Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, Cham, 2022, pp. 23-33: Springer Nature Switzerland.

Y. Zhang, "Pathological brain detection in MRI scanning by wavelet packet Tsallis entropy and fuzzy support vector machine," SpringerPlus, vol. 4, no. 1, 2015, Art. no. 716.

Y. D. Zhang and S. Satapathy, "A seven-layer convolutional neural network for chest CT-based COVID-19 diagnosis using stochastic pooling," IEEE Sensors Journal, vol. 22, no. 18, pp. 17573 - 17582, 2022.

I. Qureshi et al., "Medical image segmentation using deep semantic-based methods: A review of techniques, applications and emerging trends," Information Fusion, vol. 90, pp. 316-352, 2023.

S. Wang, "Grad-CAM: understanding AI models," Computers, Materials & Continua, vol. 76, no. 2, pp. 1321-1324, 2023.

M. Arabahmadi, R. Farahbakhsh, and J. Rezazadeh, Deep Learning for Smart Healthcare—A Survey on Brain Tumor Detection from Medical Imaging (no. 5). 2022, p. 1960.

K. Treb, Y. Hämisch, C. Ullberg, R. Zhang, and K. Li, "Photon counting-energy integrating hybrid flat panel detector systems for image-guided interventions: an experimental proof-of-concept," Physics in Medicine & Biology, vol. 68, no. 13, p. 135009, 2023.

T. Zhou, S. Ruan, and S. Canu, "A review: Deep learning for medical image segmentation using multi-modality fusion," Array, vol. 3-4, p. 100004, 2019.

A. Lin, B. Chen, J. Xu, Z. Zhang, G. Lu, and D. Zhang, "DS-TransUNet: Dual Swin Transformer U-Net for Medical Image Segmentation," IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1-15, 2022.

S. Mishra, Y. Zhang, D. Z. Chen, and X. S. Hu, "Data-Driven Deep Supervision for Medical Image Segmentation," IEEE Transactions on Medical Imaging, vol. 41, no. 6, pp. 1560-1574, 2022.

D. Karimi, H. Dou, and A. Gholipour, "Medical Image Segmentation Using Transformer Networks," IEEE Access, vol. 10, pp. 29322-29332, 2022.

S. Niyas, S. J. Pawan, M. Anand Kumar, and J. Rajan, "Medical image segmentation with 3D convolutional neural networks: A survey," Neurocomputing, vol. 493, pp. 397-413, 2022.