Superresolution Reconstruction of Magnetic Resonance Images Based on a Nonlocal Graph Network
DOI:
https://doi.org/10.4108/eetiot.v8i29.769Keywords:
Magnetic resonance imaging, Superresolution reconstruction, Nonlocal operation, Nonlocal self-similarity, Graph attentionAbstract
INTRODUCTION: High-resolution (HR) medical images are very important for doctors when diagnosing the internal pathological structures of patients and formulating precise treatment plans.
OBJECTIVES: Other methods of superresolution cannot adequately capture nonlocal self-similarity information of images. To solve this problem, we proposed using graph convolution to capture non-local self-similar information.
METHODS: This paper proposed a nonlocal graph network (NLGN) to perform single magnetic resonance (MR) image SR. Specifically, the proposed network comprises a nonlocal graph module (NLGM) and a nonlocal graph attention block (NLGAB). The NLGM is designed with densely connected residual blocks, which can fully explore the features of input images and prevent the loss of information. The NLGAB is presented to efficiently capture the dependency relationships among the given data by merging a nonlocal operation (NL) and a graph attention layer (GAL). In addition, to enable the current node to aggregate more beneficial information, when information is aggregated, we aggregate the neighbor nodes that are closest to the current node.
RESULTS: For the scale r=2, the proposed NLGN achieves PSNR of 38.54 dB and SSIM of 0.9818 on the T(T1, BD) dataset, and yielding a 0.27 dB and 0.0008 improvement over the CSN method, respectively.
CONCLUSION: The experimental results obtained on the IXI dataset show that the proposed NLGN performs better than the state-of-the-art methods.
Downloads
References
Li, M. A coarse-to-fine face hallucination method by exploiting facial prior knowledge. In C. N. Proceedings. Proceedings of the 25th IEEE International Conference on Image Processing; October 7–10, 2018; Athens, Greece. Piscataway, NJ: IEEE: 2018. pp. 61–65.
Zhang, Z., Shi, Y., Zhou, X., Kan, H. and Wen, J. Shuffle block srgan for face image super-resolution reconstruction. Measurement and Control. 2020; 53(7-8): 1429–1439.
Pan, Z., Yu, J., Huang, H., Hu, S., Zhang, A., Ma, H. and Sun, W. (2013) Super-resolution based on compressive sensing and structural self-similarity for remote sensing images. IEEE Transactions on Geoscience and Remote Sensing. 2013; 51(9): 4864–4876.
Zhang, S., Yuan, Q., Li, J., Sun, J. and Zhang, X. Scene-adaptive remote sensing image super-resolution using a multiscale attention network. IEEE Transactions on Geoscience and Remote Sensing. 2013; 58(7): 4764–4779.
Hu, J. (2016) Single image super resolution of 3D MRI using local regression and intermodality priors. In Falco, C.M. Proceedings Volume 10033. Proceedings of the Eighth International Conference on Digital Image Processing; 2016; Chengdu, China. US; SPIE;2016. 10033. 866–871.
Zhao, X., Zhang, Y., Zhang, T. and Zou, X. Channel splitting network for single MR image superresolution. IEEE transactions on image processing. 2019; 28(11): 5649—5662.
Shi, J., Li, Z., Ying, S., Wang, C., Liu, Q., Zhang, Q. and Yan, P. Mr image super-resolution via wide residual networks with fixed skip connection. IEEE Journal of Biomedical and Health Informatics. 2019; 23(3): 1129–1140.
Song, T.A., Chowdhury, S.R., Yang, F. and Dutta, J. Super-resolution pet imaging using convolutional neural networks. IEEE Transactions on Computational Imaging. 2020; 6(1): 518–528.
Keys, R. Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing. 1981; 29(6): 1153–1160.
Medoff, B.P., Brody, W.R., Nassi, M. and Macovski, A. Iterative convolution backprojection algorithms for image reconstruction from limited data. J. Opt. Soc. Am. 1983; 73(11): 1493–1500.
Stark, H. and Oskoui, P. High-resolution image recovery from image-plane arrays, using convex projections. J. Opt. Soc. Am. A. 1989; 6(11): 1715–1726.
Yang, J., Wright, J., Huang, T.S. and Ma, Y. Image super-resolution via sparse representation. IEEE Transactions on Image Processing. 2010; 19(11): 2861–2873.
Zhu, Q. Image superresolution via sparse embedding. In Amir, A. Proceedings. Proceeding of the 11th World Congress on Intelligent Control and Automation; 29 June-4 July 2014; Shengyang, China. Piscataway, NJ: IEEE; 2014. 5673–5676.
Yang, L., Wang, S.H. and Zhang, Y.D. (2022) Ednc: Ensemble deep neural network for covid-19 recognition. Tomography. 2020; 8(2): 869–890.
Zhou, Q., Zhang, X. and Zhang, Y.D. Ensemble learning with attention-based multiple instance pooling for classification of spt. IEEE Transactions on Circuits and Systems II: Express Briefs. 2022; 69(3): 1927–1931.
Zhang, Y.D., Satapathy, S.C. and Wang, S.H. (2022) Fruit category classification by fractional fourier entropy with rotation angle vector grid and stacked sparse autoencoder. Expert Systems. 2022; 39(3), Article: e12701.
Wang, S.H., Satapathy, S.C., Zhou, Q., Zhang, X. and Zhang, Y.D. Secondary pulmonary tuberculosis identifica-tion via pseudo-zernike moment and deep stacked sparse autoencoder. Journal of Grid Computing. 2022; 20(1): 1–16.
Shui-Hua, W., Khan, M.A., Govindaraj, V., Fernandes, S.L., Zhu, Z. and Yu-Dong, Z. Deep rankbased average pooling network for covid-19 recognition. Computers, Materials, & Continua. 2022; 70(2): 2797–2813.
Shui-Hua Wang, Muhammad Attique Khan, Y.D.Z. Vispnn: Vgg-inspired stochastic pooling neural network. Computers, Materials & Continua. 2022; 70(2): 3081–3097.
Redmon, J. Yolo9000: better, faster, stronger. In G. L. Proceedings. Proceedings of the IEEE conference on computer vision and pattern recognition; July 21-26 2017; Honolulu, HI, USA. Piscataway, NJ: IEEE; 2017. pp. 7263–7271.
Bochkovskiy, A., Wang, C.Y. and Liao, H.Y.M. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv: 2004.10934, 2020.
Lin, T.Y. Feature pyramid networks for object detection. In G. L. Proceedings. Proceedings of the IEEE conference on computer vision and pattern recognition; July 21-26 2017; Honolulu, HI, USA. Piscataway, NJ: IEEE; 2017. pp. 936–944.
Long, J. Fully convolutional networks for semantic segmentation. In Los Alamitos. Proceedings. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition; June 7-15 2015; Boston, MA, USA. Piscataway, NJ: IEEE; 2015. pp. 3431–3440.
Ronneberger, O., Fischer, P. and Brox, T. U-net: Convolu-tional networks for biomedical image segmentation. arXiv preprint arXiv:1505.04597, 2015.
Dong, C. Learning a deep convolutional network for image superresolution. In Fleet, D. Proceedings. Proceedings of the 2014 European Conference on Computer Vision; Sep 6-12, 2014; Zurich, Switzerland.2014 Cham: Springer International Publishing; 2014, pp. 184–199.
Kim, J., Lee, J.K. and Lee, K.M. Accurate image super-resolution using very deep convolutional networks. arXiv preprint arXiv: 1511.04587, 2015.
He, K. Deep residual learning for image recognition. In R. B. Proceedings. Proceedings of the IEEE conference on computer vision and pattern recognition; June 27-30, 2016; Las Vegas, NV, USA. Piscataway, NJ: IEEE; 2016. pp. 770–778.
Lim, B., Son, S., Kim, H., Nah, S. and Lee, K.M. (2017), Enhanced deep residual networks for single image super-resolution. arXiv preprint arXiv: 1707.02921, 2017.
Hu, Y., Gao, X., Li, J., Huang, Y. and Wang, H. Single image super-resolution with multi-scale information cross-fusion network. Signal Processing. 2021; 179: 107831.
Tai, Y. Image superresolution via deep recursive residual network. In G. L. Proceedings of the IEEE conference on computer vision and pattern recognition; July 21-26 2017; Honolulu, HI, USA. Piscataway, NJ: IEEE; 2017. pp. 2790–2798.
Zhang, Y. Residual dense network for image super-resolution. In Brown. M. Proceedings. the IEEE conference on computer vision and pattern recognition; Jun 18-22 2018, Salt Lake City, Utah, USA. Piscataway, NJ: IEEE; 2019. pp. 2472–2481.
Liu D, Wen B, Fan Y, et al. Non-local recurrent network for image restoration. Advances in neural information processing systems, 2018, 31: 1680–1689.
Zhang, Y., Li, K., Li, K., Zhong, B. and Fu, Y. Residual non-local attention networks for image restoration. arXiv preprint arXiv: 1903.10082, 2019
Dai, T. Second-order attention network for single image superresolution. In Davis L. Proceedings. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition; Jun 16-20, 2019; Long Beach, CA, USA. Piscataway, NJ: IEEE; 2019. pp. 11057–11066.
Bruna J, Zaremba W, Szlam A, et al. Spectral networks and locally connected networks on graphs[J]. arXiv preprint arXiv:1312.6203, 2013.
Defferrard, M., Bresson, X. and Vandergheynst, P. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, 2016, 29: 3844–3852.
Kipf, T.N. and Welling, M. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: 1609. 02907, 2019.
Veli ˇ ckovi ´ c, P., Cucurull, G., Casanova, A., Romero, A., Lio˙, P. and Bengio, Y. Graph attention networks. arXiv preprint arXiv: 1710.10903, 2017.
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C. and Yu, P.S. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems. 2021; 32(1): 4–24.
Chami, I., Abu-El-Haija, S., Perozzi, B., Ré, C. and Murphy, K. (2021), Machine learning on graphs: A model and comprehensive taxonomy. arXiv preprint arXiv: 2005.03675, 2020.
Xu, B. and Yin, H. Graph convolutional networks in feature space for image deblurring and super-resolution. arXiv preprint arXiv: 2105.10465, 2105.
Yan, Y., Ren, W., Hu, X., Li, K., Shen, H. and Cao, X. SRGAT: Single image super-resolution with graph attention network. IEEE Transactions on Image Processing. 2021; 30: 4905–4918.
Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D. et al. Real-time single image and video super-resolution using an efficient subpixel convolutional neural network. arXiv preprint arXiv: 1609.05158, 2016.
Anwar, S. and Barnes, N. Densely residual laplacian super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022; 44(3): 1192–11204.
Wang, Z., Bovik, A., Sheikh, H. and Simoncelli, E. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing. 2004; 13(4): 600–612.
Downloads
Published
Issue
Section
License
Copyright (c) 2022 EAI Endorsed Transactions on Internet of Things
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 4.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
