Superresolution Reconstruction of Magnetic Resonance Images Based on a Nonlocal Graph Network

Authors

DOI:

https://doi.org/10.4108/eetiot.v8i29.769

Keywords:

Magnetic resonance imaging, Superresolution reconstruction, Nonlocal operation, Nonlocal self-similarity, Graph attention

Abstract

INTRODUCTION: High-resolution (HR) medical images are very important for doctors when diagnosing the internal pathological structures of patients and formulating precise treatment plans.
OBJECTIVES: Other methods of superresolution cannot adequately capture nonlocal self-similarity information of images. To solve this problem, we proposed using graph convolution to capture non-local self-similar information.
METHODS: This paper proposed a nonlocal graph network (NLGN) to perform single magnetic resonance (MR) image SR. Specifically, the proposed network comprises a nonlocal graph module (NLGM) and a nonlocal graph attention block (NLGAB). The NLGM is designed with densely connected residual blocks, which can fully explore the features of input images and prevent the loss of information. The NLGAB is presented to efficiently capture the dependency relationships among the given data by merging a nonlocal operation (NL) and a graph attention layer (GAL). In addition, to enable the current node to aggregate more beneficial information, when information is aggregated, we aggregate the neighbor nodes that are closest to the current node.
RESULTS: For the scale r=2, the proposed NLGN achieves PSNR of 38.54 dB and SSIM of 0.9818 on the T(T1, BD) dataset, and yielding a 0.27 dB and 0.0008 improvement over the CSN method, respectively.
CONCLUSION: The experimental results obtained on the IXI dataset show that the proposed NLGN performs better than the state-of-the-art methods.

Downloads

Download data is not yet available.
<br data-mce-bogus="1"> <br data-mce-bogus="1">

References

Li, M. A coarse-to-fine face hallucination method by exploiting facial prior knowledge. In C. N. Proceedings. Proceedings of the 25th IEEE International Conference on Image Processing; October 7–10, 2018; Athens, Greece. Piscataway, NJ: IEEE: 2018. pp. 61–65. DOI: https://doi.org/10.1109/ICIP.2018.8451122

Zhang, Z., Shi, Y., Zhou, X., Kan, H. and Wen, J. Shuffle block srgan for face image super-resolution reconstruction. Measurement and Control. 2020; 53(7-8): 1429–1439. DOI: https://doi.org/10.1177/0020294020944969

Pan, Z., Yu, J., Huang, H., Hu, S., Zhang, A., Ma, H. and Sun, W. (2013) Super-resolution based on compressive sensing and structural self-similarity for remote sensing images. IEEE Transactions on Geoscience and Remote Sensing. 2013; 51(9): 4864–4876. DOI: https://doi.org/10.1109/TGRS.2012.2230270

Zhang, S., Yuan, Q., Li, J., Sun, J. and Zhang, X. Scene-adaptive remote sensing image super-resolution using a multiscale attention network. IEEE Transactions on Geoscience and Remote Sensing. 2013; 58(7): 4764–4779. DOI: https://doi.org/10.1109/TGRS.2020.2966805

Hu, J. (2016) Single image super resolution of 3D MRI using local regression and intermodality priors. In Falco, C.M. Proceedings Volume 10033. Proceedings of the Eighth International Conference on Digital Image Processing; 2016; Chengdu, China. US; SPIE;2016. 10033. 866–871. DOI: https://doi.org/10.1117/12.2243617

Zhao, X., Zhang, Y., Zhang, T. and Zou, X. Channel splitting network for single MR image superresolution. IEEE transactions on image processing. 2019; 28(11): 5649—5662. DOI: https://doi.org/10.1109/TIP.2019.2921882

Shi, J., Li, Z., Ying, S., Wang, C., Liu, Q., Zhang, Q. and Yan, P. Mr image super-resolution via wide residual networks with fixed skip connection. IEEE Journal of Biomedical and Health Informatics. 2019; 23(3): 1129–1140. DOI: https://doi.org/10.1109/JBHI.2018.2843819

Song, T.A., Chowdhury, S.R., Yang, F. and Dutta, J. Super-resolution pet imaging using convolutional neural networks. IEEE Transactions on Computational Imaging. 2020; 6(1): 518–528. DOI: https://doi.org/10.1109/TCI.2020.2964229

Keys, R. Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing. 1981; 29(6): 1153–1160. DOI: https://doi.org/10.1109/TASSP.1981.1163711

Medoff, B.P., Brody, W.R., Nassi, M. and Macovski, A. Iterative convolution backprojection algorithms for image reconstruction from limited data. J. Opt. Soc. Am. 1983; 73(11): 1493–1500. DOI: https://doi.org/10.1364/JOSA.73.001493

Stark, H. and Oskoui, P. High-resolution image recovery from image-plane arrays, using convex projections. J. Opt. Soc. Am. A. 1989; 6(11): 1715–1726. DOI: https://doi.org/10.1364/JOSAA.6.001715

Yang, J., Wright, J., Huang, T.S. and Ma, Y. Image super-resolution via sparse representation. IEEE Transactions on Image Processing. 2010; 19(11): 2861–2873. DOI: https://doi.org/10.1109/TIP.2010.2050625

Zhu, Q. Image superresolution via sparse embedding. In Amir, A. Proceedings. Proceeding of the 11th World Congress on Intelligent Control and Automation; 29 June-4 July 2014; Shengyang, China. Piscataway, NJ: IEEE; 2014. 5673–5676.

Yang, L., Wang, S.H. and Zhang, Y.D. (2022) Ednc: Ensemble deep neural network for covid-19 recognition. Tomography. 2020; 8(2): 869–890. DOI: https://doi.org/10.3390/tomography8020071

Zhou, Q., Zhang, X. and Zhang, Y.D. Ensemble learning with attention-based multiple instance pooling for classification of spt. IEEE Transactions on Circuits and Systems II: Express Briefs. 2022; 69(3): 1927–1931. DOI: https://doi.org/10.1109/TCSII.2021.3124165

Zhang, Y.D., Satapathy, S.C. and Wang, S.H. (2022) Fruit category classification by fractional fourier entropy with rotation angle vector grid and stacked sparse autoencoder. Expert Systems. 2022; 39(3), Article: e12701. DOI: https://doi.org/10.1111/exsy.12701

Wang, S.H., Satapathy, S.C., Zhou, Q., Zhang, X. and Zhang, Y.D. Secondary pulmonary tuberculosis identifica-tion via pseudo-zernike moment and deep stacked sparse autoencoder. Journal of Grid Computing. 2022; 20(1): 1–16. DOI: https://doi.org/10.1007/s10723-021-09596-6

Shui-Hua, W., Khan, M.A., Govindaraj, V., Fernandes, S.L., Zhu, Z. and Yu-Dong, Z. Deep rankbased average pooling network for covid-19 recognition. Computers, Materials, & Continua. 2022; 70(2): 2797–2813. DOI: https://doi.org/10.32604/cmc.2022.020140

Shui-Hua Wang, Muhammad Attique Khan, Y.D.Z. Vispnn: Vgg-inspired stochastic pooling neural network. Computers, Materials & Continua. 2022; 70(2): 3081–3097. DOI: https://doi.org/10.32604/cmc.2022.019447

Redmon, J. Yolo9000: better, faster, stronger. In G. L. Proceedings. Proceedings of the IEEE conference on computer vision and pattern recognition; July 21-26 2017; Honolulu, HI, USA. Piscataway, NJ: IEEE; 2017. pp. 7263–7271. DOI: https://doi.org/10.1109/CVPR.2017.690

Bochkovskiy, A., Wang, C.Y. and Liao, H.Y.M. Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv: 2004.10934, 2020.

Lin, T.Y. Feature pyramid networks for object detection. In G. L. Proceedings. Proceedings of the IEEE conference on computer vision and pattern recognition; July 21-26 2017; Honolulu, HI, USA. Piscataway, NJ: IEEE; 2017. pp. 936–944. DOI: https://doi.org/10.1109/CVPR.2017.106

Long, J. Fully convolutional networks for semantic segmentation. In Los Alamitos. Proceedings. Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition; June 7-15 2015; Boston, MA, USA. Piscataway, NJ: IEEE; 2015. pp. 3431–3440. DOI: https://doi.org/10.1109/CVPR.2015.7298965

Ronneberger, O., Fischer, P. and Brox, T. U-net: Convolu-tional networks for biomedical image segmentation. arXiv preprint arXiv:1505.04597, 2015. DOI: https://doi.org/10.1007/978-3-319-24574-4_28

Dong, C. Learning a deep convolutional network for image superresolution. In Fleet, D. Proceedings. Proceedings of the 2014 European Conference on Computer Vision; Sep 6-12, 2014; Zurich, Switzerland.2014 Cham: Springer International Publishing; 2014, pp. 184–199. DOI: https://doi.org/10.1007/978-3-319-10593-2_13

Kim, J., Lee, J.K. and Lee, K.M. Accurate image super-resolution using very deep convolutional networks. arXiv preprint arXiv: 1511.04587, 2015. DOI: https://doi.org/10.1109/CVPR.2016.182

He, K. Deep residual learning for image recognition. In R. B. Proceedings. Proceedings of the IEEE conference on computer vision and pattern recognition; June 27-30, 2016; Las Vegas, NV, USA. Piscataway, NJ: IEEE; 2016. pp. 770–778. DOI: https://doi.org/10.1109/CVPR.2016.90

Lim, B., Son, S., Kim, H., Nah, S. and Lee, K.M. (2017), Enhanced deep residual networks for single image super-resolution. arXiv preprint arXiv: 1707.02921, 2017. DOI: https://doi.org/10.1109/CVPRW.2017.151

Hu, Y., Gao, X., Li, J., Huang, Y. and Wang, H. Single image super-resolution with multi-scale information cross-fusion network. Signal Processing. 2021; 179: 107831. DOI: https://doi.org/10.1016/j.sigpro.2020.107831

Tai, Y. Image superresolution via deep recursive residual network. In G. L. Proceedings of the IEEE conference on computer vision and pattern recognition; July 21-26 2017; Honolulu, HI, USA. Piscataway, NJ: IEEE; 2017. pp. 2790–2798. DOI: https://doi.org/10.1109/CVPR.2017.298

Zhang, Y. Residual dense network for image super-resolution. In Brown. M. Proceedings. the IEEE conference on computer vision and pattern recognition; Jun 18-22 2018, Salt Lake City, Utah, USA. Piscataway, NJ: IEEE; 2019. pp. 2472–2481.

Liu D, Wen B, Fan Y, et al. Non-local recurrent network for image restoration. Advances in neural information processing systems, 2018, 31: 1680–1689.

Zhang, Y., Li, K., Li, K., Zhong, B. and Fu, Y. Residual non-local attention networks for image restoration. arXiv preprint arXiv: 1903.10082, 2019

Dai, T. Second-order attention network for single image superresolution. In Davis L. Proceedings. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition; Jun 16-20, 2019; Long Beach, CA, USA. Piscataway, NJ: IEEE; 2019. pp. 11057–11066. DOI: https://doi.org/10.1109/CVPR.2019.01132

Bruna J, Zaremba W, Szlam A, et al. Spectral networks and locally connected networks on graphs[J]. arXiv preprint arXiv:1312.6203, 2013.

Defferrard, M., Bresson, X. and Vandergheynst, P. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, 2016, 29: 3844–3852.

Kipf, T.N. and Welling, M. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv: 1609. 02907, 2019.

Veli ˇ ckovi ´ c, P., Cucurull, G., Casanova, A., Romero, A., Lio˙, P. and Bengio, Y. Graph attention networks. arXiv preprint arXiv: 1710.10903, 2017.

Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C. and Yu, P.S. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems. 2021; 32(1): 4–24. DOI: https://doi.org/10.1109/TNNLS.2020.2978386

Chami, I., Abu-El-Haija, S., Perozzi, B., Ré, C. and Murphy, K. (2021), Machine learning on graphs: A model and comprehensive taxonomy. arXiv preprint arXiv: 2005.03675, 2020.

Xu, B. and Yin, H. Graph convolutional networks in feature space for image deblurring and super-resolution. arXiv preprint arXiv: 2105.10465, 2105.

Yan, Y., Ren, W., Hu, X., Li, K., Shen, H. and Cao, X. SRGAT: Single image super-resolution with graph attention network. IEEE Transactions on Image Processing. 2021; 30: 4905–4918. DOI: https://doi.org/10.1109/TIP.2021.3077135

Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A.P., Bishop, R., Rueckert, D. et al. Real-time single image and video super-resolution using an efficient subpixel convolutional neural network. arXiv preprint arXiv: 1609.05158, 2016. DOI: https://doi.org/10.1109/CVPR.2016.207

Anwar, S. and Barnes, N. Densely residual laplacian super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2022; 44(3): 1192–11204. DOI: https://doi.org/10.1109/TPAMI.2020.3021088

Wang, Z., Bovik, A., Sheikh, H. and Simoncelli, E. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing. 2004; 13(4): 600–612. DOI: https://doi.org/10.1109/TIP.2003.819861

Downloads

Published

04-05-2022

How to Cite

[1]
Y. Li, X. Li, Y. Yan, and C. Hu, “Superresolution Reconstruction of Magnetic Resonance Images Based on a Nonlocal Graph Network”, EAI Endorsed Trans IoT, vol. 8, no. 29, p. e2, May 2022.

Issue

Vol. 8 No. 29 (2022): EAI Endorsed Transactions on Internet of Things

Section

Research article

License

Copyright (c) 2022 EAI Endorsed Transactions on Internet of Things

Creative Commons License

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.

Funding data