A survey on graph neural networks

Authors

DOI:

https://doi.org/10.4108/eetel.3466

Keywords:

Deep Learning, Graph Neural Networks, Euclidean domains, Non-Euclidean domains, Graph data

Abstract

In recent years, we have witnessed the developments that deep learning has brought to machine learning. It has solved many problems in the areas of computer vision, speech recognition, natural language processing, and various other tasks with state-of-the-art performance. However, the data in these tasks is typically represented in Euclidean space. As technology develops, more and more applications are generating data from non-Euclidean domains and representing them as graphs with complex relationships and interdependencies between objects. This poses a significant challenge to deep learning algorithms. This is because, due to the uniqueness of graphs, applying deep learning to the ubiquitous graph data is not an easy task. To solve the problem in non-Euclidean domains, Graph Neural Networks (GNNs) have emerged. A Graph Neural Network (GNN) is a neural model that captures dependencies between graphs by passing messages between graph nodes. This paper introduces commonly used graph neural networks, their learning methods, and common datasets for graph neural networks. It also provides an outlook on the future of Graph Neural Networks.

References

LeCun, Y., et al., Deep learning. Nature, 2015. 521(7553): p. 436-44

Zhang, Y.-D., et al., Deep learning for computer-aided medical diagnosis. Multimedia Tools and Applications, 2020. 79(21): p. 15073-15073

Wu, S., et al., Session-based recommendation with graph neural networks. 2019

Michael, et al., Geometric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine, 2017. 34(4): p. 18-42

Sperduti, A., et al., Supervised neural networks for the classification of structures. IEEE Transactions on Neural Networks, 1997. 8(3): p. 714

Gori, M., et al. A new model for learning in graph domains. in IEEE International Joint Conference on Neural Networks. 2005.

Scarselli, F., et al., The graph neural network model. IEEE Transactions on Neural Networks, 2009. 20(1): p. 61

Gallicchio, C., et al. Graph echo state networks. in International Joint Conference on Neural Networks. 2010.

Wu, Z., et al., A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 2021(1): p. 32

Zhang, Z., et al., Deep learning on graphs: A survey. 2018

Thomas, J.M., et al., Graph neural networks designed for different graph types: A survey. 2022

Waikhom, L., et al., Graph neural networks: Methods, applications, and opportunities. 2021

Zhou, J., et al., Graph neural networks: A review of methods and applications. AI Open, 2020. 1: p. 57-81

Cao, Z., et al., Unsupervised feature learning by autoencoder and prototypical contrastive learning for hyperspectral classification. 2020

Yang, L., et al. Toward unsupervised graph neural network: Interactive clustering and embedding via optimal transport. in 2020 IEEE International Conference on Data Mining (ICDM). 2020.

Okuda, M., et al. Unsupervised common particular object discovery and localization by analyzing a match graph. in IEEE International Conference on Acoustics, Speech and Signal Processing. 2021.

Du, L., et al. Dynamic network embedding : An extended approach for skip-gram based network embedding. in Twenty-Seventh International Joint Conference on Artificial Intelligence {IJCAI-18}. 2018.

Liu, R., et al., Federated graph neural networks: Overview, techniques and challenges. 2022

Zheng, X., et al., Graph neural networks for graphs with heterophily: A survey. 2022

Wang, S.-H., Covid-19 classification by fgcnet with deep feature fusion from graph convolutional network and convolutional neural network. Information Fusion, 2021. 67: p. 208-229

Zhou, L., et al. A weighted gcn with logical adjacency matrix for relation extraction. in ECAI 2020 - 24th European Conference on Artificial Intelligence. 2020.

You, J., et al., Identity-aware graph neural networks. 2021

Guttery, D.S., Improved breast cancer classification through combining graph convolutional network and convolutional neural network. Information Processing and Management, 2021. 58: Article ID. 102439

Atwood, J., et al., Diffusion-convolutional neural networks. Computer Science, 2015

Niepert, M., et al., Learning convolutional neural networks for graphs. JMLR.org, 2016

Gilmer, J., et al., Neural message passing for quantum chemistry. 2017

Kipf, T.N., et al., Semi-supervised classification with graph convolutional networks. 2016

Monti, F., et al. Geometric deep learning on graphs and manifolds using mixture model cnns. in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.

Liu, Z., et al., Geniepath: Graph neural networks with adaptive receptive paths. Proceedings of the AAAI Conference on Artificial Intelligence, 2018. 33

Gao, H., et al., Large-scale learnable graph convolutional networks. 2018, ACM.

Defferrard, M., et al., Convolutional neural networks on graphs with fast localized spectral filtering. 2016.

Henaff, M., et al., Deep convolutional networks on graph-structured data. Computer Science, 2015

Li, R., et al., Adaptive graph convolutional neural networks. 2018.

Bianchi, F.M., et al., Graph neural networks with convolutional arma filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021. PP(99): p. 1-1

Dong, J., et al., Global neighbor sampling for mixed cpu-gpu training on giant graphs. 2021

Wang, X., et al., Neural graph collaborative filtering. ACM, 2019

Krizhevsky, A., et al., Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 2012. 25(2)

Velikovi, P., et al., Graph attention networks. 2017

Vaswani, A., et al., Attention is all you need. arXiv, 2017

Merity, S., Single headed attention rnn: Stop thinking with your head. 2019

Kipf, T.N., et al., Variational graph auto-encoders. 2016

Badrinarayanan, V., et al., Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017: p. 1-1

Cho, K., et al., Learning phrase representations using rnn encoder-decoder for statistical machine translation. Computer Science, 2014

Liu, Z., et al. Towards explainable representation of time-evolving graphs via spatial-temporal graph attention networks. in Conference on Information and Knowledge Management. 2019.

Schulman, J., et al., High-dimensional continuous control using generalized advantage estimation. Computer ence, 2015

Ng, I., et al., A graph autoencoder approach to causal structure learning. 2019

Satapathy, S.C., Secondary pulmonary tuberculosis identification via pseudo-zernike moment and deep stacked sparse autoencoder. Journal of Grid Computing, 2022. 20: p. 1: Article ID. 1

Zhang, Y.-D., Pseudo zernike moment and deep stacked sparse autoencoder for covid-19 diagnosis. CMC-Computers, Materials & Continua, 2021. 69(3): p. 3145–3162

Wang, S.-H., Dssae: Deep stacked sparse autoencoder analytical model for covid-19 diagnosis by fractional fourier entropy. ACM Transactions on Management Information Systems, 2021. 13(1): Article ID. 2

Peng, C., et al., A survey on network embedding. IEEE Transactions on Knowledge and Data Engineering, 2017. PP(99): p. 1-1

B, J.X.A., et al., Attention adjacency matrix based graph convolutional networks for skeleton-based action recognition - sciencedirect. Neurocomputing, 2021. 440: p. 230-239

Dickson, M.C., et al., Hybridised loss functions for improved neural network generalisation. 2022

Mirza, M., et al., Conditional generative adversarial nets. Computer Science, 2014: p. 2672-2680

Radford, A., et al., Unsupervised representation learning with deep convolutional generative adversarial networks. Computer ence, 2015

Salimans, T., et al., Improved techniques for training gans. 2016

Karras, T., et al. A style-based generator architecture for generative adversarial networks. in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019.

Zhang, Y., Deep learning in food category recognition. Information Fusion, 2023. 98: p. 101859

Wang, S., Advances in data preprocessing for biomedical data fusion: An overview of the methods, challenges, and prospects. Information Fusion, 2021. 76: p. 376-421

Zhang, Y.-D., et al., Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation. Information Fusion, 2020. 64: p. 149-187

Mackay, D.J.C., The humble gaussian distribution. 2006

Mohan, R., Collective classification in network data.

Downloads

Published

10-07-2023

How to Cite

[1]
J. Wang, “A survey on graph neural networks”, EAI Endorsed Trans e-Learn, vol. 8, no. 3, p. e6, Jul. 2023.