Knowledge Graph Fusion for Cross-Modal Semantic Communication
DOI:
https://doi.org/10.4108/eetsis.9216Keywords:
Knowledge graph, cross-modal, semantic communication, performance evaluationAbstract
This paper proposes a knowledge graph-enhanced multi-source fusion (KG-MSF) scheme, a novel cross-modal semantic communication system to robustly fuse visual and textual data for tasks such as visual question answering (VQA) over wireless channels. The proposed KG-MSF scheme integrates knowledge graph reasoning into a multi-stage fusion and encoding pipeline, utilizing bidirectional cross attention between modalities and structured semantic triplets to enhance semantic preservation and resilience to channel impairments. Specifically, image objects and question tokens are first aligned via cross-modal attention, then enriched with shallow and deep semantic triplets extracted through knowledge graphs, which are subsequently fused and transmitted using joint source-channel coding. Extensive simulation results are provided to demonstrate that the proposed KG-MSF scheme significantly outperforms the competing ones under both AWGN and Rayleigh fading channels, indicating KG-MSF’s superior semantic robustness and efficient cross-modal reasoning in wireless environments.
References
[1] L. Bai, X. Song, and L. Zhu, “Joint multi-feature information entity alignment for cross-lingual temporal knowledge graph with BERT,” IEEE Trans. Big Data, vol. 11, no. 2, pp. 345–358, 2025.
[2] F. Yang,W. Chen, H. Lin, S.Wu, X. Li, Z. Li, and Y.Wang, “Task-oriented tool manipulation with robotic dexterous hands: A knowledge graph approach from fingers to functionality,” IEEE Trans. Cybern., vol. 55, no. 1, pp. 395–408, 2025.
[3] T. Zhang, J. Cheng, L. Miao, H. Chen, Q. Li, Q. He, J. Lyu, and L. Ma, “Multi-hop reasoning with relation based node quality evaluation for sparse medical knowledge graph,” IEEE Trans. Emerg. Top. Comput. Intell., vol. 9, no. 2, pp. 1805–1816, 2025.
[4] H. Sun, J. Wang, J. Weng, and W. Tan, “KG-ID: knowledge graph-based intrusion detection on in vehicle network,” IEEE Trans. Intell. Transp. Syst., vol. 26, no. 4, pp. 4988–5000, 2025.
[5] L. Liao, L. Zheng, J. Shang, X. Li, and F. Chen, “ATPF: an adaptive temporal perturbation framework for adversarial attacks on temporal knowledge graph,” IEEE Trans. Knowl. Data Eng., vol. 37, no. 3, pp. 1091-1104, 2025.
[6] T. Song, L. Yin, Y. Liu, L. Liao, J. Luo, and Z. Xu, “Expressiveness analysis and enhancing framework for geometric knowledge graph embedding models,” IEEE Trans. Knowl. Data Eng., vol. 37, no. 1, pp. 306-318, 2025.
[7] Z. Liu, T. Sun, X. Sun, andW. Cui, “Estimating remaining useful life of aircraft engine system via a novel graph tensor fusion network based on knowledge of physical structure and thermodynamics,” IEEE Trans. Instrum. Meas., vol. 74, pp. 1–14, 2025.
[8] S. Bi, Z. Miao, and Q. Min, “LEMON: A knowledge enhanced, type-constrained, and grammar-guided model for question generation over knowledge graphs,” IEEE Trans. Learn. Technol., vol. 18, pp. 256–272, 2025.
[9] C. Mai, Y. Chang, C. Chen, and Z. Zheng, “Enhanced scalable graph neural network via knowledge distillation,” IEEE Trans. Neural Networks Learn. Syst., vol. 36, no. 1, pp. 1258–1271, 2025.
[10] V. Hoang, V. Nguyen, R. Chang, P. Lin, R. Hwang, and T. Q. Duong, “Adversarial attacks against shared knowledge interpretation in semantic communications,” IEEE Trans. Cogn. Commun. Netw., vol. 11, no. 2, pp. 1024–1040, 2025.
[11] Y. Bo, S. Shao, and M. Tao, “Deep learning-based superposition coded modulation for hierarchical semantic communications over broadcast channels,” IEEE Trans. Commun., vol. 73, no. 2, pp. 1186–1200, 2025.
[12] Y. Rong, G. Nan, M. Zhang, S. Chen, S. Wang, X. Zhang, N. Ma, S. Gong, Z. Yang, Q. Cui, X. Tao, and T. Q. S. Quek, “Semantic entropy can simultaneously benefit transmission efficiency and channel security of wireless semantic communications,” IEEE Trans. Inf. Forensics Secur., vol. 20, pp. 2067–2082, 2025.
[13] R. Cheng, Y. Sun, D. Niyato, L. Zhang, L. Zhang, and M. A. Imran, “A wireless ai-generated content (AIGC) provisioning framework empowered by semantic communication,” IEEE Trans. Mob. Comput., vol. 24, no. 3, pp. 2137–2150, 2025.
[14] X. Liu, H. Liang, Z. Bao, C. Dong, and X. Xu, “A semantic communication system for point cloud,” IEEE Trans. Veh. Technol., vol. 74, no. 1, pp. 894–910, 2025.
[15] L. Wang, W. Wu, F. Zhou, Z. Qin, and Q. Wu, “Irs-enhanced secure semantic communication networks: Cross-layer and context-awared resource allocation,” IEEE Trans. Wirel. Commun., vol. 24, no. 1, pp. 494–508, 2025.
[16] Z. Wan, S. Liu, Z. Xu, W. Ni, Z. Chen, and F. Wang, “Semantic communication method based on compression ratio optimization for vision tasks in the artificial intelligence of things,” IEEE Trans. Consumer Electron., vol. 70, no. 2, pp. 4934–4944, 2024.
[17] P. Wang, J. Li, C. Liu, X. Fan, M. Ma, and Y. Wang, “Distributed semantic communications for multimodal audio-visual parsing tasks,” IEEE Trans. Green Commun. Netw., vol. 8, no. 4, pp. 1707–1716, 2024.
[18] Y. Tang, N. Zhou, Q. Yu, D. Wu, C. Hou, G. Tao, and M. Chen, “Intelligent fabric enabled 6g semantic communication system for in-cabin scenarios,” IEEE Trans. Intell. Transp. Syst., vol. 24, no. 1, pp. 1153–1162, 2023.
[19] C. Sun, D. Ming, L. Xu, S. Xie, R. Liu, and X. Ling, “A hybrid damaged building sample generation method based on cross-scale fusion generative model for destroyed building detection after earthquake,” IEEE Trans. Geosci. Remote. Sens., vol. 63, pp. 1–15, 2025.
[20] M. Liu, H. Liu, and T. Guo, “Cross-model cross-stream learning for self-supervised human action recognition,” IEEE Trans. Hum. Mach. Syst., vol. 54, no. 6, pp. 743–752, 2024
[21] J. Wang, Y. Xie, S. Xie, and X. Chen, “Dual cross-attention transformer networks for temporal predictive modeling of industrial process,” IEEE Trans. Instrum. Meas., vol. 73, pp. 1–11, 2024.
[22] P. Miao, W. Su, G. Wang, X. Li, and X. Li, “Self-paced multi-grained cross-modal interaction modeling for referring expression comprehension,” IEEE Trans. Image Process., vol. 33, pp. 1497–1507, 2024.
[23] J. Ding, W. Li, L. Pei, M. Yang, A. Tian, and B. Yuan, “Novel pipeline integrating cross-modality and motion model for nearshore multi-object tracking in optical video surveillance,” IEEE Trans. Intell. Transp. Syst., vol. 25, no. 9, pp. 12 464–12 476, 2024.
[24] W. Li, H. Zhou, C. Zhang, W. Nie, X. Li, and A. Liu, “Dual-stage uncertainty modeling for unsupervised cross-domain 3d model retrieval,” IEEE Trans. Multim., vol. 26, pp. 8996–9007, 2024.
[25] J. Xu and L. Cao, “Copula variational LSTM for high-dimensional cross-market multivariate dependence modeling,” IEEE Trans. Neural Networks Learn. Syst., vol. 35, no. 11, pp. 16 233–16 247, 2024.
[26] Y. He and W. Shen, “Msit: A cross-machine fault diagnosis model for machine-level CNC spindle motors,” IEEE Trans. Reliab., vol. 73, no. 1, pp. 792–802, 2024.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Yanrong Yang, Tianxiang Zhong, Mengting Chen
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open access article distributed under the terms of the CC BY-NC-SA 4.0, which permits copying, redistributing, remixing, transformation, and building upon the material in any medium so long as the original work is properly cited.
