Leveraging Relation Attention Mechanisms for Enhanced Knowledge Graph Completion with Embedding Translation: Leveraging Relation Attention Mechanisms

Jiahao Shi; Zhengping Lin; Yuzhong Zhou; Yuliang Yang; Jie Lin

doi:10.4108/eetsis.9117

Authors

Jiahao Shi Electric Power Research Institute of China Southern Power Grid Company
Zhengping Lin Electric Power Research Institute of China Southern Power Grid Company
Yuzhong Zhou Electric Power Research Institute of China Southern Power Grid Company
Yuliang Yang Electric Power Research Institute of China Southern Power Grid Company
Jie Lin Electric Power Research Institute of China Southern Power Grid Company

DOI:

https://doi.org/10.4108/eetsis.9117

Keywords:

Knowledge graph, relation attention mechanism, embedding translation, performance evaluation

Abstract

In this paper, we propose a novel knowledge graph completion framework to leverage a relation-specific attention mechanism integrated with an embedding translation strategy to improve the accuracy and contextual understanding of link prediction tasks. Unlike traditional models that rely on fixed transformation spaces, the proposed method dynamically captures fine-grained relational semantics by combining hierarchical candidate categorization, relation-guided entity projection, and asymmetric score functions. Specifically, the proposed model employs K-means clustering and principal component analysis (PCA) to identify semantically consistent entity sets, and integrates attention-weighted multi-attribute embeddings to construct robust relational representations. A margin-based ranking loss with normalized embedding constraints ensures effective optimization, further supported by Xavier initialization and stochastic gradient descent. Extensive experiments on two benchmark datasets, WN18 and FB15K, demonstrate the superiority of the proposed method. Specifically, on WN18, the proposed method achieves the lowest mean rank (MR) of 144, with competitive results in mean reciprocal rank (MRR) (0.902), Hits@1 (89.0%), Hits@3 (90.4%), and Hits@10 (96.3%), closely rivaling state- of-the-art models like QuatE and ComplEx. On FB15K, the proposed method again delivers the best Mean Rank of 21, along with strong scores in MRR (0.831), Hits@1 (72.2%), Hits@3 (88.4%), and the highest Hits@10 (92.5%) among all compared methods.

References

[1] S. Kekki, W. Featherstone, Y. Fang, P. Kuure, A. Li, A. Ranjan, D. Purkayastha, F. Jiangping, D. Frydman, G. Verin et al., “Mec in 5g networks,” ETSI white paper, vol. 28, no. 28, pp. 1–28, 2018.

[2] S. Guo and X. Zhao, “Multi-agent deep reinforcement learning based transmission latency minimization for delay-sensitive cognitive satellite-UAV networks,” IEEE Trans. Commun., vol. 71, no. 1, pp. 131–144, 2023.

[3] H. Hui and W. Chen, “Joint scheduling of proactive pushing and on-demand transmission over shared spectrum for profit maximization,” IEEE Trans. Wirel. Commun., vol. 22, no. 1, pp. 107–121, 2023.

[4] M. Rahim, G. Kaddoum, and T. N. Do, “Joint devices and IRSs association for terahertz communications in industrial IoT networks,” IEEE Trans. Green Commun. Netw., vol. 8, no. 1, pp. 375–390, 2024.

[5] M. Alawida, “A novel image encryption algorithm based on cyclic chaotic map in industrial IoT environments,” IEEE Trans. Ind. Informatics, vol. 20, no. 8, pp. 10530–10541, 2024.

[6] Y. Ge, K. Xiong, Q. Wang, Q. Ni, P. Fan, and K. B. Letaief, “Aoi-minimal power adjustment in RF-EH-powered industrial IoT networks: A soft actor-critic-based method,” IEEE Trans. Mob. Comput., vol. 23, no. 9, pp. 8729–8741, 2024.

[7] C. Hu, J. Liu, H. Xia, S. Deng, and J. Yu, “A lightweight mutual privacy preserving K-means clustering in industrial IoT,” IEEE Trans. Netw. Sci. Eng., vol. 11, no. 2, pp. 2138–2152, 2024.

[8] J. Wang, Y. Hu, Y. Zhu, T. Wang, and A. Schmeink, “Reliability-optimal offloading for mobile edge-computing in low-latency industrial IoT networks,” IEEE Trans. Wirel. Commun., vol. 23, no. 10, pp. 12765– 12781, 2024.

[9] Q. Xiong, X. Zhu, Y. Jiang, J. Cao, X. Xiong, and H. Wang,“Status prediction and data aggregation for AoI-oriented short-packet transmission in industrial IoT,” IEEE Trans. Commun., vol. 71, no. 1, pp. 611–625, 2023.

[10] L. Bai, X. Song, and L. Zhu, “Joint multi-feature information entity alignment for cross-lingual temporal knowledge graph with BERT,” IEEE Trans. Big Data, vol. 11, no. 2, pp. 345–358, 2025.

[11] F. Yang, W. Chen, H. Lin, S. Wu, X. Li, Z. Li, and Y. Wang, “Task-oriented tool manipulation with robotic dexterous hands: A knowledge graph approach from fingers to functionality,” IEEE Trans. Cybern., vol. 55, no. 1, pp. 395–408, 2025.

[12] T. Zhang, J. Cheng, L. Miao, H. Chen, Q. Li, Q. He, J. Lyu, and L. Ma, “Multi-hop reasoning with relation based node quality evaluation for sparse medical knowledge graph,” IEEE Trans. Emerg. Top. Comput. Intell., vol. 9, no. 2, pp. 1805–1816, 2025.

[13] Z. Liu, T. Sun, X. Sun, and W. Cui, “Estimating remaining useful life of aircraft engine system via a novel graph tensor fusion network based on knowledge of physical structure and thermodynamics,” IEEE Trans. Instrum. Meas., vol. 74, pp. 1–14, 2025.

[14] H. Sun, J. Wang, J. Weng, and W. Tan, “KG-ID: knowledge graph-based intrusion detection on in-vehicle network,” IEEE Trans. Intell. Transp. Syst., vol. 26, no. 4, pp. 4988–5000, 2025.

[15] L. Liao, L. Zheng, J. Shang, X. Li, and F. Chen, “ATPF: an adaptive temporal perturbation framework for adversarial attacks on temporal knowledge graph,” IEEE Trans. Knowl. Data Eng., vol. 37, no. 3, pp. 1091–1104, 2025.

[16] T. Song, L. Yin, Y. Liu, L. Liao, J. Luo, and Z. Xu, “Expressiveness analysis and enhancing framework for geometric knowledge graph embedding models,” IEEE Trans. Knowl. Data Eng., vol. 37, no. 1, pp. 306–318, 2025.

[17] S. Bi, Z. Miao, and Q. Min, “LEMON: A knowledge-enhanced, type-constrained, and grammar-guided model for question generation over knowledge graphs,” IEEE Trans. Learn. Technol., vol. 18, pp. 256–272, 2025.

[18] C. Mai, Y. Chang, C. Chen, and Z. Zheng, “Enhanced scalable graph neural network via knowledge distillation,” IEEE Trans. Neural Networks Learn. Syst., vol. 36, no. 1, pp. 1258–1271, 2025.

[19] X. Sun, B. Zhang, Y. Wang, J. Mai, Y. Wang, J. Tan, and W. Wang, “A multiscale attention mechanism super-resolution confocal microscopy for wafer defect detection,” IEEE Trans Autom. Sci. Eng., vol. 22, pp. 1016–1027, 2025.

[20] L. Jin and L. Guo, “Facial expression recognition based on feature representation learning and clustering-based attention mechanism,” IEEE Trans. Biom. Behav. Identity Sci., vol. 7, no. 2, pp. 182–194, 2025.

[21] Z. Wang, G. Wang, and G. He, “COSA plus: Enhanced co-operative systolic arrays for attention mechanism in transformers,” IEEE Trans. Comput. Aided Des. Integr. Circuits Syst., vol. 44, no. 2, pp. 723–736, 2025.

[22] L. Zhang, J. Wang, Y. Guo, H. Zhang, L. Cheng, and W. Xia, “A self-attention mechanism-based model to detect ipv6 multi-field covert channels,” IEEE Trans. Cogn. Commun. Netw., vol. 11, no. 1, pp. 258–273, 2025.

[23] S. Li, L. Zhang, H. Guo, J. Li, J. Yu, X. He, Y. Zhao, and X. He, “CSA-FCN: channel- and spatial-gated attention mechanism based fully complex-valued neural network for system matrix calibration in magnetic particle imaging,” IEEE Trans. Computational Imaging, vol. 11, pp. 65–76, 2025.

[24] X. Liu, R. Dong, and H. Yang, “DHHG-TAC: fusion of dynamic heterogeneous hypergraphs and transformer attention mechanism for visual question answering tasks,” IEEE Trans. Ind. Informatics, vol. 21, no. 1, pp. 545–553, 2025.

[25] M. Li, Z. Wang, K. Li, and X. Liao, “Multiagent-system-based attention mechanism for predicting product popularity: Handling positive-negative diffusion over social networks,” IEEE Trans. Neural Networks Learn. Syst., vol. 36, no. 1, pp. 1925–1932, 2025.

[26] Z. Fu, K. Jiang, Y. Xu, Y. Wang, T. Wen, H. Gao, Z. Zhong, and D. Yang, “Top-down attention-based mechanisms for interpretable autonomous driving,” IEEE Trans. Intell. Transp. Syst., vol. 26, no. 2, pp. 2212–2226, 2025.

[27] X. Li, W. Jiao, Q. Han, and Z. Lu, “Detection of FDIA in power grid based on hypergraph and attention mechanism,” IEEE Trans. Smart Grid, vol. 16, no. 2, pp. 1862–1871, 2025.

[28] X. Xu, W. Lv, W. Wang, Y. Zhang, and J. Chen, “Empowering semantic segmentation with selective frequency enhancement and attention mechanism for tampering detection,” IEEE Trans. Artif. Intell., vol. 5, no. 6, pp. 3270–3283, 2024.