A Novel Hybrid Transformer for RUL Prediction in Predictive Maintenance for Smart Manufacturing
DOI:
https://doi.org/10.4108/eetinis.132.12274Keywords:
Remaining Useful Lifetime, Bayesian optimization,, Transformer, Fourier transform, Convolutional Neural networkAbstract
Ensuring continuous operation and minimizing unexpected failures are critical priorities in industrial manufacturing. Due to this requirement, smart manufacturing has transformed maintenance strategies, moving from traditional scheduled approaches to predictive maintenance (PdM), which leverages actual machine health conditions. A powerful technique that enables accurate PdM is Remaining Useful Life (RUL) estimation. Accurate RUL prediction allows the assessment of machine health, thereby facilitating timely and appropriate maintenance decisions to sustain continuous operation and reduce repair costs. While several existing models have been developed for RUL estimation, they often struggle to capture long-term dependencies in time series data, limiting their predictive accuracy. In this study, we propose a novel architecture that combines a one-dimensional Convolutional Neural Network (1D-CNN) with two consecutive transformer encoders enhanced by Fourier transforms to address these challenges. The performance of the proposed model was evaluated based on two well-known benchmark datasets, C-MAPSS and its improved version, N-CMAPSS. The experiment results show that our approach outperforms current state-of-the-art methods in both prediction accuracy and computational efficiency, demonstrating its potential for practical applications.
Downloads
References
[1] M. Xiong, H. Wang, Q. Fu, and Y. Xu, “Digital twin– driven aero-engine intelligent predictive maintenance,” The International Journal of Advanced Manufacturing Technology, vol. 114, no. 11, pp. 3751–3761, 2021.
[2] X.-S. Si, W. Wang, C.-H. Hu, and D.-H. Zhou, “Remaining useful life estimation – a review on the statistical data driven approaches,” European Journal of Operational Research, vol. 213, no. 1, pp. 1–14, 2011. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0377221710007903
[3] A. Meddaoui, M. Hain, and A. Hachmoud, “The benefits of predictive maintenance in manufacturing excellence: a case study to establish reliable methods for predicting failures,” The International Journal of Advanced Manufacturing Technology, vol. 128, no. 7, pp. 3685–3690, 2023.
[4] K. Medjaher, D. A. Tobon-Mejia, and N. Zerhouni, “Remaining useful life estimation of critical components with application to bearings,” IEEE Transactions on Reliability, vol. 61, no. 2, pp. 292–302, 2012.
[5] X. Li, F. Elasha, S. Shanbr, and D. Mba, “Remaining useful life prediction of rolling element bearings using supervised machine learning,” Energies, vol. 12, no. 14, 2019. [Online]. Available: https: //www.mdpi.com/1996-1073/12/14/2705
[6] T. Benkedjouh, K. Medjaher, N. Zerhouni, and S. Rechak, “Remaining useful life estimation based on nonlinear feature reduction and support vector regression,” Engineering Applications of Artificial Intelligence, vol. 26, no. 7, pp. 1751–1760, 2013. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0952197613000365
[7] X. Li, Q. Ding, and J.-Q. Sun, “Remaining useful life estimation in prognostics using deep convolution neural networks,” Reliability Engineering & System Safety, vol. 172, pp. 1–11, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832017307779
[8] J. Liu, F. Lei, C. Pan, D. Hu, and H. Zuo, “Prediction of remaining useful life of multi-stage aero-engine based on clustering and lstm fusion,” Reliability Engineering & System Safety, vol. 214, p. 107807, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832021003306
[9] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
[10] X. Li, J. Li, L. Zuo, L. Zhu, and H. T. Shen, “Domain adaptive remaining useful life prediction with transformer,” IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1–13, 2022.
[11] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
[12] H. D. Nguyen, N. H. Long, K. N. Ha, N. V. Hoang, T. T. Huong, and K. P. Tran, “Translighter: A light-weight federated learning-based architecture for remaining useful lifetime prediction,” Computers in Industry, vol. 148, p. 103888, 2023. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S0166361523000386
[13] J. Lee-Thorp, J. Ainslie, I. Eckstein, and S. Ontanon, “Fnet: Mixing tokens with fourier transforms,” arXiv [Online]. 2019. Available: https://arxiv.org/abs/2105.03824
[14] F. Zeng, Y. Li, Y. Jiang, and G. Song, “A deep attention residual neural networkbased remaining useful life prediction of machinery,” Measurement, vol. 181, p. 109642, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0263224121006114
[15] Y. Song, S. Gao, Y. Li, L. Jia, Q. Li, and F. Pang, “Distributed attention-based temporal convolutional network for remaining useful life prediction,” IEEE Internet of Things Journal, vol. 8, no. 12, pp. 9594–9602, 2021.
[16] L. Biggio, A. Wieland, M. A. Chao, I. Kastanis, and O. Fink, “Uncertainty-aware prognosis via deep gaussian process,” IEEE Access, vol. 9, pp. 123 517–123 527, 2021.
[17] S. Zheng, K. Ristovski, A. Farahat, and C. Gupta, “Long short-term memory network for remaining useful life estimation,” in 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), 2017, pp. 88–95.
[18] J. Yao, B. Lu, and J. Zhang, “Tool remaining useful life prediction using deep transfer reinforcement learning based on long short-term memory networks,” The International Journal of Advanced Manufacturing Technology, vol. 118, no. 3, pp. 1077–1086, 2022.
[19] Y. Zhang, Y. Xin, Z. wei Liu, M. Chi, and G. Ma, “Health status assessment and remaining useful life prediction of aero-engine based on bigru and mmoe,” Reliability Engineering & System Safety, vol. 220, p. 108263, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832021007389
[20] J. Zeng and Z. Liang, “A deep gaussian process approach for predictive maintenance,” IEEE Transactions on Reliability, 2022.
[21] H. Mo and G. Iacca, “Evolutionary neural architecture search on transformers for rul prediction,” Materials and Manufacturing Processes, vol. 0, no. 0, pp. 1–18, 2023. [Online]. Available: https://doi.org/10.1080/10426914. 2023.2199499
[22] Y. Zhu, Z. Liu, Z. Luo, C. Du, and H. Wang, “Aircraft engine remaining life prediction method with deep learning,” in 2022 International Conference on Artificial Intelligence and Computer Information Technology (AICIT), 2022, pp. 1–6.
[23] H. Liu, Z. Liu, W. Jia, and X. Lin, “Remaining useful life prediction using a novel feature-attention-based end-to-end approach,” IEEE Transactions on Industrial Informatics, vol. 17, no. 2, pp. 1197–1207, 2021.
[24] D. Kapoor, D. Gupta, S. AgarwaI, M. Uppal, S. Juneja, and M. K. Sharma, “Starnet: Stacked transfer-aware for robust remaining useful life prediction for c-mapss multi-regime engines,” IEEE Access, 2026.
[25] Y. Mo, Q. Wu, X. Li, and B. Huang, “Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unit,” Journal of Intelligent Manufacturing, vol. 32, pp. 1997 – 2006, 2021. [Online]. Available: https://api.semanticscholar.org/ CorpusID:233649678
[26] T. Jing, P. Zheng, L. Xia, and T. Liu, “Transformer-based hierarchical latent space vae for interpretable remaining useful life prediction,” Advanced Engineering Informatics, vol. 54, p. 101781, 2022. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1474034622002397
[27] Z. Zhang, W. Song, and Q. Li, “Dual-aspect selfattention based on transformer for remaining useful life prediction,” IEEE Transactions on Instrumentation and Measurement, vol. 71, pp. 1–11, 2022.
[28] G. S. Chadha, S. R. B. Shah, A. Schwung, and S. X. Ding, “Shared temporal attention transformer for remaining useful lifetime estimation,” IEEE Access, vol. 10, pp. 74 244–74 258, 2022.
[29] H.-K. Wang, Y. Cheng, and K. Song, “Remaining useful life estimation of aircraft engines using a joint deep learning model based on tcnn and transformer,” Computational Intelligence and Neuroscience, vol. 2021, 2021.
[30] S. Kamei and S. Taghipour, “A comparison study of centralized and decentralized federated learning approaches utilizing the transformer architecture for estimating remaining useful life,” Reliability Engineering & System Safety, vol. 233, p. 109130, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0951832023000455
[31] Z. Xu, Y. Zhang, J. Miao, and Q. Miao, “Global attention mechanism based deep learning for remaining useful life prediction of aero-engine,” Measurement, vol. 217, p. 113098, 2023.
[32] F. Xiang, Y. Zhang, S. Zhang, Z. Wang, L. Qiu, and J.-H. Choi, “Bayesian gated-transformer model for risk-aware prediction of aero-engine remaining useful life,” Expert Systems with Applications, p. 121859, 2023.
[33] S. Lv, S. Liu, and H. Li, “New method for remaining useful life prediction based on recurrence multiinformation time-frequency transformer networks: Rul prediction with recurrence multi-information tf transformers,” Quality and Reliability Engineering International, vol. 41, no. 5, pp. 1643–1663, 2025.
[34] C. Zhao, H. Shi, X. Huang, and Y. Zhang, “A multiple conditions dual inputs attention network remaining useful life prediction method,” Engineering Applications of Artificial Intelligence, vol. 133, p. 108160, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S095219762400318X
[35] Z. Huang, Y. He, and B. Sick, “Spatio-temporal attention graph neural network for remaining useful life prediction,” arXiv [Online]. 2024. Available: https://arxiv.org/abs/2401.15964
[36] P. R. de O. da Costa, A. Akcay, Y. Zhang, and U. Kaymak, “Remaining useful lifetime prediction via deep domain adaptation,” arXiv [Online]. 2019. Available: https://arxiv.org/abs/1907.07480
[37] H. Li, Y. Li, Z. Wang, and Z. Li, “Remaining useful life prediction of aero-engine based on pca-lstm,” in 2021 7th International Conference on Condition Monitoring of Machinery in Non-Stationary Operations (CMMNO), 2021, pp. 63–66.
[38] Y. He, H. Su, E. Zio, S. Peng, L. Fan, Z. Yang, Z. Yang, and J. Zhang, “A systematic method of remaining useful life estimation based on physics-informed graph neural networks with multisensor data,” Reliability Engineering & System Safety, vol. 237, p. 109333, 2023.
[39] J. Chen, Z. Chen, J. Xia, R. Huang, and W. Li, “Multigranularity cross-domain temporal regression network for remaining useful life estimation of aero engines,” in 2022 International Conference on Sensing, Measurement & Data Analytics in the era of Artificial Intelligence (ICSMD), 2022, pp. 1–6.
[40] G. Kim, J. G. Choi, and S. Lim, “Using transformer and a reweighting technique to develop a remaining useful life estimation method for turbofan engines,” Engineering Applications of Artificial Intelligence, vol. 133, p. 108475, 2024.
[41] S. Deng and J. Zhou, “Prediction of remaining useful life of aero-engines based on cnn-lstm-attention,” International Journal of Computational Intelligence Systems, vol. 17, no. 1, p. 232, 2024.
[42] T. Zhang, L. Jiang, R. Huang, and X. Zhang, “A multiscale cross-channel attention network for remaining useful life prediction with variable sensors,” IEEE Transactions on Instrumentation and Measurement, 2025.
[43] S. Kiranyaz, O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj, and D. J. Inman, “1d convolutional neural networks and applications: A survey,” Mechanical systems and signal processing, vol. 151, p. 107398, 2021.
[44] M. Arias Chao, C. Kulkarni, K. Goebel, and O. Fink, “Aircraft engine run-to-failure dataset under real flight conditions for prognostics and diagnostics,” Data, vol. 6, no. 1, p. 5, 2021.
[45] A. Saxena, K. Goebel, D. Simon, and N. Eklund, “Damage propagation modeling for aircraft engine runto- failure simulation,” in 2008 International Conference on Prognostics and Health Management, 2008, pp. 1–9.
[46] Y. Mo, Q. Wu, X. Li, and B. Huang, “Remaining useful life estimation via transformer encoder enhanced by a gated convolutional unit,” Journal of Intelligent Manufacturing, vol. 32, pp. 1997–2006, 2021.
[47] H. Tian, L. Yang, and B. Ju, “Spatial correlation and temporal attention-based lstm for remaining useful life prediction of turbofan engine,” Measurement, vol. 214, p. 112816, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0263224123003809
[48] H. Du Nguyen, K. P. Tran, P. Castagliola, and F. M. Megahed, “Enabling smart manufacturing with artificial intelligence and big data: a survey and perspective,” in Advanced Manufacturing Methods. CRC Press, 2022, pp. 1–26.
[49] K. T. Nguyen, K. Medjaher, and C. Gogu, “Probabilistic deep learning methodology for uncertainty quantification of remaining useful lifetime of multi-component systems,” Reliability Engineering & System Safety, vol. 222, p. 108383, 2022.
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2026 Nguyen Huu Du, Nguyen Xuan Hoang, Dao Bich Thuong, Do Cong Ngon, Truong Thu Huong
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
