Deep Reinforcement Learning-Driven Adaptive Control for Intelligent Manufacturing Systems: A Multi-Sensor Fusion Framework for Real-Time Anomaly Detection
DOI:
https://doi.org/10.4108/eetsis.11781Keywords:
Deep Reinforcement Learning, Intelligent Manufacturing Systems, Multi-Sensor Fusion, Residual-Based Anomaly Detection, Adaptive Control, Proximal Policy OptimizationAbstract
One of the principal requirements for Intelligent Manufacturing Systems that are working in complicated, nonlinear, and uncertain environments is the robust real-time monitoring and an adaptive control system. In this paper, a deep reinforcement learning-based adaptive control framework is proposed that combines multi-sensor fusion, residual-based anomaly detection, and robust state estimation. The framework is tested on a multi-sensor manufacturing dataset that consists of a total of 10,000 time steps of four different heterogeneous sensor streams collected under both normal and anomalous operating conditions. The sensor data is initially preprocessed by noise filtering, normalization, synchronization, and feature extraction, after which an Unscented Kalman Filter (UKF) is used for sensor fusion and state estimation. Residual analysis along with Support Vector Machine (SVM) allows for real-time anomaly detection and classification, while Multi-Model Adaptive Estimation (MMAE) technique improves the robustness of the state estimation. A Proximal Policy Optimization (PPO) agent uses the updated system state and the anomaly information for adaptive control. The results of the experiments reveal perfect anomaly classification with an ROC-AUC of 1.000, highly stable state estimation in all state dimensions, and continuous control performance increase with the average reward of 1.79, thereby validating the proposed framework's suitability for anomaly-aware adaptive control in intelligent manufacturing systems.
References
[1] R. Sahba, R. Radfar, A. Rajabzadeh Ghatari, and A. Pour Ebrahimi, “Development of Industry 4.0 predictive maintenance architecture for broadcasting chain,” Adv. Eng. Inform., vol. 49, p. 101324, Aug. 2021, doi: 10.1016/j.aei.2021.101324.
[2] V. Varriale, A. Cammarano, F. Michelino, and M. Caputo, “Critical analysis of the impact of artificial intelligence integration with cutting-edge technologies for production systems,” J. Intell. Manuf., vol. 36, no. 1, pp. 61–93, Jan. 2025, doi: 10.1007/s10845-023-02244-8.
[3] Q. Lawal, “Leveraging Artificial Intelligence to Enhance Process Control and Improve Efficiency in Manufacturing Industries,” Int. J. Comput. Appl. Technol. Res., vol. 14, no. 2, Jan. 2025, doi: 10.7753/IJCATR1402.1002.
[4] D. Frempong et al., “Real-Time Analytics Dashboards for Decision-Making Using Tableau in Public Sector and Business Intelligence Applications,” J. Front. Multidiscip. Res., vol. 3, pp. 65–80, Jan. 2022, doi: 10.54660/.IJFMR.2022.3.2.65-80.
[5] M. A. Rawajbeh et al., “Trustworthy Adaptive AI for Real-Time Intrusion Detection in Industrial IoT Security,” IoT, vol. 6, no. 3, Sep. 2025, doi: 10.3390/iot6030053.
[6] A. Siriweera and K. Naruse, “QoS-Aware Federated Crosschain-Based Model-Driven Reference Architecture for IIoT Sensor Networks in Distributed Manufacturing,” IEEE Sens. J., vol. 23, no. 23, pp. 29630–29644, Dec. 2023, doi: 10.1109/JSEN.2023.3325342.
[7] G. Mattera, A. Caggiano, and L. Nele, “Optimal data-driven control of manufacturing processes using reinforcement learning: an application to wire arc additive manufacturing,” J. Intell. Manuf., vol. 36, no. 2, pp. 1291–1310, Feb. 2025, doi: 10.1007/s10845-023-02307-w.
[8] I. Podder, T. Fischl, U. Bub, I. Podder, T. Fischl, and U. Bub, “Artificial Intelligence Applications for MEMS-Based Sensors and Manufacturing Process Optimization,” Telecom, vol. 4, no. 1, pp. 165–197, Mar. 2023, doi: 10.3390/telecom4010011.
[9] F. J. Maseda et al., “Sensors Data Analysis in Supervisory Control and Data Acquisition (SCADA) Systems to Foresee Failures with an Undetermined Origin,” Sensors, vol. 21, no. 8, Apr. 2021, doi: 10.3390/s21082762.
[10] W. Strielkowski et al., “Prospects and Challenges of the Machine Learning and Data-Driven Methods for the Predictive Analysis of Power Systems: A Review,” Energies, vol. 16, no. 10, May 2023, doi: 10.3390/en16104025.
[11] F. Ale, I. Daniyan, O. Aderoba, and A. A. Adediran, “An Artificial Intelligence-based Integrated Framework for Lean and Smart Manufacturing: A Case Study of the Rail Industry,” in 2024 International Conference on Science, Engineering and Business for Driving Sustainable Development Goals (SEB4SDG), Omu-Aran, Nigeria: IEEE, Apr. 2024, pp. 1–10. doi: 10.1109/SEB4SDG60871.2024.10629752.
[12] A. Corallo, M. Lazoi, M. Lezzi, and P. Pontrandolfo, “Cybersecurity Challenges for Manufacturing Systems 4.0: Assessment of the Business Impact Level,” IEEE Trans. Eng. Manag., vol. 70, no. 11, pp. 3745–3765, Nov. 2023, doi: 10.1109/TEM.2021.3084687.
[13] L. Rojas, Á. Peña, J. Garcia, L. Rojas, Á. Peña, and J. Garcia, “AI-Driven Predictive Maintenance in Mining: A Systematic Literature Review on Fault Detection, Digital Twins, and Intelligent Asset Management,” Appl. Sci., vol. 15, no. 6, Mar. 2025, doi: 10.3390/app15063337.
[14] B. Xia, K. Wang, A. Xu, P. Zeng, N. Yang, and B. Li, “Intelligent Fault Diagnosis for Bearings of Industrial Robot Joints Under Varying Working Conditions Based on Deep Adversarial Domain Adaptation,” IEEE Trans. Instrum. Meas., vol. 71, pp. 1–13, 2022, doi: 10.1109/TIM.2022.3158996.
[15] J. Hu, C. Fan, M. Ozay, Q. Gao, Y. Guo, and T. L. Lam, “Robust Depth Estimation Under Sensor Degradations: A Multi-Sensor Fusion Perspective,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 47, no. 10, pp. 8691–8707, Oct. 2025, doi: 10.1109/TPAMI.2025.3581311.
[16] M. Shahin, A. Hosseinzadeh, and F. F. Chen, “Generative artificial intelligence in manufacturing: applications, case studies, and future directions for next-generation intelligent production systems,” Int. J. Adv. Manuf. Technol., vol. 141, no. 3, pp. 1159–1265, Nov. 2025, doi: 10.1007/s00170-025-16667-5.
[17] Z. Lian, Z. Zhou, C. Hu, Z. Ming, J. Wang, and Y. Zhao, “A Belief Rule-Based Performance Evaluation Model for Complex Systems Considering Sensors Disturbance,” IEEE Trans. Reliab., vol. 73, no. 2, pp. 1245–1257, Jun. 2024, doi: 10.1109/TR.2023.3311436.
[18] D. A. Bhanage, A. V. Pawar, and K. Kotecha, “IT Infrastructure Anomaly Detection and Failure Handling: A Systematic Literature Review Focusing on Datasets, Log Preprocessing, Machine & Deep Learning Approaches and Automated Tool,” IEEE Access, vol. 9, pp. 156392–156421, 2021, doi: 10.1109/ACCESS.2021.3128283.
[19] J. Desikan et al., “Hybrid Machine Learning-Based Fault-Tolerant Sensor Data Fusion and Anomaly Detection for Fire Risk Mitigation in IIoT Environment,” Sensors, vol. 25, no. 7, Mar. 2025, doi: 10.3390/s25072146.
[20] X. Xiao, J. Dufek, and R. R. Murphy, “Robot Risk-Awareness by Formal Risk Reasoning and Planning,” IEEE Robot. Autom. Lett., vol. 5, no. 2, pp. 2856–2863, Apr. 2020, doi: 10.1109/LRA.2020.2974434.
[21] M. Orabi, K. P. Tran, P. Egger, and S. Thomassey, “Anomaly detection in smart manufacturing: An Adaptive Adversarial Transformer-based model,” J. Manuf. Syst., vol. 77, pp. 591–611, Dec. 2024, doi: 10.1016/j.jmsy.2024.09.021.
[22] C.-C. Wong, H.-M. Feng, and K.-L. Kuo, “Multi-Sensor Fusion Simultaneous Localization Mapping Based on Deep Reinforcement Learning and Multi-Model Adaptive Estimation,” Sensors, vol. 24, no. 1, p. 48, Dec. 2023, doi: 10.3390/s24010048.
[23] T. Zhou, M. Chen, and J. Zou, “Reinforcement Learning Based Data Fusion Method for Multi-Sensors,” IEEECAA J. Autom. Sin., vol. 7, no. 6, pp. 1489–1497, 2020, doi: 10.1109/JAS.2020.1003180.
[24] Zhe Sun, Minru Kong, and Guiqi Zhu, “Multi-source Sensor Data Fusion and Deep Reinforcement Learning Anomaly Detection Method for Power UAV Inspection,” Int. J. Hous. Sci. Its Appl., vol. 46, no. 3, 2025.
[25] Y. Yu and M. Liu, “Deep Reinforcement Learning-Based Multisensor Control for Labeled Multi-Bernoulli Filtering,” IEEE Trans. Aerosp. Electron. Syst., vol. 61, no. 5, pp. 13548–13564, Oct. 2025, doi: 10.1109/TAES.2025.3579476.
[26] Y. M. Alginahi, O. Sabri, and W. Said, “Reinforcement Learning for Industrial Automation: A Comprehensive Review of Adaptive Control and Decision-Making in Smart Factories,” Machines, vol. 13, no. 12, p. 1140, Dec. 2025, doi: 10.3390/machines13121140.
[27] R. Anwar, J.-W. Kwon, and W.-T. Kim, “A Deep Reinforcement Learning-Based Concurrency Control of Federated Digital Twin for Software-Defined Manufacturing Systems,” Appl. Sci., vol. 15, no. 15, p. 8245, Jul. 2025, doi: 10.3390/app15158245.
[28] J. Huang et al., “Deep Reinforcement Learning-Based Dynamic Reconfiguration Planning for Digital Twin-Driven Smart Manufacturing Systems With Reconfigurable Machine Tools,” IEEE Trans. Ind. Inform., vol. 20, no. 11, pp. 13135–13146, Nov. 2024, doi: 10.1109/TII.2024.3431095.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Shanshan Kong, Peng Zhou
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open access article distributed under the terms of the CC BY-NC-SA 4.0, which permits copying, redistributing, remixing, transformation, and building upon the material in any medium so long as the original work is properly cited.
