Comparative Analysis of Transformer and LSTM Architectures for Cybersecurity Threat Detection Using Machine Learning
DOI:
https://doi.org/10.4108/airo.9759Keywords:
Cybersecurity, Machine learning, LSTM, transformer, threat classification, emerging threats, predictive analyticsAbstract
The growing prevalence of advanced persistent threats (APTs), zero-day exploits, and the rapid proliferation of IoT devices have exposed limitations in traditional cybersecurity approaches. In response, this study presents a comparative analysis of deep learning models—specifically Long Short-Term Memory (LSTM) and Transformer-based architectures—for cybersecurity threat classification from textual data. Leveraging a standardized dataset and consistent preprocessing pipeline, both models are evaluated across key performance metrics, including accuracy, precision, recall, and F1-score. The results demonstrate that Transformer models significantly outperform LSTM-based approaches, exhibiting superior capacity to capture long-range dependencies, handle complex threat narratives, and generalize to previously unseen data. These findings offer valuable insights into the practical application of modern deep learning techniques in cybersecurity and provide a foundation for designing more robust and adaptive threat detection systems.
Downloads
References
[1] W. E. Forum. "Global Risks Report 2023." The World Economic Forum. https://www.weforum.org/publications/global-risks-report-2023/digest/ (accessed 10 Sept, 2024).
[2] D. B. Davis. "ISTR 2019: Internet of Things Cyber Attacks Grow More Diverse." https://symantec-enterprise-blogs.security.com/expert-perspectives/istr-2019-internet-things-cyber-attacks-grow-more-diverse (accessed 30 Aug, 2024).
[3] P. Estes. "Cybercrime Costs Skyrocket to $10.5 Trillion: AI in Cybersecurity Fights Back." virtasant. https://www.virtasant.com/ai-today/cybercrime-costs-skyrocket-to-10-5-trillion-ai-in-cybersecurity-fights-back (accessed Sep 1, 2024, 2024).
[4] D. D. Jim Boehm, Charlie Lewis, Kathleen Li, Daniel Wallance. "Cybersecurity trends: Looking over the horizon." McKinsey & Company. https://www.mckinsey.com/capabilities/risk-and-resilience/our-insights/cybersecurity/cybersecurity-trends-looking-over-the-horizon (accessed.
[5] IBM. "Cost of a Data Breach Report 2021." IBM. https://www.ibm.com/reports/data-breach (accessed 28 Aug, 2024).
[6] I. F. Kilincer, F. Ertam, and A. J. C. N. Sengur, "Machine learning methods for cyber security intrusion detection: Datasets and comparative study," vol. 188, p. 107840, 2021.
[7] B. Naik, A. Mehta, H. Yagnik, M. J. C. Shah, and I. Systems, "The impacts of artificial intelligence techniques in augmentation of cybersecurity: a comprehensive review," vol. 8, no. 2, pp. 1763-1780, 2022.
[8] H. J. a. p. a. Kheddar, "Transformers and large language models for efficient intrusion detection systems: A comprehensive survey," 2024.
[9] T. M. Chen and J.-M. Robert, "The evolution of viruses and worms," in Statistical methods in computer security: CRC press, 2004, pp. 289-310.
[10] Symantec. "Symantec Global Internet Security Threat Report Trends for 2008." Symantec enterprise security. https://docs.broadcom.com/doc/istr-global-09-april-volume-xiv-en (accessed 28 Aug, 2024).
[11] N. Sfetcu, Advanced Persistent Threats in Cybersecurity–Cyber Warfare. MultiMedia Publishing, 2024.
[12] L. Bilge and T. Dumitraş, "Before we knew it: an empirical study of zero-day attacks in the real world," in Proceedings of the 2012 ACM conference on Computer and communications security, 2012, pp. 833-844.
[13] M. Brundage et al., "The malicious use of artificial intelligence: Forecasting, prevention, and mitigation," 2018.
[14] S. Sicari, A. Rizzardi, L. A. Grieco, and A. J. C. n. Coen-Porisini, "Security, privacy and trust in Internet of Things: The road ahead," vol. 76, pp. 146-164, 2015.
[15] K. Hashizume, D. G. Rosado, E. Fernández-Medina, E. B. J. J. o. i. s. Fernandez, and applications, "An analysis of security issues for cloud computing," vol. 4, pp. 1-13, 2013.
[16] H.-J. Liao, C.-H. R. Lin, Y.-C. Lin, K.-Y. J. J. o. N. Tung, and C. Applications, "Intrusion detection system: A comprehensive review," vol. 36, no. 1, pp. 16-24, 2013.
[17] S. Furnell, Computer insecurity: Risking the system. Springer Science & Business Media, 2005.
[18] K. J. N. s. p. SCARFONE, "Guide to Intrusion Detection and Prevention Systems (IDPS)," 2007.
[19] I. Androutsopoulos, J. Koutsias, K. V. Chandrinos, G. Paliouras, and C. D. J. a. p. c. Spyropoulos, "An evaluation of naive bayesian anti-spam filtering," 2000.
[20] A. Yaqoob. "Watson for Cyber Security." IBM. https://www.ibm.com/blogs/nordic-msp/watson-cyber-security/ (accessed 2 September 2024).
[21] A. L. Buczak, E. J. I. C. s. Guven, and tutorials, "A survey of data mining and machine learning methods for cyber security intrusion detection," vol. 18, no. 2, pp. 1153-1176, 2015.
[22] M. Alabadi and Y. Celik, "Anomaly detection for cyber-security based on convolution neural network: A survey," in 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), 2020: IEEE, pp. 1-14.
[23] D. Dasgupta, Z. Akhtar, S. J. T. J. o. D. M. Sen, and Simulation, "Machine learning in cybersecurity: a comprehensive survey," vol. 19, no. 1, pp. 57-106, 2022.
[24] G. Kim, S. Lee, and S. J. E. S. w. A. Kim, "A novel hybrid intrusion detection method integrating anomaly detection with misuse detection," vol. 41, no. 4, pp. 1690-1700, 2014.
[25] V. Sowinski-Mydlarz, J. Li, K. Ouazzane, V. J. T. o. C. S. Vassilev, and C. Intelligence, "Threat intelligence using machine learning packet dissection," 2021.
[26] M. Ring, S. Wunderlich, D. Scheuring, D. Landes, A. J. C. Hotho, and security, "A survey of network-based intrusion detection data sets," vol. 86, pp. 147-167, 2019.
[27] B. Biggio and F. Roli, "Wild patterns: Ten years after the rise of adversarial machine learning," in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 2018, pp. 2154-2156.
[28] R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. J. A. c. s. Pedreschi, "A survey of methods for explaining black box models," vol. 51, no. 5, pp. 1-42, 2018.
[29] B. D. Mittelstadt, P. Allo, M. Taddeo, S. Wachter, L. J. B. D. Floridi, and Society, "The ethics of algorithms: Mapping the debate," vol. 3, no. 2, p. 2053951716679679, 2016.
[30] F. Ramoliya, R. Kakkar, R. Gupta, S. Tanwar, and S. Agrawal, "SEAM: Deep Learning-based Secure Message Exchange Framework For Autonomous EVs," in 2023 IEEE Globecom Workshops (GC Wkshps), 2023: IEEE, pp. 80-85.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Jobanpreet Kaur, Mani Prabha, Md Samiun, Syed Nazmul Hasan, Rakibul Hasan, Hammed Esa, Md Fakhrul Hasan Bhuiyan, Md Abdur Rob, Durga Shahi
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open access article distributed under the terms of the CC BY-NC-SA 4.0, which permits copying, redistributing, remixing, transformation, and building upon the material in any medium so long as the original work is properly cited.
