Comparative Analysis of Transformer and LSTM Architectures for Cybersecurity Threat Detection Using Machine Learning

Jobanpreet Kaur; Mani Prabha; Md Samiun; Syed Nazmul Hasan; Rakibul Hasan; Hammed Esa; Md Fakhrul Hasan Bhuiyan; Md Abdur Rob; Durga Shahi

doi:10.4108/airo.9759

Authors

Jobanpreet Kaur Westcliff University
Mani Prabha International American University
Md Samiun International American University
Syed Nazmul Hasan Westcliff University
Rakibul Hasan Westcliff University
Hammed Esa International American University
Md Fakhrul Hasan Bhuiyan Trine University
Md Abdur Rob Ohio University
Durga Shahi Westcliff University

DOI:

https://doi.org/10.4108/airo.9759

Keywords:

Cybersecurity, Machine learning, LSTM, transformer, threat classification, emerging threats, predictive analytics

Abstract

The growing prevalence of advanced persistent threats (APTs), zero-day exploits, and the rapid proliferation of IoT devices have exposed limitations in traditional cybersecurity approaches. In response, this study presents a comparative analysis of deep learning models—specifically Long Short-Term Memory (LSTM) and Transformer-based architectures—for cybersecurity threat classification from textual data. Leveraging a standardized dataset and consistent preprocessing pipeline, both models are evaluated across key performance metrics, including accuracy, precision, recall, and F1-score. The results demonstrate that Transformer models significantly outperform LSTM-based approaches, exhibiting superior capacity to capture long-range dependencies, handle complex threat narratives, and generalize to previously unseen data. These findings offer valuable insights into the practical application of modern deep learning techniques in cybersecurity and provide a foundation for designing more robust and adaptive threat detection systems.

Downloads

Download data is not yet available.

Author Biographies

Jobanpreet Kaur, Westcliff University

College of Technology & Engineering, Westcliff University, CA 92614, USA

Mani Prabha, International American University

Department of Business Administration, International American University, Los Angeles, CA 90010, USA

Md Samiun, International American University

Department of Business Administration, International American University, Los Angeles, CA 90010, USA

Syed Nazmul Hasan, Westcliff University

College of Technology & Engineering, Westcliff University, CA 92614, USA

Rakibul Hasan, Westcliff University

Department of Business Administration, Westcliff University, Irvine, CA 92614, USA

References

[1] W. E. Forum. "Global Risks Report 2023." The World Economic Forum. https://www.weforum.org/publications/global-risks-report-2023/digest/ (accessed 10 Sept, 2024).

[2] D. B. Davis. "ISTR 2019: Internet of Things Cyber Attacks Grow More Diverse." https://symantec-enterprise-blogs.security.com/expert-perspectives/istr-2019-internet-things-cyber-attacks-grow-more-diverse (accessed 30 Aug, 2024).

[3] P. Estes. "Cybercrime Costs Skyrocket to $10.5 Trillion: AI in Cybersecurity Fights Back." virtasant. https://www.virtasant.com/ai-today/cybercrime-costs-skyrocket-to-10-5-trillion-ai-in-cybersecurity-fights-back (accessed Sep 1, 2024, 2024).

[4] D. D. Jim Boehm, Charlie Lewis, Kathleen Li, Daniel Wallance. "Cybersecurity trends: Looking over the horizon." McKinsey & Company. https://www.mckinsey.com/capabilities/risk-and-resilience/our-insights/cybersecurity/cybersecurity-trends-looking-over-the-horizon (accessed.

[5] IBM. "Cost of a Data Breach Report 2021." IBM. https://www.ibm.com/reports/data-breach (accessed 28 Aug, 2024).

[6] I. F. Kilincer, F. Ertam, and A. J. C. N. Sengur, "Machine learning methods for cyber security intrusion detection: Datasets and comparative study," vol. 188, p. 107840, 2021.

[7] B. Naik, A. Mehta, H. Yagnik, M. J. C. Shah, and I. Systems, "The impacts of artificial intelligence techniques in augmentation of cybersecurity: a comprehensive review," vol. 8, no. 2, pp. 1763-1780, 2022.

[8] H. J. a. p. a. Kheddar, "Transformers and large language models for efficient intrusion detection systems: A comprehensive survey," 2024.

[9] T. M. Chen and J.-M. Robert, "The evolution of viruses and worms," in Statistical methods in computer security: CRC press, 2004, pp. 289-310.

[10] Symantec. "Symantec Global Internet Security Threat Report Trends for 2008." Symantec enterprise security. https://docs.broadcom.com/doc/istr-global-09-april-volume-xiv-en (accessed 28 Aug, 2024).

[11] N. Sfetcu, Advanced Persistent Threats in Cybersecurity–Cyber Warfare. MultiMedia Publishing, 2024.

[12] L. Bilge and T. Dumitraş, "Before we knew it: an empirical study of zero-day attacks in the real world," in Proceedings of the 2012 ACM conference on Computer and communications security, 2012, pp. 833-844.

[13] M. Brundage et al., "The malicious use of artificial intelligence: Forecasting, prevention, and mitigation," 2018.

[14] S. Sicari, A. Rizzardi, L. A. Grieco, and A. J. C. n. Coen-Porisini, "Security, privacy and trust in Internet of Things: The road ahead," vol. 76, pp. 146-164, 2015.

[15] K. Hashizume, D. G. Rosado, E. Fernández-Medina, E. B. J. J. o. i. s. Fernandez, and applications, "An analysis of security issues for cloud computing," vol. 4, pp. 1-13, 2013.

[16] H.-J. Liao, C.-H. R. Lin, Y.-C. Lin, K.-Y. J. J. o. N. Tung, and C. Applications, "Intrusion detection system: A comprehensive review," vol. 36, no. 1, pp. 16-24, 2013.

[17] S. Furnell, Computer insecurity: Risking the system. Springer Science & Business Media, 2005.

[18] K. J. N. s. p. SCARFONE, "Guide to Intrusion Detection and Prevention Systems (IDPS)," 2007.

[19] I. Androutsopoulos, J. Koutsias, K. V. Chandrinos, G. Paliouras, and C. D. J. a. p. c. Spyropoulos, "An evaluation of naive bayesian anti-spam filtering," 2000.

[20] A. Yaqoob. "Watson for Cyber Security." IBM. https://www.ibm.com/blogs/nordic-msp/watson-cyber-security/ (accessed 2 September 2024).

[21] A. L. Buczak, E. J. I. C. s. Guven, and tutorials, "A survey of data mining and machine learning methods for cyber security intrusion detection," vol. 18, no. 2, pp. 1153-1176, 2015.

[22] M. Alabadi and Y. Celik, "Anomaly detection for cyber-security based on convolution neural network: A survey," in 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), 2020: IEEE, pp. 1-14.

[23] D. Dasgupta, Z. Akhtar, S. J. T. J. o. D. M. Sen, and Simulation, "Machine learning in cybersecurity: a comprehensive survey," vol. 19, no. 1, pp. 57-106, 2022.

[24] G. Kim, S. Lee, and S. J. E. S. w. A. Kim, "A novel hybrid intrusion detection method integrating anomaly detection with misuse detection," vol. 41, no. 4, pp. 1690-1700, 2014.

[25] V. Sowinski-Mydlarz, J. Li, K. Ouazzane, V. J. T. o. C. S. Vassilev, and C. Intelligence, "Threat intelligence using machine learning packet dissection," 2021.

[26] M. Ring, S. Wunderlich, D. Scheuring, D. Landes, A. J. C. Hotho, and security, "A survey of network-based intrusion detection data sets," vol. 86, pp. 147-167, 2019.

[27] B. Biggio and F. Roli, "Wild patterns: Ten years after the rise of adversarial machine learning," in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 2018, pp. 2154-2156.

[28] R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. J. A. c. s. Pedreschi, "A survey of methods for explaining black box models," vol. 51, no. 5, pp. 1-42, 2018.

[29] B. D. Mittelstadt, P. Allo, M. Taddeo, S. Wachter, L. J. B. D. Floridi, and Society, "The ethics of algorithms: Mapping the debate," vol. 3, no. 2, p. 2053951716679679, 2016.

[30] F. Ramoliya, R. Kakkar, R. Gupta, S. Tanwar, and S. Agrawal, "SEAM: Deep Learning-based Secure Message Exchange Framework For Autonomous EVs," in 2023 IEEE Globecom Workshops (GC Wkshps), 2023: IEEE, pp. 80-85.