Deep Reinforcement Learning Approaches Against Jammers with Unequal Sweeping Probability Attacks
DOI:
https://doi.org/10.4108/eetinis.v12i4.10461Keywords:
Jamming Attacks, Markov Decision Process, Double Deep Q-Networks, Data Rate Game, Q-learning, Reinforcement Learning, Deep Q-NetworksAbstract
This paper investigates deep reinforcement learning (DRL) approaches designed to counter jammers that maximize disruption by employing unequal sweeping probabilities. We first propose a model and defense action based on a Markov Decision Process (MDP) under non-uniform attacks. A key drawback of the standard MDP model, however, is its assumption that the defending agent can acquire sufficient information about the jamming patterns to determine the transition probability matrix. In a dynamic environment, the attacker’s patterns and models are often unknown or difficult to obtain. To overcome this limitation, RL techniques such as Q-learning, deep Q-network (DQN), and double deep Q-network (DDQN) have been considered effective defense strategies that operate without an explicit jamming model. With Q-learning, defense strategies can still be computationally expensive and require long time to learn the optimal policy. This limitation arises because a large state space or a substantial number of actions causes the Q-table to grow exponentially. Leveraging the flexibility, adaptability, and scalability of RL, we first propose a DQN framework designed to handle large-scale action spaces across expanded channels and jammers. Furthermore, to overcome the inherent overestimation bias present in Q-learning and DQN algorithms, we investigate a DDQN framework. Assuming the estimation error of the action value in DQN follows a zero-mean Gaussian distribution, we then analytically derive the expected loss. Numerical examples are finally presented to characterize the performances of the proposed algorithms and the superiority of DDQN over DQN and Q-learning approaches.
Downloads
References
[1] L. Jia, N. Qi, Z. Su, F. Chu, S. Fang, K.-K. Wong, and C.-B. Chae, “Game theory and reinforcement learning for anti-jamming defense in wireless communications: Current research, challenges, and solutions,” IEEE Communications Surveys & Tutorials, 2024.
[2] Y. Liu, B. Zhang, D. Guo, H. Wang, G. Ding, N. Yang, and J. Gu, “A game theoretical anti-jamming beamforming approach for integrated sensing and communications systems,” IEEE Transactions on Vehicular Technology, vol. 73, no. 10, pp. 15780–15785, 2024.
[3] X. Guan, Y. Hu, and K. Peng, “Bayesian-Stackelberg-gamebased finite-time sliding mode fault-tolerant secure control for cyber–physical systems under jamming attacks and multiple physical faults,” IEEE Transactions on Cybernetics, 2025.
[4] Y. Li, W. Miao, Z. Gao, and G. Lv, “Intelligent jamming strategy for wireless communications based on game theory,” IEEE Access, 2024.
[5] D. Yang, G. Xue, J. Zhang, A. Richa, and X. Fang, “Coping with a smart jammer in wireless networks: A Stackelberg game approach,” IEEE Transactions on Wireless Communications, vol. 12, pp. 4038–4047, Aug. 2013.
[6] A. Garnaev, Y. Liu, and W. Trappe, “Anti-jamming strategy versus a low-power jamming attack when intelligence of adversary’s attack type is unknown,” IEEE Transactions on Signal and Information Processing over Networks, vol. 2, pp. 49–56, Mar. 2016.
[7] L. Jia, F. Yao, Y. Sun, Y. Niu, and Y. Zhu, “Bayesian Stackelberg game for antijamming transmission with incomplete information,” IEEE Communications Letters, vol. 20, pp. 1991–1994, Oct. 2016.
[8] L. Jia, Y. Xu, Y. Sun, S. Feng, and A. Anpalagan, “Stackelberg game approaches for anti-jamming defence in wireless networks,” IEEE Wireless Communications, vol. 25, pp. 120–128, Dec. 2018.
[9] L. Jia, Y. Xu, Y. Sun, S. Feng, L. Yu, and A. Anpalagan, “A game-theoretic learning approach for anti-jamming dynamic spectrum access in dense wireless networks,” IEEE Transactions on Vehicular Technology, vol. 68, pp. 1646–1656, Feb. 2019.
[10] C. Han, A. Liu, H. Wang, L. Huo, and X. Liang, “Dynamic anti-jamming coalition for satellite-enabled Army IoT: A distributed game approach,” to appear in IEEE Internet of Things Journal, 2020.
[11] Q. Wang, T. Nguyen, K. Pham, and H. Kwon, “Mitigating jamming attack: A game-theoretic perspective,” IEEE Transactions on Vehicular Technology, vol. 67, pp. 6063–6074, July 2018.
[12] Y. Wu, B. Wang, K. J. R. Liu, and T. C. Clancy, “Antijamming games in multi-channel cognitive radio networks,” IEEE Journal on Selected Areas in Communications, vol. 30, pp. 4–15, Jan. 2012.
[13] M. K. Hanawal, M. J. Abdel-Rahman, and M. Krunz, “Joint adaptation of frequency hopping and transmission rate for anti-jamming wireless systems,” IEEE Transactions on Mobile Computing, vol. 15, no. 9, pp. 2247–2259, 2016.
[14] M. K. Hanawal, M. J. Abdel-Rahman, and M. Krunz, “Game theoretic anti-jamming dynamic frequency hopping and rate adaptation in wireless systems,” in 2014 12th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), pp. 247–254, 2014.
[15] Z. Yin, J. Li, Z. Wang, Y. Qian, Y. Lin, F. Shu, and W. Chen, “UAV communication against intelligent jamming: A Stackelberg game approach with federated reinforcement learning,” IEEE Transactions on Green Communications and Networking, vol. 8, no. 4, pp. 1796–1808, 2024.
[16] Y. Qin, J. Tang, F. Tang, M. Zhao, and N. Kato, “Multi-agent reinforcement learning in adversarial game environments: Personalized anti-interference strategies for heterogeneous uav communication,” IEEE Transactions on Mobile Computing, 2025.
[17] Z. Lin, L. Xiao, H. Chen, and Z. Lv, “Reinforcement learning based environment-aware v2i anti-jamming communications,” IEEE Transactions on Vehicular Technology, 2024.
[18] B. He, N. Yang, X. Zhang, and W. Wang, “Game theory and reinforcement learning in cognitive radar game modeling and algorithm research: A review,” IEEE Sensors Journal, 2024.
[19] M. Chen, F. Shu, M. Zhu, D. Wu, Y. Yao, and Q. Zhang, “Reinforcement-learning-based uav 3-d target tracking and digital-twin-assisted collision avoidance with integrated sensing and communication,” IEEE Internet of Things Journal, 2025.
[20] Y. Ma, K. Liu, Y. Liu, X. Wang, and Z. Zhao, “An intelligent game-based anti-jamming solution using adversarial populations for aerial communication networks,” IEEE Transactions on Cognitive Communications and Networking, vol. 11, no. 3, pp. 1981–1995, 2025.
[21] Y. Gwon, S. Dastangoo, C. Fossa, and H. T. Kung, “Competing mobile network game: Embracing antijamming and jamming strategies with reinforcement learning,” in IEEE Conference on Communications and Network Security (CNS), pp. 28–36, 2013.
[22] N. Adem and B. Hamdaoui, “Jamming resiliency and mobility management in cognitive communication networks,” in IEEE International Conference on Communications (ICC), pp. 1–6, 2017.
[23] M. A. Aref, S. K. Jayaweera, and S. Machuzak, “Multi-agent reinforcement learning based cognitive anti-jamming,” in 2017 IEEE Wireless Communications and Networking Conference (WCNC), pp. 1–6, 2017.
[24] X. He, H. Dai, and P. Ning, “Faster learning and adaptation in security games by exploiting information asymmetry,” IEEE Transactions on Signal Processing, vol. 64, no. 13, pp. 3429–3443, 2016.
[25] L. K. Nguyen, D. H. N. Nguyen, N. H. Tran, C. Bosler, and D. Brunnenmeyer, “SATCOM jamming resiliency under non-uniform probability of attacks,” in IEEE Military Communications Conference (MILCOM), pp. 85–90, 2021.
[26] L. K. Nguyen, D. H. N. Nguyen, N. H. Tran, C. Bosler, and D. Brunnenmeyer, “Coordinated multi-agent q-learning for resilient SATCOM against smart jammers,” in IEEE Military Communications Conference (MILCOM) (Restricted Access), pp. 1–6, 2022.
[27] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. A. Riedmiller, “Playing atari with deep reinforcement learning,” CoRR, vol. abs/1312.5602, 2013.
[28] L. Xiao, D. Jiang, D. Xu, H. Zhu, Y. Zhang, and H. V. Poor, “Two-dimensional antijamming mobile communication based on reinforcement learning,” IEEE Transactions on Vehicular Technology, vol. 67, pp. 9499–9512, Oct. 2018.
[29] X. Liu, Y. Xu, L. Jia, Q. Wu, and A. Anpalagan, “Antijamming communications using spectrum waterfall: A deep reinforcement learning approach,” IEEE Communications Letters, vol. 22, pp. 998–1001, May 2018.
[30] N. Gao, Z. Qin, X. Jing, Q. Ni, and S. Jin, “Antiintelligent UAV jamming strategy via deep Q-networks,” IEEE Transactions on Communications, vol. 68, pp. 569–581, Jan. 2020.
[31] S. Thrun and A. Schwartz, “Issues in using function approximation for reinforcement learning,” 10 1993.
[32] H. v. Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double Q-Learning,” in Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp. 2094––2100, AAAI Press, 2016.
[33] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015. Software available from tensorflow.org.
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2025 Lan Nguyen, Duy Nguyen, Nghi Tran, David Brunnenmeyer
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
