Massively Parallel Evasion Attacks and the Pitfalls of Adversarial Retraining
DOI:
https://doi.org/10.4108/eetiot.6652Keywords:
Machine Learning, Support Vector Machines, Trustworthy AI, Anomaly Detection, AI for CybersecurityAbstract
Even with widespread adoption of automated anomaly detection in safety-critical areas, both classical and advanced machine learning models are susceptible to first-order evasion attacks that fool models at run-time (e.g. an automated firewall or an anti-virus application). Kernelized support vector machines (KSVMs) are an especially useful model because they combine a complex geometry with low run-time requirements (e.g. when compared to neural networks), acting as a run-time lower bound when compared to contemporary models (e.g. deep neural networks), to provide a cost-efficient way to measure model and attack run-time costs. To properly measure and combat adversaries, we propose a massively parallel projected gradient descent (PGD) evasion attack framework. Through theoretical examinations and experiments carried out using linearly-separable Gaussian normal data, we present (i) a massively parallel naive attack, we show that adversarial retraining is unlikely to be an effective means to combat an attacker even on linearly separable datasets, (ii) a cost effective way of evaluating models defences and attacks, and an extensible code base for doing so, (iii) an inverse relationship between adversarial robustness and benign accuracy, (iv) the lack of a general relationship between attack time and efficacy, and (v) that adversarial retraining increases compute time exponentially while failing to reliably prevent highly-confident false classifications.
Downloads
References
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. and Fergus, R. (2013) Intriguing properties of neural networks. International Conference on Learning Representations.
Madry, A., Makelov, A., Schmidt, L., Tsipras, D. and Vladu, A. (2017) Towards deep learning models resistant to adversarial attacks. International Conference on Machine Learning.
Croce, F. and Hein, M. (2020) Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. International Conference on Machine Learning.
Deka, P.K., Bhuyan, M.H., Kadobayashi, Y. and Elmroth, E. (2019) Adversarial impact on anomaly detection in cloud datacenters. In 2019 IEEE 24th Pacific Rim International Symposium on Dependable Computing (PRDC) (IEEE): 188–18809. DOI: https://doi.org/10.1109/PRDC47002.2019.00049
Kim, D.S. and Park, J.S. (2003) Network-based intrusion detection with support vector machines. In International Conference on Information Networking (Springer): 747–756. DOI: https://doi.org/10.1007/978-3-540-45235-5_73
Mehmood, T. and Rais, H.B.M. (2015) Svm for network anomaly detection using aco feature subset. In 2015 International symposium on mathematical sciences and computing research (iSMSC) (IEEE): 121–126. DOI: https://doi.org/10.1109/ISMSC.2015.7594039
Tzotsos, A. and Argialas, D. (2008) Support vector machine classification for object-based image analysis. In Object-Based Image Analysis (Springer), 663–677. DOI: https://doi.org/10.1007/978-3-540-77058-9_36
Uesato, J., O’Donoghue, B., Oord, A.v.d. and Kohli, P. (2018) Adversarial risk and the dangers of evaluating against weak attacks. Proceedings of Machine Learning Research.
Carlini, N., Athalye, A., Papernot, N., Brendel, W., Rauber, J., Tsipras, D., Goodfellow, I. et al. (2019) On evaluating adversarial robustness. arXiv:1902.06705.
Li, B., Vorobeychik, Y. and Chen, X. (2016) A general retraining framework for scalable adversarial classification. Workshop on Adversarial Training, Neural Information Processing Systems.
Stutz, D., Hein, M. and Schiele, B. (2019) Confidencecalibrated adversarial training: Towards robust models generalizing beyond the attack used during training. International Conference on Machine Learning.
Demontis, A., Melis, M., Pintor, M., Jagielski, M., Biggio, B., Oprea, A., Nita-Rotaru, C. et al. (2019) Why do adversarial attacks transfer? explaining transferability of evasion and poisoning attacks. In 28th {USENIX} Security Symposium: 321–338.
Desislavov, R., Martínez-Plumed, F. and Hernández- Orallo, J. (2021) Compute and energy consumption trends in deep learning inference. arXiv:2109.05472.
Tsipras, D., Santurkar, S., Engstrom, L., Turner, A. and Madry, A. (2018) Robustness may be at odds with accuracy. Int’l Conference on Learning Representations.
Raghunathan, A., Xie, S.M., Yang, F., Duchi, J. and Liang, P. (2020) Understanding and mitigating the tradeoff between robustness and accuracy. International Conference on Machine Learning.
Dohmatob, E. (2019) Generalized no free lunch theorem for adversarial robustness. In International Conference on Machine Learning (PMLR): 1646–1654.
Carlini, N. and Wagner, D. (2017) Towards evaluating the robustness of neural networks. In IEEE symposium on security and privacy (sp) (IEEE): 39–57. DOI: https://doi.org/10.1109/SP.2017.49
Athalye, A., Carlini, N. and Wagner, D. (2018) Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. Int. Conference on Machine Learning .
Cortes, C. and Vapnik, V. (1995) Support-vector networks. Machine learning 20(3): 273–297. DOI: https://doi.org/10.1007/BF00994018
Trafalis, T.B. and Gilbert, R.C. (2007) Robust support vector machines for classification and computational issues. Optimisation Methods and Software 22(1): 187–198. DOI: https://doi.org/10.1080/10556780600883791
Bordes, A., Ertekin, S.,Weston, J. and Bottou, L. (2005) Fast kernel classifiers with online and active learning. Journal of Machine Learning Research 6(Sep): 1579–1619.
Christmann, A. and Steinwart, I. (2004) On robustness properties of convex risk minimization methods for pattern recognition. The Journal of Machine Learning Research 5: 1007–1034.
Bienstock, D., Muñoz, G. and Pokutta, S. (2018) Principled deep neural network training through linear programming. arXiv:1810.03218 .
Falco, J.A., Hurd, S. and Teumim, D. (2006) Using hostbased anti-virus software on industrial control systems: Integration guidance and a test methodology for assessing performance impacts (NIST). DOI: https://doi.org/10.6028/NIST.SP.1058
Organization, I.S. (2018), ISO 26262- 1:2011, road vehicles — functional safety, https://www.iso.org/standard/43464.html (visited 2022-04-20).
Su, J., Vargas, D.V. and Sakurai, K. (2019) One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation 23(5): 828–841. DOI: https://doi.org/10.1109/TEVC.2019.2890858
Chen, J., Jordan, M.I. and Wainwright, M.J. (2020) Hopskipjumpattack: A query-efficient decision-based attack. In 2020 ieee symposium on security and privacy (sp) (IEEE): 1277–1294. DOI: https://doi.org/10.1109/SP40000.2020.00045
Brown, T.B., Mané, D., Roy, A., Abadi, M. and Gilmer, J. (2017) Adversarial patch. arXiv:1712.09665 .
Brendel,W., Rauber, J. and Bethge, M. (2017) Decisionbased adversarial attacks: Reliable attacks against blackbox machine learning models. arXiv:1712.04248 .
Liu, X., Yang, H., Liu, Z., Song, L., Li, H. and Chen, Y. (2018) Dpatch: An adversarial patch attack on object detectors. arXiv:1806.02299.
Qin, Y., Carlini, N., Cottrell, G., Goodfellow, I. and Raffel, C. (2019) Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. In International conference on machine learning (PMLR): 5231–5240.
Grosse, K., Pfaff, D., Smith, M.T. and Backes, M. (2018) The limitations of model uncertainty in adversarial settings. arXiv:1812.02606.
Kotyan, S. and Vargas, D.V. (2019) Adversarial robustness assessment: Why both l0 and l∞ attacks are necessary. arXiv:1906.06026 .
Chen, P.Y., Zhang, H., Sharma, Y., Yi, J. and Hsieh, C.J. (2017) Zoo: Zeroth order optimization based blackbox attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security: 15–26. DOI: https://doi.org/10.1145/3128572.3140448
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A. and Talwalkar, A. (2017) Hyperband: A novel bandit-based approach to hyperparameter optimization. The Journal of Machine Learning Research 18(1): 6765–6816.
Hansen, N. (2016) The cma evolution strategy: A tutorial. arXiv:1604.00772 .
Ozaki, Y., Tanigaki, Y., Watanabe, S., Nomura, M. and Onishi, M. (2022) Multiobjective tree-structured parzen estimator. Journal of Artificial Intelligence Research 73: 1209–1250. DOI: https://doi.org/10.1613/jair.1.13188
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M. et al. (2011) Scikitlearn: Machine learning in Python. Journal of Machine Learning Research 12: 2825–2830.
Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G. et al. (2013) Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery in databases (Springer): 387–402. DOI: https://doi.org/10.1007/978-3-642-40994-3_25
Liu, T.,Wang, H., Li, L., Luo, X., Dong, F., Guo, Y.,Wang, L. et al. (2020) MadDroid: Characterizing and detecting devious ad contents for android apps. Proceedings of The Web Conference 2020 . DOI: https://doi.org/10.1145/3366423.3380242
Meidan, Y., Bohadana, M., Mathov, Y., Mirsky, Y., Shabtai, A., Breitenbacher, D. and Elovici, Y. (2018) N-BaIoT—network-based detection of IoT botnet attacks using deep autoencoders. IEEE Pervasive Computing 17(3): 12–22. DOI: https://doi.org/10.1109/MPRV.2018.03367731
Wang, X., Li, J., Kuang, X., Tan, Y.a. and Li, J. (2019) The security of machine learning in an adversarial setting: A survey. Journal of Parallel and Distributed Computing 130: 12–23. DOI: https://doi.org/10.1016/j.jpdc.2019.03.003
Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A. and Mukhopadhyay, D. (2018) Adversarial attacks and defences: A survey. arXiv:1810.00069 .
Biggio, B., Nelson, B. and Laskov, P. (2012) Poisoning attacks against support vector machines. International Conference on Machine Learning .
Tramèr, F., Zhang, F., Juels, A., Reiter, M.K. and Ristenpart, T. (2016) Stealing machine learning models via prediction apis. In 25th {USENIX} Security Symposium Security 16): 601–618.
Fredrikson, M., Jha, S. and Ristenpart, T. (2015) Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security: 1322–1333. DOI: https://doi.org/10.1145/2810103.2813677
Ateniese, G., Mancini, L.V., Spognardi, A., Villani, A., Vitali, D. and Felici, G. (2015) Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. International Journal of Security and Networks 10(3): 137–150. DOI: https://doi.org/10.1504/IJSN.2015.071829
Shokri, R., Stronati, M., Song, C. and Shmatikov, V. (2017) Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP) (IEEE): 3–18. DOI: https://doi.org/10.1109/SP.2017.41
Tavallaee, M., Bagheri, E., Lu, W. and Ghorbani, A.A. (2009) A detailed analysis of the kdd cup 99 data set. In 2009 IEEE symposium on computational intelligence for security and defense applications (Ieee): 1–6. DOI: https://doi.org/10.1109/CISDA.2009.5356528
Dua, D. and Graff, C. (2017), UCI machine learning repository. URL http://archive.ics.uci.edu/ml.
Dadkhah, S., Zhang, X., Weismann, A.G., Firouzi, A. and Ghorbani, A.A. (2023) TruthSeeker: The Largest Social Media Ground-Truth Dataset for Real/Fake Content doi:10.36227/techrxiv.22795130.v1, URL https://www.techrxiv.org/articles/preprint/TruthSeeker_The_Largest_Social_Media_Ground-Truth_Dataset_for_Real_Fake_Content/22795130. DOI: https://doi.org/10.36227/techrxiv.22795130.v1
Akiba, T., Sano, S., Yanase, T., Ohta, T. and Koyama, M. (2019) Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining: 2623–2631. DOI: https://doi.org/10.1145/3292500.3330701
Chang, C.C. and Lin, C.J. (2011) Libsvm: a library for support vector machines. ACM transactions on intelligent systems and technology (TIST) 2(3): 1–27. DOI: https://doi.org/10.1145/1961189.1961199
Meyers, C., Löfstedt, T. and Elmroth, E. (2023) Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems. Artificial Intelligence Review : 1–35. DOI: https://doi.org/10.1007/s10462-023-10521-4
Jakubovitz, D. and Giryes, R. (2018) Improving dnn robustness to adversarial attacks using jacobian regularization. In Proceedings of the European Conference on Computer Vision (ECCV): 514–529. DOI: https://doi.org/10.1007/978-3-030-01258-8_32
Ross, A. and Doshi-Velez, F. (2018) Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients. In Proceedings of the AAAI Conference on Artificial Intelligence, 32. DOI: https://doi.org/10.1609/aaai.v32i1.11504
Colbrook, M.J., Antun, V. and Hansen, A.C. (2021) Can stable and accurate neural networks be computed. On the barriers of deep learning and Smale’s 18th problem. arXiv 2101. DOI: https://doi.org/10.1073/pnas.2107151119
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 EAI Endorsed Transactions on Internet of Things
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
Funding data
-
Knut och Alice Wallenbergs Stiftelse
Grant numbers 2019.0352
