MFRLMO: Model-free reinforcement learning for multi-objective optimization of apache spark
DOI:
https://doi.org/10.4108/eetsis.4764Abstract
Hyperparameter optimization (HO) is a must to figure out to what extent can a specific configuration of hyperparameters contribute to the performance of a machine learning task. The hardware and MLlib library of Apache Spark have the potential to improve big data processing performance when a tuning operation is combined with the exploitation of hyperparameters. To the best of our knowledge, the most of existing studies employ a black-box approach that results in misleading results due to ignoring the interior dynamics of big data processing. They suffer from one or more drawbacks including high computational cost, large search space, and sensitivity to the dimension of multi-objective functions. To address the issues above, this work proposes a new model-free reinforcement learning for multi-objective optimization of Apache Spark, thereby leveraging reinforcement learning (RL) agents to uncover the internal dynamics of Apache Spark in HO. To bridge the gap between multi-objective optimization and interior constraints of Apache Spark, our method runs a lot of iterations to update each cell of the RL grid. The proposed model-free learning mechanism achieves a tradeoff between three objective functions comprising time, memory, and accuracy. To this end, optimal values of the hyperparameters are obtained via an ensemble technique that analyzes the individual results yielded by each objective function. The results of the experiments show that the number of cores has not a direct effect on $speedup$. Further, although grid size has an impact on the time passed between two adjoining iterations, it is negligible in the computational burden. Dispersion and risk values of model-free RL differ when the size of the data is small. On average, MFRLMO produced $speedup$ that is 37% better than those of the competitors. Last, our approach is very competitive in terms of converging to a high accuracy when optimizing Convolutional Neural networks (CNN).
References
Meng, X., Bradley, J., Yavuz, B., Sparks, E., Venkataraman, S., Liu, D., Freeman, J. et al. (2016) Mllib: Machine learning in apache spark. The Journal of Machine Learning Research 17(1): 1235–1241.
Xie, F. (2023) Monitoring and quality evaluation method of english teaching in machine manufacturing based on machine learning and internet of things. EAI Endorsed Transactions on Scalable Information Systems 10(6).
Morfino, V. and Rampone, S. (2020) Towards near-realtime intrusion detection for iot devices using supervised learning and apache spark. Electronics 9(3): 444.
Cheng, Y., Yu, N., Foggo, B. and Yamashita, K. (2022) Online power system event detection via bidirectional generative adversarial networks. IEEE Transactions on Power Systems .
de Souza Neto, J.B., Martins Moreira, A., Vargas-Solar, G. and Musicante, M.A. (2022) Transmut-spark: Transformation mutation for apache spark. Software Testing, Verification and Reliability : e1809.
Wang, G., Xu, J. and He, B. (2016) A novel method for tuning configuration parameters of spark based on machine learning. In 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC/SmartCity/DSS) (IEEE): 586–593.
Baldacci, L. and Golfarelli, M. (2018) A cost model for spark sql. IEEE Transactions on Knowledge and Data Engineering 31(5): 819–832.
Zhu, Y., Liu, J., Guo, M., Bao, Y., Ma, W., Liu, Z., Song, K. et al. (2017) Bestconfig: tapping the performance potential of systems via automatic configuration tuning. In Proceedings of the 2017 Symposium on Cloud Computing: 338–350.
Ge, Y.F., Wang, H., Bertino, E., Zhan, Z.H., Cao, J., Zhang, Y. and Zhang, J. (2023) Evolutionary dynamic database partitioning optimization for privacy and utility. IEEE Transactions on Dependable and Secure Computing .
Cheng, G., Ying, S. andWang, B. (2021) Tuning configuration of apache spark on public clouds by combining multi-objective optimization and performance prediction model. Journal of Systems and Software 180: 111028.
Turner, R., Eriksson, D., McCourt, M., Kiili, J., Laaksonen, E., Xu, Z. and Guyon, I. (2021) Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the blackbox optimization challenge 2020. In NeurIPS 2020 Competition and Demonstration Track (PMLR): 3–26.
Shekar, B. and Dagnew, G. (2019) Grid search-based hyperparameter tuning and classification of microarray cancer data. In 2019 second international conference on advanced computational and communication paradigms (ICACCP) (IEEE): 1–8.
Lindauer, M., Eggensperger, K., Feurer, M., Biedenkapp, A., Deng, D., Benjamins, C., Ruhkopf, T. et al. (2022) Smac3: A versatile bayesian optimization package for hyperparameter optimization. J. Mach. Learn. Res. 23: 54–1.
Binder, M., Moosbauer, J., Thomas, J. and Bischl, B. (2020) Multi-objective hyperparameter tuning and feature selection using filter ensembles. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference: 471–479.
Meister, M., Sheikholeslami, S., Payberah, A.H., Vlassov, V. and Dowling, J. (2020) Maggy: Scalable asynchronous parallel hyperparameter search. In Proceedings of the 1stWorkshop on Distributed Machine Learning: 28–33.
Trotter, M., Wood, T. and Hwang, J. (2019) Forecasting a storm: Divining optimal configurations using genetic algorithms and supervised learning. In 2019 IEEE international conference on autonomic computing (ICAC) (IEEE): 136–146.
Liu, J., Ravi, N., Chakradhar, S. and Kandemir, M. (2012) Panacea: Towards holistic optimization of mapreduce applications. In Proceedings of the Tenth International Symposium on Code Generation and Optimization: 33–43.
Lin, J.C., Lee, M.C., Yu, I.C. and Johnsen, E.B. (2018) Modeling and simulation of spark streaming. In 2018 IEEE 32nd International Conference on Advanced Information Networking and Applications (AINA) (IEEE): 407–413.
Petridis, P., Gounaris, A. and Torres, J. (2016) Spark parameter tuning via trial-and-error. In INNS Conference on Big Data (Springer): 226–237.
Li, J.Y., Du, K.J., Zhan, Z.H., Wang, H. and Zhang, J. (2022) Distributed differential evolution with adaptive resource allocation. IEEE transactions on cybernetics .
Fu, T.Z., Ding, J., Ma, R.T., Winslett, M., Yang, Y. and Zhang, Z. (2015) Drs: dynamic resource scheduling for real-time analytics over fast streams. In 2015 IEEE 35th International Conference on Distributed Computing Systems (IEEE): 411–420.
Petrov, M., Butakov, N., Nasonov, D. and Melnik, M. (2018) Adaptive performance model for dynamic scaling apache spark streaming. Procedia Computer Science 136: 109–117.
Venkataraman, S., Panda, A., Ousterhout, K., Armbrust, M., Ghodsi, A., Franklin, M.J., Recht, B. et al. (2017) Drizzle: Fast and adaptable stream processing at scale. In Proceedings of the 26th Symposium on Operating Systems Principles: 374–389.
Zacheilas, N., Kalogeraki, V., Zygouras, N., Panagiotou, N. and Gunopulos, D. (2015) Elastic complex event processing exploiting prediction. In 2015 IEEE International Conference on Big Data (Big Data) (IEEE): 213–222.
Vaquero, L.M. and Cuadrado, F. (2018) Auto-tuning distributed stream processing systems using reinforcement learning. arXiv preprint arXiv:1809.05495 .
Bhatia, A., Svegliato, J., Nashed, S.B. and Zilberstein, S. (2022) Tuning the hyperparameters of anytime planning: A metareasoning approach with deep reinforcement learning. In Proceedings of the International Conference on Automated Planning and Scheduling, 32: 556–564.
Liu, X., Wu, J. and Chen, S. (2022) A contextbased meta-reinforcement learning approach to efficient hyperparameter optimization. Neurocomputing 478: 89–103.
Yu, W., You, J., Niu, X., He, J. and Zhang, Y. (2023) Rboira: Integrating rules and reinforcement learning to improve index recommendation. EAI Endorsed Transactions on Scalable Information Systems 10(6).
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P. and Levine, S. (2021) How to train your robot with deep reinforcement learning: lessons we have learned. The International Journal of Robotics Research 40(4-5): 698– 721.
Fan, T., Long, P., Liu, W. and Pan, J. (2020) Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios. The International Journal of Robotics Research 39(7): 856–892.
Hu, J., Niu, H., Carrasco, J., Lennox, B. and Arvin, F. (2020) Voronoi-based multi-robot autonomous exploration in unknown environments via deep reinforcement learning. IEEE Transactions on Vehicular Technology 69(12): 14413–14423.
Oliff, H., Liu, Y., Kumar, M., Williams, M. and Ryan, M. (2020) Reinforcement learning for facilitating human-robot-interaction in manufacturing. Journal of Manufacturing Systems 56: 326–340.
Wang, D., Deng, H. and Pan, Z. (2020) Mrcdrl: Multirobot coordination with deep reinforcement learning. Neurocomputing 406: 68-76.
Wu, Y.H., Yu, Z.C., Li, C.Y., He, M.J., Hua, B. and Chen, Z.M. (2020) Reinforcement learning in dualarm trajectory planning for a free-floating space robot. Aerospace Science and Technology 98: 105657.
Hu, H., Zhang, K., Tan, A.H., Ruan, M., Agia, C. and Nejat, G. (2021) A sim-to-real pipeline for deep reinforcement learning for autonomous robot navigation in cluttered rough terrain. IEEE Robotics and Automation Letters 6(4): 6569–6576.
Rajeswaran, A., Mordatch, I. and Kumar, V. (2020) A game theoretic framework for model based reinforcement learning. In International conference on machine learning (PMLR): 7953–7963.
Albaba, B.M. and Yildiz, Y. (2019) Modeling cyberphysical human systems via an interplay between reinforcement learning and game theory. Annual Reviews in Control 48: 1–21.
Xu, Q., Su, Z. and Lu, R. (2020) Game theory and reinforcement learning based secure edge caching in mobile social networks. IEEE Transactions on Information Forensics and Security 15: 3415–3429.
Albaba, B.M. and Yildiz, Y. (2021) Driver modeling through deep reinforcement learning and behavioral game theory. IEEE Transactions on Control Systems Technology 30(2): 885–892.
Ahad, A., Tahir, M., Sheikh, M.A., Ahmed, K.I. and Mughees, A. (2021) An intelligent clustering-based routing protocol (crp-gr) for 5g-based smart healthcare using game theory and reinforcement learning. Applied Sciences 11(21): 9993.
Bui, V.H., Hussain, A. and Su, W. (2022) A dynamic internal trading price strategy for networked microgrids: A deep reinforcement learning based game-theoretic approach. IEEE Transactions on Smart Grid .
Wu, J., Chen, S. and Liu, X. (2020) Efficient hyperparameter optimization through model-based reinforcement learning. Neurocomputing 409: 381–393.
Jomaa, H.S., Grabocka, J. and Schmidt-Thieme, L. (2019) Hyp-rl: Hyperparameter optimization by reinforcement learning. arXiv preprint arXiv:1906.11527 .
Chen, S., Wu, J. and Liu, X. (2021) Emorl: Effective multi-objective reinforcement learning method for hyperparameter optimization. Engineering Applications of Artificial Intelligence 104: 104315.
García, S., Luengo, J. and Herrera, F. (2015) Data preprocessing in data mining, 72 (Springer).
Aragón-Royón, F., Jiménez-Vílchez, A., Arauzo-Azofra, A. and Benítez, J.M. (2020) Fsinr: an exhaustive package for feature selection. arXiv preprint arXiv:2002.10330 .
Nguyen, N., Khan, M.M.H. and Wang, K. (2018) Towards automatic tuning of apache spark configuration. In 2018 IEEE 11th International Conference on Cloud Computing (CLOUD) (IEEE): 417–425.
Arnold, T.B. (2017) kerasr: R interface to the keras deep learning library. J. Open Source Softw. 2(14): 296.
Ozturk, M.M. Tuning parameters of apache spark with gauss pareto based multi-objective optimization.
Chan, S.C., Fishman, S., Canny, J., Korattikara, A. and Guadarrama, S. (2019) Measuring the reliability of reinforcement learning algorithms. arXiv preprint arXiv:1912.05663 .
Altunkaynak, B. and Gamgam, H. (2019) Bootstrap confidence intervals for the coefficient of quartile variation. Communications in Statistics-Simulation and Computation 48(7): 2138–2146.
Jung, K., Lee, J., Gupta, V. and Cho, G. (2019) Comparison of bootstrap confidence interval methods for gsca using a monte carlo simulation. Frontiers in psychology 10: 2215.
Chapman, M.P., Bonalli, R., Smith, K.M., Yang, I., Pavone, M. and Tomlin, C.J. (2021) Risk-sensitive safety analysis using conditional value-at-risk. IEEE Transactions on Automatic Control .
Mhammedi, Z., Guedj, B. and Williamson, R.C. (2020) Pac-bayesian bound for the conditional value at risk. Advances in Neural Information Processing Systems 33: 17919–17930.
Soma, T. and Yoshida, Y. (2020) Statistical learning with conditional value at risk. arXiv preprint arXiv:2002.05826 .
Guo, Y., Shan, H., Huang, S., Hwang, K., Fan, J. and Yu, Z. (2021) Gml: Efficiently auto-tuning flink’s configurations via guided machine learning. IEEE Transactions on Parallel and Distributed Systems 32(12): 2921–2935.
Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O. (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 .
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Muhammed Maruf Öztürk
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open access article distributed under the terms of the CC BY-NC-SA 4.0, which permits copying, redistributing, remixing, transformation, and building upon the material in any medium so long as the original work is properly cited.
