Comparative analysis of performance of AutoML algorithms: Classification model of payment arrears in students of a private university
DOI:
https://doi.org/10.4108/eetsis.4550Keywords:
payment arrears, AutoML, AutoKeras, AutoGluon, HyperOPT, MLJar and H2OAbstract
The impact of artificial intelligence in our society is important due to the innovation of processes through data science to know the academic and sociodemographic factors that contribute to late payments in university students, to identify them and make timely decisions for implementing prevention and correction programs, avoiding student dropout due to this economic problem, and ensuring success in their education in a meaningful and focused way. In this sense, the research aims to compare the performance metrics of classification models for late payments in students of a private university by using AutoML algorithms from various existing platforms and solutions such as AutoKeras, AutoGluon, HyperOPT, MLJar, and H2O in a data set consisting of 8,495 records and the application of data balancing techniques. From the implementation and execution of various algorithms, similar metrics have been obtained based on the parameters and optimization functions used automatically by each tool, providing better performance to the H2O platform through the Stacked Ensemble algorithm with metrics accuracy = 0.778. F1 = 0.870, recall = 0.904 and precision = 0.839. The research can be extended to other contexts or areas of knowledge due to the growing interest in automated machine learning, providing researchers with a valuable tool in data science without the need for deep knowledge.
References
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1). https://doi.org/10.1186/s41239-019-0171-0
Ramani, P. (2022). Artificial Intelligence in Higher Education and Changing roles of Educators. World Journal of Educational Research. https://doi.org/10.22158/wjer.v9n2p56
Salas-Pilco, S. Z., & Yang, Y. (2022). Artificial intelligence applications in Latin American higher education: a systematic review. International Journal of Educational Technology in Higher Education, 19(1). https://doi.org/10.1186/s41239-022-00326-w
Al, M. (2023). Higher Education and the Challenges of Artificial Intelligence. Russian Law Journal; LLC V.Em Publishing. https://doi.org/10.52783/rlj.v11i6s.1489
Alqahtani, T., Badreldin, H. A., Alrashed, M., Alshaya, A. I., Alghamdi, S. S., Bin Saleh, K., Alowais, S. A., Alshaya, O. A., Rahman, I., Al Yami, M. S., & Albekairy, A. M. (2023). The emergent role of artificial intelligence, natural learning processing, and large language models in higher education and research. Research in social & administrative pharmacy: RSAP, 19(8), 1236–1242. https://doi.org/10.1016/j.sapharm.2023.05.016
Okagbue, E. F., Ezeachikulo, U. P., Akintunde, T. Y., Tsakuwa, M. B., Ilokanulo, S. N., Obiasoanya, K. M., ... & Ouattara, C. A. T. (2023). A comprehensive overview of artificial intelligence and machine learning in education pedagogy: 21 Years (2000–2021) of research indexed in the scopus database. Social Sciences & Humanities Open, 8(1), 100655. https://doi.org/10.1016/j.ssaho.2023.100655
Quadri, A. T., & Shukor, N. A. (2021). The Benefits of Learning Analytics to Higher Education Institutions: A Scoping Review. International Journal of Emerging Technologies in Learning (Ijet); kassel university press. https://doi.org/10.3991/ijet.v16i23.27471
Aldowah, H., Al-Samarraie, H., & Fauzy, W. M. (2019). Educational data mining and learning analytics for 21st century higher education: A review and synthesis. Telematics and Informatics, 37, 13-49. https://doi.org/10.1016/j.tele.2019.01.007
Al Ka’bi, A. (2023). Proposed artificial intelligence algorithm and deep learning techniques for development of higher education. Int J Intell Netw. https://doi.org/10.1016/j.ijin.2023.03.002
Sollosy, M., & McInerney, M. (2022). Artificial intelligence and business education: What should be taught. The International Journal of Management Education, 20(3), 100720. https://doi.org/10.1016/j.ijme.2022.100720
Wang, C., Chen, Z., & Zhou, M. (2023, April). AutoML from Software Engineering Perspective: Landscapes and Challenges. In Proceedings of the 20th International Conference on Mining Software Repositories. MSR. https://chenzhenpeng18.github.io/papers/MSR23.pdf
Zhang, D. (2022). Analysis of University Management Model of National Higher Education Institutions Based on Machine Learning Algorithm. Mobile Information Systems, 2022, 1–7. https://doi.org/10.1155/2022/4553185
Iatrellis, O., Savvas, I., Fitsilis, P., & Gerogiannis, V. C. (2020). A two-phase machine learning approach for predicting student outcomes. Education and Information Technologies; Springer Science+Business Media. https://doi.org/10.1007/s10639-020-10260-x
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415
Fahd, K., Venkatraman, S., Miah, S. J., & Ahmed, K. (2022). Application of machine learning in higher education to assess student academic performance, at-risk, and attrition: A meta-analysis of literature. Education and Information Technologies; Springer Science+Business Media. https://doi.org/10.1007/s10639-021-10741-7
Janiesch, C., Zschech, P., & Heinrich, K. (2021). Machine learning and deep learning. Electronic Markets; Springer Science+Business Media. https://doi.org/10.1007/s12525-021-00475-2
Oqaidi, K., Aouhassi, S., & Mansouri, K. (2022). Towards a Students’ Dropout Prediction Model in Higher Education Institutions Using Machine Learning Algorithms. International Journal of Emerging Technologies in Learning (Ijet); kassel university press. https://doi.org/10.3991/ijet.v17i18.25567
Oladipupo, T. (2010). Types of Machine Learning Algorithms. New Advances in Machine Learning. https://doi.org/10.5772/9385
Manduchi, E., Romano, J. D., & Moore, J. H. (2021). The promise of automated machine learning for the genetic analysis of complex traits. Human Genetics, 141(9), 1529–1544. https://doi.org/10.1007/s00439-021-02393-x
Waring, J., Lindvall, C., & Umeton, R. (2020). Automated machine learning: Review of the state-of-the-art and opportunities for healthcare. Artificial Intelligence in Medicine, 104, 101822. https://doi.org/10.1016/j.artmed.2020.101822
Wever, M., Tornede, A., Mohr, F., & Hullermeier, E. (2021). AutoML for Multi-Label Classification: Overview and Empirical Evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9), 3037–3054. https://doi.org/10.1109/tpami.2021.3051276
Zender, A., & Humm, B. G. (2022). Ontology-based Meta AutoML. Integrated Computer-Aided Engineering, 29(4), 351–366. https://doi.org/10.3233/ica-220684
Bahri, M., Salutari, F., Putina, A., & Sozio, M. (2022). AutoML: state of the art with a focus on anomaly detection, challenges, and research directions. International Journal of Data Science and Analytics, 14(2), 113–126. https://doi.org/10.1007/s41060-022-00309-0
Musigmann, M., Akkurt, B. H., Krähling, H., Nacul, N. G., Remonda, L., Sartoretti, T., Henssen, D., Brokinkel, B., Stummer, W., Heindel, W., & Mannil, M. (2022). Testing the applicability and performance of Auto ML for potential applications in diagnostic neuroradiology. Scientific reports, 12(1), 13648. https://doi.org/10.1038/s41598-022-18028-8
Cerrada, M., Trujillo, L., Hernández, D. E., Correa Zevallos, H. A., Macancela, J. C., Cabrera, D., & Vinicio Sánchez, R. (2022). AutoML for Feature Selection and Model Tuning Applied to Fault Severity Diagnosis in Spur Gearboxes. Mathematical and Computational Applications, 27(1), 6. https://doi.org/10.3390/mca27010006
Choi, W., Choi, T., & Heo, S. (2023). A Comparative Study of Automated Machine Learning Platforms for Exercise Anthropometry-Based Typology Analysis: Performance Evaluation of AWS SageMaker, GCP VertexAI, and MS Azure. Bioengineering, 10(8), 891. https://doi.org/10.3390/bioengineering10080891
Frank, F. & Bacao, F. (2023). Advanced Genetic Programming vs. State-of-the-Art AutoML in Imbalanced Binary Classification. Emerging Science Journal, 7(4), 1349–1363. https://doi.org/10.28991/esj-2023-07-04-021
Neverov, E. A., Viksnin, I. I., & Chuprov, S. S. (2023). The Research of AutoML Methods in the Task of Wave Data Classification. 2023 XXVI International Conference on Soft Computing and Measurements (SCM). https://doi.org/10.1109/scm58628.2023.10159058
Mueller, J., Shi, X., & Smola, A. (2020). Faster, Simpler, More Accurate. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. https://doi.org/10.1145/3394486.3406706
Jin, H., Chollet, F., Song, Q., & Hu, X. (2023). Autokeras: An automl library for deep learning. Journal of Machine Learning Research, 24(6), 1-6. https://www.jmlr.org/papers/volume24/20-1355/20-1355.pdf
Alaiad, A., Migdady, A., Al-Khatib, R. M., AlZoubi, O., Zitar, R. A., & Abualigah, L. (2023). Autokeras Approach: A Robust Automated Deep Learning Network for Diagnosis Disease Cases in Medical Images. Journal of Imaging; Multidisciplinary Digital Publishing Institute. https://doi.org/10.3390/jimaging9030064
Paldino, G. M., De Stefani, J., De Caro, F., & Bontempi, G. (2021, July 5). Does AutoML Outperform Naive Forecasting? The 7th International Conference on Time Series and Forecasting. https://doi.org/10.3390/engproc2021005036
Shchur, O., Turkmen, C., Erickson, N., Shen, H., Shirkov, A., Hu, T., & Wang, B. (2023). AutoGluon-TimeSeries: AutoML for Probabilistic Time Series Forecasting. ArXiv, abs/2308.05566. https://doi.org/10.48550/arXiv.2308.05566
Erickson, N., Mueller, J.W., Shirkov, A., Zhang, H., Larroy, P., Li, M., & Smola, A. (2020). AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data. ArXiv, abs/2003.06505. https://doi.org/10.48550/arXiv.2003.06505
Komer, B., Bergstra, J., Eliasmith, C. (2019). Hyperopt-Sklearn. In: Hutter, F., Kotthoff, L., Vanschoren, J. (eds) Automated Machine Learning. The Springer Series on Challenges in Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-05318-5_5
Dumitrache, A., Melian, D. M., Bălăcian, D., Nastu, A., & Stancu, S. (2020). Churn prepaid customers classified by HyperOpt techniques. Proceedings of the International Conference on Applied Statistics. https://doi.org/10.2478/icas-2021-0010
Bergstra, J., Yamins, D., & Cox, D. D. (2013). Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms. THe 12th Python in Science Conference.
Bergstra, J., Komer, B., Eliasmith, C., Yamins, D., & Cox, D. D. (2015). Hyperopt: a Python library for model selection and hyperparameter optimization. Computational Science & Discovery, 8(1), 014008. https://doi.org/10.1088/1749-4699/8/1/014008
Ma, J., Xu, H., Wang, A., Wang, A., Gao, L., & Ding, M. (2023). Machine learning-guided underlying decisive factors of high-performance membrane distillation system: Membrane properties, operation conditions and solution composition. Separation and Purification Technology, 327, 124964. https://doi.org/10.1016/j.seppur.2023.124964
Płońska, A., & Płoński, P. (2021). MLJAR: State-of-the-art Automated Machine Learning Framework for Tabular Data. Version 0.10.3. [Computer software]. MLJAR, https://github.com/mljar/mljar-supervised
Vázquez, F. (2023, October 6). Entrenando Tu Propio LLM Sin Programación. H2O.ai. Retrieved October 23, 2023, from https://h2o.ai/blog/entrenando-tu-propio-llm-sin-programacion/
LeDell, E., & Poirier, S. (2020). H2o automl: Scalable automatic machine learning. In Proceedings of the AutoML Workshop at ICML (Vol. 2020). ICML.
Kochura, Y., Stirenko, S., & Gordienko, Y. (2017). Comparative performance analysis of neural networks architectures on H2O platform for various activation functions. 2017 IEEE International Young Scientists Forum on Applied Physics and Engineering (YSF). doi:10.1109/ysf.2017.8126654
Saucedo, M. L., Sánchez, R. L., Becerra, E. E., & Puican, V. H. (2023). New E-government Strategies in Peruvian Universities. https://doi.org/10.55908/sdgs.v11i2.703
Salas‐Pilco, S. Z., Yang, Y., & Zhang, Z. (2022). Student engagement in online learning in Latin American higher education during the COVID‐19 pandemic: A systematic review. British Journal of Educational Technology, 53(3), 593–619. https://doi.org/10.1111/bjet.13190
Bates, T., Cobo, C., Mariño, O., & Wheeler, S. (2020). Can artificial intelligence transform higher education? International Journal of Educational Technology in Higher Education, 17(1). https://doi.org/10.1186/s41239-020-00218-x
Nuankaew, P., Nasa-Ngium, P., Kunasit, T., & Nuankaew, W. (2023). Implementation of Data Analytics and Machine Learning in Thailand Education Sector. International Journal of Emerging Technologies in Learning (Ijet); kassel university press. https://doi.org/10.3991/ijet.v18i05.36871
Callender, C., & Dougherty, K. J. (2018, October 9). Student Choice in Higher Education—Reducing or Reproducing Social Inequalities? Social Sciences, 7(10), 189. https://doi.org/10.3390/socsci7100189
Wadesango, N., Maphosa, C., & Moyo, G. (2014). An Academic Development Agenda for Postgraduate Research Students. Mediterranean Journal of Social Sciences. https://doi.org/10.5901/mjss.2014.v5n11p49
Ilie, S., Rose, P., & Vignoles, A. (2021). Understanding higher education access: Inequalities and early learning in low and lower‐middle‐income countries. British Educational Research Journal, 47(5), 1237–1258. https://doi.org/10.1002/berj.3723
Villarreal-Torres, H., Ángeles-Morales, J., Marín-Rodriguez, W., Andrade-Girón, D., Cano-Mejía, J., Mejía-Murillo, C., Flores-Reyes, G., & Palomino-Márquez, M. (2023a). Classification model for student dropouts using machine learning: A case study. EAI Endorsed Transactions on Scalable Information Systems, 10(5). https://doi.org/10.4108/eetsis.vi.3455
Villarreal-Torres, H., Ángeles-Morales, J., Marín-Rodriguez, W., Andrade-Girón, D., Carreño-Cisneros, E., Cano-Mejía, J., Mejía-Murillo, C., Boscán-Carroz, M. C., Flores-Reyes, G., & Cruz-Cruz, O. (2023b). Development of a Classification Model for Predicting Student Payment Behavior Using Artificial Intelligence and Data Science Techniques. EAI Endorsed Transactions on Scalable Information Systems, 10(5). https://doi.org/10.4108/eetsis.3489
El Peruano. (2022, 19 de agosto). Retrocedió el índice de morosidad. https://www.elperuano.pe/noticia/183969-retrocedio-el-indice-de-morosidad
Ferreira, L., Pilastri, A., Romano, F., & Cortez, P. (2022). Using supervised and one-class automated machine learning for predictive maintenance. Applied Soft Computing, 131, 109820. https://doi.org/10.1016/j.asoc.2022.109820
Gijsbers, P., Bueno, M. L., Coors, S., LeDell, E., Poirier, S., Thomas, J., ... & Vanschoren, J. (2022). AMLB: an automl benchmark. https://doi.org/10.48550/arxiv.2207.12560
Abaimov, S., & Martellini, M. (2022). Understanding Machine Learning. Advanced Sciences and Technologies for Security Applications. https://doi.org/10.1007/978-3-030-91585-8_2
Lázaro, L. M. (2022). La UNESCO y los futuros de la educación superior hasta 2050. Por una ampliación del derecho a la educación que incluya a la educación superior. Revista Española De Educación Comparada, (41), 271–280. https://doi.org/10.5944/reec.41.2022.33879
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Henry Villarreal-Torres, Julio Ángeles-Morales , Jenny Cano-Mejía, Carmen Mejía-Murillo, Gumercindo Flores-Reyes, Oscar Cruz-Cruz, Manuel Urcia-Quispe, Manuel Palomino-Márquez, Miguel Solar-Jara, Reyna Escobedo-Zarzosa
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open access article distributed under the terms of the CC BY-NC-SA 4.0, which permits copying, redistributing, remixing, transformation, and building upon the material in any medium so long as the original work is properly cited.
