Comparative analysis of performance of AutoML algorithms: Classification model of payment arrears in students of a private university

Authors

DOI:

https://doi.org/10.4108/eetsis.4550

Keywords:

payment arrears, AutoML, AutoKeras, AutoGluon, HyperOPT, MLJar and H2O

Abstract

The impact of artificial intelligence in our society is important due to the innovation of processes through data science to know the academic and sociodemographic factors that contribute to late payments in university students, to identify them and make timely decisions for implementing prevention and correction programs, avoiding student dropout due to this economic problem, and ensuring success in their education in a meaningful and focused way. In this sense, the research aims to compare the performance metrics of classification models for late payments in students of a private university by using AutoML algorithms from various existing platforms and solutions such as AutoKeras, AutoGluon, HyperOPT, MLJar, and H2O in a data set consisting of 8,495 records and the application of data balancing techniques. From the implementation and execution of various algorithms, similar metrics have been obtained based on the parameters and optimization functions used automatically by each tool, providing better performance to the H2O platform through the Stacked Ensemble algorithm with metrics accuracy = 0.778. F1 = 0.870, recall = 0.904 and precision = 0.839. The research can be extended to other contexts or areas of knowledge due to the growing interest in automated machine learning, providing researchers with a valuable tool in data science without the need for deep knowledge.

References

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16(1). https://doi.org/10.1186/s41239-019-0171-0

Ramani, P. (2022). Artificial Intelligence in Higher Education and Changing roles of Educators. World Journal of Educational Research. https://doi.org/10.22158/wjer.v9n2p56

Salas-Pilco, S. Z., & Yang, Y. (2022). Artificial intelligence applications in Latin American higher education: a systematic review. International Journal of Educational Technology in Higher Education, 19(1). https://doi.org/10.1186/s41239-022-00326-w

Al, M. (2023). Higher Education and the Challenges of Artificial Intelligence. Russian Law Journal; LLC V.Em Publishing. https://doi.org/10.52783/rlj.v11i6s.1489

Alqahtani, T., Badreldin, H. A., Alrashed, M., Alshaya, A. I., Alghamdi, S. S., Bin Saleh, K., Alowais, S. A., Alshaya, O. A., Rahman, I., Al Yami, M. S., & Albekairy, A. M. (2023). The emergent role of artificial intelligence, natural learning processing, and large language models in higher education and research. Research in social & administrative pharmacy: RSAP, 19(8), 1236–1242. https://doi.org/10.1016/j.sapharm.2023.05.016

Okagbue, E. F., Ezeachikulo, U. P., Akintunde, T. Y., Tsakuwa, M. B., Ilokanulo, S. N., Obiasoanya, K. M., ... & Ouattara, C. A. T. (2023). A comprehensive overview of artificial intelligence and machine learning in education pedagogy: 21 Years (2000–2021) of research indexed in the scopus database. Social Sciences & Humanities Open, 8(1), 100655. https://doi.org/10.1016/j.ssaho.2023.100655

Quadri, A. T., & Shukor, N. A. (2021). The Benefits of Learning Analytics to Higher Education Institutions: A Scoping Review. International Journal of Emerging Technologies in Learning (Ijet); kassel university press. https://doi.org/10.3991/ijet.v16i23.27471

Aldowah, H., Al-Samarraie, H., & Fauzy, W. M. (2019). Educational data mining and learning analytics for 21st century higher education: A review and synthesis. Telematics and Informatics, 37, 13-49. https://doi.org/10.1016/j.tele.2019.01.007

Al Ka’bi, A. (2023). Proposed artificial intelligence algorithm and deep learning techniques for development of higher education. Int J Intell Netw. https://doi.org/10.1016/j.ijin.2023.03.002

Sollosy, M., & McInerney, M. (2022). Artificial intelligence and business education: What should be taught. The International Journal of Management Education, 20(3), 100720. https://doi.org/10.1016/j.ijme.2022.100720

Wang, C., Chen, Z., & Zhou, M. (2023, April). AutoML from Software Engineering Perspective: Landscapes and Challenges. In Proceedings of the 20th International Conference on Mining Software Repositories. MSR. https://chenzhenpeng18.github.io/papers/MSR23.pdf

Zhang, D. (2022). Analysis of University Management Model of National Higher Education Institutions Based on Machine Learning Algorithm. Mobile Information Systems, 2022, 1–7. https://doi.org/10.1155/2022/4553185

Iatrellis, O., Savvas, I., Fitsilis, P., & Gerogiannis, V. C. (2020). A two-phase machine learning approach for predicting student outcomes. Education and Information Technologies; Springer Science+Business Media. https://doi.org/10.1007/s10639-020-10260-x

Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415

Fahd, K., Venkatraman, S., Miah, S. J., & Ahmed, K. (2022). Application of machine learning in higher education to assess student academic performance, at-risk, and attrition: A meta-analysis of literature. Education and Information Technologies; Springer Science+Business Media. https://doi.org/10.1007/s10639-021-10741-7

Janiesch, C., Zschech, P., & Heinrich, K. (2021). Machine learning and deep learning. Electronic Markets; Springer Science+Business Media. https://doi.org/10.1007/s12525-021-00475-2

Oqaidi, K., Aouhassi, S., & Mansouri, K. (2022). Towards a Students’ Dropout Prediction Model in Higher Education Institutions Using Machine Learning Algorithms. International Journal of Emerging Technologies in Learning (Ijet); kassel university press. https://doi.org/10.3991/ijet.v17i18.25567

Oladipupo, T. (2010). Types of Machine Learning Algorithms. New Advances in Machine Learning. https://doi.org/10.5772/9385

Manduchi, E., Romano, J. D., & Moore, J. H. (2021). The promise of automated machine learning for the genetic analysis of complex traits. Human Genetics, 141(9), 1529–1544. https://doi.org/10.1007/s00439-021-02393-x

Waring, J., Lindvall, C., & Umeton, R. (2020). Automated machine learning: Review of the state-of-the-art and opportunities for healthcare. Artificial Intelligence in Medicine, 104, 101822. https://doi.org/10.1016/j.artmed.2020.101822

Wever, M., Tornede, A., Mohr, F., & Hullermeier, E. (2021). AutoML for Multi-Label Classification: Overview and Empirical Evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(9), 3037–3054. https://doi.org/10.1109/tpami.2021.3051276

Zender, A., & Humm, B. G. (2022). Ontology-based Meta AutoML. Integrated Computer-Aided Engineering, 29(4), 351–366. https://doi.org/10.3233/ica-220684

Bahri, M., Salutari, F., Putina, A., & Sozio, M. (2022). AutoML: state of the art with a focus on anomaly detection, challenges, and research directions. International Journal of Data Science and Analytics, 14(2), 113–126. https://doi.org/10.1007/s41060-022-00309-0

Musigmann, M., Akkurt, B. H., Krähling, H., Nacul, N. G., Remonda, L., Sartoretti, T., Henssen, D., Brokinkel, B., Stummer, W., Heindel, W., & Mannil, M. (2022). Testing the applicability and performance of Auto ML for potential applications in diagnostic neuroradiology. Scientific reports, 12(1), 13648. https://doi.org/10.1038/s41598-022-18028-8

Cerrada, M., Trujillo, L., Hernández, D. E., Correa Zevallos, H. A., Macancela, J. C., Cabrera, D., & Vinicio Sánchez, R. (2022). AutoML for Feature Selection and Model Tuning Applied to Fault Severity Diagnosis in Spur Gearboxes. Mathematical and Computational Applications, 27(1), 6. https://doi.org/10.3390/mca27010006

Choi, W., Choi, T., & Heo, S. (2023). A Comparative Study of Automated Machine Learning Platforms for Exercise Anthropometry-Based Typology Analysis: Performance Evaluation of AWS SageMaker, GCP VertexAI, and MS Azure. Bioengineering, 10(8), 891. https://doi.org/10.3390/bioengineering10080891

Frank, F. & Bacao, F. (2023). Advanced Genetic Programming vs. State-of-the-Art AutoML in Imbalanced Binary Classification. Emerging Science Journal, 7(4), 1349–1363. https://doi.org/10.28991/esj-2023-07-04-021

Neverov, E. A., Viksnin, I. I., & Chuprov, S. S. (2023). The Research of AutoML Methods in the Task of Wave Data Classification. 2023 XXVI International Conference on Soft Computing and Measurements (SCM). https://doi.org/10.1109/scm58628.2023.10159058

Mueller, J., Shi, X., & Smola, A. (2020). Faster, Simpler, More Accurate. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. https://doi.org/10.1145/3394486.3406706

Jin, H., Chollet, F., Song, Q., & Hu, X. (2023). Autokeras: An automl library for deep learning. Journal of Machine Learning Research, 24(6), 1-6. https://www.jmlr.org/papers/volume24/20-1355/20-1355.pdf

Alaiad, A., Migdady, A., Al-Khatib, R. M., AlZoubi, O., Zitar, R. A., & Abualigah, L. (2023). Autokeras Approach: A Robust Automated Deep Learning Network for Diagnosis Disease Cases in Medical Images. Journal of Imaging; Multidisciplinary Digital Publishing Institute. https://doi.org/10.3390/jimaging9030064

Paldino, G. M., De Stefani, J., De Caro, F., & Bontempi, G. (2021, July 5). Does AutoML Outperform Naive Forecasting? The 7th International Conference on Time Series and Forecasting. https://doi.org/10.3390/engproc2021005036

Shchur, O., Turkmen, C., Erickson, N., Shen, H., Shirkov, A., Hu, T., & Wang, B. (2023). AutoGluon-TimeSeries: AutoML for Probabilistic Time Series Forecasting. ArXiv, abs/2308.05566. https://doi.org/10.48550/arXiv.2308.05566

Erickson, N., Mueller, J.W., Shirkov, A., Zhang, H., Larroy, P., Li, M., & Smola, A. (2020). AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data. ArXiv, abs/2003.06505. https://doi.org/10.48550/arXiv.2003.06505

Komer, B., Bergstra, J., Eliasmith, C. (2019). Hyperopt-Sklearn. In: Hutter, F., Kotthoff, L., Vanschoren, J. (eds) Automated Machine Learning. The Springer Series on Challenges in Machine Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-05318-5_5

Dumitrache, A., Melian, D. M., Bălăcian, D., Nastu, A., & Stancu, S. (2020). Churn prepaid customers classified by HyperOpt techniques. Proceedings of the International Conference on Applied Statistics. https://doi.org/10.2478/icas-2021-0010

Bergstra, J., Yamins, D., & Cox, D. D. (2013). Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms. THe 12th Python in Science Conference.

Bergstra, J., Komer, B., Eliasmith, C., Yamins, D., & Cox, D. D. (2015). Hyperopt: a Python library for model selection and hyperparameter optimization. Computational Science & Discovery, 8(1), 014008. https://doi.org/10.1088/1749-4699/8/1/014008

Ma, J., Xu, H., Wang, A., Wang, A., Gao, L., & Ding, M. (2023). Machine learning-guided underlying decisive factors of high-performance membrane distillation system: Membrane properties, operation conditions and solution composition. Separation and Purification Technology, 327, 124964. https://doi.org/10.1016/j.seppur.2023.124964

Płońska, A., & Płoński, P. (2021). MLJAR: State-of-the-art Automated Machine Learning Framework for Tabular Data. Version 0.10.3. [Computer software]. MLJAR, https://github.com/mljar/mljar-supervised

Vázquez, F. (2023, October 6). Entrenando Tu Propio LLM Sin Programación. H2O.ai. Retrieved October 23, 2023, from https://h2o.ai/blog/entrenando-tu-propio-llm-sin-programacion/

LeDell, E., & Poirier, S. (2020). H2o automl: Scalable automatic machine learning. In Proceedings of the AutoML Workshop at ICML (Vol. 2020). ICML.

Kochura, Y., Stirenko, S., & Gordienko, Y. (2017). Comparative performance analysis of neural networks architectures on H2O platform for various activation functions. 2017 IEEE International Young Scientists Forum on Applied Physics and Engineering (YSF). doi:10.1109/ysf.2017.8126654

Saucedo, M. L., Sánchez, R. L., Becerra, E. E., & Puican, V. H. (2023). New E-government Strategies in Peruvian Universities. https://doi.org/10.55908/sdgs.v11i2.703

Salas‐Pilco, S. Z., Yang, Y., & Zhang, Z. (2022). Student engagement in online learning in Latin American higher education during the COVID‐19 pandemic: A systematic review. British Journal of Educational Technology, 53(3), 593–619. https://doi.org/10.1111/bjet.13190

Bates, T., Cobo, C., Mariño, O., & Wheeler, S. (2020). Can artificial intelligence transform higher education? International Journal of Educational Technology in Higher Education, 17(1). https://doi.org/10.1186/s41239-020-00218-x

Nuankaew, P., Nasa-Ngium, P., Kunasit, T., & Nuankaew, W. (2023). Implementation of Data Analytics and Machine Learning in Thailand Education Sector. International Journal of Emerging Technologies in Learning (Ijet); kassel university press. https://doi.org/10.3991/ijet.v18i05.36871

Callender, C., & Dougherty, K. J. (2018, October 9). Student Choice in Higher Education—Reducing or Reproducing Social Inequalities? Social Sciences, 7(10), 189. https://doi.org/10.3390/socsci7100189

Wadesango, N., Maphosa, C., & Moyo, G. (2014). An Academic Development Agenda for Postgraduate Research Students. Mediterranean Journal of Social Sciences. https://doi.org/10.5901/mjss.2014.v5n11p49

Ilie, S., Rose, P., & Vignoles, A. (2021). Understanding higher education access: Inequalities and early learning in low and lower‐middle‐income countries. British Educational Research Journal, 47(5), 1237–1258. https://doi.org/10.1002/berj.3723

Villarreal-Torres, H., Ángeles-Morales, J., Marín-Rodriguez, W., Andrade-Girón, D., Cano-Mejía, J., Mejía-Murillo, C., Flores-Reyes, G., & Palomino-Márquez, M. (2023a). Classification model for student dropouts using machine learning: A case study. EAI Endorsed Transactions on Scalable Information Systems, 10(5). https://doi.org/10.4108/eetsis.vi.3455

Villarreal-Torres, H., Ángeles-Morales, J., Marín-Rodriguez, W., Andrade-Girón, D., Carreño-Cisneros, E., Cano-Mejía, J., Mejía-Murillo, C., Boscán-Carroz, M. C., Flores-Reyes, G., & Cruz-Cruz, O. (2023b). Development of a Classification Model for Predicting Student Payment Behavior Using Artificial Intelligence and Data Science Techniques. EAI Endorsed Transactions on Scalable Information Systems, 10(5). https://doi.org/10.4108/eetsis.3489

El Peruano. (2022, 19 de agosto). Retrocedió el índice de morosidad. https://www.elperuano.pe/noticia/183969-retrocedio-el-indice-de-morosidad

Ferreira, L., Pilastri, A., Romano, F., & Cortez, P. (2022). Using supervised and one-class automated machine learning for predictive maintenance. Applied Soft Computing, 131, 109820. https://doi.org/10.1016/j.asoc.2022.109820

Gijsbers, P., Bueno, M. L., Coors, S., LeDell, E., Poirier, S., Thomas, J., ... & Vanschoren, J. (2022). AMLB: an automl benchmark. https://doi.org/10.48550/arxiv.2207.12560

Abaimov, S., & Martellini, M. (2022). Understanding Machine Learning. Advanced Sciences and Technologies for Security Applications. https://doi.org/10.1007/978-3-030-91585-8_2

Lázaro, L. M. (2022). La UNESCO y los futuros de la educación superior hasta 2050. Por una ampliación del derecho a la educación que incluya a la educación superior. Revista Española De Educación Comparada, (41), 271–280. https://doi.org/10.5944/reec.41.2022.33879

Downloads

Published

06-12-2023

How to Cite

1.
Villarreal-Torres H, Ángeles-Morales J, Cano-Mejía J, Mejía-Murillo C, Flores-Reyes G, Cruz-Cruz O, Urcia-Quispe M, Palomino-Márquez M, Solar-Jara M Ángel, Escobedo-Zarzosa R. Comparative analysis of performance of AutoML algorithms: Classification model of payment arrears in students of a private university. EAI Endorsed Scal Inf Syst [Internet]. 2023 Dec. 6 [cited 2024 Jun. 23];11(4). Available from: https://publications.eai.eu/index.php/sis/article/view/4550

Most read articles by the same author(s)