From Voluntary Practice Towards Actionable Learning Analytics: Architecture, Feature-Level Use, and Design Implications for an AI-Supported Learning Platform
DOI:
https://doi.org/10.4108/dtip.14898Keywords:
AI-supported learning, learning analytics, higher education, event logging, formative feedback, teacher-facing dashboardsAbstract
INTRODUCTION: Voluntary use of AI-supported learning platforms may differ across features, while aggregate logs cannot show whether teachers act on analytics.
OBJECTIVES: To evaluate AI Sensei’s architecture, logged activity across platform features, self-reported feature use and student feedback.
METHODS: An observational pilot analysed logs for 205 students and an anonymous questionnaire (N = 100). Training quizzes and feedback were programmed; chat used generative AI.
RESULTS: All 205 students submitted training quizzes and self-check tests (1,999 and 481 submissions, respectively); 164 (80.0%) sent at least one chat message. In the separate anonymous questionnaire, 91% selected quizzes/tests and 17% the chat assistant; 94% agreed that combining teacher support with AI Sensei was more beneficial than teaching without the platform.
CONCLUSION: Structured practice merits refinement; teacher-facing analytics require evaluation for actionability.
Downloads
References
[1] Shute VJ. Focus on formative feedback. Rev Educ Res. 2008;78(1):153–189. https://doi.org/10.3102/0034654307313795
[2] Brame CJ, Biel R. Test-enhanced learning: The potential for testing to promote greater learning in undergraduate science courses. CBE Life Sci Educ. 2015;14(2):es4. https://doi.org/10.1187/cbe.14-11-0208
[3] van Wijk EV, de Jonge M, van Blankenstein FM, Janse RJ, Langers AMJ. The battle of question formats: a comparative study of retrieval practice using very short answer questions and multiple choice questions. BMC Med Educ. 2024;24(1):1547. https://doi.org/10.1186/s12909-024-06538-0
[4] Panadero E. A review of self-regulated learning: Six models and four directions for research. Front Psychol. 2017;8:422. https://doi.org/10.3389/fpsyg.2017.00422
[5] Li Y, Sueb R, Said Hashim K. The relationship between parental autonomy support, teacher autonomy support, peer support, and university students’ academic engagement: the mediating roles of basic psychological needs and autonomous motivation. Front Psychol. 2025;16:1503473. https://doi.org/10.3389/fpsyg.2025.1503473
[6] Teo T, Scherer R, Fung ASK, Fung CSL. Factors associated with students’ adoption of artificial intelligence technology in tertiary education: A meta-analytic review. Educ Res Rev. 2026;52:100804. https://doi.org/10.1016/j.edurev.2026.100804
[7] Annapureddy R, Fornaroli A, Gatica-Perez D. Generative AI literacy: Twelve defining competencies. Digit Gov Res Pract. 2025;6(1):Article 13, 1–21. https://doi.org/10.1145/3685680
[8] Zhang G, Yu T. Association between Generative AI self-efficacy and Generative AI acceptance: The mediating role of Generative AI trust and the moderating role of Generative AI risk perception. Acta Psychol (Amst). 2025;261:105791. https://doi.org/10.1016/j.actpsy.2025.105791
[9] Tong A, Zainol Z, Chong TS, Renganathan K. AI governance on young consumers in higher education: a content analysis of policies for generative AI. Young Consum. 2025;26(5):865–885. https://doi.org/10.1108/YC-10-2024-2303
[10] Alfredo R, Echeverria V, Jin Y, Yan L, Swiecki Z, Gašević D, Martinez-Maldonado R. Human-centred learning analytics and AI in education: A systematic literature review. Comput Educ Artif Intell. 2024;6:100215. https://doi.org/10.1016/j.caeai.2024.100215
[11] Du J, Hew KF, Liu L. What can online traces tell us about students’ self-regulated learning? A systematic review of online trace data analysis. Comput Educ. 2023;201:104828. https://doi.org/10.1016/j.compedu.2023.104828
[12] Hao C, Zhang F. Understanding self-regulated grammar learning with LLM chatbot support: An epistemic network analysis of grammar learning strategy patterns. System. 2026;136:103879. https://doi.org/10.1016/j.system.2025.103879
[13] Kaliisa R, Jivet I, Prinsloo P. A checklist to guide the planning, designing, implementation, and evaluation of learning analytics dashboards. Int J Educ Technol High Educ. 2023;20:28. https://doi.org/10.1186/s41239-023-00394-6
[14] Paulsen L, Lindsay E. Learning analytics dashboards are increasingly becoming about learning and not just analytics - A systematic review. Educ Inf Technol. 2024;29(11):14279–14308. https://doi.org/10.1007/s10639-023-12401-4
[15] Bai H, Lui WC, Khiatani PV. Promoting student engagement with GPTutor: An intelligent tutoring system powered by generative AI. Int J Educ Technol High Educ. 2025;22:77. https://doi.org/10.1186/s41239-025-00571-9
[16] Mask J. Does AI help economics students learn or just finish? A classroom field study. Int Rev Econ Educ. 2026;52:100350. https://doi.org/10.1016/j.iree.2026.100350
[17] Stolariková R, Vadovičová A, Dobiášová M, Valášek P, Votava J. AI Sensei: Design and pilot evaluation of an event-logged learning platform with analytics for higher education. In: Machado J, Trojanowska J, Antosz K, Leão CP, Knapcikova L, Sover A, editors. Innovations in Industrial Engineering V. Lecture Notes in Mechanical Engineering. Cham: Springer Nature; 2026. In press.
[18] Wise AF, Jung Y. Teaching with analytics: Towards a situated model of instructional decision-making. J Learn Anal. 2019;6(2):53–69. https://doi.org/10.18608/jla.2019.62.4
[19] Kaliisa R, Dolonen JA. CADA: A teacher-facing learning analytics dashboard to foster teachers’ awareness of students’ participation and discourse patterns in online discussions. Technol Knowl Learn. 2023;28:937–958. https://doi.org/10.1007/s10758-022-09598-7
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Radka Stolariková, Aneta Vadovičová, Monika Dobiášová, Petr Valášek, Jiří Votava
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open access article distributed under the terms of the CC BY-NC-SA 4.0, which permits copying, redistributing, remixing, transformation, and building upon the material in any medium so long as the original work is properly cited.
