Development and Perceived Usability Evaluation of a Mobile application for Notetaking

H. Demirelli; Y. Isler; Yilmaz Kemal Yuce

doi:10.4108/eetel.4538

Authors

H. Demirelli Alanya Alaaddin Keykubat Üniversitesi
Y. Isler Izmir Kâtip Çelebi University
Yilmaz Kemal Yuce Alanya Alaaddin Keykubat Üniversitesi

DOI:

https://doi.org/10.4108/eetel.4538

Keywords:

notetaking, Cornell Technique, e-learning, mobile application

Abstract

INTRODUCTION: Notetaking is considered, by many educators, as one of the critical actions of learning. There are several note-taking methods and approaches. Based on these methods and approaches, various applications, whether mobile, desktop or -Web-based, were developed.

OBJECTIVES: In this paper, a novel note-taking application based on Cornell Technique, is presented. Its development process and user acceptance trend are exhibited and results for user evaluation based on user satisfaction are presented.

METHODS: For the software development process, Incremental Model was adopted. Requirement Analysis included, aside from examining principles and related note-taking structure of Cornell Technique, investigating (i) how to perform notetaking as an activity of learning, (ii) its product and (iii) relationship of notes for the purpose of storage. Models containing sub-activities, such as reviewing note have been identified and some were selectively adopted and related functions such as review alert (tickler) and collaboration on notetaking have been implemented. To the purpose of storage, a tree-based scheme called collection was modelled. User interfaces were first designed as mockups and click-through pro-totype using Adobe XD. The mobile application was implemented in Dart programming language. Google’s Firebase Service and Flutter Framework was adopted. The mobile application was compared with its equivalents in the Google Play Store and user statistics were investigated. To evaluate perceived usability, the System Usability Scale is adopted and applied to 14 university students conforming to determined persona.

RESULTS: The application has been published in Google Play Store for users to install for free on 18^th March 2022. As of 10^th September 2023, total number of downloads is 5K and the Cornell Note mobile app is currently installed on 1108 devices. For the last three-month period (from 11^th June to 10^th September 2023), the active users per month changed in an increasing trend from 450 to 589. The average engagement time on 11^th of April 2023 was 28 minutes 00 seconds. As the number of monthly active users increased, the average engagement time measured on 10^th September 2023 decreased to 23 minutes 31 seconds. However, engagement rates measured were 76.91% and 77.19%, respectively. The mean SUS score was found to be equal to 79.5.

CONCLUSION: The user statistics and comparison with equivalent mobile applications reveal that Cornell Note has potential to grow as a mobile application for notetaking since it has a good perceived usability, however, there is room for improvement. Considering any extra marketing effort was not spent for the application such as application store optimization, the statistics are another evidence for user appeal and acceptance. However, it is important to add new functionality without complicating the user experience so that user appeal and acceptance boosts.

References

Hartley, J., Note taking in non-academic settings: a review. Applied Cognitive Psychology. 2023; 16(5): 559-574.

Brobst, K.E., The process of integrating information from two sources, lecture and text. Dissertation (1996).

Di Vesta, F.J., Gray, G.S. Listening and note taking. J Educ Psychol. 1972; 63(1): 8–14.

Craik, F. I. M., Tulving, E. Depth of processing and the retention of words in episodic memory. Journal of Experimental Psychology: General. 1975; 104(3): 268–294. https://doi.org/10.1037/0096-3445.104.3.268

Kiewra, K.A. Note taking and review: The research and its implications, Journal of Instructional Science. 1987; 16, 233-249.

Kiewra, K.A. Cognitive Aspects of Autonomous Note Taking: Control Processes, Learning Strategies, and Prior Knowledge, Educational Psychologist. 1988; 23(1): 39-56. DOI: 10.1207/s15326985ep2301_3.

Van Meter, P. N., Yokoi, L., Pressley, M. College Students' Theory of Notetaking Derived from Their Perceptions of Notetaking. Journal of Educational Psychology. 1994; 86(3): 323-338. https://doi.org/10.1037/0022-0663.86.3.323.

Carrier, C. A., Williams, M. D., Dalgaard, B. R. College Students’ Perceptions of Notetaking and Their Relationship to Selected Learner Characteristics and Course Achievement. Research in Higher Education. 1988; 28(3): 223–239. http://www.jstor.org/stable/40195863.

Kiewra, K.A. A Review of Note-taking: Encoding-Storage Paradigm and Beyond. Educational Psychology Review. 1989; 1(2): 147-173.

Kiewra, K. A. The process of review: A levels of processing approach, Contemporary Educational Psychology. 1983; 8(4): 366-374. https://doi.org/10.1016/0361-476X(83)90023-1.

Kiewra, K. A., Benton, S. L. The relationship between information-processing ability and notetaking. Contemporary Educational. Psychology. 1988; 13: 33-44.

Kiewra, K. A. Investigating notetaking and review: A depth of processing alternative. Educational Psychologist. 1985; 20(1): 23-32.

Kiewra, K. A., DuBois, N. F., Christian, D., McShane, A., Meyerhoffer, M., Roskelley, D. Note-taking functions and techniques. Journal of Educational Psychology. 1991; 83(2): 240–245. https://doi.org/10.1037/0022-0663.83.2.240.

Kılıçkaya, F., Çokal-karadaş, D. The Effect Of Note-Taking On University Students’ Listening Comprehension Of Lectures, Kastamonu Eğitim Dergisi. 2009; 17 (1): 47-56.

Kiewra, K. A. Aids to lecture learning. Educational Psychologist. 1991; 26(1): 37–54.

Pressley, M., Yokoi, L., van Meter, P. et al. Some of the Reasons Why Preparing for Exams Is So Hard: What Can Be Done to Make It Easier?, Educational Psychology Review. 1997 9: 1–38. https://doi.org/10.1023/A:1024796622045

Kiewra, K. A. Learning from a lecture: An investigation of notetaking, review and attendance at a lecture. Human Learning: Journal of Practical Research & Applications. 1985; 4(1): 73–77.

Kiewra, K.A. Students' note-taking behaviors and the efficacy of providing the instructor's notes for review, Contemporary Educational Psychology. 1985; 10(4): 378-386. ISSN 0361-476X. https://doi.org/10.1016/0361-476X(85)90034-7.

Baker, L., Lombardi B. R. Students' Lecture Notes and Their Relation to Test Performance. Teaching of Pschology. 1985; 12(1): 28-32.

Kiewra, K. A. Acquiring effective notetaking skills: An alternative to professional notetaking. Journal of Reading. 1984; 27(1): 299–302.

Kiewra, K. A. Implications for note taking based on relationships between note taking variables and achievement measures. Reading Improvement. 1984; 21: 145–149.

Kiewra, K. A., & Benton, S. L. The relationship between information-processing ability and notetaking. Contemporary Educational Psychology. 1988; 13(1): 33–44.

Kiewra, K. A., Benton, S. L., Lewis, L. B. Qualitative aspects of notetaking and their relationship with information-processing ability and academic achievement. Journal of Instructional Psychology. 1987; 14(3): 110–117.

Collingwood, V., Hughes, D. C. Effects of three types of university lecture notes on student achievement. Journal of Educational Psychology. 1978; 70(2): 175–179. https://doi.org/10.1037/0022-0663.70.2.175.

Morgan, C. H., Lilley, J. D., Boreham, N. C. Learning from lectures: The effect of varying the detail in lecture handouts on notetaking and recall. Applied Cognitive Psychology. 1988; 2(1): 115–122.

Kiewra, K. A., Kauffman, D. F., Robinson, D., DuBois, N., & Staley, R. K. Supplementing floundering text with adjunct displays. Journal of Instructional Science. 1999; 27(1): 373–401.

Robinson, D., & Kiewra, K. A. Visual argument: Graphic organizers are superior to outlines in improving learning from text. Journal of Educational Psychology. 1995; 87(1): 455–467.

Tee, T.K., Azman, M.N., Mohamed, S.B., Muhammad, Mohamad, M.M., Yunos, J.B., Yee, M.H., Othman, W. Buzan Mind Mapping: An Efficient Technique for Notetaking. World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 2014; 8(1): 28-31.

Readence, J.E., Bean, T., Baldwin, R.S. Content area reading: An integrated approach, 4th Edition. Iowa, USA: Kendall-Hunt Publishing; 1989. 421 pages.

Jairam, D., Kiewra, K. A. An investigation of the SOAR study method. Journal of Advanced Academics. 2009; 20(4): 602–629. https://doi.org/10.1177/1932202X0902000403.

Daher, T.A., Kiewra, K.A. An investigation of SOAR study strategies for learning from multiple online resources, Contemporary Educational Psychology. 2016; 46, 10-21. ISSN 0361-476X https://doi.org/10.1016/j.cedpsych.2015.12.004.

Pauk, W., & Owens, R. J. Q. How to study in college. 10th Ed. Boston, USA: Wadsworth Cengage Learning; 2011. 406 pages. ISBN-13: 978-1-4390-8446-5.

Stacy, E.M., & Cain, J.J. Notetaking and Handouts in The Digital Age. American Journal of Pharmaceutical Education. 2015; 79(7): 107-112.

Samsung Notes. https://play.google.com/store/apps/details?id=com.samsung.android.app.notes, Last accessed 10.01.2023.

Google Keep, https://play.google.com/store/apps/details?id=com.google.android.keep, Last accessed 10.01.2023.

Microsoft OneNote. https://play.google.com/store/apps/details?id=com.microsoft.office.onenote, Last accessed 10.01.2023.

ColorNote Notepad Notes. https://play.google.com/store/apps/details?id=com.socialnmobile.dictapps.notepad.color.note&gl=TR, Last accessed 10.01.2023.

Evernote. https://play.google.com/store/apps/details?id=com.evernote , Last accessed 10.01.2023.

Pyörälä, E., Mäenpää, S., Heinonen, L. et al. The art of note taking with mobile devices in medical education. BMC Med Educ. 2019; 19(1): 96-105. https://doi.org/10.1186/s12909-019-1529-7.

Shen, H., Reilly, M. Personalized Multi-User View and Content Synchronization and Retrieval in Real-Time Mobile Social Software Applications. Journal of Computer and System Sciences. 2012; 78(4): 1185-1203.

Popescu, E., Stefan, C., Ilie, S., Ivanović, M. (2016). EduNotes – A Mobile Learning Application for Collaborative Notetaking in Lecture Settings. In: Chiu, D., Marenzi, I., Nanni, U., Spaniol, M., Temperini, M. (eds): Advances in Web-Based Learning – ICWL 2016. ICWL 2016. Lecture Notes in Computer Science, 10013. Springer, Cham. https://doi.org/10.1007/978-3-319-47440-3_15

Şahin, A., Aydın, G., Sevim, O. The Effect of Cornell Method on Understanding and Retention of the Text Dictated. Dumlupınar Üniversitesi Sosyal Bilimler Dergisi. 2011; 29(1): 29-36.

Valtonen, T., Havu-Nuutinen, S., Dillon, P., Vesisenaho, M. Facilitating Collaboration in Lecture-based Learning through Shared Notes Using Wireless Technologies. Journal of Computer Assisted Learning. 2011; 27(6): 575 – 586.

Kolb, D.A., Rubin, I.M., & McIntyre, J. Organizational psychology: An experiential approach. 4th Ed. New Jersey, USA: Prentice Hall; 1984. 496 pages.

Kolb, A.Y., Kolb, D.A. The Kolb Learning Style Inventory 4.0: A Comprehensive Guide to the Theory, Psychometrics, Research on Validity and Educational Applications. Kaunakakai, Hawaii, USA: Experience Based Learning Systems Inc.; 2013. 233 pages.

Smith, D.M., Kolb, D.A. User's Guide for the Learning Style Inventory. Boston, USA: McBer and Company; 1986. 100 pages.

Kiewra, K.A. Providing the Instructor's Notes: An Effective Addition to Student Notetaking, Educational Psychologist. 1985; 20(1): 33-39. DOI: 10.1207/s15326985ep2001_5.

Link, J. Software Engineering and Programming, Unit Testing in Java. Massachusetts, USA: Morgan Kaufmann; 2003. Chapter 14, The Role of Unit Tests in the Software Process [291-312]. ISBN 9781558608689, https://doi.org/10.1016/B978-155860868-9/50016-X.

Arifoğlu, A., Doğru, A. Yazılım Mühendisliği (Yöntemler, Metodolojiler, CASE Ortamları, Günün Teknolojisi), Ankara, Turkey: SaS Publishing; 2001. 361 pages.

Adobe XD, https://www.adobe.com/tr/products/xd.html , Last Accessed 10.02.2023.

A Comprehensive Guide to Wireframing and Prototyping, https://www.smashingmagazine.com/2018/03/guide-wireframing-prototyping/ , Christopher Murhpy, Last Accessed 10.01.2023.

McElroy, K. Prototyping for Designers: Developing the Best Digital and Physical Products. California, USA: O’Reilly; 2017. 324 pages. ISBN-13: 978-1491954089.

Android Studio. https://developer.android.com/studio . Last Accessed 10.01.2023.

IntelliJ Idea. https://www.jetbrains.com/idea/. Last Accessed 10.01.2023.

Google Firestore. https://firebase.google.com/products/firestore?gclid=CjwKCAjwjZmTBhB4EiwAynRmD0uFN9Kh-qQ_S3NSUmvuy4N9uEWtHT_QI2xOks59VnOvvICuN3C4qRoCxeAQAvD_BwE&gclsrc=aw.ds. Last Accessed 10.01.2023

What is Three-tier Architecture? https://www.ibm.com/cloud/learn/three-tier-architecture. IBM, Last Accessed 10.01.2023.

Google Firebase. https://firebase.google.com/. Last Accessed 10.01.2023.

Google OAuth API. https://developers.google.com/identity/protocols/oauth2. Last Accessed 10.01.2023.

JavaScript Object Notation (JSON). https://www.json.org/json-en.html. Last Accessed 10.01.2023.

The Midas Touch Effect. https://uxdesign.cc/the-midas-touch-effect-the-most-unknown-phenomenon-in-ux-design-36827204edd. Last Accessed 10.01.2023.

Haans A., Usselsteijn, W.A. The Virtual Midas Touch: Helping Behavior After a Mediated Social Touch. IEEE Transactions on Haptics; 2009; 2(3): 136-140. doi: 10.1109/TOH.2009.20.

Cornell Notes Mobile Application. https://play.google.com/store/apps/details?id=com.aipcsoft.cornell_notes_mobile&hl=en_US&gl=US. Last Accessed 10.01.2023.

Columns – Mobile application for Cornell Note Technique. https://apps.apple.com/us/app/columns-cornell-notes/id1493839821?mt=12. Last Accessed 06.01.2023.

Speech-to-text Cornell Notes Mobile Application. https://play.google.com/store/apps/details?id=com.cornell.voice.notepad&pcampaignid=web_share. Last Accessed 10.01.2023.

Cornell Note Mobile Application. https://play.google.com/store/apps/details?id=com.cornell.not&gl=TR. Last Accessed 06.01.2023.

ISO 9241-11:1998. Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on Usability.

Brooke, J. Usability Evaluation in Industry. London, UK: Taylor & Francis; 1996. Chapter 21, SUS: A quick and dirty usability scale [4-7].

Kirakowski, J., Corbett, M. “SUMI: the Software Usability Measurement Inventory”, British Journal of Educational Technology, 1993; 24(3): 210-212.

Lewis, J.R. “IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use”, International Journal of Human-Computer Interaction. 1995; 7(1): 57-78.

Lewis, J.R. Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies, International Journal of Human–Computer Interaction. 2002; 14:(3): 463-488.

Bangor, A., Kortum, P., & Miller, J. Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of usability studies. 2009; 4(3): 114-123.

Sauro, J. A practical guide to the System Usability Scale: Background, benchmarks, & best practices. Denver, USA: Measuring Usability LLC; 2011. 150 pages.

Lewis, J.R., Sauro, J. The Factor Structure of the System Usability Scale. In: Kurosu, M. (eds) Human Centered Design. HCD 2009. Lecture Notes in Computer Science, vol 5619. Springer, Berlin, Heidelberg.

Borsci, S., Federici, S., & Lauriola, M. On the dimensionality of the System Usability Scale: A test of alternative measurement models.Cognitive Processing. 2009; 10 (3): 193–197.

Bangor, A., Kortum, P. T., & Miller, J. T. An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction. 2008; 24(6): 574-594.

Orfanou, K., Tselios, N. & Katsanos, C. Perceived usability evaluation of learning management systems: Empirical evaluation of the System Usability Scale. The International Review Of Research In Open And Distributed Learning. 16

Tullis, T. S., Stetson, J. N.: A Comparison of Questionnaires for Assessing Website Usability (2004), unpublished presentation given at the UPA Annual Conference, http://home.comcast.net/~tomtullis/publications/UPA2004TullisStetson.pdf

Landauer, T. K. Handbook of Human-Computer Interaction. North-Holland: Elsevier; 1997. Chapter 9, Behavioral Research Methods in Human-Computer Interaction In: Helander, M., Landauer, T., Prabhu, P. (eds.) pp. 203--227. Elsevier, Amsterdam, Netherlands (1997)

Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. Intl. Journal of Human–Computer Interaction, 24(6), 574-594.

Demi̇rkol, D., & Şeneler, Ç.A Turkish translation of the system usability scale: The SUS-TR. Uşak Üniversitesi Sosyal Bilimler Dergisi. 2018; 11(3): 237-253.

Martins, A. I., Rosa, A. F., Queirós, A., Silva, A., & Rocha, N. P. European Portuguese validation of the system usability scale (SUS). Procedia computer science. 2015; 67: 293-300.

Sevilla-Gonzalez, M. D. R., Loaeza, L. M., Lazaro-Carrera, L. S., Ramirez, B. B., Rodríguez, A. V., Peralta-Pedrero, M. L., & Almeda-Valdes, P. (2020). Spanish version of the system usability scale for the assessment of electronic tools: development and validation. JMIR Human Factors, 7(4), e21161.

Katsanos, C., Tselios, N., & Xenos, M.N. (2012). Perceived Usability Evaluation of Learning Management Systems: A First Step towards Standardization of the System Usability Scale in Greek. 2012 16th Panhellenic Conference on Informatics, 302-307.

Blažica, B., & Lewis, J. R. A slovene translation of the system usability scale: The SUS-SI. International Journal of Human-Computer Interaction. 2015; 31(2): 112-117.

Condit F., J., Mandernach, M., Nelson, C. S., Paulo, J. R., & Saunders, G. Usability test results for a discovery tool in an academic library. Information Technology & Libraries. 2015; 31(1): 83–112.

Perrin, J. M., Clark, M., De-Leon, E., & Edgar, L. Usability testing for greater impact: A Primo case study. Information Technology & Libraries. 2014; 33(4): 57–66.

Yüce YK, Tezcan E, Uyguralp C, Yolcular BO, Akça M. On the usability of snapchat: what do users think about it. Akıllı Sistemler ve Uygulamaları Dergisi (Journal of Intelligent Systems with Applications). 2019; 2(2): 148-156. DOI: 10.54856/jiswa.201912085

Kaya, A., Ozturk, R., Altin Gumussoy, C. Usability Measurement of Mobile Applications with System Usability Scale (SUS). In: Calisir, F., Cevikcan, E., Camgoz Akdag, H. (eds) Industrial Engineering in the Big Data Era. 2019. Lecture Notes in Management and Industrial Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-03317-0_32

Liang, J., Xian, D., Liu, X., Fu, J., Zhang, X., Tang, B., & Lei, J. Usability study of mainstream wearable fitness devices: feature analysis and system usability scale evaluation. JMIR mHealth and uHealth. 2018; 6(11): e11066.

SUS Calculator, https://measuringu.com/product/suscalc/. Measuring Usability. Last accessed 03.10.2023

Padilla-Piernas, J.M., Parra-Meroño, M.C., Beltrán-Bueno, M.Á. The Importance of App Store Optimization (ASO) for Hospitality Applications. In: Rana, N.P., et al. Digital and Social Media Marketing. 2020. Advances in Theory and Practice of Emerging Markets. Springer, Cham. https://doi.org/10.1007/978-3-030-24374-6_11

Karagkiozidou, M., Ziakis, C., Vlachopoulou, M., Kyrkoudis, T. App Store Optimization Factors for Effective Mobile App Ranking. In: Kavoura, A., Kefallonitis, E., Giovanis, A. (eds) Strategic Innovative Marketing and Tourism 2019. Springer Proceedings in Business and Economics. Springer, Cham. https://doi.org/10.1007/978-3-030-12453-3_54.

Lewis, J.R. and Sauro, J. Item benchmarks for the system usability scale. J. Usability Studies. 2018; 13(3): 158–167.