Field report for Platform mBox: Designing an Open MMLA Platform
Pages 785 - 791
Abstract
Multimodal Learning Analytics (MMLA) is an evolving sector within learning analytics that has become increasingly useful for examining complex learning and collaboration dynamics for group work across all educational levels. The availability of low-cost sensors and affordable computational power allows researchers to investigate different modes of group work. However, the field faces challenges stemming from the complexity and specialization of the systems required for capturing diverse interaction modalities, with commercial systems often being expensive or narrow in scope and researcher-developed systems needing to be more specialized and difficult to deploy. Therefore, more user-friendly, adaptable, affordable, open-source, and easy-to-deploy systems are needed to advance research and application in the MMLA field. The paper presents a field report on the design of mBox that aims to support group work across different contexts. We share the progress of mBox, a low-cost, easy-to-use platform grounded on learning theories to investigate collaborative learning settings. Our approach has been guided by iterative design processes that let us rapidly prototype different solutions for these settings.
References
[1]
Dor Abrahamson, Marcelo Worsley, Zachary A. Pardos, and Lu Ou. 2022. Learning analytics of embodied design: Enhancing synergy. International Journal of Child-Computer Interaction 32 (2022), 100409. https://doi.org/10.1016/j.ijcci.2021.100409
[2]
Haifa Alwahaby, Mutlu Cukurova, Zacharoula Papamitsiou, and Michail Giannakos. 2022. The evidence of impact and ethical considerations of Multimodal Learning Analytics: A Systematic Literature Review. The Multimodal Learning Analytics Handbook (2022), 289–325.
[3]
Saskia Bakker, Alissa N. Antle, and Elise Van Den Hoven. 2012. Embodied metaphors in tangible interaction design. Personal and Ubiquitous Computing 16, 4 (2012), 433–449. https://doi.org/10.1007/s00779-011-0410-4
[4]
Paulo Blikstein and Marcelo Worsley. 2016. Multimodal Learning Analytics and Education Data Mining: using computational technologies to measure complex learning tasks. Journal of Learning Analytics 3, 2 (2016), 220–238. https://doi.org/10.18608/jla.2016.32.11
[5]
Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Mihai Jalobeanu, Patrick Sweeney, Anne Loomis Thompson, and Eric Horvitz. 2021. Platform for situated intelligence. arXiv preprint arXiv:2103.15975 (2021).
[6]
Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design. Morgan kaufmann.
[7]
Oscar Canovas and Felix J. Garcia. 2023. Analysis of Classroom Interaction Using Speaker Diarization and Discourse Features from Audio Recordings. In Learning in the Age of Digital and Green Transition, Michael E. Auer, Wolfgang Pachatz, and Tiia Rüütmann (Eds.). Springer International Publishing, Cham, 67–74. https://doi.org/10.1007/978-3-031-26190-9_7
[8]
John M. Carroll. 2003. Making use: scenario-based design of human-computer interactions. MIT press.
[9]
Pankaj Chejara, Luis P. Prieto, Adolfo Ruiz-Calleja, María Jesús Rodríguez-Triana, Shashi Kant Shankar, and Reet Kasepalu. 2020. Quantifying Collaboration Quality in Face-to-Face Classroom Settings Using MMLA. Springer International Publishing, 159–166. https://doi.org/10.1007/978-3-030-58157-2_11
[10]
Ottavio Colazingari. 2021. Empowering human behavior research: the case of iMotions. Ph. D. Dissertation.
[11]
Alexandre Défossez, Gabriel Synnaeve, and Yossi Adi. 2020. Real Time Speech Enhancement in the Waveform Domain. In Proc. Interspeech 2020. 3291–3295. https://doi.org/10.21437/Interspeech.2020-2409
[12]
Jon Echeverria and Olga C. Santos. 2021. KUMITRON: Artificial intelligence system to monitor karate fights that synchronize aerial images with physiological and inertial signals. In 26th International Conference on Intelligent User Interfaces-Companion. 37–39.
[13]
Michail Giannakos and Mutlu Cukurova. 2023. The role of learning theory in multimodal learning analytics. British Journal of Educational Technology 54, 5 (2023), 1246–1267. https://doi.org/10.1111/bjet.13320
[14]
Shirley Gregor, Leona Kruse, and Stefan Seidel. 2020. Research Perspectives: The Anatomy of a Design Principle. Journal of the Association for Information Systems 21 (2020), 1622–1652. https://doi.org/10.17705/1jais.00649
[15]
A.R. Hevner and S.T. March. 2003. IT systems perspectives - the information systems research cycle. Computer 36, 11 (2003), 111–113. https://doi.org/10.1109/mc.2003.1244541
[16]
Asami Ito-Masui, Eiji Kawamoto, Ryo Esumi, Hiroshi Imai, and Motomu Shimaoka. 2021. Sociometric wearable devices for studying human behavior in corporate and healthcare workplaces. BioTechniques 71, 1 (2021), 392–399. https://doi.org/10.2144/btn-2020-0160
[17]
Mohammad Khalil, Paul Prinsloo, and Sharon Slade. 2022. The use and application of learning theory in learning analytics: a scoping review. Journal of Computing in Higher Education (2022). https://doi.org/10.1007/s12528-022-09340-3
[18]
Nithin Rao Koluguri, Taejin Park, and Boris Ginsburg. 2022. TitaNet: Neural Model for Speaker Representation with 1D Depth-Wise Separable Convolutions and Global Context. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 8102–8106. https://doi.org/10.1109/ICASSP43922.2022.9746806
[19]
Oleksii Kuchaiev, Jason Li, Huyen Nguyen, Oleksii Hrinchuk, Ryan Leary, Boris Ginsburg, Samuel Kriman, Stanislav Beliaev, Vitaly Lavrukhin, Jack Cook, Patrice Castonguay, Mariya Popova, Jocelyn Huang, and Jonathan M. Cohen. 2019. NeMo: a toolkit for building AI applications using Neural Modules. arxiv:1909.09577 [cs.LG]
[20]
Oren Lederman, Akshay Mohan, Dan Calacci, and Alex Sandy Pentland. 2018. Rhythm: A Unified Measurement Platform for Human Organizations. IEEE MultiMedia 25, 1 (2018), 26–38. https://doi.org/10.1109/mmul.2018.112135958
[21]
Roberto Martinez-Maldonado, Vanessa Echeverria, Gloria Fernandez-Nieto, Lixiang Yan, Linxuan Zhao, Riordan Alfredo, Xinyu Li, Samantha Dix, Hollie Jaggard, Rosie Wotherspoon, 2023. Lessons learnt from a multimodal learning analytics deployment in-the-wild. ACM Transactions on Computer-Human Interaction (2023).
[22]
Xavier Ochoa. 2022. Multimodal systems for automated oral presentation feedback: A comparative analysis. In The Multimodal Learning Analytics Handbook. Springer, 53–78.
[23]
Xavier Ochoa, Simon Knight, and Alyssa Friend Wise. 2020. Learning Analytics Impact: Critical Conversations on Relevance and Social Responsibility. Journal of Learning Analytics 7, 3 (2020), 1–5. https://doi.org/10.18608/jla.2020.73.1
[24]
Edwin Olson. 2011. AprilTag: A robust and flexible visual fiducial system. In 2011 IEEE International Conference on Robotics and Automation. 3400–3407. https://doi.org/10.1109/ICRA.2011.5979561
[25]
Hamza Ouhaichi, Daniel Spikol, and Bahtijar Vogel. 2023. Research trends in multimodal learning analytics: A systematic mapping study. Computers and Education: Artificial Intelligence (2023), 100136.
[26]
Sambit Praharaj, Maren Scheffel, Hendrik Drachsler, and Marcus Specht. 2021. Literature review on co-located collaboration modeling using multimodal learning analytics—Can we go the whole nine yards?IEEE Transactions on Learning Technologies 14, 3 (2021), 367–385.
[27]
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine Mcleavey, and Ilya Sutskever. 2023. Robust Speech Recognition via Large-Scale Weak Supervision. In Proceedings of the 40th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 202). PMLR, 28492–28518. https://proceedings.mlr.press/v202/radford23a.html
[28]
Despoina Schina, Vanessa Esteve-González, Mireia Usart, José-Luis Lázaro-Cantabrana, and Mercè Gisbert. 2020. The Integration of Sustainable Development Goals in Educational Robotics: A Teacher Education Experience. Sustainability 12, 23 (2020), 10085. https://doi.org/10.3390/su122310085
[29]
Jan Schneider, Daniele Di Mitri, Bibeg Limbu, and Hendrik Drachsler. 2018. Multimodal Learning Hub: A Tool for Capturing Customizable Multimodal Learning Experiences. Springer International Publishing, 45–58. https://doi.org/10.1007/978-3-319-98572-5_4
[30]
Silero Team. 2021. Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier. https://github.com/snakers4/silero-vad
[31]
Vijay K Vaishnavi. 2007. Design science research methods and patterns: innovating information and communication technology. Auerbach Publications.
[32]
Johannes Wagner, Florian Lingenfelser, Tobias Baur, Ionut Damian, Felix Kistler, and Elisabeth André. 2013. The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time. In Proceedings of the 21st ACM international conference on Multimedia. 831–834.
[33]
Alyssa Friend Wise and David Williamson Shaffer. 2015. Why theory matters more than ever in the age of big data. Journal of Learning Analytics 2, 2 (2015), 5–13.
[34]
Shunpei Yamaguchi, Motoki Nagano, Shunpei Ohira, Ritsuko Oshima, Jun Oshima, Takuya Fujihashi, Shunsuke Saruwatari, and Takashi Watanabe. 2022. Web Services for Collaboration Analysis With IoT Badges. IEEE Access 10 (2022), 121318–121328. https://doi.org/10.1109/access.2022.3222562
[35]
Lixiang Yan, Linxuan Zhao, Dragan Gasevic, and Roberto Martinez-Maldonado. 2022. Scalability, Sustainability, and Ethicality of Multimodal Learning Analytics. In LAK22: 12th International Learning Analytics and Knowledge Conference. ACM. https://doi.org/10.1145/3506860.3506862
[36]
Patrick H Zimmerman, J Elizabeth Bolhuis, Albert Willemsen, Erik S Meyer, and Lucas PJJ Noldus. 2009. The Observer XT: A tool for the integration and synchronization of multimodal signals. Behavior research methods 41 (2009), 731–735.
Index Terms
- Field report for Platform mBox: Designing an Open MMLA Platform
Recommendations
Bringing Collaborative Analytics using Multimodal Data to Masses: Evaluation and Design Guidelines for Developing a MMLA System for Research and Teaching Practices in CSCL
LAK '24: Proceedings of the 14th Learning Analytics and Knowledge ConferenceThe Multimodal Learning Analytics (MMLA) research community has significantly grown in the past few years. Researchers in this field have harnessed diverse data collection devices such as eye-trackers, motion sensors, and microphones to capture rich ...
Rethinking MMLA: Design Considerations for Multimodal Learning Analytics Systems
L@S '23: Proceedings of the Tenth ACM Conference on Learning @ ScaleDesigning MMLA systems is a complex task requiring a wide range of considerations. In this paper, we identify key considerations that are essential for designing MMLA systems. These considerations include data management, human factors, sensors and ...
Classifying student dialogue acts with multimodal learning analytics
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And KnowledgeSupporting learning with rich natural language dialogue has been the focus of increasing attention in recent years. Many adaptive learning environments model students' natural language input, and there is growing recognition that these systems can be ...
Comments
Information & Contributors
Information
Published In
March 2024
962 pages
ISBN:9798400716188
DOI:10.1145/3636555
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 March 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
LAK '24
Acceptance Rates
Overall Acceptance Rate 236 of 782 submissions, 30%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 133Total Downloads
- Downloads (Last 12 months)133
- Downloads (Last 6 weeks)42
Reflects downloads up to 10 Aug 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in