The Multimodal Learning Analytics Handbook

This handbook is the first book ever covering the area of Multimodal Learning Analytics (MMLA). The field of MMLA is an emerging domain of Learning Analytics and plays an important role in expanding the Learning Analytics goal of understanding and improving learning in all the different environments where it occurs. The challenge for research and practice in this field is how to develop theories about the analysis of human behaviors during diverse learning processes and to create useful tools that could augment the capabilities of learners and instructors in a way that is ethical and sustainable. Behind this area, the CrossMMLA research community exchanges ideas on how we can analyze evidence from multimodal and multisystem data and how we can extract meaning from this increasingly fluid and complex data coming from different kinds of transformative learning situations and how to best feed back the results of these analyses to achieve positive transformative actions on those learning processes.

This handbook also describes how MMLA uses the advances in machine learning and affordable sensor technologies to act as a virtual observer/analyst of learning activities. The book describes how this “virtual nature” allows MMLA to provide new insights into learning processes that happen across multiple contexts between stakeholders, devices and resources. Using such technologies in combination with machine learning, Learning Analytics researchers can now perform text, speech, handwriting, sketches, gesture, affective, or eye-gaze analysis, improve the accuracy of their predictions and learned models and provide automated feedback to enable learner self-reflection. However, with this increased complexity in data, new challenges also arise. Conducting the data gathering, pre-processing, analysis, annotation and sense-making, in a way that is meaningful for learning scientists and other stakeholders (e.g., students or teachers), still pose challenges in this emergent field. This handbook aims to serve as a unique resource for state of the art methods and processes.

Chapter 11 of this book is available open access under a CC BY 4.0 license at link.springer.com.

Introduction to Multimodal Learning Analytics.- Multimodal Learning Analytics and the Design of Learning Spaces.- Multimodal Systems for Automated Oral Presentation Feedback: A Comparative Analysis.- Modeling the Complex Interplay Between Monitoring Events for Regulated Learning with Psychological Networks.- The role of metacognition and self-regulation on clinical reasoning: Leveraging multimodal learning analytics to transform medical education.- Intermodality in Multimodal Learning Analytics for Cognitive Theory Development: A Case from Embodied Design for Mathematics Learning.- Bridging the Gap between Informal Learning Pedagogy and Multimodal Learning Analytics.- Multimodal Learning Experience for Deliberate Practice.- CDM4MMLA: Contextualized Data Model for MultiModal Learning Analytics.- A Physiology-Aware Learning Analytics Framework.- Once more with Feeling - Emotions in Multimodal Learning Analytics.- The evidence of impact and ethical considerations of Multimodal Learning Analytics: A Systematic Literature Review.- Sensor-based analytics in education: Lessons learned from research in Multimodal Learning Analytics.- Framing the future of Multimodal Learning Analytics.