Evaluation of a collaborative reading annotation system through multimodal data analysis

Technological advances and the explosion of epidemics have contributed to a surge in the number of online learning platforms. Because single modal data is often not enough in evaluating the usability of interface interaction design for online learning platforms and multimodal data (eye movement, Electroencephalogram, skin conductance response) with advanced sensing technologies provide new possibilities to address this issue, this case study explores how multimodal data can be used to evaluate the interface design for our self-developed collaborative reading system. The results of our randomized between-subject experiments showed that, from eye movement analysis, constructive-level annotations prompt students to allocate more attention to the annotation area than active level annotations and facilitate the transition between the annotation and reading areas. From EEG data analysis, all the students stayed high concentration levels no matter the types of annotations they were reading. From SCRs analysis, although no significant difference in the level of excitement between experimental conditions was identified, students showed great individual difference within the same conditions. This study illustrated how multimodal data can be applied to interface design evaluation.