- University of Genova, DIBRIS - Department of Computer Science, Bioengineering, Robotics and Systems Engineering, Department MemberUniversità degli Studi di Milano - State University of Milan (Italy), Cognitive Science, Graduate StudentUniversità degli Studi di Milano - State University of Milan (Italy), Philosophy, Department Memberadd
- Neuroscience, Cognitive Science, Cognitive Psychology, Philosophy of cognitive neuroscience, Philosophy of Mind, Philosophy of Science, and 13 moreCognitive Neuroscience, Pattern Recognition, Affective Computing, Computer Science, Computer Science and Engineering, Computer Arts, Media Arts & Computer Science, Computer Music, Human Computer Interaction, Brain-Computer Music Interfaces, Music Technology, Music, and performing art and technologyedit
- Postdoc researcher, based in Genoa, Italy, I am a cognitive scientist with a Ph.D. in Computer Science. My research ... morePostdoc researcher, based in Genoa, Italy, I am a cognitive scientist with a Ph.D. in Computer Science.
My research interests cover multisensory and cross-modal integration and sonification design, particularly applied to multimodal and interactive technologies, for education and rehabilitation purposes and music therapy.
In last years, my focus has been mainly devoted on developmental age and sonification strategies to improve learning outcomes and executive functions development in both typical and sensory impaired users.edit
Learning to play a musical instrument is a difficult task, mostly based on the master-apprentice model. Technologies are rarely employed and are usually restricted to audio and video recording and playback. Nevertheless, multimodal... more
Learning to play a musical instrument is a difficult task, mostly based on the master-apprentice model. Technologies are rarely employed and are usually restricted to audio and video recording and playback. Nevertheless, multimodal interactive systems can complement actual learning and teaching practice, by offering students guidance during self-study and by helping teachers and students to focus on details that would be otherwise difficult to appreciate from usual audiovisual recordings. This paper introduces a multimodal corpus consisting of the recordings of expert models of success, provided by four professional violin performers. The corpus is publicly available on the repoVizz platform, and includes synchronized audio, video, motion capture, and physiological (EMG) data. It represents the reference archive for the EU-H2020-ICT Project TELMI, an international research project investigating how we learn musical instruments from a ped-agogical and scientific perspective and how to develop new interactive, assistive, self-learning, augmented-feedback, and social-aware systems to support musical instrument learning and teaching.
Research Interests:
Recent results from psychophysics and developmental psychology show that children have a preferential sensory channel to learn specific concepts. In this work, we explore the possibility of developing and evaluating novel multisensory... more
Recent results from psychophysics and developmental psychology show that children have a preferential sensory channel to learn specific concepts. In this work, we explore the possibility of developing and evaluating novel multisensory technologies for deeper learning of arithmetic and geometry. The main novelty of such new technologies comes from the renewed understanding of the role of communication between sensory modalities during development that is that specific sensory systems have specific roles for learning specific concepts. Such understanding suggests that it is possible to open a new teaching/learning channel, personalized for each student based on the child's sensory skills. Multisensory interactive technologies exploiting full-body movement interaction and including a hardware and software platform to support this approach will be presented and discussed. The platform is part of a more general framework developed in the context of the EU-ICT-H2020 weDRAW Project that aims to develop new multimodal technologies for multisensory serious-games to teach mathematics concepts in the primary school.
Research Interests:
This paper discusses self-efficacy, curiosity, and reflectivity as cognitive and affective states that are critical to learning but are overlooked in the context of affect-aware technology for learning. This discussion sits within the... more
This paper discusses self-efficacy, curiosity, and reflectivity as cognitive and affective states that are critical to learning but are overlooked in the context of affect-aware technology for learning. This discussion sits within the opportunities offered by the weDRAW project aiming at an embodied approach to the design of technology to support exploration and learning of mathematical concepts. We first review existing literature to clarify how the three states facilitate learning and how, if not supported, they may instead hinder learning. We then review the literature to understand how bodily expressions communicate these states and how technology could be used to monitor them. We conclude by presenting initial movement cues currently explored in the context of weDRAW.
Research Interests:
Multisensory learning is considered a relevant pedagogical framework for education since a very long time and several authors support the use of a multisensory and kinesthetic approach in children learning. Moreover, results from... more
Multisensory learning is considered a relevant pedagogical framework for education since a very long time and several authors support the use of a multisensory and kinesthetic approach in children learning. Moreover, results from psychophysics and developmental psychology show that children have a preferential sensory channel to learn specific concepts (spatial and/or temporal), hence a further evidence for the need of a multisensory approach. In this work, we present an example of serious game for learning a particularly complicated mathematical concept: fractions. The main novelty of our proposal comes from the role covered by the communication between sensory modalities in particular, movement, vision, and sound. The game has been developed in the context of the EU-ICT-H2020 weDRAW Project aiming at developing new multimodal technologies for multisensory serious-games on mathematical concepts for primary school children.
Research Interests:
Violin performance is characterized by an intimate connection be-tween the player and her instrument that allows her a continuous control of sound through a sophisticated bowing technique. A great importance in violin pedagogy is, then,... more
Violin performance is characterized by an intimate connection be-tween the player and her instrument that allows her a continuous control of sound through a sophisticated bowing technique. A great importance in violin pedagogy is, then, given to techniques of the right hand, responsible of most of the sound produced. This study analyses the bowing trajectory in three different classical violin exercises from audio and motion capture recordings to classify, using ma-chine learning techniques, the different kinds of bowing techniques used. Our results show that a clustering algorithm is able to appropriately group together the different shapes produced by the bow trajectories.
Research Interests:
Recent results from psychophysics and developmental psychology show that children have a preferential sensory channel to learn specific concepts (spatial and/or temporal). At school, the visual channel is often the one most frequently... more
Recent results from psychophysics and developmental psychology show that children have a preferential sensory channel to learn specific concepts (spatial and/or temporal). At school, the visual channel is often the one most frequently exploited for teaching, and the other channels are left a marginal role only. However, the visual signal is not always the more powerful and effective channel for perception. In this work we explore the possibility of creating and evaluating a new methodology to teaching and a novel technology for deeper learning of arithmetic (time) and geometry (space). The main novelty of such a new technology comes from the renewed understanding of the role of communication between sensory modalities during development, that is, that specific sensory systems have specific roles for learning specific concepts. Results suggest that it is possible to open a new teaching/learning channel, personalized for each student based on child sensory skills. Multisensory interactive technology including a hardware and software platform to support this approach will be presented and discussed.
Research Interests:
Research Interests:
Research Interests:
Musical communication can be considered as a form of social interaction and it requires a certain degree of inter-individual cortical synchronization. Scientific literature has shown that visual experience is necessary for social... more
Musical communication can be considered as a form of social interaction and it requires a certain degree of inter-individual cortical synchronization. Scientific literature has shown that visual experience is necessary for social interaction and visual impairment negatively influences nonverbal communication both in children and adults. In this study, we present a pilot multimodal dataset to investigate the role that visual experience plays in the context of musical interaction. We selected a mixed interaction scenario (i.e., sighted and blind performers) to investigate non-verbal communication patterns. In particular, we recorded motion tracking data to analyze the interaction between two musicians in three different contexts with the same cellist as soloist accompanied by a (1) blind pianist, (2) sighted pianist, and (3) sighted but blindfolded pianist. Recordings comprised upper-body motion capture and audio data. We also investigated human perception of changes in non-verbal beh...
A poster abstract for the Consortium of European Research on Emotion (CERE) 2018.
Research Interests:
Psychophysical and developmental psychology evidence shows that children have a preferential sensory channel to learn specific concepts, highlighting the need of a multisensory approach to education. Despite such a potential for a... more
Psychophysical and developmental psychology evidence shows that children have a preferential sensory channel to learn specific concepts, highlighting the need of a multisensory approach to education. Despite such a potential for a positive impact, in education technology research multisensory learning has been often penalised and limited by design choices. In this work, we present the requirements and the design of an implementable prototype of serious-game aimed at teaching children a precise mathematical concept: angles. The game is based on the interaction between different sensory modalities, in particular movement, vision, and sound.
Research Interests:
Research Interests:
e-Health technologies may potentially improve healthcare assistance for frail elderly subjects. However, despite the interest and potential benefits of the field, research highlighted that several challenges remain, such as scarce wide... more
e-Health technologies may potentially improve healthcare assistance for frail elderly subjects. However, despite the interest and potential benefits of the field, research highlighted that several challenges remain, such as scarce wide spreading of information and communication technology technologies among the elderly and infrastructural difficulties. The aim of this survey was to assess to what extent digital technologies are widespread and used among Italian geriatricians. We conducted an online survey addressed to geriatricians, from the SIGOT (The Italian Geriatric Society) website, collecting the 78 completed questionnaires. We then analyzed with descriptive statistics the data. Results showed a still scarce use of digital technologies in geriatric practice in Italy. However, more than 60% of the SIGOT geriatricians who participated in the survey responded that the use of digital technology had increased significantly due to COVID-19 pandemic and for 80% of the members, the pr...
Research Interests:
Learning to playa music instrument is a complex task, requiring continuous practice and the development of sophisticated motor control techniques. The traditional model of music learning is based on a master-apprentice relationship,... more
Learning to playa music instrument is a complex task, requiring continuous practice and the development of sophisticated motor control techniques. The traditional model of music learning is based on a master-apprentice relationship, leading often to a solitary learning process, in which the time spent with the teacher is usually limited to weekly lessons and a long period of self-study is needed. Moreover, a large amount of time passes from the teacher's feedback and the student's proprioceptive perception while studying, requiring a big effort in developing an efficient and healthy technique. In this paper, we present our recent developments concerning an assistive and adaptive technology to help violin students overcoming all these difficulties, and developing their technique and repertoire properly and sefely. In particular, we focus on the multimodal corpus of violin performances which was collected for the purpose, and on the analysis of such data to compute postural and gestural features characterizing the performance under a biomechanical perspective and in terms of movement quality. Analysis is expected to provide students with feedback for reaching a physically accurate performance, maximizing efficiency and minimizing injuries.
Research Interests:
Sensory cues enable navigation through space, as they inform us about movement properties, such as the amount of travelled distance and the heading direction. In this study, we focused on the ability to spatially update one's position... more
Sensory cues enable navigation through space, as they inform us about movement properties, such as the amount of travelled distance and the heading direction. In this study, we focused on the ability to spatially update one's position when only proprioceptive and vestibular information is available. We aimed to investigate the effect of yaw rotation on path integration across development in the absence of visual feedback. To this end, we utilized the triangle completion task: participants were guided through two legs of a triangle and asked to close the shape by walking along its third imagined leg. To test the influence of yaw rotation across development, we tested children between 6 and 11 years old (y.o.) and adults on their perceptions of angles of different degrees. Our results demonstrated that the amount of turn while executing the angle influences performance at all ages, and in some aspects, also interacted with age. Indeed, whilst adults seemed to adjust their heading towards the end of their walked path, younger children took less advantage of this strategy. The amount of disorientation the path induced also affected participants' full maturational ability to spatially navigate with no visual feedback. Increasing induced disorientation required children to be older to reach adult-level performance. Overall, these results provide novel insights on the maturation of spatial navigation-related processes.
Research Interests:
Demographers and criminologists from the Netherlands, US, Norway, UK and Italy will discuss and compare their findings cross-national to assess the effects that offending can have on issues such as marriage chances, mortality and the... more
Demographers and criminologists from the Netherlands, US, Norway, UK and Italy will discuss and compare their findings cross-national to assess the effects that offending can have on issues such as marriage chances, mortality and the chance to become a parent. Thereby, they will not only assess the influence criminal behavior as on an individual’s own likelihood of experiencing demographic transitions, but also take into account intergenerational effects: how criminal behavior of the parents may affect their children’s demographic transitions.
Research Interests:
The rising popularity of learning techniques in data analysis has recently led to an increased need of large-scale datasets. In this study, we propose a system consisting of a VR game and a software platform designed to collect the... more
The rising popularity of learning techniques in data analysis has recently led to an increased need of large-scale datasets. In this study, we propose a system consisting of a VR game and a software platform designed to collect the player’s multimodal data, synchronized with the VR content, with the aim of creating a dataset for emotion detection and recognition. The game was implemented ad-hoc in order to elicit joy and frustration, following the emotion elicitation process described by Roseman’s appraisal theory. In this preliminary study, 5 participants played our VR game along with pre-existing ones and self-reported experienced emotions.
Research Interests:
Research Interests:
Recent results from psychophysics and developmental psychology show that children do not integrate and use the same sensory modalities in the same way, but they rather have a preferential sensory channel to learn specific concepts. In... more
Recent results from psychophysics and developmental psychology show that children do not integrate and use the same sensory modalities in the same way, but they rather have a preferential sensory channel to learn specific concepts. In schools, however, the visual channel is often the only one exploited for teaching, whereas the other channels are left to a marginal role. The weDRAW project, an ongoing European two-years project, explores the chance to create and evaluate new methodologies to teaching, grounded on multisensory technologies. In particular, the project targets a deeper understanding of arithmetic and geometry. A major novelty of such a new technology is that it is based on the renewed understanding of what communication between senses is and how it occurs during child development, that is, that specific sensory systems have specific roles for learning specific concepts. Neuroscientific prospectives suggest that it is possible to develop new teaching/learning channels, ...
For technology (like serious games) that aims to deliver interactive learning, it is important to address relevant mental experiences such as reflective thinking during problem solving. To facilitate research in this direction, we present... more
For technology (like serious games) that aims to deliver interactive learning, it is important to address relevant mental experiences such as reflective thinking during problem solving. To facilitate research in this direction, we present the weDraw-1 Movement Dataset of body movement sensor data and reflective thinking labels for 26 children solving mathematical problems in unconstrained settings where the body (full or parts) was required to explore these problems. Further, we provide qualitative analysis of behaviours that observers used in identifying reflective thinking moments in these sessions. The body movement cues from our compilation informed features that lead to average F1 score of 0.73 for binary classification of problem-solving episodes by reflective thinking based on Long Short-Term Memory neural networks. We further obtained 0.79 average F1 score for end-to-end classification, i.e. based on raw sensor data. Finally, the algorithms resulted in 0.64 average F1 score ...
Research Interests:
The main purpose of my work is to investigate multisensory and multimodal integration in the design and development of adaptive systems and interfaces for game-based learning applications in the areas of education and rehabilitation. To... more
The main purpose of my work is to investigate multisensory and multimodal integration in the design and development of adaptive systems and interfaces for game-based learning applications in the areas of education and rehabilitation. To this aim, I contributed to the creation of a multimodal dataset of violin performances, integrating motion capture, video, audio, and on-body sensors (accelerometers and EMG), and I worked closely with psychophysicists and educators on the design of paradigms and technologies for multisensory and embodied learning of mathematics in primary school children. Main theoretical foundations of my research are multisensory processing and integration, psychophysics analysis, embodied cognition theories, computational models of non-verbal and emotion communication in full-body movement, and human-computer interaction models for adaptive interfaces and serious-games.
Research Interests:
This paper investigates how unitizing affects external observers' annotation of group cohesion. We compared unitizing techniques belonging to these categories: interval coding, continuous coding, and a technique inspired by a... more
This paper investigates how unitizing affects external observers' annotation of group cohesion. We compared unitizing techniques belonging to these categories: interval coding, continuous coding, and a technique inspired by a cognitive theory on event perception. We applied such techniques for sampling coding units from a set of recordings of social interactions rich in behaviors related to cohesion. Then, we compared the cohesion scores the observers assigned to each coding unit. Results show that the three techniques can lead to suitable ratings and that the technique inspired to cognitive theories leads to scores reflecting variability in cohesion better than the other ones.
Research Interests:
Research Interests:
Recent scientific results show that audio feedback associated with body movements can be fundamental during the development to learn new spatial concepts [1], [2]. Within the weDraw project [3], [4], we have investigated how this link can... more
Recent scientific results show that audio feedback associated with body movements can be fundamental during the development to learn new spatial concepts [1], [2]. Within the weDraw project [3], [4], we have investigated how this link can be useful to learn mathematical concepts. Here we present a study investigating how mathematical skills changes after multisensory training based on human-computer interaction (RobotAngle and BodyFraction activities). We show that embodied angle and fractions exploration associated with audio and visual feedback can be used in typical children to improve cognition of spatial mathematical concepts. We finally present the exploitation of our results: an online, optimized version of one of the tested activity to be used at school. The training result suggests that audio and visual feedback associated with body movements is informative for spatial learning and reinforces the idea that spatial representation development is based on sensory-motor interactions.
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Learning to play and perform a music instrument is a complex cognitive task, requiring high conscious control and coordination of an impressive number of cognitive and sensorimotor skills. For professional violinists, there exists a... more
Learning to play and perform a music instrument is a complex cognitive task, requiring high conscious control and coordination of an impressive number of cognitive and sensorimotor skills. For professional violinists, there exists a physical connection with the instrument allowing the player to continuously manage the sound through sophisticated bowing techniques and fine hand movements. Hence, it is not surprising that great importance in violin training is given to right hand techniques, responsible for most of the sound produced. In this paper, our aim is to understand which motion features can be used to efficiently and effectively distinguish a professional performance from that of a student without exploiting sound-based features. We collected and made freely available a dataset consisting of motion capture recordings of different violinists with different skills performing different exercises covering different pedagogical and technical aspects. We then engineered peculiar fe...