The Role of Eye Gaze and Body Movements in Turn-Taking during a Contemporary Dance Improvisation Vito Evola, Joanna Skubisz and Carla Fernandes BlackBox Project, Faculdade de Ciências Sociais e Humanas, Universidade Nova de Lisboa Humans regulate their contributions in social interactions using practices, norms, and rules depending on the nature of their exchanges (inter alia Sacks, Schegloff & Jefferson 1974), whether it be by using prosody to solicit a reply to a question or understanding who goes next around the table during a hand at poker. Understanding how humans manage turns, and “the floor”, provides a deeper insight into how humans dynamically interact in various social contexts, which is of interest to linguists, sociologists, anthropologists, and computer scientists among others. Previous research has sufficiently investigated the phenomenon in conversational groups (Petukhova & Bunt 2009; Bohus & Horvitz 2011) but mainly between dyads (Kendon 1967; Ochs et al. 1996), describing various strategies deployed (e.g. primarily via speech [Duncan 1972; Ward & Tsukahara 2000], but also eye gaze [Bavelas 2005; Peters et al. 2005; Brône et al. 2013]; bodily motions [Mondada 2007]). The present study intends to contribute to the multimodal turn-taking literature by presenting data collected in an improvisation session in the context of the performing arts and its quali-quantitative analysis, where the focus is on how the body participates in the interaction (rather than the more traditional conversation-focused corpus of analyses). Five expert performers joined Portuguese contemporary choreographer, João Fiadeiro, in practicing his Composição em Tempo Real Method (CTR, or Real-Time Composition; Fiadeiro 2007). Fiadeiro created the so-called “CTR Game” in 1995, and has been systematizing it since then, in order to provide choreographers and performers a methodological tool for composing artistic works through exercising mindful decision-making. Performers take turns in building on what other performers generate in a delimited space in the studio, following a process of creating relations with previous actions. Although his method invites performers to use their bodies on a stage floor, he also uses a variation using objects on a table. As the performers sit around the table, through means of self-selection, they perform a single action at a time on the table with the objects to develop compositions. Creative and innovative ideas for stage compositions and performances are generated collaboratively through what emerges throughout the Game. Unlike previous studies on turn-taking, the context of this inquiry is linguistically independent, where practitioners do not talk during the improvisation act (unless their speech is being used as material), and there are no “turns” (performers are free to choose to perform or not, but only a single action at a time, and not twice in a row). Nonetheless, there is social communication: turns are regulated by the information “given” (e.g. moving towards the action) and information “given off” (e.g. via gaze or bodily motions; Goffman 1963) only visually by means of their bodies. Although a silent performance, CTR, like language, is a symbolic system, where the purpose of each action is “understood to be understood” across practitioners. It is a socially organized and coordinated activity situated in a shared perceptual space, which can be manipulated (Clark 2005). It involves “embodied minds” in a process of world-making (Goodman 1978). However, unlike language, the turns in CTR are only self-selected, and there is no explicit communication involved. From a semiotic perspective, the Game is meant to be based on iconic and indexical relations, where symbols, narration and narratives are excluded. One 2.5-hour session of the CTR Game, elaborated by the six expert CTR performers (including the choreographer), was recorded and coded for study. A macro-analysis of the session indicates a set of gestural strategies deployed by the performers in deciding what to do next and exactly when to do it, and in a socially coordinated way without spoken language; in this sense, this study suggests to breakdown the notion of “performers’ intuition” in terms consonant with Cognitive Science. A micro-analysis of portions of the session was conducted using the ELAN video annotation tool (Lausberg & Sloetjes 2009). The annotation scheme codes for: a) directedness behavior (spatial location and orientation of the body, gaze points, object interaction); b) a formal description of movement units, or MUs (i.e. gestural complexes marked by the distinct change of the articulator’s configuration or position in space) of the head/face, upper-body, and lower-body articulators; and c) a hermeneutic tier categorizing the functional-semiotic interpretation of the MUs (following a hierarchical taxonomy: self-focused, context-focused; communication-focused). The first two levels of annotation have an objective quality, whereas the last level, based on the previous ones, describes raters’ subjective interpretation of the performers’ movements before, during and after their actions. According to high inter-rater agreement, we extrapolate that attentive co-participants interpreted each other’s movements in a similar way. Unlike in more common, everyday social interactions, we found that intersubjectivity was avoided during this performance of Contemporary Dance Improvisation, both in the performers’ bodily movements and mutual gaze; we extrapolate that peripheral vision (e.g. Gullberg & Holmqvist 1999) was chiefly deployed as a regulating strategy by these experts during the performance to coordinate turn-taking. The data provides zero cases of communication-focused movements. Although contextand communication-focused movements were controlled, self-focused movements seemed less monitored and were in fact overwhelmingly present, a further indication that these bodily movements are produced as neurophysiological responses to a cognitive load. Furthermore, we identified a class of gestures occurring in decision-making contexts that we have dubbed “precursory gestures”, and we describe the anatomy, function and timing of these bodily movements. The subjects’ control of their gestures and gazes will be further examined within the framework of practice theory. Finally, from a methodological perspective, we argue for using Cohen’s kappa (notwithstanding its shortcomings) and contingency tables as a mean to correct for “annotator fatigue”, and their importance not only to calculate inter-rater agreement, but as a tool to achieve inter-rater reliability by highlighting gross errors. The results of these analyses and their implications for computational modeling of turns in the context of multimodality, as well as the relevance with questions of embodiment, creativity, and performance will be discussed. Preliminary findings contrasting the results of this group with a group of expert performers having irrelevant dance improvisation expertise and a third group of nonperformers will also be described. Works Cited Bavelas, J. B. (2005). Appreciating face-to-face dialogue. In Auditory-Visual Speech Processing, 1. Bohus, D., & Horvitz, E. (2011). Multiparty turn taking in situated dialog: Study, lessons, and directions. In Proceedings of the SIGDIAL 2011 Conference, 98-10. Brône, G., Feyaerts, K., & Oben, B. (2013). On the interplay between eye gaze and speech. In Aflico V. Clark, H. H. (2005). Coordinating with each other in a material world. Discourse Studies, 7(4-5), 507-525. Duncan, S. (1972). Some signals and rules for taking speaking turns in conversations. Journal of Personality and Social Psychology, (23), 283-292. Fiadeiro, J. (2007). Wenn Du das nicht weißt, warum fragst Du dann? In: Gehm, S., Husemann, P., & von Wilke, K. (eds.). Wissen in Bewegung. Perspektiven der künstlerischen und wissenschaftlichen Forschung im Tanz, 103-112. Bielefeld: transcript Verlag. Goffman, E. (1963). Behavior in Public Places. New York: Free Press. Goodman, N. (1978). Ways of Worldmaking. Indianapolis: Hackett Publishing. Gullberg, M., & Holmqvist, K. (1999). Keeping an Eye on Gestures: Visual Perception of Gestures in Face-toFace Communication. Pragmatics & Cognition 7, 35-63. Kendon, A. (1967). Some functions of gaze direction in social interaction. Acta Psychologica, (26), 22-63. Lausberg, H., & Sloetjes, H. (2009). Coding gestural behavior with the NEUROGES-ELAN system. Behavior Research Methods, Instruments, & Computers, 41(3), 841-849. Mondada, L. (2007). Multimodal resources for turn-taking: pointing and emergence of possible next speakers. Discourse Studies, 194-225. Ochs, E., Schegloff, E. A., & Thompson, S. A. (1996). Interaction and grammar (Vol. 13). Cambridge: Cambridge University Press. Peters, C., Pelachaud, C., Bevacqua, E., Mancini, M., & Poggi, I. (2005). A model of attention and interest using gaze behavior. In Proceedings of the 5th International Working Conference on Intelligent Virtual Agents, 229-240. Petukhova, V., & Bunt, H. (2009). Who’s next? Speaker-selection mechanisms in multiparty dialogue. In: Proceedings of the Workshop on the Semantics and Pragmatics of Dialogue, 19-26. Sacks, H., Schegloff, E. A., & Jefferson, G. (1974). A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language, 50(4), 696-735. Ward, N., & Tsukahara, W. (2000). Prosodic features which cue back-channel responses in English and Japanese. Journal of pragmatics, 32(8), 1177-1207.