The CorpAGEst project aims to study the pragmatic competence of very old people (75 y. old and mo... more The CorpAGEst project aims to study the pragmatic competence of very old people (75 y. old and more), by looking at their use of verbal and gestural pragmatic markers in real-world settings (vs. laboratory conditions). More precisely, we hypothesize that identifying multimodal pragmatic patterns in language use, as produced by older adults at the gesture-speech interface, helps to better characterize language variation and communication abilities in later life. The underlying assumption is that discourse markers (e.g., tu sais 'you know') and pragmatic gestures (e.g., an exaggerated opening of the eyes) are relevant indicators of stance in discourse. The present paper's objective is mainly methodological. It aims to demonstrate how the pragmatic profile of older adults can be established by analyzing audio and video data. After a brief theoretical introduction, we describe the annotation protocol that has been developed to explore issues in multimodal pragmatics and aging. Lastly, first results from a corpus-based study are given, showing how multimodal approaches can tackle important aspects of communication abilities, at the crossroads of language and aging research in linguistics.
Pragmatics is concerned with the relation of linguistic items to their context of utterance, thei... more Pragmatics is concerned with the relation of linguistic items to their context of utterance, their users and interpreters, that is, with context-dependent aspects of meaning. Studies in pragmatics also recognize that non-verbal communication mechanisms (including gestures), alongside verbal aspects of communication, are at the core of the creation of meaning in the interaction context (Payrató, 2009: 175). Given this context-sensitive and multimodal view of pragmatics, ‘pragmatic gestures’ will be defined here as formally heterogeneous, multifunctional, (mostly) non-representational and (often) unintentional visible actions in language interaction, which express “aspects of utterance structure, including the status of discourse segments with respect to one another, and the character of the ‘speech act’ or interactional move of the utterance” (Kendon, 1995: 247). The aim of the present talk is to answer the following question: To what extent non-representational gestures with a pragm...
The medium we use to communicate (oral, written, or even gestural) plays an important role in the... more The medium we use to communicate (oral, written, or even gestural) plays an important role in the way we structure and organize our discourse. To do this, we can draw on linguistic markers, such as connectives, discourse markers or frame markers, or on (marked) information structure constructions. What is the impact of the nature of the medium (spoken vs. written vs. gestural) and of the style of the discourse at hand (formal vs. informal) on the choice of one linguistic expression over the other? While medium seems to play a role in the discrimination between text types (e.g., casual coffee conversation between colleagues, business meeting, e-novel), it is less clear what the potential impact is of extra-linguistic parameters, such as emotional weight or spatiotemporal distance between the interlocutors, on the structuring of those texts. These questions bring us face to face with the limits of the traditional dichotomic representation opposing speech and writing on the sole basis ...
Taxonomies and categorizing models are numerous, but often criticized for their lack of cognitive... more Taxonomies and categorizing models are numerous, but often criticized for their lack of cognitive realism (see Arppe et al. 2010). Still, a large body of research grounded in cognitive theories advocates for valid categories that reflect internal processing (Taylor 2003). Adequate categories are especially useful for the comparison of languages and also, in our case, of modalities, by providing valid tertia comparationes, which apply to different language systems. Our contribution to the field is a revised taxonomy of functional categories in discursive pragmatics (Zienkowski et al. 2011), after critical discussion of existing models and confrontation to authentic data. The purpose is to show how far different empirical methods can combine and help validate theory-driven functional categories across languages (spoken English vs. French) and across modalities (speech vs. gesture): “[c]ategorization, the process by which distinguishable objects or events are treated equivalently, is a...
Dans cette communication, nous nous proposons d'étudier, par le biais d'une analyse sys... more Dans cette communication, nous nous proposons d'étudier, par le biais d'une analyse systématique de données de corpus en diachronie, l'évolution de la locution conjonctive vu que depuis le français préclassique jusqu'au français contemporain. Pour ce faire, nous nous appuierons, d'une part, sur les études préexistantes ayant trait à l'évolution des conjonctions en français (Bat-Zeev Shyldkrot & Kemmer 1988, Bertin 2003) et, d'autre part, sur les études relevant du domaine de la grammaticalisation (Marchello-Nizia 2006).
L’objet de cette communication porte sur l’annotation multimodale et, en particulier, sur l’évalu... more L’objet de cette communication porte sur l’annotation multimodale et, en particulier, sur l’évaluation des outils et méthodes appliqués au corpus CorpAGEst (Corpus sur le langage mimogestuel des personnes âgées). Actuellement en cours de constitution, le corpus se compose d’entretiens filmés semi-dirigés en face à face avec une personne très âgée (> 75 ans) vivant à domicile ou en maison de repos. Ayant une visée ouvertement écologique (Chafe 1992), il vise à refléter l’usage langagier des personnes très âgées appréhendées dans leur milieu naturel, autrement dit dans leur environnement de vie quotidienne. La constitution du corpus multimodal CorpAGEst s’inscrit dans le cadre plus large d’un projet de recherche portant sur l’étude de la compétence pragmatique (Kasher 1991; Rose & Kasper 2002) des personnes très âgées saines. Ce projet implique le développement d’une méthode d’annotation pragmatique (Prévot et al. 2010) et multimodale (Mondada 2006, 2007) de l’interaction, qui perm...
Starting from the common observation that there is no recognized closed class of Discourse Marker... more Starting from the common observation that there is no recognized closed class of Discourse Markers (DMs) and that their definition may vary from one theoretical framework to another, the aim of the MDMA project (“Model for Discourse Marker Annotation”) is to establish an empirical method for the identification and annotation of DMs in spoken French. Central to our proposal is that DMs may be described as clusters of features that, in specific patterns of combination, allow distinguishing between more or less prototypical uses of DMs in context. We proceeded in three steps : (i) manual identification of all so-called “potential” DMs in a balanced corpus of spoken French (5,000 words; Belgium and France) ; (ii) automatic extraction from the corpus of every token corresponding to the DMs candidates previously identified (1,181 tokens) ; and (iii) parameter analysis of a random sample of 200 potential DMs (syntactic, formal and semantic-pragmatic variables). The hypothesis is that the statistical analysis – based on the distributional constraints of the potential DMs at stake - should uncover a certain hierarchy between the different features under scrutiny, regarding their relevance, reliability, and generalizability (or even specificity). In the present paper, we first present the annotation procedure, then we discuss several aspects of inter-rater agreement, and finally discuss the results from the in-depth corpus-based and statistical analyses.
Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication (August 6-8, Tartu, Estonia), 2015
This pilot study aims at reconstructing the empathic profile of conversation participants from th... more This pilot study aims at reconstructing the empathic profile of conversation participants from their inter-action in real-world settings. It addresses the question of how verbal and nonverbal modes converge in conveying information about the emotional and attitudinal behavior in everyday communication. In par-ticular, the empathic ability of older people is explored studying physiological patterning from nonver-bal resources, in relation to emotions expressed through the face. In addition, the IRI psychometric test of empathy provides the participant’s overall empathic profile. The data is taken from the CorpAGEst multimodal corpus, and focuses on the language of four healthy very old women who obtained a normal score on the MoCA cognitive test. Preliminary results indicate that, despite the highly idiosyncratic use of nonverbal resources, some inter- and intra-individual tendencies seem to emerge.
Yearbook of Phraseology 5, 25-56. (ISSN: 1868-6338), 2014
The assumption made in the present paper is that phraseology and grammaticalization are distinct ... more The assumption made in the present paper is that phraseology and grammaticalization are distinct areas, which nevertheless have many linguistic criteria in common. To support this view, particular attention is paid to parentheticals that are characterized by mutual association between their lexical constituents and grammatical patterning. More particularly, the paper is devoted to the constructional approach of (reduced) parenthetical clauses in present-day French, with a focus on the tu vois ‘you see’ construction. Parentheticals are defined here in terms of synchronic gradience, according to their degree of association strength observable in present-day use, and in terms of diachronic gradualness, resulting from successive evolving steps in language change. Questioning the nature of the link that exists between the various processes of language change at play for the construction, the concept of functionalization is also introduced, in line with emergentist and lexico-grammatical view of language.
M. Van Peteghem, P. Lauwers, E. Tobback, A. Demol & L. De Wilde (eds.), Le verbe en verve. Réflexions sur la syntaxe et la sémantique verbales. Gent: Academia Press, 611-634, 2012
V. Conti, G. Corminbœuf & L. A. Jonhsen (eds.), Entre syntaxe et discours. Eclairages épistémologiques et descriptions linguistiques, 2010, LINX 62-63, 11-39, 2012
F. Neveu, J. Durand, T. Klingler, S. Prévost & V. Muni-Toké (eds.), Actes du CMLF 2010 (2e Congrès Mondial de Linguistique Française, 12-15 July 2010, New Orleans (United States of America), 2010
Partant du postulat selon lequel donc aurait un statut «plurifonctionnel» en situation de discour... more Partant du postulat selon lequel donc aurait un statut «plurifonctionnel» en situation de discours, nous montrerons par le biais d'une analyse de données authentiques comment ce marqueur discursif peut structurer le discours à différents niveaux langagiers. Notre hypothèse de travail est la suivante: donc pourrait acquérir, en situation de discours, des fonctions proprement discursives dépassant sa fonction logique de connecteur conséquentiel.
The CorpAGEst project aims to study the pragmatic competence of very old people (75 y. old and mo... more The CorpAGEst project aims to study the pragmatic competence of very old people (75 y. old and more), by looking at their use of verbal and gestural pragmatic markers in real-world settings (vs. laboratory conditions). More precisely, we hypothesize that identifying multimodal pragmatic patterns in language use, as produced by older adults at the gesture-speech interface, helps to better characterize language variation and communication abilities in later life. The underlying assumption is that discourse markers (e.g., tu sais 'you know') and pragmatic gestures (e.g., an exaggerated opening of the eyes) are relevant indicators of stance in discourse. The present paper's objective is mainly methodological. It aims to demonstrate how the pragmatic profile of older adults can be established by analyzing audio and video data. After a brief theoretical introduction, we describe the annotation protocol that has been developed to explore issues in multimodal pragmatics and aging. Lastly, first results from a corpus-based study are given, showing how multimodal approaches can tackle important aspects of communication abilities, at the crossroads of language and aging research in linguistics.
Pragmatics is concerned with the relation of linguistic items to their context of utterance, thei... more Pragmatics is concerned with the relation of linguistic items to their context of utterance, their users and interpreters, that is, with context-dependent aspects of meaning. Studies in pragmatics also recognize that non-verbal communication mechanisms (including gestures), alongside verbal aspects of communication, are at the core of the creation of meaning in the interaction context (Payrató, 2009: 175). Given this context-sensitive and multimodal view of pragmatics, ‘pragmatic gestures’ will be defined here as formally heterogeneous, multifunctional, (mostly) non-representational and (often) unintentional visible actions in language interaction, which express “aspects of utterance structure, including the status of discourse segments with respect to one another, and the character of the ‘speech act’ or interactional move of the utterance” (Kendon, 1995: 247). The aim of the present talk is to answer the following question: To what extent non-representational gestures with a pragm...
The medium we use to communicate (oral, written, or even gestural) plays an important role in the... more The medium we use to communicate (oral, written, or even gestural) plays an important role in the way we structure and organize our discourse. To do this, we can draw on linguistic markers, such as connectives, discourse markers or frame markers, or on (marked) information structure constructions. What is the impact of the nature of the medium (spoken vs. written vs. gestural) and of the style of the discourse at hand (formal vs. informal) on the choice of one linguistic expression over the other? While medium seems to play a role in the discrimination between text types (e.g., casual coffee conversation between colleagues, business meeting, e-novel), it is less clear what the potential impact is of extra-linguistic parameters, such as emotional weight or spatiotemporal distance between the interlocutors, on the structuring of those texts. These questions bring us face to face with the limits of the traditional dichotomic representation opposing speech and writing on the sole basis ...
Taxonomies and categorizing models are numerous, but often criticized for their lack of cognitive... more Taxonomies and categorizing models are numerous, but often criticized for their lack of cognitive realism (see Arppe et al. 2010). Still, a large body of research grounded in cognitive theories advocates for valid categories that reflect internal processing (Taylor 2003). Adequate categories are especially useful for the comparison of languages and also, in our case, of modalities, by providing valid tertia comparationes, which apply to different language systems. Our contribution to the field is a revised taxonomy of functional categories in discursive pragmatics (Zienkowski et al. 2011), after critical discussion of existing models and confrontation to authentic data. The purpose is to show how far different empirical methods can combine and help validate theory-driven functional categories across languages (spoken English vs. French) and across modalities (speech vs. gesture): “[c]ategorization, the process by which distinguishable objects or events are treated equivalently, is a...
Dans cette communication, nous nous proposons d'étudier, par le biais d'une analyse sys... more Dans cette communication, nous nous proposons d'étudier, par le biais d'une analyse systématique de données de corpus en diachronie, l'évolution de la locution conjonctive vu que depuis le français préclassique jusqu'au français contemporain. Pour ce faire, nous nous appuierons, d'une part, sur les études préexistantes ayant trait à l'évolution des conjonctions en français (Bat-Zeev Shyldkrot & Kemmer 1988, Bertin 2003) et, d'autre part, sur les études relevant du domaine de la grammaticalisation (Marchello-Nizia 2006).
L’objet de cette communication porte sur l’annotation multimodale et, en particulier, sur l’évalu... more L’objet de cette communication porte sur l’annotation multimodale et, en particulier, sur l’évaluation des outils et méthodes appliqués au corpus CorpAGEst (Corpus sur le langage mimogestuel des personnes âgées). Actuellement en cours de constitution, le corpus se compose d’entretiens filmés semi-dirigés en face à face avec une personne très âgée (> 75 ans) vivant à domicile ou en maison de repos. Ayant une visée ouvertement écologique (Chafe 1992), il vise à refléter l’usage langagier des personnes très âgées appréhendées dans leur milieu naturel, autrement dit dans leur environnement de vie quotidienne. La constitution du corpus multimodal CorpAGEst s’inscrit dans le cadre plus large d’un projet de recherche portant sur l’étude de la compétence pragmatique (Kasher 1991; Rose & Kasper 2002) des personnes très âgées saines. Ce projet implique le développement d’une méthode d’annotation pragmatique (Prévot et al. 2010) et multimodale (Mondada 2006, 2007) de l’interaction, qui perm...
Starting from the common observation that there is no recognized closed class of Discourse Marker... more Starting from the common observation that there is no recognized closed class of Discourse Markers (DMs) and that their definition may vary from one theoretical framework to another, the aim of the MDMA project (“Model for Discourse Marker Annotation”) is to establish an empirical method for the identification and annotation of DMs in spoken French. Central to our proposal is that DMs may be described as clusters of features that, in specific patterns of combination, allow distinguishing between more or less prototypical uses of DMs in context. We proceeded in three steps : (i) manual identification of all so-called “potential” DMs in a balanced corpus of spoken French (5,000 words; Belgium and France) ; (ii) automatic extraction from the corpus of every token corresponding to the DMs candidates previously identified (1,181 tokens) ; and (iii) parameter analysis of a random sample of 200 potential DMs (syntactic, formal and semantic-pragmatic variables). The hypothesis is that the statistical analysis – based on the distributional constraints of the potential DMs at stake - should uncover a certain hierarchy between the different features under scrutiny, regarding their relevance, reliability, and generalizability (or even specificity). In the present paper, we first present the annotation procedure, then we discuss several aspects of inter-rater agreement, and finally discuss the results from the in-depth corpus-based and statistical analyses.
Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication (August 6-8, Tartu, Estonia), 2015
This pilot study aims at reconstructing the empathic profile of conversation participants from th... more This pilot study aims at reconstructing the empathic profile of conversation participants from their inter-action in real-world settings. It addresses the question of how verbal and nonverbal modes converge in conveying information about the emotional and attitudinal behavior in everyday communication. In par-ticular, the empathic ability of older people is explored studying physiological patterning from nonver-bal resources, in relation to emotions expressed through the face. In addition, the IRI psychometric test of empathy provides the participant’s overall empathic profile. The data is taken from the CorpAGEst multimodal corpus, and focuses on the language of four healthy very old women who obtained a normal score on the MoCA cognitive test. Preliminary results indicate that, despite the highly idiosyncratic use of nonverbal resources, some inter- and intra-individual tendencies seem to emerge.
Yearbook of Phraseology 5, 25-56. (ISSN: 1868-6338), 2014
The assumption made in the present paper is that phraseology and grammaticalization are distinct ... more The assumption made in the present paper is that phraseology and grammaticalization are distinct areas, which nevertheless have many linguistic criteria in common. To support this view, particular attention is paid to parentheticals that are characterized by mutual association between their lexical constituents and grammatical patterning. More particularly, the paper is devoted to the constructional approach of (reduced) parenthetical clauses in present-day French, with a focus on the tu vois ‘you see’ construction. Parentheticals are defined here in terms of synchronic gradience, according to their degree of association strength observable in present-day use, and in terms of diachronic gradualness, resulting from successive evolving steps in language change. Questioning the nature of the link that exists between the various processes of language change at play for the construction, the concept of functionalization is also introduced, in line with emergentist and lexico-grammatical view of language.
M. Van Peteghem, P. Lauwers, E. Tobback, A. Demol & L. De Wilde (eds.), Le verbe en verve. Réflexions sur la syntaxe et la sémantique verbales. Gent: Academia Press, 611-634, 2012
V. Conti, G. Corminbœuf & L. A. Jonhsen (eds.), Entre syntaxe et discours. Eclairages épistémologiques et descriptions linguistiques, 2010, LINX 62-63, 11-39, 2012
F. Neveu, J. Durand, T. Klingler, S. Prévost & V. Muni-Toké (eds.), Actes du CMLF 2010 (2e Congrès Mondial de Linguistique Française, 12-15 July 2010, New Orleans (United States of America), 2010
Partant du postulat selon lequel donc aurait un statut «plurifonctionnel» en situation de discour... more Partant du postulat selon lequel donc aurait un statut «plurifonctionnel» en situation de discours, nous montrerons par le biais d'une analyse de données authentiques comment ce marqueur discursif peut structurer le discours à différents niveaux langagiers. Notre hypothèse de travail est la suivante: donc pourrait acquérir, en situation de discours, des fonctions proprement discursives dépassant sa fonction logique de connecteur conséquentiel.
Cognitively speaking, repetition can work as a facilitator for both the planning and the understa... more Cognitively speaking, repetition can work as a facilitator for both the planning and the understanding of spoken language. It can work as a cohesive and interactive device or, at a higher-level function of discourse, to engage in interaction by creating interpersonal involvement: repetition always conveys altered meaning, as it has to be re-interpreted by the interlocutor “in light of the accretion, juxtaposition, or expansion” (Tannen 2007: 62). Repetition is thus useful for the building of shared representation and organization of discourse, especially in the elderly’s speech where it may serve as a glue to engage in social interaction (Davis & Maclagan 2014).
In this line, the aim of our talk is to determine whether self-repetitions fulfill a specific function in the elderly’s discourse. To reach this goal, corpus-based analyses will be carried out based on multimodal (audio and video) data taken from CLARe’s corpora (Corpora for Language and Aging Research) consisting of face-to-face conversations in French, between a young adult and a very old speaker.
First, we will give an overview of the types of verbatim self-repetitions (affirmation/negation, false starts, word retrieval, anaphors, intensifiers/mitigators), comparing older and younger people’s speech (CLARe corpora vs. C-ORAL-ROM and VALIBEL corpora).
We will then have a closer look at the role of prosody in the discourse of elderly speakers (LangAge/CLARe corpus). It has been evidenced that prosody plays an important role for the interpretation of repetitions (Couper-Kuhlen 1996; Curl, Local & Walker 2006). While this was examined for dialogues, we will focus here on monological sequences of verbatim self-repetitions with a personal meaning (e.g., et qu'on avait faim \ et qu'on avait faim \ ‘we suffered from hunger’). The phonetic-prosodic analysis shows that there are minimal differences in the original and the repeated elements, which leads us to argue that the repetitions reinforce the position of the speaker, instead of contributing to the (dialogical) discursive flow of the interaction.
Next, the multimodal approach will explore the function of repetition in gestures (CorpAGEst/CLARe corpus). In line with form-based approaches to gesture (Müller, Bressem & Ladewig 2013), relations that exist between speech and nonverbal resources (including hand gestures, facial displays, gaze, head, and shoulders) will be investigated. We will highlight recurrent cases combining repeated word-sequences and clusters of nonverbal parameters (e.g., l’amour ‘love’ [gaze towards the interlocutor, eyebrows raising, wide opening of the eyes] / en un mot l’amour surtout ‘in one word love above all’ [repeated head turns] / l’amour ‘love’ [repeated head turns, head leaning]). The focus will thus be on self-repetitions that have an expressive or a self-adapting function (Ekman & Friesen 1972: 382), which are linked to the affective and intimate component of communication.
Such prosodic and gestural phenomena, which contribute to the subjective expression of the self and to the speaker’s stance taking in discourse (Du Bois 2007) will give new insight in the way elderly people actually engage in dialogues in a context-sensitive manner.
Some authors (e.g. Hansen 2006) claim that Discourse Markers (DMs) function as cues to situate up... more Some authors (e.g. Hansen 2006) claim that Discourse Markers (DMs) function as cues to situate upcoming speech into the listener’s mental representation of ongoing discourse, either by signaling a semantic relation between the connected units (Ex. 1), by planning and structuring discourse (Ex. 2), or by managing the interactional and intersubjective dimension of communication (Ex. 3). In this perspective, DMs can be seen as “lexical markers of common ground” (Fetzer & Fischer 2007), since they form explicit contextualization cues which instruct the listener on how to retrieve and re-construct speaker-intended meaning. In particular, their procedural meaning of “inference facilitators” creates a bridge between an utterance and its cognitive context (Gonzalez 2005).
(1) “on ne sait pas faire ce qu’on veut / donc on doit / on doit s’adapter”
we can’t do what we want / so we have to / we have to adapt
(VALIBEL; Corpage corpus; 2012; spk: ageNM1)
(2) “je vais commencer par ma première question // euh // ben / d’après…”
I will start with my first question // uh // well / according to…
(VALIBEL; Corpage corpus; 2012; spk: ageFB0)
(3) “ce n’était pas le / le / le grand bonheur si tu veux”
it wasn’t the / the / the happiest time if you will
(VALIBEL; Corpage corpus; 2012; spk: ageNM1)
Our corpus-based study reports the methods and results of two joint endeavours related to the parameter and functional description of DMs in spoken French (corpora: CLAPI, CorpAGEst, VALIBEL). The first research project (MDMA Working Group, see Bolly et al. 2014) aimed at reaching feature-based criteria for the identification of DMs by disambiguating the uses of DM candidates, through cotextual and surface features mainly (e.g. position, co-occurrence) as well as some contextual interpretation (e.g. procedural vs. conceptual meaning). The second research focuses on the functional annotation of DMs based on operational categories validated over languages and modalities (see Crible & Bolly (submitted)). Here, context is extended to situational considerations, including in-depth pragmatic interpretation and multimodal features (prosody, gestures). In both projects, emphasis is put on the constant relation between a DM occurrence and its co(n)text, be it to identify the relevant factors that help classify an item as discursive or non-discursive, or to select the appropriate function the DM is performing in its particular utterance, among the range of “meaning potentials” (Aijmer 2013) it can convey.
We will first show that, despite the great grammatical diversity of the DM class, recurrent patterns of features can be revealed through multivariate analysis of several parameters annotated by four different coders. These results confirm that the distinction between DMs and their non-discursive forms is motivated by syntactic (position, mobility) and semantic (procedurality, prototypical meaning) criteria.
Secondly, the functional annotation applied to these “confirmed” DMs reveals in its turn how DMs relate to their (extra)linguistic context by signaling a number of discourse relations and/or structural, metadiscursive and interpersonal functions. We claim that our revised taxonomy of DM functions allows for an operational annotation of pragmatic features, thus providing a suitable model to account for the different domains covered by DMs in speech.
To sum up, our contribution will provide a comprehensive view on the role of DMs as key signals of the interactive process of co-building interpretation, by highlighting the constant back-and-forth between utterance and context.
According to their traditional definition, gestures are visible bodily actions that are intention... more According to their traditional definition, gestures are visible bodily actions that are intentional and meaningful in a communicative context (Kendon 2004). As such, they can be attributed with the following functions (Colletta et al. 2009): (i) a reference function (deictic or representational); (ii) a discourse-structuring function (e.g., beats or cohesive devices); (iii) an expressive function (oriented towards attitudes, mental states, stance or emotions); (iv) an interactive function (oriented towards the interlocutor and the regulation of speech).
In contrast to representational gestures, the hypothesis is that non-representational gestures are visible bodily actions that are idiosyncratic, (mostly) unintentional and serving pragmatic purposes in language interaction. As such, they play a role similar to that of pragmatic markers in speech (Aijmer 2013): they are metalinguistic indicators of the speaker’s mental processes and, at the same time, help the addressee to build a meaningful holistic representation of the information conveyed. A particular attention will thus be paid here to non-representational spontaneous gestures, which act as emphasizing, mitigating or punctuating devices in language communication (called adaptors, beats, batons, or motor movements – see Ekman & Friesen 1969, McNeill 1992, Krauss et al. 2000).
The following research questions will be addressed: How can we decide which nonverbal units must be accounted for to reach a better understanding of pragmatic competence in human-human interaction? To what extent is it possible (or even, necessary) to integrate non-representational gestures into a consistent model for the annotation of multimodal communication?
The present study is part of the CorpAGEst project (2013-2015), which aims to establish the gestural and verbal profile of very old people, looking at their pragmatic competence from a naturalistic perspective. Within this context, a multimodal corpus has been created, which is comprised of 18 semi-directed, face-to-face interviews between an adult and a very old subject (9 subjects; 16.8 hrs; approx. 250,000 words). This corpus served as a basis for the annotation of nonverbal data (hand gestures, body gestures and facial expressions). Hand gestures were decomposed into phases and annotated in terms of physical parameters (configuration, orientation, movement and position) (Bressem 2008). Body gestures were annotated taking into account the following articulators: head, shoulders, arms, trunk, legs, and feet. It is worth noting that all potential meaningful units were identified as strokes in the first step of the annotation process, including micro-movements (Ex. 1) and activities (Ex. 2).
In line with the MUMIN project (Allwood et al., 2004), facial expressions were identified according to their location in the face (eyebrow, eye movement, gaze, mouth, lips) and then annotated in terms of physiological features. They were also attributed with an emotion label recognized from the face (see Bolly, to appear in 2014).
Preliminary results indicate that the use of nonverbal resources is highly idiosyncratic. For instance, it appeared from a functional analysis of hand gestures that the distribution is not homogeneous among the participants. In addition, focusing on physiological patterning from face and gaze expressions in one of the speaker’s speech, no clear physiological pattern seems to be emotion-specific. Some regularity has nevertheless been noticed for the most frequent emotions used (e.g., surprise is mainly expressed by means of eyebrow raising, often combined with an exaggerated opening of the eyes).
This multimodal and multi-level approach will give new insight into the use of (non)verbal pragmatic markers in relation to the participants’ emotional and attitudinal behavior in intergenerational interaction.
Starting from the common observation that there is no recognized closed class of discourse marker... more Starting from the common observation that there is no recognized closed class of discourse markers (DMs) and that a number of linguistic markers may or may not count as DMs according to the definitions at stake (Schourup 1999: 228), we aim to present an empirical method for the identification and annotation of DMs in spontaneous spoken French (MDMA project). Central to our proposal is that DMs may be described as clusters of features that, in specific patterns of combination, allow distinguishing DM use from other uses. We proceeded in three steps: (i) using a very broad definition of DMs – i.e. items that “provide instructions to the hearer on how to integrate their host utterance into a developing mental model of the discourse in such a way as to make that utterance appear optimally coherent” (Hansen 2006: 25) – three analysts identified all potential DMs in an 800 words transcript; (ii) all types found were then extracted from a balanced 10,000 words corpus; and (iii) analyzed according to more than 10 features (including syntactic, semantic, collocational, and prosodic features). The hypothesis underlying our annotation experiment is that the analysis of the distributional constraints imposed on specific markers should uncover reliable features for the identification and categorization of DMs.
The potential DMs extracted at the first step of analysis refer to those linguistic expressions that can, in one context or another, fulfill a DM function, i.e. be used at either of the following “levels” or “domains”: “the sequential structure of the dialogue, the turn-taking system, speech management, interpersonal management, the topic structure, and participation frameworks” (Fischer 2006: 9). For example, tu vois ‘you see’ is defined as a potential DM because it can occur in contexts where it serves to manage the relationship between speaker and hearer (1), although in other contexts it does not (2).
In line with the objectives of the workshop, our endeavor seeks to establish (more) reliable criteria for the categorization as DMs, and how to distinguish them from other linguistic items fulfilling a non-propositional function, such as modal particles (Degand et al. 2013) or pragmatic markers (Brinton 1996). Disagreement between annotators also reveals borderline cases where, although some discursive, pragmatic, indexical function is commonly detected, there seems to be some hesitation as to what category these items belong to (such as c'est ça ‘that’s it’, quand même ‘still’ in examples 3-4).
In this presentation, we first briefly go over our methodological choices and issues, and then uncover problematic examples of inter-coder disagreement, as well as the first results of the statistical analysis of clusters of features. The latter suggests that there is a certain hierarchy between the different features under scrutiny, regarding their relevance, reliability, or usefulness in the process of identifying DMs in context.
La pragmatique se définit comme une sous-discipline de la linguistique qui prend pour objets d’ét... more La pragmatique se définit comme une sous-discipline de la linguistique qui prend pour objets d’étude des phénomènes langagiers dont le sens est intimement lié au contexte dans lequel ils apparaissent. Mais quelle vision nouvelle cette discipline peut-elle apporter aux études sur le langage des personnes âgées ? Nous verrons que la prise en compte de (micro-)phénomènes langagiers peut constituer une aide pour décoder les attitudes et les émotions chez la personne âgée (par ex., le marqueur de discours ben ou un hochement de tête). Après avoir présenté les enjeux des recherches sur corpus dans le domaine du vieillissement, les corpus Corpage et CorpAGEst, constitués de dialogues entre un adulte et une personne très âgée, seront présentés. Nous verrons ensuite comment une approche multimodale de l’interaction (combinant gestes et parole) peut mener à dresser un profil pragmatique individuel de la personne âgée en situation de communication réelle. Nous nous arrêterons en particulier sur la relation entre les traits du visage et l’expression d’émotions, ainsi que sur le rôle joué par la répétition de micro-gestes dans l’interaction langagière.
The main objective of this paper is to give a clearer view on the way we can reconstruct the emot... more The main objective of this paper is to give a clearer view on the way we can reconstruct the emotional and attitudinal profile of healthy very old people when communicating in their eve-ryday environment. At the core of the study, the following question is addressed: To what ex-tent do the verbal and nonverbal modes converge in the information they convey about the authentic emotional and attitudinal states of older people in their everyday communication? Combining a psychometric measurement test with a multimodal corpus-based approach, this research proposal aims to mirror the authentic empathic profile of communicating subjects as closely as possible.
Annual VALS/ASLA Conference: What is the relevance of linguistic research for society? Questioning the notion of “impact”, 13 May 2013, University of Basel, Basel (Switzerland), 2013
International Conference Corpus Linguistics: How to bind concepts, technique and application?, 14-15 December 2012, Université Bordeaux 3 - EA TELEM - CLLE-ERSSàB, Bordeaux (France), 2012
The CorpAGEst multimodal corpus is comprised of interviews with elderly subjects living at home o... more The CorpAGEst multimodal corpus is comprised of interviews with elderly subjects living at home or in a residential home. It is comprised of transversal (16.8 hrs of video recordings) and longitudinal audio-video data.
To cite: Bolly, Catherine T. (2013). CorpAGEst. Multimodal corpus for the elderly’s language. Louvain-la-Neuve, Paris: Université catholique de Louvain (Valibel – Discours et Variation) and CNRS (UMR 7023 Structures Formelles du Langage)
The Corpage reference corpus is comprised of semi-directed, face-to-face conversations in French ... more The Corpage reference corpus is comprised of semi-directed, face-to-face conversations in French (approx. 144 hrs of audio media recordings) between an adult and a very old healthy subject living at home (> 75 years old).
To cite: Bolly, Catherine T., Masse, Marie & Meire, Philippe (2012). Corpage. Reference corpus for the elderly’s language. Louvain-la-Neuve: Université catholique de Louvain (Valibel – Discours et Variarion and Psychological Sciences Research Institute).
Starting from the common observation that there is no recognized closed class of Discourse Marker... more Starting from the common observation that there is no recognized closed class of Discourse Markers (DMs) and that their definition may vary from one theoretical framework to another (Schourup 1999: 228), the MDMA project (“Model for Discourse Marker Annotation”) has established an empirical corpus-based method for the identification and annotation of DMs in spoken French. Central to our proposal is that DMs may be described as clusters of features that, in specific patterns of combination, allow distinguishing between more or less prototypical uses of DMs in context. The general goal of MDMA is to cover every step of the analysis of DMs from identification to parameter and functional description, while its ultimate perspective is to provide reliable clusters to feed machine-learning systems for semi-automatic identification of DMs in authentic data (cf. Fischer 2000 for a similar approach to functional category assignment). We proceeded in three steps : (i) manual identification of all so-called “potential” DMs in a balanced corpus of spoken French (5,000 words; Belgium and France) ; (ii) automatic extraction from the corpus of every token corresponding to the candidate DMs previously identified (1,181 tokens) ; and (iii) parameter analysis of a random sample of 200 potential DMs (syntactic, formal and semantic-pragmatic variables). In line with the objectives of this conference, our hypothesis is that statistical analysis – based on the distributional constraints of the potential DMs at stake – should uncover a certain hierarchy between the different features under scrutiny, regarding their relevance, reliability, and generalizability (or even specificity). Our results show that syntactic position is the most predictive feature that correctly identifies a potential DM (which is encouraging for NLP purposes), and that the statistically most prototypical profile of DM is the following: autonomous, post-final, pre-initial or indeterminable position; with a procedural meaning; in initial or medial position in the conversational turn; not expressing its coded meaning; not mobile. This pattern is exemplified below: (1) bon moi heureusement je n’ai pas eu de choses très graves mais enfin quand même / on sent / on commence à se sentir qu’on / qu’on / qu’on diminue / qu’on ne sait pas faire ce qu’on veut (Corpage : corpus ageNM1) Less prototypical profiles either correspond to propositional equivalents of DM tokens (example 2), or borderline expressions at the edges of several pragmatic categories such as modal particles or response signals (example 3): (2) le loup s’empresse/ chemin pour arriver chez la mère-grand avant le petit chaperon rouge mais là faut mettre pas pris le bon chemin /euh/ (Clapi : corpus Chaperon Rouge, “Jean-Pierre et Magali”) (3) ah à cause des sans papiers à Saint Bernard CEC voilà Saint Bernard CG d’accord (Clapi : corpus Adi étudiants) In this poster, we first describe the annotation scheme and procedure, then we illustrate several issues of inter-rater agreement, and finally present the main results from the statistical analyses. Pointers to some (functional and multimodal) perspectives of the MDMA project will also be mentioned.
References Balthasar L. & Bert M. 2005. La plateforme « Corpus de langues parlées en interaction » (CLAPI), Lidil, 31. URL : http://lidil.revues.org/139 Bolly C. T., Masse M. & Meire P. 2012. Corpage. Reference corpus for the elderly’s language. Louvain-la-Neuve: Université catholique de Louvain. Fischer, K. 2000. From cognitive semantics to lexical pragmatics. Berlin: Mouton de Gruyter. Schourup, L. 1999. Discourse markers. Lingua 107: 227-265.
Uploads
Papers by Catherine Bolly
In this line, the aim of our talk is to determine whether self-repetitions fulfill a specific function in the elderly’s discourse. To reach this goal, corpus-based analyses will be carried out based on multimodal (audio and video) data taken from CLARe’s corpora (Corpora for Language and Aging Research) consisting of face-to-face conversations in French, between a young adult and a very old speaker.
First, we will give an overview of the types of verbatim self-repetitions (affirmation/negation, false starts, word retrieval, anaphors, intensifiers/mitigators), comparing older and younger people’s speech (CLARe corpora vs. C-ORAL-ROM and VALIBEL corpora).
We will then have a closer look at the role of prosody in the discourse of elderly speakers (LangAge/CLARe corpus). It has been evidenced that prosody plays an important role for the interpretation of repetitions (Couper-Kuhlen 1996; Curl, Local & Walker 2006). While this was examined for dialogues, we will focus here on monological sequences of verbatim self-repetitions with a personal meaning (e.g., et qu'on avait faim \ et qu'on avait faim \ ‘we suffered from hunger’). The phonetic-prosodic analysis shows that there are minimal differences in the original and the repeated elements, which leads us to argue that the repetitions reinforce the position of the speaker, instead of contributing to the (dialogical) discursive flow of the interaction.
Next, the multimodal approach will explore the function of repetition in gestures (CorpAGEst/CLARe corpus). In line with form-based approaches to gesture (Müller, Bressem & Ladewig 2013), relations that exist between speech and nonverbal resources (including hand gestures, facial displays, gaze, head, and shoulders) will be investigated. We will highlight recurrent cases combining repeated word-sequences and clusters of nonverbal parameters (e.g., l’amour ‘love’ [gaze towards the interlocutor, eyebrows raising, wide opening of the eyes] / en un mot l’amour surtout ‘in one word love above all’ [repeated head turns] / l’amour ‘love’ [repeated head turns, head leaning]). The focus will thus be on self-repetitions that have an expressive or a self-adapting function (Ekman & Friesen 1972: 382), which are linked to the affective and intimate component of communication.
Such prosodic and gestural phenomena, which contribute to the subjective expression of the self and to the speaker’s stance taking in discourse (Du Bois 2007) will give new insight in the way elderly people actually engage in dialogues in a context-sensitive manner.
(1) “on ne sait pas faire ce qu’on veut / donc on doit / on doit s’adapter”
we can’t do what we want / so we have to / we have to adapt
(VALIBEL; Corpage corpus; 2012; spk: ageNM1)
(2) “je vais commencer par ma première question // euh // ben / d’après…”
I will start with my first question // uh // well / according to…
(VALIBEL; Corpage corpus; 2012; spk: ageFB0)
(3) “ce n’était pas le / le / le grand bonheur si tu veux”
it wasn’t the / the / the happiest time if you will
(VALIBEL; Corpage corpus; 2012; spk: ageNM1)
Our corpus-based study reports the methods and results of two joint endeavours related to the parameter and functional description of DMs in spoken French (corpora: CLAPI, CorpAGEst, VALIBEL). The first research project (MDMA Working Group, see Bolly et al. 2014) aimed at reaching feature-based criteria for the identification of DMs by disambiguating the uses of DM candidates, through cotextual and surface features mainly (e.g. position, co-occurrence) as well as some contextual interpretation (e.g. procedural vs. conceptual meaning). The second research focuses on the functional annotation of DMs based on operational categories validated over languages and modalities (see Crible & Bolly (submitted)). Here, context is extended to situational considerations, including in-depth pragmatic interpretation and multimodal features (prosody, gestures). In both projects, emphasis is put on the constant relation between a DM occurrence and its co(n)text, be it to identify the relevant factors that help classify an item as discursive or non-discursive, or to select the appropriate function the DM is performing in its particular utterance, among the range of “meaning potentials” (Aijmer 2013) it can convey.
We will first show that, despite the great grammatical diversity of the DM class, recurrent patterns of features can be revealed through multivariate analysis of several parameters annotated by four different coders. These results confirm that the distinction between DMs and their non-discursive forms is motivated by syntactic (position, mobility) and semantic (procedurality, prototypical meaning) criteria.
Secondly, the functional annotation applied to these “confirmed” DMs reveals in its turn how DMs relate to their (extra)linguistic context by signaling a number of discourse relations and/or structural, metadiscursive and interpersonal functions. We claim that our revised taxonomy of DM functions allows for an operational annotation of pragmatic features, thus providing a suitable model to account for the different domains covered by DMs in speech.
To sum up, our contribution will provide a comprehensive view on the role of DMs as key signals of the interactive process of co-building interpretation, by highlighting the constant back-and-forth between utterance and context.
In contrast to representational gestures, the hypothesis is that non-representational gestures are visible bodily actions that are idiosyncratic, (mostly) unintentional and serving pragmatic purposes in language interaction. As such, they play a role similar to that of pragmatic markers in speech (Aijmer 2013): they are metalinguistic indicators of the speaker’s mental processes and, at the same time, help the addressee to build a meaningful holistic representation of the information conveyed. A particular attention will thus be paid here to non-representational spontaneous gestures, which act as emphasizing, mitigating or punctuating devices in language communication (called adaptors, beats, batons, or motor movements – see Ekman & Friesen 1969, McNeill 1992, Krauss et al. 2000).
The following research questions will be addressed: How can we decide which nonverbal units must be accounted for to reach a better understanding of pragmatic competence in human-human interaction? To what extent is it possible (or even, necessary) to integrate non-representational gestures into a consistent model for the annotation of multimodal communication?
The present study is part of the CorpAGEst project (2013-2015), which aims to establish the gestural and verbal profile of very old people, looking at their pragmatic competence from a naturalistic perspective. Within this context, a multimodal corpus has been created, which is comprised of 18 semi-directed, face-to-face interviews between an adult and a very old subject (9 subjects; 16.8 hrs; approx. 250,000 words). This corpus served as a basis for the annotation of nonverbal data (hand gestures, body gestures and facial expressions). Hand gestures were decomposed into phases and annotated in terms of physical parameters (configuration, orientation, movement and position) (Bressem 2008). Body gestures were annotated taking into account the following articulators: head, shoulders, arms, trunk, legs, and feet. It is worth noting that all potential meaningful units were identified as strokes in the first step of the annotation process, including micro-movements (Ex. 1) and activities (Ex. 2).
In line with the MUMIN project (Allwood et al., 2004), facial expressions were identified according to their location in the face (eyebrow, eye movement, gaze, mouth, lips) and then annotated in terms of physiological features. They were also attributed with an emotion label recognized from the face (see Bolly, to appear in 2014).
Preliminary results indicate that the use of nonverbal resources is highly idiosyncratic. For instance, it appeared from a functional analysis of hand gestures that the distribution is not homogeneous among the participants. In addition, focusing on physiological patterning from face and gaze expressions in one of the speaker’s speech, no clear physiological pattern seems to be emotion-specific. Some regularity has nevertheless been noticed for the most frequent emotions used (e.g., surprise is mainly expressed by means of eyebrow raising, often combined with an exaggerated opening of the eyes).
This multimodal and multi-level approach will give new insight into the use of (non)verbal pragmatic markers in relation to the participants’ emotional and attitudinal behavior in intergenerational interaction.
The potential DMs extracted at the first step of analysis refer to those linguistic expressions that can, in one context or another, fulfill a DM function, i.e. be used at either of the following “levels” or “domains”: “the sequential structure of the dialogue, the turn-taking system, speech management, interpersonal management, the topic structure, and participation frameworks” (Fischer 2006: 9). For example, tu vois ‘you see’ is defined as a potential DM because it can occur in contexts where it serves to manage the relationship between speaker and hearer (1), although in other contexts it does not (2).
In line with the objectives of the workshop, our endeavor seeks to establish (more) reliable criteria for the categorization as DMs, and how to distinguish them from other linguistic items fulfilling a non-propositional function, such as modal particles (Degand et al. 2013) or pragmatic markers (Brinton 1996). Disagreement between annotators also reveals borderline cases where, although some discursive, pragmatic, indexical function is commonly detected, there seems to be some hesitation as to what category these items belong to (such as c'est ça ‘that’s it’, quand même ‘still’ in examples 3-4).
In this presentation, we first briefly go over our methodological choices and issues, and then uncover problematic examples of inter-coder disagreement, as well as the first results of the statistical analysis of clusters of features. The latter suggests that there is a certain hierarchy between the different features under scrutiny, regarding their relevance, reliability, or usefulness in the process of identifying DMs in context.
To cite: Bolly, Catherine T. (2013). CorpAGEst. Multimodal corpus for the elderly’s language. Louvain-la-Neuve, Paris: Université catholique de Louvain (Valibel – Discours et Variation) and CNRS (UMR 7023 Structures Formelles du Langage)
To cite: Bolly, Catherine T., Masse, Marie & Meire, Philippe (2012). Corpage. Reference corpus for the elderly’s language. Louvain-la-Neuve: Université catholique de Louvain (Valibel – Discours et Variarion and Psychological Sciences Research Institute).
We proceeded in three steps : (i) manual identification of all so-called “potential” DMs in a balanced corpus of spoken French (5,000 words; Belgium and France) ; (ii) automatic extraction from the corpus of every token corresponding to the candidate DMs previously identified (1,181 tokens) ; and (iii) parameter analysis of a random sample of 200 potential DMs (syntactic, formal and semantic-pragmatic variables). In line with the objectives of this conference, our hypothesis is that statistical analysis – based on the distributional constraints of the potential DMs at stake – should uncover a certain hierarchy between the different features under scrutiny, regarding their relevance, reliability, and generalizability (or even specificity). Our results show that syntactic position is the most predictive feature that correctly identifies a potential DM (which is encouraging for NLP purposes), and that the statistically most prototypical profile of DM is the following: autonomous, post-final, pre-initial or indeterminable position; with a procedural meaning; in initial or medial position in the conversational turn; not expressing its coded meaning; not mobile. This pattern is exemplified below:
(1) bon moi heureusement je n’ai pas eu de choses très graves mais enfin quand même / on sent / on commence à se sentir qu’on / qu’on / qu’on diminue / qu’on ne sait pas faire ce qu’on veut (Corpage : corpus ageNM1)
Less prototypical profiles either correspond to propositional equivalents of DM tokens (example 2), or borderline expressions at the edges of several pragmatic categories such as modal particles or response signals (example 3):
(2) le loup s’empresse/ chemin pour arriver chez la mère-grand avant le petit chaperon rouge mais là faut mettre pas pris le bon chemin /euh/ (Clapi : corpus Chaperon Rouge, “Jean-Pierre et Magali”)
(3) ah à cause des sans papiers à Saint Bernard CEC voilà Saint Bernard CG d’accord (Clapi : corpus Adi étudiants)
In this poster, we first describe the annotation scheme and procedure, then we illustrate several issues of inter-rater agreement, and finally present the main results from the statistical analyses. Pointers to some (functional and multimodal) perspectives of the MDMA project will also be mentioned.
References
Balthasar L. & Bert M. 2005. La plateforme « Corpus de langues parlées en interaction » (CLAPI), Lidil, 31. URL : http://lidil.revues.org/139
Bolly C. T., Masse M. & Meire P. 2012. Corpage. Reference corpus for the elderly’s language. Louvain-la-Neuve: Université catholique de Louvain.
Fischer, K. 2000. From cognitive semantics to lexical pragmatics. Berlin: Mouton de Gruyter.
Schourup, L. 1999. Discourse markers. Lingua 107: 227-265.