Proceedings of the 7th Speech Prosody Conference 2014, May 2014
There is an increasing consensus to regard gesture and speech as parts of an integrated communica... more There is an increasing consensus to regard gesture and speech as parts of an integrated communication system, in part because of the findings related to their temporal coordination at different levels. In general, results for different types of gestures show that the most prominent part of the gesture (the apex) is typically aligned with accented syllables [6, 10-12, 14, 17]. The aim of the present study is to test for this coordination by focusing on head movements taken from a semi-spontaneous setting in order to look at the effects of upcoming phrase boundaries on their timing. Our results show that while apexes of head gestures are synchronized with accented syllables, upcoming phrase boundaries have an effect on the timing of three gestural points, namely the start, apex, and end time of head gestures. Crucially, these points are aligned differently with respect to the stressed syllable for trochees as compared with iambs/monosyllables, showing that head nods are retracted before upcoming phrase boundaries. This result corroborates previous results by Esteve-Gibert & Prieto [17] for pointing gestures in laboratory settings.
Interlocutors sometimes repeat each other's representational hand gestures. We investigated i... more Interlocutors sometimes repeat each other's representational hand gestures. We investigated if this is a case of direct mimicry of form, or whether perceiving a gesture gives rise to a semantic representation, which subsequently informs gesture production. For this we used an interactive route description task, in which a confederate's gestures indicated the route in either the vertical or the horizontal plane and either with one or four fingers extended as an index. We found that perceiving vertical gestures led to an increase not only in participants' production of vertical gestures, but also in their use of one finger as an index, suggesting that seeing vertical gestures caused participants to think of the route as on a map, which led them to point with one finger (as is common on a map) rather than four. Our results support the notion that repetition of meaningful gesture forms results from converging semantic representations.
Does language influence the production and perception of gestures? The metaphorical use of langua... more Does language influence the production and perception of gestures? The metaphorical use of language in representing time is deeply interlinked with actions in space, such as gestures. In Chinese, speakers can talk and gesture about time as if it were horizontal, sagittal, or vertical. In English, speakers rarely employ the vertical plane. Two experiments showed that the verbal use of vertical spatial metaphors had an online influence on the production and perception of gestures by late Chinese-English bilinguals. Participants produced more vertical gestures when talking about time references by use of vertical spatial metaphors, e.g. 'shàng-zhōu' (literally: 'above week', meaning 'last week'), and they preferred vertical gestures to horizontal gestures when perceiving time references with vertical spatial metaphors. Gestures are not only shaped by the language specific conceptualisation, but are also sensitive to the changes in linguistic choices, both in pro...
General rights Copyright and moral rights for the publications made accessible in the public port... more General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.? Users may download and print one copy of any publication from the public portal for the purpose of private study or research? You may not further distribute the material or use it for any profit-making activity or commercial gain? You may freely distribute the URL identifying the publication in the public portal Take down policy If you believe that this document breaches copyright, please contact us providing details, and we will remove access to the work immediately and investigate your claim. Download date: 17. mei. 2016
This paper describes the effect of deviance in focus marking by means of pitch accent distributio... more This paper describes the effect of deviance in focus marking by means of pitch accent distributions in Dutch on L1 perceptions of accentedness, nativeness and comprehensibility in L1 and L2 speech. On a rating task Dutch natives demonstrate that they have unambiguous intuitions concerning L2 speech by Spanish learners of Dutch by categorically rating it as more accented, more difficult to comprehend and less typical of an L1 speaker than L1 speech, with proficiency factor as a modulating factor. Interestingly, accentedness and nativeness are rated more extremely than comprehensibility, suggesting that non-native, foreign accented speech can still be highly comprehensible. A preference task reveals that Dutch natives prefer prosodically accurate utterances to prosodically inaccurate ones, when making nativeness judgments based on prosodic cues only, for both L1 and proficient L2 speakers.
The Journal of the Acoustical Society of America, 2017
This study examines the influence of the position of prosodic heads (accented syllables) and pros... more This study examines the influence of the position of prosodic heads (accented syllables) and prosodic edges (prosodic word and intonational phrase boundaries) on the timing of head movements. Gesture movements and prosodic events tend to be temporally aligned in the discourse, the most prominent part of gestures typically being aligned with prosodically prominent syllables in speech. However, little is known about the impact of the position of intonational phrase boundaries on gesture-speech alignment patterns. Twenty-four Catalan speakers produced spontaneous (experiment 1) and semi-spontaneous head gestures with a confirmatory function (experiment 2), along with phrase-final focused words in different prosodic conditions (stress-initial, stress-medial, and stress-final). Results showed (a) that the scope of head movements is the associated focused prosodic word, (b) that the left edge of the focused prosodic word determines where the interval of gesture prominence starts, and (c) ...
The importance of intonation in communication cannot be denied (Gussenhoven, 2004; Ladd, 2008). S... more The importance of intonation in communication cannot be denied (Gussenhoven, 2004; Ladd, 2008). Since the goal of most foreign language learners is to successfully communicate in a language other than their mother tongue, more research on the way they are perceived by native speakers and the role of intonation in these interactions in highly relevant. For one, the correct use of pitch accents in the marking of information status is essential in many languages. For instance, in Dutch the answer "I would like red WINE" (pitch accent on the last word) would be confusing if the question was "Would you like RED or WHITE wine?", but not if the question was "Would you like red WINE or BEER?". Thus the use of an inappropriate intonation pattern may lead to miscommunication or incomprehensibility, or make interaction between communication partners more cumbersome, both in the processing of the speech (Munro & Derwing, 1999; Terken & Nooteboom, 1987; Van Leeuwen ...
Repeated references have been found to be reduced as compared to references that are not repeated... more Repeated references have been found to be reduced as compared to references that are not repeated, both in speech and in gesture. In the present study we wanted to see whether certain factors can inhibit this reduction in repeated references. In a production experiment, speakers were confronted with negative feedback after an initial description of an object, indicating that the communication was unsuccessful. We found that after initial negative feedback, second references were reduced with regard to all speech variables. When the speakers were confronted with additional negative feedback, the ensuing third references were increased in the number of words and the duration, as compared to the second references, but further reduced in their speech rate. Gesture rate increased in third references as compared to initial references. After (repeated) instances of unsuccessful communication, speakers speak slower and increase their gesture rate, thereby making their repeated references cl...
Proceedings of the 7th Speech Prosody Conference 2014, May 2014
There is an increasing consensus to regard gesture and speech as parts of an integrated communica... more There is an increasing consensus to regard gesture and speech as parts of an integrated communication system, in part because of the findings related to their temporal coordination at different levels. In general, results for different types of gestures show that the most prominent part of the gesture (the apex) is typically aligned with accented syllables [6, 10-12, 14, 17]. The aim of the present study is to test for this coordination by focusing on head movements taken from a semi-spontaneous setting in order to look at the effects of upcoming phrase boundaries on their timing. Our results show that while apexes of head gestures are synchronized with accented syllables, upcoming phrase boundaries have an effect on the timing of three gestural points, namely the start, apex, and end time of head gestures. Crucially, these points are aligned differently with respect to the stressed syllable for trochees as compared with iambs/monosyllables, showing that head nods are retracted before upcoming phrase boundaries. This result corroborates previous results by Esteve-Gibert & Prieto [17] for pointing gestures in laboratory settings.
Interlocutors sometimes repeat each other's representational hand gestures. We investigated i... more Interlocutors sometimes repeat each other's representational hand gestures. We investigated if this is a case of direct mimicry of form, or whether perceiving a gesture gives rise to a semantic representation, which subsequently informs gesture production. For this we used an interactive route description task, in which a confederate's gestures indicated the route in either the vertical or the horizontal plane and either with one or four fingers extended as an index. We found that perceiving vertical gestures led to an increase not only in participants' production of vertical gestures, but also in their use of one finger as an index, suggesting that seeing vertical gestures caused participants to think of the route as on a map, which led them to point with one finger (as is common on a map) rather than four. Our results support the notion that repetition of meaningful gesture forms results from converging semantic representations.
Does language influence the production and perception of gestures? The metaphorical use of langua... more Does language influence the production and perception of gestures? The metaphorical use of language in representing time is deeply interlinked with actions in space, such as gestures. In Chinese, speakers can talk and gesture about time as if it were horizontal, sagittal, or vertical. In English, speakers rarely employ the vertical plane. Two experiments showed that the verbal use of vertical spatial metaphors had an online influence on the production and perception of gestures by late Chinese-English bilinguals. Participants produced more vertical gestures when talking about time references by use of vertical spatial metaphors, e.g. 'shàng-zhōu' (literally: 'above week', meaning 'last week'), and they preferred vertical gestures to horizontal gestures when perceiving time references with vertical spatial metaphors. Gestures are not only shaped by the language specific conceptualisation, but are also sensitive to the changes in linguistic choices, both in pro...
General rights Copyright and moral rights for the publications made accessible in the public port... more General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.? Users may download and print one copy of any publication from the public portal for the purpose of private study or research? You may not further distribute the material or use it for any profit-making activity or commercial gain? You may freely distribute the URL identifying the publication in the public portal Take down policy If you believe that this document breaches copyright, please contact us providing details, and we will remove access to the work immediately and investigate your claim. Download date: 17. mei. 2016
This paper describes the effect of deviance in focus marking by means of pitch accent distributio... more This paper describes the effect of deviance in focus marking by means of pitch accent distributions in Dutch on L1 perceptions of accentedness, nativeness and comprehensibility in L1 and L2 speech. On a rating task Dutch natives demonstrate that they have unambiguous intuitions concerning L2 speech by Spanish learners of Dutch by categorically rating it as more accented, more difficult to comprehend and less typical of an L1 speaker than L1 speech, with proficiency factor as a modulating factor. Interestingly, accentedness and nativeness are rated more extremely than comprehensibility, suggesting that non-native, foreign accented speech can still be highly comprehensible. A preference task reveals that Dutch natives prefer prosodically accurate utterances to prosodically inaccurate ones, when making nativeness judgments based on prosodic cues only, for both L1 and proficient L2 speakers.
The Journal of the Acoustical Society of America, 2017
This study examines the influence of the position of prosodic heads (accented syllables) and pros... more This study examines the influence of the position of prosodic heads (accented syllables) and prosodic edges (prosodic word and intonational phrase boundaries) on the timing of head movements. Gesture movements and prosodic events tend to be temporally aligned in the discourse, the most prominent part of gestures typically being aligned with prosodically prominent syllables in speech. However, little is known about the impact of the position of intonational phrase boundaries on gesture-speech alignment patterns. Twenty-four Catalan speakers produced spontaneous (experiment 1) and semi-spontaneous head gestures with a confirmatory function (experiment 2), along with phrase-final focused words in different prosodic conditions (stress-initial, stress-medial, and stress-final). Results showed (a) that the scope of head movements is the associated focused prosodic word, (b) that the left edge of the focused prosodic word determines where the interval of gesture prominence starts, and (c) ...
The importance of intonation in communication cannot be denied (Gussenhoven, 2004; Ladd, 2008). S... more The importance of intonation in communication cannot be denied (Gussenhoven, 2004; Ladd, 2008). Since the goal of most foreign language learners is to successfully communicate in a language other than their mother tongue, more research on the way they are perceived by native speakers and the role of intonation in these interactions in highly relevant. For one, the correct use of pitch accents in the marking of information status is essential in many languages. For instance, in Dutch the answer "I would like red WINE" (pitch accent on the last word) would be confusing if the question was "Would you like RED or WHITE wine?", but not if the question was "Would you like red WINE or BEER?". Thus the use of an inappropriate intonation pattern may lead to miscommunication or incomprehensibility, or make interaction between communication partners more cumbersome, both in the processing of the speech (Munro & Derwing, 1999; Terken & Nooteboom, 1987; Van Leeuwen ...
Repeated references have been found to be reduced as compared to references that are not repeated... more Repeated references have been found to be reduced as compared to references that are not repeated, both in speech and in gesture. In the present study we wanted to see whether certain factors can inhibit this reduction in repeated references. In a production experiment, speakers were confronted with negative feedback after an initial description of an object, indicating that the communication was unsuccessful. We found that after initial negative feedback, second references were reduced with regard to all speech variables. When the speakers were confronted with additional negative feedback, the ensuing third references were increased in the number of words and the duration, as compared to the second references, but further reduced in their speech rate. Gesture rate increased in third references as compared to initial references. After (repeated) instances of unsuccessful communication, speakers speak slower and increase their gesture rate, thereby making their repeated references cl...
Uploads
Abstracts and Proceedings by Marc Swerts
Papers by Marc Swerts