Maciej Karpiński

W obecnym tekście skupiamy się na wybranych aspektach segmentacji ruchów rąk i jednostkach stosowanych w tradycyjnym podejściu do opisu gestykulacji (sekwencja gestów, fraza i faza gestowa). Ich omówienie poprzedzamy szkicem problematyki segmentacji wypowiedzi ustnych. Przedstawiamy wyniki badania empirycznego, w którym zadaniem nieprzeszkolonych uczestników było wskazanie w materiale filmowym granic gestów mówcy. Zwracamy uwagę na rozbieżności między formalnym, naukowym pojmowaniem gestu, a jego potocznym rozumieniem, znajdującym wyraz w opisie wykonanym przez uczestników przedstawionego badania. Wskazujemy problemy związane z implementacją naukowych definicji jednostek analizy. Proponujemy uporządkowanie podejść do definicji gestu poprzez jasne odnoszenie się do poziomu abstrakcji, na którym operują.

"The paralinguistic component of communication attracted a great deal of attention from contemporary linguists in the 1960s. The seminal works written then by Trager, Crystal and others had a powerful influence on the concept of paralanguage that lasted for many years. But, with the focus shifting towards the socio-psychological context of communication in the 1970s, the development of spoken corpora and databases and the significant progress in speech technology in the 1980s and 1990s, the need has arisen for a more comprehensive, coherent and formalised – but also flexible – approach to paralinguistic features. This study advances some preliminary proposals for a revised treatment of paralanguage that
would meet some of these requirements and provide a conceptual basis for a new system of annotation
for paralinguistic features. A range of views on paralinguistic features, which come mostly from the fields of speech prosody and gesture analysis, are briefly discussed. A number of assumptions and postulates are formulated to allow for a more consistent approach to paralinguistic features. The study
suggests that there should be more reliance on continua than on binary categorisations of features, that multi-functionality and multimodality should be fully acknowledged and that clear distinctions should be made among the levels of description, and between the properties of speakers and the speech signal
itself."

The holistic approach to interpersonal communication in dialogue, involving the analysis of multiple sensory modalities and channels, poses a serious challenge not only in terms of research techniques and methods but also from the viewpoint of data infrastructure. In the Borderland project, the process of communicative accommodation is studied in young people in the intercultural context of the Polish-German boundary region. In order to collect, annotate, and analyse research material, a new technical and analytic infrastructure has been developed. Centred around a data management system, it incorporates well-known annotation and transcription tools along with custom-designed transcription and annotation tagsets and procedures. Data and metadata formats are designed to enable further corpus-based analyses of
accommodation-related processes, mostly in the paralinguistic domain of prosody and gestures. Data formats used in the project ensure wide interchangeability and usage of almost any analytic software. Previously tested methods of quantitative accommodation analysis are adjusted, supplemented with new custom procedures, and applied to each channel under study as well as to the cross-modal (e.g., prosody - gesture) accommodation processes.

The tendency to use a lower voice in public speeches may be well justified by evolutionary and social factors. Low, stable voice is often associated with authority and persuasive power while high-pitched voices are linked to intensive active emotions. In this light, the occurrence of extremely high intonation peaks or falsetto voice in public speeches by male speakers is puzzling and requires explanation. In this study we analyse the usage of high-pitched intonation peaks in ten male members of the Polish parliament based on recordings selected from the MuMo Stance Corpus of German and Polish Parliamentary Speeches. The speeches are scrutinized for high pitched prominences using the criteria of (a) relative pitch range and (b) top pitch value relative to the mean pitch of each speaker. The material includes also prominence areas annotations for: hand gestures, head, body movements, and the discourse functions realized by the respective utterances. We qualitatively explore on the distribution, form, and function of high-pitched prominences, and their co-occurrence with gestures and body movements. We find the usage of high-pitched prominences to significantly differ among and within speeches. While they tend to co-occur with gestures, the connection of their characteristics with gestural features is not straightforward.

Aspects of gestural alignment in task-oriented dialoguesInterlocutors in a conversation influence each other in a number of dimensions. This process may lead to observable changes in their communicative behaviour. The directions and profiles of these changes are often correlated with the quality of interaction and may predict its success. In the present study, the gestural component of communication is scrutinised for changes that may reflect the process of alignment. Two types of task-oriented dialogues between teenagers are recorded and annotated for gestures and their features. We hypothesize that the dialogue task type (collaborative vs. competitive), as well as certain culture-specific properties of alignment that differ between German and Polish pairs, may significantly influence the process of communication. In order to explore the data and detect tendencies in gestural behaviour, automatised annotation mining and statistical exploration have been used, including a moving fra...

Conversational parties tend to mutually adapt their communicative behaviour in a number of dimensions, from the level of physical aspects of speech signal and gesture, utterance properties, up to the level of mental representations. In the present study, an attempt is made to track the process of convergence in the temporal domain both as a global tendency and a local phenomenon. The material under study consists of two sets of task-oriented dialogues recorded with or without eye contact (telephone conversations) between the speakers. All the recordings were segmented into syllables and analysed in terms of speech rate and nPVI for each speaker as well as for the correlations between the speakers in each pair. Global convergence tendencies were proven to be weak but some influence of dialogue settings and gender was found. The results seem to support the hypotheses that the alignment- related processes remain under the influence of many factors related to the dialogue flow and canno...

Intonation is often considered the most problematic component of prosody. It may largely contribute to the meaning of an utterance and provide rich indexical information. The complexity of the intonationrelated phenomena, from the acoustic level up to the cognitive process of semiosis, resulted in a variety of conceptualisations and divergent theoretical approaches. In the present text, an attempt is made to sketch the late Wiktor Jassem’s contribution to intonation studies, starting from his early work on colloquial British English to the studies focused on the Polish language as well as his general perspective on prosody and on the methods of its exploration. Some of his unpublished contribution to research projects on various aspects of prosody is also acknowledged. The present overview is based both on available written materials (publications, references, reports, reviews) and the author’s personal communication with Wiktor Jassem and his collaborators.

This paper describes the design, development and preliminary verification of a new tool created for the purpose of annotation of spoken language recordings. The software extends the potential of a typical multi-layer annotation system with a new component based on the graphical representation of feature space that supports annotation of continuous and noncategorial features. Apart from the annotation options, the program provides a flexible perception experiment framework aimed especially at testing hypotheses related to continuous and non-categorial features. The tool was initially tested and first applied for the annotation of a speech corpus composed of conversational and emotionally marked speech data within a larger project confessed to speaker characterisation and recognition.

Filled pauses (FPs) have proved to be more than valuable cues to speech production processes and important units in discourse analysis. Some aspects of their form and occurrence patterns have been shown to be speaker- and language-specific. In the present study, basic acoustic properties of FPs in Polish task-oriented dialogues are explored. A set of FPs was extracted from a corpus of twenty task- oriented dialogues on the basis of available annotations. After initial scrutiny and selection, a subset of the signals underwent a series of pitch, formant frequency and voice quality analyses. A significant amount of variation found in the realisations of FPs justifies their potential application in speaker recognition systems. Regular monosegmental FPs were confirmed to show relatively stable basic acoustic parameters, which allows for their easy identification and measurements but it may result in less significant differences among the speakers.

In the present text, some recent changes in the perspective taken by psycholinguists in the study of language and communication are discussed. T heir interests seem to gradually shift from the study of language processing as an isolated and independent phenomenon towards inclusion of more interactional factors being indispensable components of interpersonal communication and involved in the process of communicative alignment. Alignment is here understood as a complex phenomenon that goes beyond increasing similarity of mental representations and related communicative behaviour. It simultaneously occurs on many levels and in various modalities, including those traditionally excluded from language study. A s a consequence, it implies not only more flexibility in the study of interpersonal communication but it also means a shift in the psycholinguistic methodology and probably also in the widely accepted picture of language and its limits.

"In the present paper, an attempt is made to track some
potential cues to intentionality that may accompany pointing
gestures in task-oriented dialogues. Twenty “origami”
dialogues are analysed, half of them in the “mutual visibility”
(MV) and half in the “lack of mutual visibility” (LV)
condition. Pointing gestures are extracted and annotated for
their internal structure. Accompanying gaze directions are
tagged and the verbal content of utterances is transcribed
orthographically. A modification to the description of gestural
phrase structure for pointing gestures is proposed. It has been
found that in mutual visibility condition, speakers tend to
gesture in the location which is potentially more visible to
their conversational partners and that the pointing holds they
perform are substantially longer than in the case of lack of
mutual visibility. However, in many cases, the gesturer did not
care to track the behaviour of his/her conversational partner in
order to make sure whether his/her gesture was noticed. Our
analyses of more complex dialogue structures do not show any
clear tendencies but allow for hypothesizing that not all
pointings are driven by communicative intentions. Some of
them seem to be performed with only little or no attention
directed to the addressee."

We compared nuclear accent production in English and Polish read speech. We investigated declaratives and three types of questions. We expected to find (a) cross-linguistic differences and (b) a cross-language generalisation which may be evidence for an intonational universal. The generalisation under investigation was a trade-off between syntactic or lexical question markers in the text and the production of high pitch. The expected cross-linguistic difference involved the production and distribution of nuclear accent types in different kinds of utterances.

In this study, we investigate the distribution and properties of hesitation markers produced in task-oriented dialogues by Polish and German teenagers.  The materialcomes from a multimodal corpus which has been collected in the Polish-German bor-der area, in the cities of Słubice and Frankfurt (Oder).  The speakers took part in twokinds of dialogue tasks:  a collaborative and a competitive one.  We report that thenumber and durational variability of hesitation markers produced by the speakers areinfluenced by dialogue task type and language configuration.  We inspect aspects ofinterlocutor alignment using automatized annotation mining. A number of patterns ofalignment can be visually traced for the study material.  However, only few of them can be confirmed by tests as statistically significant.

Laboratory speech is often implicitly assumed to be a stable and reliable source of data for phonetic studies and perception tests. At the same time, researchers are conscious that some of its parameters may significantly change due to minute changes in the internal states of the speaker or in recording conditions, including the environment and its acoustic properties. While we are able to detect tiny changes in the spectral and prosodic parameters of speech, it is usually more difficult to decide on their perception and communicative relevance. In the present study an attempt is made to find out whether speaking style changes occurring due to the change of experimental setting (type of microphone), related to the dialogue task stage (e.g. initial vs. final) as well as those resulting from explicit evaluation of the behavior of the participants, can be perceived in short samples of speech. Experimental stimuli are extracted from task-oriented dialogues and used in a perception test involving same-different and two-dimensional evaluation paradigms. The results show that the influence of the experimental setting factor can be detected, especially when combined with speaker factor, i.e. the specific realization of the differences does appear to be setting-dependent but also highly individual.

Gussenhoven [1] suggests that the intonation of any language involves universal and language-specific components. Following earlier work by Ohala [2], Gussenhoven argues that the phonetic implementation of fundamental frequency is affected by three biological ...

With the increasing awareness of neural links between gesture and language, the traditional distinction between “linguistic” and “gestural” behaviour has become less clear. Gestures share many traits with certain components of speech (especially prosody) and some of their aspects are studied on a relatively similar methodological basis (Gibbon 2011). Yet, the integration of gesture studies and linguistics remains a challenging task. It is not only the question of unified terminology. Nicla Rossini, the author of the book under review, has a strong academic background in linguistics but most of her research has been devoted to non-verbal communication. Her work clearly draws on the pioneers of gesture studies (McNeill, Kendon) and traditional linguistics, but, simultaneously, it is strongly driven by the cognitivist way of thinking and recent advances in neuroscience. The cognitive-neuroscientific perspective seems to offer a platform where speech and gesture can be studied jointly i...

In this demo presentation, we discuss tools and techniques developed within Borderland: a project dedicated to the analysis of paralin-guistic phenomena in the conversations of Polish and German teenagers, entitled: Language of Boundaries and Boundaries of Language (see more at: http://borderland.amu.edu.pl/).

In this text, an analysis of the lexical material obtained from the recordings of Polish task-oriented dialogues is presented. Some findings regarding quasi- and non-lexical units are also mentioned. The dialogue task involved the reconstruction of a figure made of paper. It was carried out in two settings: Mutual visibility and limited visibility. The analyses of the vocabulary are focused on finding its specific features and the differences between the structures of the vocabularies of instruction givers and instruction followers in the two settings. Frequency lists as well as most frequent bigrams extracted from the material are discussed in terms of their relatedness to the settings, the roles of the speakers and some other factors.

Infant-directed speech (IDS) is reported to differ significantly from adult-directed speech (ADS) in its acoustic-phonetic properties. In IDS, phonetic features of individual speech sounds tend to be intensified [6, 14, 20]. An example phenomenon documented for IDS in several languages is vowel hyperarticulation [25]. Thus, the formant frequency values (F1, F2) vary in the two target speaking styles. Other modifications can be observed in F0 levels (e.g., [5, 11]. Due to infants' preference towards IDS [5], laboratory-elicited IDS recordings are often used as stimuli in infant speech perception studies, aiming for example at the investigation of the effects of short-term exposure to foreign-language stimuli in early infancy and its potential contribution to the development of language learning skills (e.g., [15]). In the present study, we compare F0, F1, F2 values, and segmental duration in vowels produced by five female speakers of Polish, reading pseudoword lists in IDS and ADS.

In this presentation a planned future extension to an on-going project is to be discussed. The project is focused on identifying, describing and annotating paralinguistic features in quasi-spontaneous speech using audio data for speaker characterisation. The extension would rely on bringing the “vision ” component to the scope of analysis and including it into the present framework, i.e. including features obtained from video recordings. Paralinguistic or non-linguistic features might be categorised or grouped in multiple ways depending on the prospective application and even their definition (although long discussed) is still not quite clear (e.g. Crystal, 1966; Schötz, 2000). Different categorisations might be expected from linguists, phoneticians, speech technologists or engineers, and further differentiation may result from distinguishing various levels of analysis. In some cases, it appears sufficient and adequate to distinguish a limited number of inherently diversified catego...

In the present study, we test whether adult listeners detect phonological contrasts faster and more accurately in non-native infant-directed speech (IDS) than in non-native adult-directed speech (ADS). 21 participants listened to pairs of speech signals and their task was to decide as quickly as possible whether the signals constitute the same or different words. Each pair of signals contained target vowels or consonants representing a certain category of contrast that was phonologically relevant in a given language but not in Polish, i.e., the native language of the participants of the listening test. The signals were presented in a random order, and each pair occurred in the material
twice. Although we demonstrated significant acoustic-phonetic differences between the utterances realized in the IDS and ADS speaking styles, the listeners in our study were not significantly more accurate or faster in the identification of contrasts in either IDS or ADS stimuli.

Interlocutors in a conversation influence each other in a number of dimensions. This process may lead to observable changes in their communicative behaviour. The directions and profiles of these changes are often correlated with the quality of interaction and may predict its success. In the present study, the gestural component of communication is scrutinised for changes that may reflect the process of alignment. Two types of task-oriented dialogues between teenagers are recorded and annotated for gestures and their features. We hypothesize that the dialogue task type (collaborative vs. competitive), as well as certain culture-specific properties of alignment that differ between German and Polish pairs, may significantly influence the process of communication. In order to explore the data and detect tendencies in gestural behaviour, automatised annotation mining and statistical exploration have been used, including a moving frame approach aimed at the investigation of co-occurring strokes as well as re-occurring strokes and their features. Significant differences between German and Polish speakers, as well as between the two dialogue types, have been found in the number of gestures, stroke duration and amplitude.

Artykuł pokazuje sposoby użycia humoru do stratyfikacji społeczności użytkowników internetu. Autor wymienia różne typy tekstów humory- stycznych. Analizuje je jako wyznaczniki istniejących podziałów oraz jako potencjalne czynniki klasyfikujące

Intonation is often considered the most problematic component of prosody. It may largely contribute to the meaning of an utterance and provide rich indexical information. The complexity of the intonation-related phenomena, from the acoustic level up to the cognitive process of semiosis, resulted in a variety of conceptualisations and divergent theoretical approaches. In the present text, an attempt is made to sketch the late Wiktor Jassem’s contribution to intonation studies, starting from his early work on colloquial British English to the studies focused on the Polish language as well as his general perspective on prosody and on the methods of its exploration. Some of his unpublished contribution to research projects on various aspects of prosody is also acknowledged. The present overview is based both on available written materials (publications, references, reports, reviews) and the author’s personal communication with Wiktor Jassem and his collaborators. OPEN ACCESS

The present study is focused on selected types of fillers, quasi-words and non-lexical words that are generally categorized as expressing positive or negative response in Polish task-oriented dialogues. Basic phonetic properties of such units are analyzed with a special focus on intonation. Some of their possible realizations are shown and some relations between their intonational form and meaning are hypothesized. A brief note on comparative background from our recent work is also provided and some implications for speech technology are mentioned.
Key words: dialogue, intonation.

In the present text, we propose a gesture annotation scheme
and procedure designed for the analysis of gestural and
gestural-prosodic entrainment in multilingual and
multicultural corpora. The annotation scheme is based on the
traditional gesture phrase structure but focused on more
detailed characteristics of the stroke. In addition, it contains
head movement, body position and gaze direction tags. The
scheme is implemented as a hierarchical, multi-tier template
for ELAN, with pop-up lists of available tags and predefined
inter-tier dependencies. It can be used with any other
annotation software that offers time-aligned multi-tier intervalbased annotation. We also propose how annotation data based on our scheme can be applied to design and calculate measures of entrainment.

" Abstract
The aim of this pilot study is a preliminary description of intra-phrasal disfluencies in Polish task-oriented dialogues.Some cues to their identification and discrimination aretentatively proposed. The data come from eight paper foldingtask dialogue sessions. Disfluencies are categorized on the basis of their acoustic-phonetic content. Selected prosodic properties of their neighborhood are analyzed, includingintonational realizations and durations of the surroundingsyllables. Accompanying hand movements of the speakers aredescribed for the alignment of their phases with respective disfluencies in speech."

In the present text, some recent changes in the perspective taken by psycholinguists in the study of language and communication are discussed. T heir interests seem to gradually shift from the study of language processing as an isolated and independent phenomenon towards inclusion of more interactional factors being indispensable components of interpersonal communication and involved in the process of communicative alignment. A lignment is here understood as a complex phenomenon that goes beyond increasing similarity of mental representations and related communicative behaviour. It simultaneously occurs on many levels and in various modalities, including those traditionally excluded
from language study. A s a consequence, it implies not only more flexibility in the study of interpersonal communication but it also means a shift in the psycholinguistic methodology and probably also in the widely accepted picture of language and its limits.
Keywords: interactivity, alignment, entrainment, dialogue

Content
Introduction.....................................................................................................7
1. Planing and design..................................................................................11
Plan and checklist.........................................................................................11
Elicitation techniques....................................................................................15
Reading....................................................................................................15
Reaction...................................................................................................17
Interaction................................................................................................18
Speakers........................................................................................................20
Preparing for recordings...............................................................................22
Deception and debriefing..............................................................................22
Legal and ethical issues................................................................................23
Rigour and flexibility....................................................................................25
2. Recording.................................................................................................27
What is to be recorded..................................................................................27
Setting (recording environment)...................................................................28
Equipment and how to use it.........................................................................30
Portable digital audio recorder................................................................30
Microphones............................................................................................33
Accessories..............................................................................................36
More on the procedure..................................................................................39
How should I speak?...............................................................................39
How should I sit?....................................................................................39
Noise is easy to produce and difficult to remove....................................40
3. Processing................................................................................................43
Introduction...................................................................................................43
Digital audio signal.......................................................................................44
Cutting and splitting speech recordings........................................................48
General rules...........................................................................................49
Cutting the speech signal where there are no pauses..............................50
Adding silence.........................................................................................51
Vowels (vocalic clusters) and approximants...........................................52
Amplitude normalization..............................................................................52
Compression.................................................................................................54
Noise gate......................................................................................................57
Advanced de-noising....................................................................................57
Reconstruction of the signal..........................................................................58
Dillemas and good practices.........................................................................59
4. Transcription, segmentation, annotation..............................................61
Transcription.................................................................................................61
Time alignment and segmentation................................................................65
(Multilayer) annotation.................................................................................68
Specifications for annotation of paralinguistic or non-linguistic features....70
Annotation mining..................................................................................71
Cross-modal interactions.........................................................................72
Labels, categories and dimensions................................................................73
Example annotation specifications................................................................76
5. Data and metadata management...........................................................83
Data and metadata.........................................................................................83
Data safety....................................................................................................84
Approaches to data management..................................................................84
An example solution................................................................................85
Data sharing and publication........................................................................88
Are the speakers really anonymous? ......................................................89
Data recycling. Interoperability and re-usability issues................................90
References.....................................................................................................95
Appendix 1: Software tools and online resources.......................................105
Appendix 2: Further reading.......................................................................108
Acknowledgements.....................................................................................110