short-paper

Prediction/Assessment of communication skill using multimodal cues in social interactions

Author:

Sowmya RasipuramAuthors Info & Claims

ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

Pages 546 - 549

https://doi.org/10.1145/2993148.2997617

Published: 31 October 2016 Publication History

Abstract

Understanding people’s behavior in social interactions is a very interesting problem in Social Computing. In this work, we automatically predict the communication skill of a person in various kinds of social interactions. We consider in particular, 1) Interview-based interactions - asynchronous interviews (web-based interview) Vs. synchronous interviews (regular face-to-face interviews) and 2) Non-interview based interactions - dyad and triad conversations (group discussions). We automatically extract multimodal cues related to verbal and non-verbal behavior content of the interaction. First, in interview-based interactions, we consider previously uninvestigated scenarios of comparing the participant’s behavioral and perceptual changes in the two contexts. Second, we address different manifestations of communication skill in different settings (face-to-face interaction vs. group). Third, the non-interview based interactions also leads to answer research questions such as “the relation between a good communicator and other group variables like dominance or leadership” Finally we look at several attributes (manually annotated) and features/feature groups (automatically extracted) that predicts communication skill well in all settings.

References

[1]

Hassle-free Efficient Hiring. https://www.talview.com/automated-video.

[2]

Batrinca, L. M. Multimodal personality recognition from audiovisual data. In Diss. University of Trento (2013).

[3]

Biel, J. I., Teijeiro-Mosquera, L., and Gatica-Perez, D. Facetube: predicting personality from facial expressions of emotion in online conversational video. In Proceedings of the 14th ACM int. COnf. on Multimodal interaction, ACM (2012), 53–56.

Digital Library

[4]

Celiktutan, O., Sariyanidi, E., and Gunes, H. Let me tell you about your personality!: Real-time personality prediction from nonverbal behavioural cues. In IEEE Int. Conf. and Workshops on Automatic Face and Gesture Recognition (FG), 2015, vol. 1, IEEE (2015), 1–1.

[5]

Giannakopoulos, T., and Pikrakis, A. Introduction to Audio Analysis: A MATLAB ® Approach. Academic Press, 2014.

Digital Library

[6]

Hoque, M. E., Courgeon, M., Martin, J.-C., Mutlu, B., and Picard, R. W. Mach: My automated conversation coach. In ACM Int. Joint Conf. on Pervasive and ubiquitous computing, ACM (2013), 697–706.

Digital Library

[7]

Hung, H., Jayagopi, D. B., Yeo, C., Friedland, G., Ba, S. O., Odobez, J.-M., Ramchandran, K., Mirghafori, N., and Gatica-Perez, D. Using audio and video features to classify the most dominant person in a group meeting. In Proc. ACM Int. Conf. on Multimedia (ACM MM) (Augsburg, Sep. 2007).

Digital Library

[8]

Keith, A., André, E., Baur, T., Bernardini, S., Chollet, M., Chryssafidou, E., Damian, I., Ennis, C., Egges, A., Gebhard, P., et al. The TARDIS framework: intelligent virtual agents for social coaching in job interviews. In Advances in Computer Entertainment. Springer, 2013, 476–491.

[9]

Littlewort, G., Whitehill, J., Wu, T., Fasel, I., Frank, M., Movellan, J., and Bartlett, M. The computer expression recognition toolbox (cert). In Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, IEEE (2011), 298–305.

[10]

Naim, I., Tanveer, M. I., Gildea, D., and Hoque, M. E. Automated prediction and analysis of job interview performance: The role of what you say and how you say it. Automatic Face and Gesture Recognition (FG) (2015).

[11]

Nguyen, L. S., and Gatica-Perez, D. I would hire you in a minute: Thin slices of nonverbal behavior in job interviews. In ACM Int. Conf. on Multimodal Interaction, ACM (2015), 51–58.

Digital Library

[12]

Rosenberg, A., and Hirschberg, J. Acoustic/prosodic and lexical correlates of charismatic speech. 513–516.

[13]

Sanchez-Cortes, D., Aran, O., Mast, M. S., and Gatica-Perez, D. A nonverbal behavior approach to identify emergent leaders in small groups. Multimedia, IEEE Transactions on 14, 3 (2012), 816–832.

Digital Library

[14]

Son, N. L. Hirability in the wild: Analysis of online conversational video resumes. vol. 18 (2016).

[15]

Spitzberg, B. H., and Adams, T. W. CSRS, the conversational skills rating scale: an instructional assessment of interpersonal competence. NCA, National Communication Association, 2007.

[16]

Teijeiro-Mosquera, L., Biel, J.-I., Alba-Castro, J. L., and Gatica-Perez, D. What your face vlogs about: Expressions of emotion and big-five traits impressions in youtube.

[17]

Weninger, F., Krajewski, J., Batliner, A., and Schuller, B. The voice of leadership: Models and performances of automatic analysis in online speeches. IEEE Transactions on Affective Computing 3, 4 (2012), 496–508.

Digital Library

[18]

Zechner, K., and Bejar, I. I. Towards automatic scoring of non-native spontaneous speech. Conf. on human language technology of the North American chapter of the association of computational linguistics. (2006), 216–223.

Digital Library

Cited By

Iharada RMuraoka TIshikawa NEgi H(2021)Analysis of Emphasis and Prosodic Features on Face-to-Face Discussions2021 IEEE International Conference on Engineering, Technology & Education (TALE)10.1109/TALE52509.2021.9678664(558-564)Online publication date: 5-Dec-2021
https://doi.org/10.1109/TALE52509.2021.9678664

Index Terms

Prediction/Assessment of communication skill using multimodal cues in social interactions
1. Applied computing
  1. Law, social and behavioral sciences
2. Information systems
  1. Information retrieval
    1. Document representation
      1. Content analysis and feature selection
    2. Specialized information retrieval
      1. Multimedia and multimodal retrieval
        Speech / audio search

Recommendations

Multimodal Analysis for Communication Skill and Self-Efficacy Level Estimation in Job Interview Scenario
MUM '22: Proceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia

An interview for a job recruiting process requires applicants to demonstrate their communication skills. Interviewees sometimes become nervous about the interview because interviewees themselves do not know their assessed score. This study investigates ...
Asynchronous video interviews vs. face-to-face interviews for communication skill measurement: a systematic study
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

Communication skill is an important social variable in em- ployment interviews. As recent trends point to, increasingly asynchronous or interface-based video interviews are becom- ing popular. Also getting increasing interest is automatic hiring ...
Multimodal analysis of body communication cues in employment interviews
ICMI '13: Proceedings of the 15th ACM on International conference on multimodal interaction

Hand gestures and body posture are intimately linked to speech as they are used to enrich the vocal content, and are therefore inherently multimodal. As an important part of nonverbal behavior, body communication carries relevant information that can ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

October 2016

605 pages

ISBN:9781450345569

DOI:10.1145/2993148

General Chairs:
Yukiko I. Nakano
Seikei University, Japan
,
Elisabeth André
Augsburg University, Germany
,
Toyoaki Nishida
Kyoto University, Japan
,
Program Chairs:
Louis-Philippe Morency
Carnegie Mellon University, USA
,
Carlos Busso
University of Texas at Dallas, USA
,
Catherine Pelachaud
ISIR, France / University of Paris6, France

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICMI '16

Sponsor:

SIGCHI

ICMI '16: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 12 - 16, 2016

Tokyo, Japan

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
134
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Iharada RMuraoka TIshikawa NEgi H(2021)Analysis of Emphasis and Prosodic Features on Face-to-Face Discussions2021 IEEE International Conference on Engineering, Technology & Education (TALE)10.1109/TALE52509.2021.9678664(558-564)Online publication date: 5-Dec-2021
https://doi.org/10.1109/TALE52509.2021.9678664

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten