short-paper

Examining linguistic content and skill impression structure for job interview analytics in hospitality

Authors:

Skanda Muralidhar,

Daniel Gatica-PerezAuthors Info & Claims

MUM '17: Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia

Pages 339 - 343

https://doi.org/10.1145/3152832.3152866

Published: 26 November 2017 Publication History

Abstract

First impressions are critical to professional interactions especially in the context of employment interviews. This work investigates connections between linguistic content and first impressions in job interviews and the structure of ten soft skills and overall impressions. Towards this, we transcribe 169 role-played job interviews conducted at a hospitality school and analyze the linguistic content using off-the-shelf software. To understand the structure of the soft skill impressions, we conduct a principal component analysis. We then develop methods to automatically infer impressions using verbal and nonverbal features and their combination. Results indicate low predictive power of verbal cues for overall impression (R² = 0.11). Combined verbal and nonverbal cues explain up to 34% of variance, a marginal improvement over R² = 0.32 using only nonverbal cues. The use of principal components reveals a major component associated to overall positive and negative impressions that when used as labels for supervised learning results in a regression performance of R² = 0.41.

References

[1]

Murray R Barrick, Jonathan A Shaffer, and Sandra W DeGrassi. 2009. What you see may not be what you get: relationships among self-presentation tactics and ratings of interview and job performance. J. Applied Psychology 94, 6 (2009), 1394.

[2]

Joan-Isaac Biel and Daniel Gatica-Perez. 2013. The YouTube lens: Crowdsourced personality impressions and audiovisual analysis of vlogs. IEEE Trans. on Multimedia 15, 1 (2013).

Digital Library

[3]

Joan-Isaac Biel, Vagia Tsiminaki, John Dines, and Daniel Gatica-Perez. 2013. Hi youtube!: Personality impressions and verbal content in social video. In Proceedings of the 15th ACM on International conference on multimodal interaction. ACM, 119--126.

Digital Library

[4]

Lei Chen, Gary Feng, Chee Wee Leong, Blair Lehman, Michelle Martin-Raugh, Harrison Kell, Chong Min Lee, and Su-Youn Yoon. 2016. Automated scoring of interview videos using Doc2Vec multimodal feature extraction paradigm. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. ACM, 161--168.

Digital Library

[5]

Cindy Chung and James W Pennebaker. 2007. The psychological functions of function words. Social communication (2007), 343--359.

[6]

Jared R Curhan and Alex Pentland. 2007. Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. J. Applied Psychology 92, 3 (2007).

[7]

Timothy DeGroot and Janaki Gooty. 2009. Can nonverbal cues be used to make meaningful personality attributions in employment interviews? J. Business and Psychology 24, 2 (2009).

[8]

Allen I Huffcutt, James M Conway, Philip L Roth, and Nancy J Stone. 2001. Identification and meta-analytic assessment of psychological constructs measured in employment interviews. J. Applied Psychology 86, 5 (2001).

[9]

Hayley Hung and Daniel Gatica-Perez. 2010. Estimating cohesion in small groups using audio-visual nonverbal behavior. IEEE Transactions on Multimedia 12, 6 (2010), 563--575.

Digital Library

[10]

Andrew S Imada and Milton D Hakel. 1977. Influence of nonverbal communication and rater proximity on impressions and decisions in simulated employment interviews. J. Applied Psychology 62, 3 (1977).

[11]

Mark Knapp, Judith Hall, and Terrence Horgan. 2013. Nonverbal communication in human interaction. Cengage Learning.

[12]

Max Kuhn. 2016. A Short Introduction to the caret Package. (2016).

[13]

Skanda Muralidhar, Laurent Son Nguyen, Denise Frauendorfer, Jean-Marc Odobez, Marianne Schimd-Mast, and Daniel Gatica-Perez. 2016. Training on the Job: Behavioral Analysis of Job Interviews in Hospitality. In Proc. 18th ACM Int. Conf. Multimodal Interact. 84--91.

Digital Library

[14]

Iftekhar Naim, M Iftekhar Tanveer, Daniel Gildea, and Mohammed Ehsan Hoque. 2015. Automated prediction and analysis of job interview performance: The role of what you say and how you say it. Proc. IEEE FG (2015).

[15]

Laurent Son Nguyen. 2015. Computational Analysis Of Behavior In Employment Interviews And Video Resumes. Ph.D. Dissertation. École Polytechnique Fédérale de Lausanne.

[16]

Laurent Son Nguyen, Denise Frauendorfer, Marianne Schmid Mast, and Daniel Gatica-Perez. 2014. Hire me: Computational inference of hirability in employment interviews based on nonverbal behavior. IEEE Trans. on Multimedia 16, 4 (2014).

Digital Library

[17]

Laurent Son Nguyen and Daniel Gatica-Perez. 2015. I Would Hire You in a Minute: Thin Slices of Nonverbal Behavior in Job Interviews. In Proc. ACM ICMI.

Digital Library

[18]

Richard E Nisbett and Timothy D Wilson. 1977. The halo effect: Evidence for unconscious alteration of judgments. Journal of personality and social psychology 35, 4 (1977), 250.

[19]

James W Pennebaker and Laura A King. 1999. Linguistic styles: language use as an individual difference. Journal of personality and social psychology 77, 6 (1999), 1296.

[20]

Dairazalia Sanchez-Cortes, Oya Aran, Marianne Schmid Mast, and Daniel Gatica-Perez. 2010. Identifying emergent leadership in small groups using nonverbal communicative cues. In Int Conf. on Multimodal interfaces and the workshop on machine learning for Multimodal Interaction. ACM, 39.

Digital Library

[21]

Dairazalia Sanchez-Cortes, Joan-Isaac Biel, Shiro Kumano, Junji Yamato, Kazuhiro Otsuka, and Daniel Gatica-Perez. 2013. Inferring mood in ubiquitous conversational video. In Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia. ACM, 22.

Digital Library

[22]

Dairazalia Sanchez-Cortes, Petr Motlicek, and Daniel Gatica-Perez. 2012. Assessing the impact of language style on emergent leadership perception from ubiquitous audio. In Proceedings of the 11th international conference on mobile and ubiquitous multimedia. ACM, 33.

Digital Library

[23]

Patrick E Shrout and Joseph L Fleiss. 1979. Intraclass correlations: uses in assessing rater reliability. Psychological bulletin 86, 2 (1979).

[24]

Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology 29, 1 (2010), 24--54.

[25]

Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemometrics and intelligent laboratory systems 2, 1--3 (1987), 37--52.

Cited By

Lv JChen CLiang Z(2024)Automated Scoring of Asynchronous Interview Videos Based on Multi-Modal Window-Consistency FusionIEEE Transactions on Affective Computing10.1109/TAFFC.2023.329433515:3(799-814)Online publication date: Jul-2024
https://doi.org/10.1109/TAFFC.2023.3294335
Muralidhar SMast MGatica-Perez D(2018)A Tale of Two InteractionsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32649392:3(1-24)Online publication date: 18-Sep-2018
https://dl.acm.org/doi/10.1145/3264939

Examining linguistic content and skill impression structure for job interview analytics in hospitality
1. Applied computing
  1. Law, social and behavioral sciences

Recommendations

Training on the job: behavioral analysis of job interviews in hospitality
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

First impressions play a critical role in the hospitality industry and have been shown to be closely linked to the behavior of the person being judged.In this work, we implemented a behavioral training framework for hospitality students with the goal of ...
Stressful first impressions in job interviews
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

Stress can impact many aspects of our lives, such as the way we interact and work with others, or the first impressions that we make. In the past, stress has been most commonly assessed through self-reported questionnaires; however, advancements in ...
How may I help you? behavior and impressions in hospitality service encounters
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal Interaction

In the service industry, customers often assess quality of service based on the behavior, perceived personality, and other attributes of the front line service employees they interact with. Interpersonal communication during these interactions is key to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

MUM '17: Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia

November 2017

567 pages

ISBN:9781450353786

DOI:10.1145/3152832

General Chairs:
Niels Henze
University of Stuttgart, Germany
,
Paweł Woźniak
University of Stuttgart, Germany
,
Kaisa Väänänen
Tampere University of Technology, Finland
,
Program Chairs:
Julie Williamson
University of Glasgow, United Kingdom
,
Stefan Schneegass
University of Duisburg-Essen, Germany

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Swiss National Science Foundation (SNSF)

Conference

MUM 2017

MUM 2017: The 16th International Conference on Mobile and Ubiquitous Multimedia

November 26 - 29, 2017

Stuttgart, Germany

Acceptance Rates

Overall Acceptance Rate 190 of 465 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
141
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lv JChen CLiang Z(2024)Automated Scoring of Asynchronous Interview Videos Based on Multi-Modal Window-Consistency FusionIEEE Transactions on Affective Computing10.1109/TAFFC.2023.329433515:3(799-814)Online publication date: Jul-2024
https://doi.org/10.1109/TAFFC.2023.3294335
Muralidhar SMast MGatica-Perez D(2018)A Tale of Two InteractionsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/32649392:3(1-24)Online publication date: 18-Sep-2018
https://dl.acm.org/doi/10.1145/3264939

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten