research-article

Don't ask me what i'm like, just watch and listen

Authors:

Ruchir Srivastava,

Terence SimAuthors Info & Claims

MM '12: Proceedings of the 20th ACM international conference on Multimedia

Pages 329 - 338

https://doi.org/10.1145/2393347.2393397

Published: 29 October 2012 Publication History

Abstract

Traditional (based on psychology) approaches for personality assessment of an individual require him/her to fill up a questionnaire. This paper presents a novel way of utilizing multimodal cues to automatically fill up the questionnaire. The contributions of this work are three-fold. (1) Novel psychology-based audio/visual/lexical features are proposed and shown to be effective in predicting answers to a personality questionnaire, Big-Five Inventory-10 (BFI- 10). (2) Extracted features are used to learn linear and kernel versions of a novel regression model, 'SLoT', to automatically predict BFI-10 answers. The model is based on Sparse and Low-rank Transformation (SLoT). (3) Predicted answers are used to compute personality scores using standard BFI-10 scoring scheme. We evaluated our approach on a dataset of 3907 clips (for 50 characters from movies of diverse genres) manually labeled with BFI-10 answers and personality scores as ground-truth. Experiments indicate that the proposed 'SLoT' model effectively automates the answering process by emulating human understanding. We also conclude that predicting personality scores through predicting answers first is better than directly predicting scores based on audio/visual features (as studied in state-of-the art methods).

References

[1]

W. Apple, L. Streeter, and R. Krauss. Effects of pitch and speech rate on personal attributions. J. Pers. Soc. Psychol., 37(5):715--727, 1979.

[2]

S. Argamon, S. Dhawle, M. Koppel, and J. Pennebaker. Lexical predictors of personality type. In Proc. 2005 Joint Annual Meeting of the Interface and the Classification Society of North America, 2005.

[3]

M. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan. Fully automatic facial action recognition in spontaneous behavior. In FG, 2006.

Digital Library

[4]

S. Benini, L. Canini, and R. Leonardi. A connotative space for supporting movie affective recommendation. IEEE T. Multimedia, 13(6):1356--1370, 2011.

Digital Library

[5]

S. Boyd and L. Vandenberghe. Convex optimization. Cambridge Univ Pr, 2004.

Digital Library

[6]

R. Calix, S. Mallepudi, B. Chen, and G. Knapp. Emotion Recognition in Text for 3-D Facial Expression Rendering. IEEE T. Multimedia, 12(6):544--551, 2010.

Digital Library

[7]

M. Everingham, J. Sivic, and A. Zisserman. "Hello! My name is... Buffy" -- automatic naming of characters in TV video. In BMVC, 2006.

[8]

L. Jensen-Campbell, J. Knack, A. Waldrip, and S. Campbell. Do big five personality traits associated with self-control influence the regulation of anger and aggression? J. Res. Pers., 41(2):403--424, 2007.

[9]

O. John, L. Naumann, and C. Soto. Paradigm shift to the integrative Big Five trait taxonomy: History, measurement, and conceptual issues. In O.P. John, R.W. Robins and L.A. Pervin (Eds.) Handbook of Personality: theory and research, pages 114--158, 2008.

[10]

C. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In CVPR 2009.

[11]

A. Lampropoulos, P. Lampropoulou, and G. Tsihrintzis. A cascade-hybrid music recommender system for mobile services based on musical genre classification and personality diagnosis. Multimed. Tools Appl., pages 1--18, 2011.

Digital Library

[12]

Z. Lin, M. Chen, L. Wu, and Y. Ma. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. Arxiv preprint arXiv:1009.5055, 2010.

[13]

P. Litle and M. Zuckerman. Sensation seeking and music preferences. Pers. Indiv. Differ., 7(4):575--578, 1986.

[14]

F. Mairesse, M. Walker, M. Mehl, and R. Moore. Using linguistic cues for the automatic recognition of personality in conversation and text. J. Artif. Intell. Res., 30(1):457--500, 2007.

[15]

R. Mattick and J. Clarke. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav. Res. Ther., 36(4):455--470, 1998.

[16]

R. McCrae and O. John. An introduction to the five-factor model and its applications. J. Pers., 60(2):175--215, 1992.

[17]

G. Mohammadi, A. Vinciarelli, and M. Mortillaro. The voice of personality: mapping nonverbal vocal behavior into trait attributions. In Proc. 2nd international workshop on Social signal processing, 2010.

Digital Library

[18]

J. Oberlander and S. Nowson. Whose thumb is it anyway?: classifying author personality from weblog text. In Proc. COLING/ACL on Main conference poster sessions, 2006.

Digital Library

[19]

W. Pavot, E. Diener, and F. Fujita. Extraversion and happiness. Pers. Indiv. Differ., 11(12):1299--1306, 1990.

[20]

F. Pianesi, N. Mana, A. Cappelletti, B. Lepri, and M. Zancanaro. Multimodal recognition of personality traits in social interactions. In ICMI, 2008.

Digital Library

[21]

R. Picard. Affective computing. The MIT press, 2000.

Digital Library

[22]

M. Power and M. Tarsia. Basic and complex emotions in depression and anxiety. Clin. Psychol. Psychot., 14(1):19--31, 2007.

[23]

B. Rammstedt and O. John. Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German. J. Res. Pers., 41(1):203--212, 2007.

[24]

P. Rentfrow and S. Gosling. The do re mi's of everyday life: The structure and personality correlates of music preferences. J. Pers. Soc. Psychol., 84(6):1236--1256, 2003.

[25]

S. Srivastava, O. John, S. Gosling, and J. Potter. Development of personality in early and middle adulthood: Set like plaster or persistent change? J. Pers. Soc. Psychol., 84(5):1041--1053.

[26]

C. Whissell. The dictionary of affect in language. Emotion: Theory, research, and experience, 4:113--131, 1989.

[27]

C. Windle. Further studies of test-retest effect on personality questionnaires. Educ. Psychol. Meas., 15(3):246--253, 1955.

[28]

P. Wojcik. Movie acting, the film reader. Psychology Press, 2004.

[29]

G. Zen, B. Lepri, E. Ricci, and O. Lanz. Space speaks: towards socially and personality aware visual surveillance. In Proc. 1st ACM international workshop on Multimodal pervasive video analysis, 2010.

Digital Library

[30]

Y. Zhai and M. Shah. Video scene segmentation using markov chain monte carlo. IEEE T. Multimedia, 8(4):686--697, 2006.

Digital Library

[31]

G. Zhu, S. Yan, and Y. Ma. Image tag refinement towards low-rank, content-tag prior and error sparsity. In ACM Multimedia, 2010.

Digital Library

Cited By

Jacques Junior JGucluturk YPerez MGuclu UAndujar CBaro XEscalante HGuyon Ivan Gerven Mvan Lier REscalera S(2022)First Impressions: A Survey on Vision-Based Apparent Personality Trait AnalysisIEEE Transactions on Affective Computing10.1109/TAFFC.2019.293005813:1(75-95)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TAFFC.2019.2930058
Guntuku SZhou JRoy SLin WTsang I(2018)`Who Likes What and, Why?ź Insights into Modeling Usersź Personality Based on Image `LikesźIEEE Transactions on Affective Computing10.5555/3195728.31957379:1(130-143)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.5555/3195728.3195737
Subramanian RWache JAbadi MVieriu RWinkler SSebe N(2018)ASCERTAIN: Emotion and Personality Recognition Using Commercial SensorsIEEE Transactions on Affective Computing10.1109/TAFFC.2016.26252509:2(147-160)Online publication date: 1-Apr-2018
https://doi.org/10.1109/TAFFC.2016.2625250
Show More Cited By

Index Terms

Don't ask me what i'm like, just watch and listen
1. Applied computing
  1. Law, social and behavioral sciences
2. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks

Recommendations

The voice of personality: mapping nonverbal vocal behavior into trait attributions
SSPW '10: Proceedings of the 2nd international workshop on Social signal processing

This paper reports preliminary experiments on automatic attribution of personality traits based on nonverbal vocal behavioral cues. In particular, the work shows how prosodic features can be used to predict, with an accuracy up to 75% depending on the ...
Automatic user preference elicitation for music recommendation
PCM'12: Proceedings of the 13th Pacific-Rim conference on Advances in Multimedia Information Processing

Recommendation Systems involve effort from the user to elicit their preference for the item to be recommended. The contribution of this paper is in eliminating such effort by automatically assessing user's personality and using the personality scores ...
Identification of personality traits from handwritten text documents using multi-label classification models

Handwriting is widely investigated to mark emotional states and personality. However, the majority of the studies are based on graphology, and do not utilise personality factor models. We use the well-known five-factor model which says that people possess ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '12: Proceedings of the 20th ACM international conference on Multimedia

October 2012

1584 pages

ISBN:9781450310895

DOI:10.1145/2393347

General Chairs:
Noboru Babaguchi
Osaka University, Japan
,
Kiyoharu Aizawa
The University of Tokyo, Japan
,
John Smith
IBM, USA
,
Program Chairs:
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Thomas Plagemann
University of Oslo, Norway
,
Xian-Sheng Hua
Microsoft, USA
,
Rong Yan
Facebook, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '12

Sponsor:

SIGMM

MM '12: ACM Multimedia Conference

October 29 - November 2, 2012

Nara, Japan

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
496
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)8

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jacques Junior JGucluturk YPerez MGuclu UAndujar CBaro XEscalante HGuyon Ivan Gerven Mvan Lier REscalera S(2022)First Impressions: A Survey on Vision-Based Apparent Personality Trait AnalysisIEEE Transactions on Affective Computing10.1109/TAFFC.2019.293005813:1(75-95)Online publication date: 1-Jan-2022
https://doi.org/10.1109/TAFFC.2019.2930058
Guntuku SZhou JRoy SLin WTsang I(2018)`Who Likes What and, Why?ź Insights into Modeling Usersź Personality Based on Image `LikesźIEEE Transactions on Affective Computing10.5555/3195728.31957379:1(130-143)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.5555/3195728.3195737
Subramanian RWache JAbadi MVieriu RWinkler SSebe N(2018)ASCERTAIN: Emotion and Personality Recognition Using Commercial SensorsIEEE Transactions on Affective Computing10.1109/TAFFC.2016.26252509:2(147-160)Online publication date: 1-Apr-2018
https://doi.org/10.1109/TAFFC.2016.2625250
Guntuku SZhou JRoy SLin WTsang I(2018)‘Who Likes What and, Why?’ Insights into Modeling Users’ Personality Based on Image ‘Likes’IEEE Transactions on Affective Computing10.1109/TAFFC.2016.25811689:1(130-143)Online publication date: 1-Jan-2018
https://doi.org/10.1109/TAFFC.2016.2581168
Zafar ZHussain Paplu SBerns K(2018)Automatic Assessment of Human Personality Traits: A Step Towards Intelligent Human-Robot Interaction2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids)10.1109/HUMANOIDS.2018.8624975(1-9)Online publication date: Nov-2018
https://doi.org/10.1109/HUMANOIDS.2018.8624975
Miranda-Correa JPatras I(2018)A Multi-Task Cascaded Network for Prediction of Affect, Personality, Mood and Social Context Using EEG Signals2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)10.1109/FG.2018.00060(373-380)Online publication date: May-2018
https://doi.org/10.1109/FG.2018.00060
Kindiroglu AAkarun LAran O(2017)Multi-domain and multi-task prediction of extraversion and leadership from meeting videosEURASIP Journal on Image and Video Processing10.1186/s13640-017-0224-z2017:1Online publication date: 21-Nov-2017
https://doi.org/10.1186/s13640-017-0224-z
Schuller B(2017)EditorialIEEE Transactions on Affective Computing10.1109/TAFFC.2017.26628588:1(1-2)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1109/TAFFC.2017.2662858
Celiktutan OGunes H(2017)Automatic Prediction of Impressions in Time and across Varying ContextIEEE Transactions on Affective Computing10.1109/TAFFC.2015.25134018:1(29-42)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1109/TAFFC.2015.2513401
Chen LZaromb FYang ZLeong CMartin-Raugh M(2017)Can a machine pass a situational judgment test measuring personality perception?2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)10.1109/ACIIW.2017.8272615(7-11)Online publication date: Oct-2017
https://doi.org/10.1109/ACIIW.2017.8272615
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents