short-paper

Trimodal Analysis of Deceptive Behavior

Authors:

Mohamed Abouelenien,

Mihai BurzoAuthors Info & Claims

WMDD '15: Proceedings of the 2015 ACM on Workshop on Multimodal Deception Detection

Pages 9 - 13

https://doi.org/10.1145/2823465.2823470

Published: 13 November 2015 Publication History

Abstract

The need arises for developing a more reliable deception detection system to address the shortcomings of the traditional polygraph tests and the dependability on physiological indicators of deceit. This paper describes a new deception detection dataset, provides a novel comparison between three modalities to identify deception including the visual, thermal, and physiological domains, and analyzes whether certain facial areas are more capable of indicating deceit. Our experimental results show a promising performance especially with the thermal modality, and provide guidelines for our data collection process and future work.

References

[1]

M. Abouelenien, V. Perez-Rosas, R. Mihalcea, and M. Burzo. Deception detection using a multimodal approach. In 16th ACM International Conference on Multimodal Interaction, ICMI 2014, November 2014.

Digital Library

[2]

M. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan. Automatic recognition of facial actions in spontaneous expressions. Journal of Multimedia, 1(6):22--35, 2006.

[3]

P. Ekman. Telling Lies: Clues to Deceit in the Marketplace, Politics and Marriage. Norton, W.W. and Company, 2001.

[4]

M. Garbey, A. Merla, and I. Pavlidis. Estimation of blood flow speed and vessel location from thermal video. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004, volume 1, pages I--356--I--363 Vol.1, June 2004.

[5]

R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge books online. Cambridge University Press, 2003.

Digital Library

[6]

U. Jain, B. Tan, and Q. Li. Concealed knowledge identification using facial thermal imaging. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1677--1680, March 2012.

[7]

G. Littlewort, J. Whitehill, T. Wu, I. Fasel, M. Frank, J. Movellan, and M. Bartlett. The computer expression recognition toolbox (cert). In 2011 IEEE International Conference on Automatic Face Gesture Recognition and Workshops (FG 2011), pages 298--305, March 2011.

[8]

T. O. Meservy, M. L. Jensen, W. Kruse, J. K. Burgoon, and J. Nunamaker, Jay F. Automatic extraction of deceptive behavioral cues from video. In Terrorism Informatics, volume 18 of Integrated Series In Information Systems, pages 495--516. Springer US, 2008.

Digital Library

[9]

M. Owayjan, A. Kashour, N. Al Haddad, M. Fadel, and G. Al Souki. The design and development of a lie detection system using facial micro-expressions. In 2012 2nd International Conference on Advances in Computational Tools for Engineering Applications (ACTEA), pages 33--38, Dec 2012.

[10]

K. K. Park, H. W. Suk, H. Hwang, and J.-H. Lee. A functional analysis of deception detection of a mock crime using infrared thermal imaging and the concealed information test. Frontiers in Human Neuroscience, 7, 2013.

[11]

I. Pavlidis and J. Levine. Thermal image analysis for polygraph testing. IEEE Engineering in Medicine and Biology Magazine, 21(6):56--64, Nov 2002.

[12]

V. Pérez-Rosas, M. Abouelenien, R. Mihalcea, Y. Xiao, C. Linton, and M. Burzo. Verbal and nonverbal clues for real-life deception detection. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2336--2346, Lisbon, Portugal, September 2015. Association for Computational Linguistics.

[13]

T. Pfister and M. Pietik\"ainen. Electronic imaging & signal processing automatic identification of facial clues to lies. SPIE Newsroom, January 2012.

[14]

T. Qin, J. K. Burgoon, J. P. Blair, and J. F. Nunamaker. Modality effects in deception detection and applications in automatic deception-detection. In Proceedings of the 38th Hawaii International Conference on System Sciences, 2005.

Digital Library

[15]

B. Rajoub and R. Zwiggelaar. Thermal facial analysis for deception detection. IEEE Transactions on Information Forensics and Security, PP(99):1--10, 2014.

Digital Library

[16]

S. N. Sinha, J.-m. Frahm, M. Pollefeys, and Y. Genc. Gpu-based video feature tracking and matching. Technical report, The University of North Carolina at Chapel Hill, 2006.

[17]

L. Warmelink, A. Vrij, S. Mann, S. Leal, D. Forrester, and R. P. Fisher. Thermal imaging as a lie detection tool at airports. Law and Human Behavior, 35(1):40--48, 2011.

[18]

M. Yancheva and F. Rudzicz. Automatic detection of deception in child-produced speech using syntactic complexity features. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 944--953, Sofia, Bulgaria, August 2013. Association for Computational Linguistics.

[19]

Y. Zhou, P. Tsiamyrtzis, P. Lindner, I. Timofeyev, and I. Pavlidis. Spatiotemporal smoothing as a basis for facial tissue tracking in thermal imaging. IEEE Transactions on Biomedical Engineering, 60(5):1280--1289, May 2013.

Cited By

King SNeal T(2024)Applications of AI-Enabled Deception Detection Using Video, Audio, and Physiological Data: A Systematic ReviewIEEE Access10.1109/ACCESS.2024.346282512(135207-135240)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3462825
Burzo MAbouelenien MPerez-Rosas VMihalcea R(2018)Multimodal deception detectionThe Handbook of Multimodal-Multisensor Interfaces10.1145/3107990.3108005(419-453)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1145/3107990.3108005
Abouelenien MBurzo MMihalcea R(2016)Human Acute Stress Detection via Integration of Physiological Signals and Thermal ImagingProceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments10.1145/2910674.2910705(1-8)Online publication date: 29-Jun-2016
https://dl.acm.org/doi/10.1145/2910674.2910705
Show More Cited By

Index Terms

Trimodal Analysis of Deceptive Behavior
1. Applied computing
  1. Law, social and behavioral sciences
    1. Psychology
  2. Life and medical sciences

Recommendations

Analyzing Thermal and Visual Clues of Deception for a Non-Contact Deception Detection Approach
PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

With increased levels of security threats and the long-term consequences of falsely accusing the innocent and freeing the guilty, there is a growing need for reliable and efficient deception detection systems. Polygraph tests are invasive and require ...
Analyzing Deceptive Speech
Proceedings, Part I, of the 10th International Conference on Engineering Psychology and Cognitive Ergonomics. Understanding Human Cognition - Volume 8019

This current work explored the speech-based attributes of participants who were being deceptive in an experimental interrogation setting. In particular, the study attempted to investigate the appropriateness of using temporal speech cues in detecting ...
The effect of conversational agent skill on user behavior during deception
Abstract
Conversational agents (CAs) are an integral component of many personal and business interactions. Many recent advancements in CA technology have attempted to make these interactions more natural and human-like. However, it is currently ...
Highlights
- People type differently when conversing with conversational agents that have more human-like conversational skill.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WMDD '15: Proceedings of the 2015 ACM on Workshop on Multimodal Deception Detection

November 2015

42 pages

ISBN:9781450339872

DOI:10.1145/2823465

General Chairs:
Mohamed Abouelenien
University of Michigan, USA
,
Mihai Burzo
University of Michigan, USA
,
Rada Mihalcea
University of Michigan, USA
,
Verónica Pérez-Rosas
University of Michigan, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICMI '15

Sponsor:

SIGCHI

ICMI '15: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 13, 2015

Washington, Seattle, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
159
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

King SNeal T(2024)Applications of AI-Enabled Deception Detection Using Video, Audio, and Physiological Data: A Systematic ReviewIEEE Access10.1109/ACCESS.2024.346282512(135207-135240)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3462825
Burzo MAbouelenien MPerez-Rosas VMihalcea R(2018)Multimodal deception detectionThe Handbook of Multimodal-Multisensor Interfaces10.1145/3107990.3108005(419-453)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1145/3107990.3108005
Abouelenien MBurzo MMihalcea R(2016)Human Acute Stress Detection via Integration of Physiological Signals and Thermal ImagingProceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments10.1145/2910674.2910705(1-8)Online publication date: 29-Jun-2016
https://dl.acm.org/doi/10.1145/2910674.2910705
Abouelenien MMihalcea RBurzo M(2016)Analyzing Thermal and Visual Clues of Deception for a Non-Contact Deception Detection ApproachProceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments10.1145/2910674.2910682(1-4)Online publication date: 29-Jun-2016
https://dl.acm.org/doi/10.1145/2910674.2910682

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten