research-article

Transductive Transfer LDA with Riesz-based Volume LBP for Emotion Recognition in The Wild

Authors:

Tong ZhangAuthors Info & Claims

ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

Pages 491 - 496

https://doi.org/10.1145/2818346.2830584

Published: 09 November 2015 Publication History

Abstract

In this paper, we propose the method using Transductive Transfer Linear Discriminant Analysis (TTLDA) and Riesz-based Volume Local Binary Patterns (RVLBP) for image based static facial expression recognition challenge of the Emotion Recognition in the Wild Challenge (EmotiW 2015). The task of this challenge is to assign facial expression labels to frames of some movies containing a face under the real word environment. In our method, we firstly employ a multi-scale image partition scheme to divide each face image into some image blocks and use RVLBP features extracted from each block to describe each facial image. Then, we adopt the TTLDA approach based on RVLBP to cope with the expression recognition task. The experiments on the testing data of SFEW 2.0 database, which is used for image based static facial expression challenge, demonstrate that our method achieves the accuracy of 50%. This result has a 10.87% improvement over the baseline provided by this challenge organizer.

References

[1]

C. Chang and C. Lin. Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(27):1--27, 2011.

Digital Library

[2]

A. Dhall, R. Goecke, J. Joshi, K. Sikka, and T. Gedeon. Emotion recognition in the wild challenge 2014: Baseline, data and protocol. In ACM International Conference on Multimodal Interaction, pages 461--466, 2014.

Digital Library

[3]

A. Dhall, R. Goecke, J. Joshi, M. Wagner, and T. Gedeon. Emotion recognition in the wild challenge 2013. In ACM International Conference on Multimodal Interaction, pages 371--372, 2013.

Digital Library

[4]

A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark. In IEEE International Conference on Computer Vision, pages 2106--2112, 2011.

[5]

A. Dhall, O. R. Murthy, R. Goecke, J. Joshi, and T. Gedeon. Video and image based emotion recognition challenges in the wild: Emotiw 2015. In ACM International Conference on Multimodal Interaction, 2015.

Digital Library

[6]

M. Felsberg and G. Sommer. The monogenic signal. IEEE Transactions on Signal Processings, 49(12):3136--3144, 2001.

Digital Library

[7]

X. Huang, A. Dhall, G. Zhao, R. Goecke, and M. Pietikäinen. Riesz-based volume local binary pattern and a novel group expression model for group happiness intensity analysis. In BMVC, pages 1--13, 2015.

[8]

X. Huang, Q. He, X. Hong, G. Zhao, and M. Pietikäinen. Improved spatiotemporal local monogenic binary pattern for emotion recognition in the wild. In ACM International Conference on Multimodal Interaction, pages 514--520, 2014.

Digital Library

[9]

X. Huang, G. Zhao, W. Zheng, and M. Pietikäinen. Spatiotemporal local monogenic binary patterns for facial expression recognition. IEEE Signal Processing Letters, 19(5):243--246, May 2012.

[10]

T. Ojala, M. Pietikäinen, and T. Mäenpää. Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. IEEE Transaction on Pattern Analysis and Machine Intelligence, 24(7):971--987, 2002.

Digital Library

[11]

V. Ojansivu and J. Heikkilä. Blur insensitive texture classification using local phase quantization. In International Conference on Image and Signal Processing, pages 236--243, 2008.

Digital Library

[12]

J. Pan and Y. Tang. Texture classification based on BIMF Monogenic signals. In Asian Conference on Computer Vision, pages 177--187, 2012.

Digital Library

[13]

S. Pan and Q. Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10):1345--1359, 2010.

Digital Library

[14]

E. Stein and G. Weiss. Introduction to Fourier analysis on Euclidean spaces. Princeton University Press, 1971.

[15]

M. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic. Avec 2013: the continuous audio/visual emotion and depression recognition challenge. In ACM International Workshop on Audio/visual emotion challenge, pages 3--10, 2013.

Digital Library

[16]

M. Yang, L. Zhang, S. Shiu, and D. Zhang. Monogenic binary coding: An efficient local feature extraction approach to face recognition. IEEE Transactions on Information Forensics and Security, 7(6):1738--1751, 2012.

Digital Library

[17]

Z. Zeng, M. Pantic, G. Roisman, and T. Huang. A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1):39--58, 2009.

Digital Library

[18]

C. Zetzsche and E. Barth. Fundamental limits of linear filters in the visual processing of two dimensional signals. Vision Research, 30(7):1111--1117, 1990.

[19]

L. Zhang and H. Li. Encoding local image patterns using Riesz transform: With applications to palmpring and finger-knuckle-print recognition. Image and Vision Computing, 30(12):1043--1051, 2012.

Digital Library

[20]

L. Zhang, D. Zhang, Z. Guo, and D. Zhang. Monogenic-LBP: A new approach for rotation invariant texture classification. In IEEE International Conference on Image Processing, pages 2677--2680, 2010.

[21]

W. Zheng. Multi-view facial expression recognition based on group sparse reduced-rank regression. IEEE Transactions on Affective Computing, 5(1):71--85, 2014.

[22]

W. Zheng, M. Xin, X. Wang, and B. Wang. A novel speech emotion recognition method via incomplete sparse least square regression. IEEE Signal Processing Letters, 21(5):569--572, 2014.

[23]

W. Zheng and X. Zhou. Cross-pose color facial expression recognition using transductive transfer linear discriminant analysis. In International Conference on Image Processing, 2015.

Cited By

LI SZONG YLU CTANG CZHAO Y(2023)Group Sparse Reduced Rank Tensor Regression for Micro-Expression RecognitionIEICE Transactions on Information and Systems10.1587/transinf.2022EDL8073E106.D:4(575-578)Online publication date: 1-Apr-2023
https://doi.org/10.1587/transinf.2022EDL8073
Cai JMeng ZKhan ALi ZO’Reilly JTong Y(2023)Probabilistic Attribute Tree Structured Convolutional Neural Networks for Facial Expression Recognition in the WildIEEE Transactions on Affective Computing10.1109/TAFFC.2022.315692014:3(1927-1941)Online publication date: 1-Jul-2023
https://doi.org/10.1109/TAFFC.2022.3156920
Zang HFoo SBernadin SMeyer-Baese A(2021)Facial Emotion Recognition Using Asymmetric Pyramidal Networks With Gradient CentralizationIEEE Access10.1109/ACCESS.2021.30753899(64487-64498)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3075389
Show More Cited By

Index Terms

Transductive Transfer LDA with Riesz-based Volume LBP for Emotion Recognition in The Wild
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Machine learning

Recommendations

Emotion Recognition with Facial Landmark Heatmaps
MultiMedia Modeling
Abstract
Facial expression recognition is a very challenging problem and has attracted more and more researchers’ attention. In this paper, considering that facial expression recognition is closely related to the features of key facial regions, we propose ...
Gaussian mixture model based estimation of the neutral face shape for emotion recognition

When the goal is to recognize the facial expression of a person given an expressive image, there are mainly two types of information encoded in the image that we have to deal with: identity-related information and expression related information. ...
Pose-Robust Facial Expression Recognition Using View-Based 2D + 3D AAM

This paper proposes a pose-robust face tracking and facial expression recognition method using a view-based 2D 3D active appearance model (AAM) that extends the 2D 3D AAM to the view-based approach, where one independent face model is used for a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

November 2015

678 pages

ISBN:9781450339124

DOI:10.1145/2818346

General Chairs:
Zhengyou Zhang
Microsoft Research, USA
,
Phil Cohen
VoiceBox Technologies, USA
,
Program Chairs:
Dan Bohus
Microsoft Research, USA
,
Radu Horaud
INRIA Grenoble Rhone-Alpes, France
,
Helen Meng
Chinese University of Hong Kong, China

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of Jiangsu Province
National Natural Science Foundation of China
Ph.D. Program Foundation of Ministry Education of China
National Basic Research Program of China
Graduate Research Innovation Project of Jiangsu Province

Conference

ICMI '15

Sponsor:

SIGCHI

ICMI '15: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 9 - 13, 2015

Washington, Seattle, USA

Acceptance Rates

ICMI '15 Paper Acceptance Rate 52 of 127 submissions, 41%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
214
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

LI SZONG YLU CTANG CZHAO Y(2023)Group Sparse Reduced Rank Tensor Regression for Micro-Expression RecognitionIEICE Transactions on Information and Systems10.1587/transinf.2022EDL8073E106.D:4(575-578)Online publication date: 1-Apr-2023
https://doi.org/10.1587/transinf.2022EDL8073
Cai JMeng ZKhan ALi ZO’Reilly JTong Y(2023)Probabilistic Attribute Tree Structured Convolutional Neural Networks for Facial Expression Recognition in the WildIEEE Transactions on Affective Computing10.1109/TAFFC.2022.315692014:3(1927-1941)Online publication date: 1-Jul-2023
https://doi.org/10.1109/TAFFC.2022.3156920
Zang HFoo SBernadin SMeyer-Baese A(2021)Facial Emotion Recognition Using Asymmetric Pyramidal Networks With Gradient CentralizationIEEE Access10.1109/ACCESS.2021.30753899(64487-64498)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3075389
Liu X(2021)Disentanglement For Discriminative Visual RecognitionRecognition and Perception of Images10.1002/9781119751991.ch5(143-187)Online publication date: 5-Mar-2021
https://doi.org/10.1002/9781119751991.ch5
Zong YZheng WCui ZZhao GHu B(2020)Toward Bridging Microexpressions From Different DomainsIEEE Transactions on Cybernetics10.1109/TCYB.2019.291451250:12(5047-5060)Online publication date: Dec-2020
https://doi.org/10.1109/TCYB.2019.2914512
Cai JMeng ZKhan ALi ZOReilly JTong Y(2018)Island Loss for Learning Discriminative Features in Facial Expression Recognition2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)10.1109/FG.2018.00051(302-309)Online publication date: May-2018
https://doi.org/10.1109/FG.2018.00051
Meng ZLiu PCai JHan STong Y(2017)Identity-Aware Convolutional Neural Network for Facial Expression Recognition2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017)10.1109/FG.2017.140(558-565)Online publication date: May-2017
https://doi.org/10.1109/FG.2017.140
Liu XKumar BYou JJia P(2017)Adaptive Deep Metric Learning for Identity-Aware Facial Expression Recognition2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW.2017.79(522-531)Online publication date: Jul-2017
https://doi.org/10.1109/CVPRW.2017.79
Yan JYan BLu GXu QLi HCheng XCai X(2017)Convolutional neural networks and feature fusion for bimodal emotion recognition on the emotiW 2016 challenge2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)10.1109/CISP-BMEI.2017.8301997(1-5)Online publication date: Oct-2017
https://doi.org/10.1109/CISP-BMEI.2017.8301997
Sherly Alphonse ADharma D(2017)A novel Monogenic Directional Pattern (MDP) and pseudo-Voigt kernel for facilitating the identification of facial emotionsJournal of Visual Communication and Image Representation10.1016/j.jvcir.2017.10.00849:C(459-470)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1016/j.jvcir.2017.10.008

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten