Article

Tracking head pose and focus of attention with multiple far-field cameras

Authors:

Rainer StiefelhagenAuthors Info & Claims

ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

Pages 281 - 286

https://doi.org/10.1145/1180995.1181050

Published: 02 November 2006 Publication History

Abstract

In this work we present our recent approach on estimating head orientations and foci of attention of multiple people in a smart room, which is equipped with several cameras to monitor the room. In our approach, we estimate each person's head orientation with respect to the room coordinate system by using all camera views. We implemented a Neural Network to estimate head pose on every single camera view, a Bayes filter is then applied to integrate every estimate into one final, joint hypothesis. Using this scheme, we can track peoples' horizontal head orientations in a full 360° range at almost all positions within the room. The tracked head orientations are then used to determine who is looking at whom, i.e. people's focus of attention. We report experimental results on one meeting video, that was recorded in the smart room.

References

[1]

S. O. Ba and J.-M. Obodez. A probabilistic framework for joint head tracking and pose estimation. In Proceedings of the 17th International Conference on Pattern Recognition, 2004.

Digital Library

[2]

A. H. Gee and R. Cipolla. Non-intrusive gaze tracking for human-computer interaction. In Proceedings of Mechatronics and Machine Vision in Practise, pages 112--117, 1994.

[3]

T. Horprasert, Y. Yacoob, and L. S. Davis. Computing 3-d head orientation from a monocular image sequence. In Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition, 1996.

Digital Library

[4]

M. Katzenmaier, R. Stiefelhagen, T. Schultz, I. Rogina, and A. Waibel. Identifying the addressee in human-human-robot interactions based on head pose and speech. In International Conference on Multimodal Interfaces ICMI, 2004.

Digital Library

[5]

S. R. Langton. The mutual influence of gaze and head orientation in the analysis of social attention direction. In The Quaterly Journal of Experimental Psychology, 53A(3):825--845, 2000.

[6]

M. C. Michael Argyle. Gaze and Mutual Gaze. Cambridge University Press, 1976.

[7]

K. Otsuka, Y. Takemae, J. Yamamoto, and H. Murase. A probabilistic inference of multiparty-conversation structure based on markov-switching models of gaze patterns, head directions and utterances. In Proceedings of the International Conference on Multimodal Interfaces - ICMI, 2005.

Digital Library

[8]

R. Pappu and P. Beardsley. A qualitative approach to classifying gaze direction. In Proceedings of FG98, pages 160--165, 1998.

Digital Library

[9]

V. B. Stephen R. H. Langton, Roger J. Watt. Do the eyes have it? cues to the direction of social attention. In Trends in Cognitive Neuroscience, 4(2):50--58, 2000.

[10]

R. Stiefelhagen. Tracking focus of attention in meetings. In IEEE International Conference on Multimodal Interfaces, pages 273--280, 2002.

Digital Library

[11]

R. Stiefelhagen, J. Yang, and A. Waibel. Simultaneous tracking of head poses in a panoramic view. In International Conference on Pattern Recognition, volume 3, pages 726--729, September 2000.

Digital Library

[12]

R. Stiefelhagen, J. Yang, and A. Waibel. Modeling focus of attention for meeting indexing based on multiple cues. IEEE Transactions on Neural Networks, 13(4), 2002.

Digital Library

[13]

R. Stiefelhagen and J. Zhu. Head orientation and gaze direction in meetings. In Conference on Human Factors in Computing Systems (CHI2002), Minneapolis, April 2002.

Digital Library

[14]

Y.-L. Tian, L. Brown, J. Connell, S. Pankanti, A. Hampapur, A. Senior, and R. Bolle. Absolute head pose estimation from overhead wide-angle cameras. In IEEE International Workshop on Analysis and Modeling of Faces and Gestures, 2003.

Digital Library

[15]

K. van Turhnout, J. Terken, I. Bakx, and B. Eggen. Identifying the intended addressee in mixed human-human and human-computer interaction from non-verbal features. In Proceedings of the International Conference on Multimodal Interfaces - ICMI, 2005.

Digital Library

[16]

R. Vertegaal. Attentive user interfaces. Communications of the ACM, 46(3), 2003.

Digital Library

[17]

M. Voit, K. Nickel, and R. Stiefelhagen. Multi-view head pose estimation using neural networks. In Second Workshop on Face Processing in Video (FPiV'05), in Proceedings of Second Canadian Conference on Computer and Robot Vision. (CRV'05), 9-11 May 2005, Victoria, BC, Canada, 2005.

Digital Library

[18]

M. L. Z. Zhang, G. Potamianos and T. Huang. Robust multi-view multi-camera face detection inside smart rooms using spatio-temporal dynamic programming. In Proceedings of Automatic Face and Gesture Recognition (FG), Southampton, United Kingdom, 2006.

Digital Library

Cited By

HATORI YNAKAJIMA TWATABE S(2022)Body Posture Analysis for the Classification of Classroom ScenesInterdisciplinary Information Sciences10.4036/iis.2022.A.0528:1(55-62)Online publication date: 2022
https://doi.org/10.4036/iis.2022.A.05
Tonsen MSteil JSugano YBulling A(2017)InvisibleEyeProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31309711:3(1-21)Online publication date: 11-Sep-2017
https://dl.acm.org/doi/10.1145/3130971
Sarfraz MConstantinescu AZuzej MStiefelhagen R(2017)A Multimodal Assistive System for Helping Visually Impaired in Social InteractionsInformatik-Spektrum10.1007/s00287-017-1077-740:6(540-545)Online publication date: 24-Oct-2017
https://doi.org/10.1007/s00287-017-1077-7
Show More Cited By

Index Terms

Tracking head pose and focus of attention with multiple far-field cameras
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding

Recommendations

3D point-of-regard, position and head orientation from a portable monocular video-based eye tracker
ETRA '08: Proceedings of the 2008 symposium on Eye tracking research & applications

To study an observer's eye movements during realistic tasks, the observer should be free to move naturally throughout our three-dimensional world. Therefore, a technique to determine an observer's point-of-regard (POR) as well as his/her motion ...
Head pose estimation with one camera, in uncalibrated environments
EGIHMI '10: Proceedings of the 2010 workshop on Eye gaze in intelligent human machine interaction

Head pose together with eye gaze are a reliable indication regarding the estimate of the focus of attention of a person standing in front of a camera, with applications ranging from driver's attention estimation to meeting environments. As gaze ...
An Affordable Solution for Binocular Eye Tracking and Calibration in Head-mounted Displays
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Immersion is the ultimate goal of head-mounted displays (HMD) for Virtual Reality (VR) in order to produce a convincing user experience. Two important aspects in this context are motion sickness, often due to imprecise calibration, and the integration ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

November 2006

404 pages

ISBN:159593541X

DOI:10.1145/1180995

General Chairs:
Francis Quek
Virginia Tech, USA
,
Jie Yang
Carnegie Mellon University, USA
,
Program Chairs:
Dominic Massaro
University of California, Santa Cruz, USA
,
Abeer Alwan
University of California, Los Angeles, USA
,
Timothy J. Hazen
Massachusetts Institute of Technology, USA

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 November 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI06

Sponsor:

ICMI06: 8th International Conference on Multimodal Interfaces 2006

November 2 - 4, 2006

Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
432
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

HATORI YNAKAJIMA TWATABE S(2022)Body Posture Analysis for the Classification of Classroom ScenesInterdisciplinary Information Sciences10.4036/iis.2022.A.0528:1(55-62)Online publication date: 2022
https://doi.org/10.4036/iis.2022.A.05
Tonsen MSteil JSugano YBulling A(2017)InvisibleEyeProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31309711:3(1-21)Online publication date: 11-Sep-2017
https://dl.acm.org/doi/10.1145/3130971
Sarfraz MConstantinescu AZuzej MStiefelhagen R(2017)A Multimodal Assistive System for Helping Visually Impaired in Social InteractionsInformatik-Spektrum10.1007/s00287-017-1077-740:6(540-545)Online publication date: 24-Oct-2017
https://doi.org/10.1007/s00287-017-1077-7
Duffner SGarcia C(2016)Visual Focus of Attention Estimation With Unsupervised Incremental LearningIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2015.250192026:12(2264-2272)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1109/TCSVT.2015.2501920
Cai QSankaranarayanan AZhang QZhang ZLiu Z(2010)Real time head pose tracking from multiple cameras with a generic model2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops10.1109/CVPRW.2010.5543822(25-32)Online publication date: Jun-2010
https://doi.org/10.1109/CVPRW.2010.5543822
Waibel AStiefelhagen RCarlson RCasas JKleindienst JLamel LLanz OMostefa DOmologo MPianesi FPolymenakos LPotamianos GSoldatos JSutschet GTerken J(2010)Computers in the Human Interaction LoopHandbook of Ambient Intelligence and Smart Environments10.1007/978-0-387-93808-0_40(1071-1116)Online publication date: 2010
https://doi.org/10.1007/978-0-387-93808-0_40
Otsuka KAraki SIshizuka KFujimoto MHeinrich MYamato JDigalakis VPotamianos ATurk MPieraccini RIvanov Y(2008)A realtime multimodal system for analyzing group meetings by combining face pose tracking and speaker diarizationProceedings of the 10th international conference on Multimodal interfaces10.1145/1452392.1452446(257-264)Online publication date: 20-Oct-2008
https://dl.acm.org/doi/10.1145/1452392.1452446
Takemura KMatsumoto YOgasawara T(2008)Estimation of group attention for automated camerawork2008 IEEE/RSJ International Conference on Intelligent Robots and Systems10.1109/IROS.2008.4651210(2317-2322)Online publication date: Sep-2008
https://doi.org/10.1109/IROS.2008.4651210
Murphy-Chutorian ETrivedi M(2008)3D tracking and dynamic analysis of human head movements and attentional targets2008 Second ACM/IEEE International Conference on Distributed Smart Cameras10.1109/ICDSC.2008.4635725(1-8)Online publication date: Sep-2008
https://doi.org/10.1109/ICDSC.2008.4635725
Stiefelhagen RBernardin KEkenel HVoit M(2008)Tracking identities and attention in smart environments - contributions and progress in the CHIL project2008 8th IEEE International Conference on Automatic Face & Gesture Recognition10.1109/AFGR.2008.4813322(1-8)Online publication date: Sep-2008
https://doi.org/10.1109/AFGR.2008.4813322
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents