Article

High accuracy optical flow serves 3-d pose tracking: exploiting contour and flow based constraints

Authors:

Bodo Rosenhahn,

Daniel Cremers,

Hans-Peter SeidelAuthors Info & Claims

ECCV'06: Proceedings of the 9th European conference on Computer Vision - Volume Part II

Pages 98 - 111

https://doi.org/10.1007/11744047_8

Published: 07 May 2006 Publication History

Abstract

Tracking the 3-D pose of an object needs correspondences between 2-D features in the image and their 3-D counterparts in the object model. A large variety of such features has been suggested in the literature. All of them have drawbacks in one situation or the other since their extraction in the image and/or the matching is prone to errors. In this paper, we propose to use two complementary types of features for pose tracking, such that one type makes up for the shortcomings of the other. Aside from the object contour, which is matched to a free-form object surface, we suggest to employ the optic flow in order to compute additional point correspondences. Optic flow estimation is a mature research field with sophisticated algorithms available. Using here a high quality method ensures a reliable matching. In our experiments we demonstrate the performance of our method and in particular the improvements due to the optic flow.

References

[1]

A. Agarwal and B. Triggs. Tracking articulated motion using a mixture of autoregressive models. In T. Pajdla and J. Matas, editors, Proc. 8th European Conference on Computer Vision, volume 3023 of LNCS, pages 54-65. Springer, May 2004.

[2]

H. Araújo, R. L. Carceroni, and C. M. Brown. A fully projective formulation to improve the accuracy of Lowe's pose-estimation algorithm. Computer Vision and Image Understanding, 70(2):227-238, May 1998.

Digital Library

[3]

C. Bregler, J. Malik, and K. Pullen. Twist based acquisition and tracking of animal and human kinematics. International Journal of Computer Vision, 56(3):179-194, 2004.

Digital Library

[4]

T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. High accuracy optical flow estimation based on a theory for warping. In T. Pajdla and J. Matas, editors, Proc. 8th European Conference on Computer Vision, volume 3024 of LNCS, pages 25-36. Springer, May 2004.

[5]

T. Brox, B. Rosenhahn, and J.Weickert. Three-dimensional shape knowledge for joint image segmentation and pose estimation. In W. Kropatsch, R. Sablatnig, and A. Hanbury, editors, Pattern Recognition, volume 3663 of LNCS, pages 109-116. Springer, Aug. 2005.

Digital Library

[6]

T. Brox and J. Weickert. A TV flow based local scale measure for texture discrimination. In T. Pajdla and J. Matas, editors, Proc. 8th European Conference on Computer Vision, volume 3022 of LNCS, pages 578-590. Springer, May 2004.

[7]

T. Chan and L. Vese. Active contours without edges. IEEE Transactions on Image Processing, 10(2):266-277, Feb. 2001.

Digital Library

[8]

P. David, D. DeMenthon, R. Duraiswami, and H. Samet. Simultaneous pose and correspondence determination using line features. In Proc. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2, pages 424-431, 2003.

Digital Library

[9]

D. DeCarlo and D. Metaxas. Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision, 38(2):99-127, July 2000.

Digital Library

[10]

P. Fua, R. Plänkers, and D. Thalmann. Tracking and modeling people in video sequences. Computer Vision and Image Understanding, 81(3):285-302, Mar. 2001.

Digital Library

[11]

J. Goddard. Pose and motion estimation from vision using dual quaternion-based extended Kalman filtering. Technical report, University of Tennessee, Knoxville, 1997.

[12]

W. E. L. Grimson. Object Recognition by Computer. The MIT Press, Cambridge, MA, 1990.

Digital Library

[13]

B. Horn and B. Schunck. Determining optical flow. Artificial Intelligence, 17:185-203, 1981.

Digital Library

[14]

R. Koch. Dynamic 3D scene analysis through synthesis feedback control. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(6):556-568, 1993.

Digital Library

[15]

D. Lowe. Solving for the parameters of object models from image descriptions. In Proc. ARPA Image Understanding Workshop, pages 121-127, 1980.

[16]

D. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91-110, 2004.

Digital Library

[17]

E. Marchand, P. Bouthemy, and F. Chaumette. A 2D-3D model-based approach to real-time visual tracking. Image and Vision Computing, 19(13):941-955, Nov. 2001.

[18]

K. Mikolajczyk and C. Schmid. A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10):1615-1630, 2005.

Digital Library

[19]

R. Murray, Z. Li, and S. Sastry. Mathematical Introduction to Robotic Manipulation. CRC Press, Boca Raton, FL, 1994.

Digital Library

[20]

S. Osher and J. A. Sethian. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. Journal of Computational Physics, 79:12-49, 1988.

Digital Library

[21]

N. Paragios and R. Deriche. Geodesic active regions: A new paradigm to deal with frame partition problems in computer vision. Journal of Visual Communication and Image Representation, 13(1/2):249-268, 2002.

Digital Library

[22]

B. Rosenhahn. Pose estimation revisited. Technical Report TR-0308, Institute of Computer Science, University of Kiel, Germany, Oct. 2003.

[23]

B. Rosenhahn, C. Perwass, and G. Sommer. Pose estimation of free-form contours. International Journal of Computer Vision, 62(3):267-289, 2005.

Digital Library

[24]

B. Rosenhahn and G. Sommer. Pose estimation of free-form objects. In T. Pajdla and J. Matas, editors, Proc. 8th European Conference on Computer Vision, volume 3021 of LNCS, pages 414-427. Springer, May 2004.

[25]

F. Shevlin. Analysis of orientation problems using Plücker lines. In International Conference on Pattern Recognition (ICPR), volume 1, pages 685-689, Brisbane, 1998.

Digital Library

[26]

L. Vacchetti, V. Lepetit, and P. Fua. Combining edge and texture information for real-time accurate 3D camera tracking. In 3rd International Symposium on Mixed and Augmented Reality, pages 48-57, 2004.

Digital Library

[27]

L. Vacchetti, V. Lepetit, and P. Fua. Stable real-time 3D tracking using online and offline information. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(10): 1391-1391, 2004.

Digital Library

[28]

Z. Zang. Iterative point matching for registration of free-form curves and surfaces. International Journal of Computer Vision, 13(2):119-152, 1999.

Digital Library

Cited By

Ji BYang CShunyu YPan YCheng WKankanhalli MWang MChu WLiu JWorring M(2021)HPOF:3D Human Pose Recovery from Monocular Video with Optical FlowProceedings of the 2021 International Conference on Multimedia Retrieval10.1145/3460426.3463605(144-154)Online publication date: 24-Aug-2021
https://dl.acm.org/doi/10.1145/3460426.3463605
Xu WChatterjee AZollhöfer MRhodin HMehta DSeidel HTheobalt C(2018)MonoPerfCapACM Transactions on Graphics10.1145/318197337:2(1-15)Online publication date: 21-May-2018
https://dl.acm.org/doi/10.1145/3181973
Huang ZLi TChen WZhao YXing JLeGendre CLuo LMa CLi H(2018)Deep Volumetric Video From Very Sparse Multi-view Performance CaptureComputer Vision – ECCV 201810.1007/978-3-030-01270-0_21(351-369)Online publication date: 8-Sep-2018
https://dl.acm.org/doi/10.1007/978-3-030-01270-0_21
Show More Cited By

High accuracy optical flow serves 3-d pose tracking: exploiting contour and flow based constraints
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
  2. Computer graphics
    1. Animation

Recommendations

Markerless tracking using Polar Correlation of camera optical flow
VR '10: Proceedings of the 2010 IEEE Virtual Reality Conference

We present a novel, real-time, markerless vision-based tracking system, employing a rigid orthogonal configuration of two pairs of opposing cameras. Our system uses optical flow over sparse features to overcome the limitation of vision-based systems ...
Real-Time Bayesian 3-D Pose Tracking

In this paper, we propose a novel approach for real-time 3-D tracking of object pose from a single camera. We formulate the 3-D pose tracking task in a Bayesian framework which fuses feature correspondence information from both previous frame and some ...
Hand-Eye Camera Calibration with an Optical Tracking System
ICDSC '18: Proceedings of the 12th International Conference on Distributed Smart Cameras

This paper presents a method for hand-eye camera calibration via an optical tracking system (OTS) faciltating robotic applications. The camera pose cannot be directly tracked via the OTS. Because of this, a transformation matrix between a marker-plate ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ECCV'06: Proceedings of the 9th European conference on Computer Vision - Volume Part II

May 2006

655 pages

ISBN:3540338349

Editors:
Aleš Leonardis
University of Ljubljana, Slovenia
,
Horst Bischof
Institute for Computer Graphics and Vision, TU Graz, Inffeldgasse 16, Graz, Austria
,
Axel Pinz
Vision-based Measurement Group, Inst. of El. Measurement and Meas. Sign. Proc. Graz, University of Technology, Inffeldgasse 16, Graz, Austria

Sponsors

University of Ljubljana: University of Ljubljana
Graz University of Technology: Graz University of Technology
Advanced Computer Vision: Advanced Computer Vision

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 07 May 2006

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ji BYang CShunyu YPan YCheng WKankanhalli MWang MChu WLiu JWorring M(2021)HPOF:3D Human Pose Recovery from Monocular Video with Optical FlowProceedings of the 2021 International Conference on Multimedia Retrieval10.1145/3460426.3463605(144-154)Online publication date: 24-Aug-2021
https://dl.acm.org/doi/10.1145/3460426.3463605
Xu WChatterjee AZollhöfer MRhodin HMehta DSeidel HTheobalt C(2018)MonoPerfCapACM Transactions on Graphics10.1145/318197337:2(1-15)Online publication date: 21-May-2018
https://dl.acm.org/doi/10.1145/3181973
Huang ZLi TChen WZhao YXing JLeGendre CLuo LMa CLi H(2018)Deep Volumetric Video From Very Sparse Multi-view Performance CaptureComputer Vision – ECCV 201810.1007/978-3-030-01270-0_21(351-369)Online publication date: 8-Sep-2018
https://dl.acm.org/doi/10.1007/978-3-030-01270-0_21
Tung HTung HYumer EFragkiadaki K(2017)Self-supervised learning of motion captureProceedings of the 31st International Conference on Neural Information Processing Systems10.5555/3295222.3295276(5242-5252)Online publication date: 4-Dec-2017
https://dl.acm.org/doi/10.5555/3295222.3295276
Behúň KHerout APáldy AGutierrez D(2014)Kinect-supported dataset creation for human pose estimationProceedings of the 30th Spring Conference on Computer Graphics10.1145/2643188.2643195(55-62)Online publication date: 28-May-2014
https://dl.acm.org/doi/10.1145/2643188.2643195
Wu CStoll CValgaerts LTheobalt C(2013)On-set performance capture of multiple actors with a stereo cameraACM Transactions on Graphics10.1145/2508363.250841832:6(1-11)Online publication date: 1-Nov-2013
https://dl.acm.org/doi/10.1145/2508363.2508418
Barranco FTomasi MDiaz JVanegas MRos E(2012)Parallel architecture for hierarchical optical flow estimation based on FPGAIEEE Transactions on Very Large Scale Integration (VLSI) Systems10.1109/TVLSI.2011.214542320:6(1058-1067)Online publication date: 1-Jun-2012
https://dl.acm.org/doi/10.1109/TVLSI.2011.2145423
De Aguiar ETheobalt CStoll CSeidel H(2007)Marker-less 3D feature tracking for mesh-based human motion captureProceedings of the 2nd conference on Human motion: understanding, modeling, capture and animation10.5555/1785357.1785359(1-15)Online publication date: 20-Oct-2007
https://dl.acm.org/doi/10.5555/1785357.1785359
Gall JRosenhahn BSeidel H(2007)Clustered stochastic optimization for object recognition and pose estimationProceedings of the 29th DAGM conference on Pattern recognition10.5555/1771530.1771535(32-41)Online publication date: 12-Sep-2007
https://dl.acm.org/doi/10.5555/1771530.1771535
de Aguiar ETheobalt CStoll CSeidel H(2007)Marker-Less 3D Feature Tracking for Mesh-Based Human Motion CaptureHuman Motion – Understanding, Modeling, Capture and Animation10.1007/978-3-540-75703-0_1(1-15)Online publication date: 20-Oct-2007
https://dl.acm.org/doi/10.1007/978-3-540-75703-0_1
Show More Cited By

View Options

View options

Figures

Tables

Media

View Table of Conten