Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
article

Learning silhouette features for control of human motion

Published: 01 October 2005 Publication History

Abstract

We present a vision-based performance interface for controlling animated human characters. The system interactively combines information about the user's motion contained in silhouettes from three viewpoints with domain knowledge contained in a motion capture database to produce an animation of high quality. Such an interactive system might be useful for authoring, for teleconferencing, or as a control interface for a character in a game. In our implementation, the user performs in front of three video cameras; the resulting silhouettes are used to estimate his orientation and body configuration based on a set of discriminative local features. Those features are selected by a machine-learning algorithm during a preprocessing step. Sequences of motions that approximate the user's actions are extracted from the motion database and scaled in time to match the speed of the user's motion. We use swing dancing, a complex human motion, to demonstrate the effectiveness of our approach. We compare our results to those obtained with a set of global features, Hu moments, and ground truth measurements from a motion capture system.

References

[1]
Arikan, O. and Forsyth, D. A. 2002. Interactive motion generation from examples. ACM Trans. Graph. 21, 3, 483--490.
[2]
Arikan, O., Forsyth, D. A., and O'Brien, J. F. 2003. Motion synthesis from annotations. ACM Trans. Graph. 22, 3, 402--408.
[3]
Brand, M. 1999. Shadow puppetry. In Proceedings of the International Conference on Computer Vision. 1237--1244.
[4]
Brand, M. and Hertzmann, A. 2000. Style machines. In Proceedings of SIGGRAPH 2000. 183--192.
[5]
Bregler, C. and Malik, J. 1998. Tracking people with twists and exponential maps. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8--15.
[6]
California Institute of Technology. 2002. Camera calibration toolbox for matlab. Available online at http://www.vision.caltech.edu/bouguetj/calib_doc/.
[7]
Carranza, J., Theobalt, C., Magnor, M. A., and Seidel, H.-P. 2003. Free-viewpoint video of human actors. ACM Trans. Graph. 22, 3, 569--577.
[8]
Chai, J. and Hodgins, J. 2005. Performance animation from low-dimensional control signals. ACM Trans. Graph. 24, 3, 686--696.
[9]
Cheung, K. M., Kanade, T., Bouguet, J.-Y., and Holler, M. 2000. A real time system for robust 3D voxel reconstruction of human motions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 714--720.
[10]
Collins, M., Schapire, R., and Singer, Y. 2000. Logistic regression, AdaBoost and Bregman distances. In Proceedings of Computational Learning Theory. 158--169.
[11]
Crow, F. C. 1984. Summed-area tables for texture mapping. In Proceedings of SIGGRAPH 1984. 207--212.
[12]
Davis, J. W. and Bobick, A. F. 1997. The representation and recognition of human movement using temporal templates. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 928--934.
[13]
Delamarre, Q. and Faugeras, O. D. 1999. 3D articulated models and multi-view tracking with silhouettes. In Proceedings of the International Conference on Computer Vision. 716--721.
[14]
Deutscher, J., Blake, A., and Reid, I. 2000. Articulated body motion capture by annealed particle filtering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 126--133.
[15]
Duda, R., Hart, P. E., and Stork, D. G. 2000. Pattern Classification. John Wiley & Sons, Inc., New York, NY.
[16]
Efros, A., Berg, A. C., Mori, G., and Malik, J. 2003. Recognizing action at a distance. In Proceedings of the IEEE International Conference on Computer Vision. 726--733.
[17]
Gavrila, D. M. 1999. The visual analysis of human movement: A survey. Comput. Vis. Image Understand. 73, 82--98.
[18]
Gionis, A., Indyk, P., and Motwani, R. 1999. Similarity search in high dimensions via hashing. In Proceedings of the 25th International Conference on Very Large Data Bases. 518--529.
[19]
Granieri, J. P., Crabtree, J., and Badler, N. I. 1995. Production and playback of human figure motion for visual simulation. ACM Trans. Model. Comput. Sim. 5, 3, 222--241.
[20]
Hu, M. K. 1962. Visual pattern recognition by moment invariants. IEEE Trans. Inform. Theor. 8, 179--187.
[21]
Jones, M. and Viola, P. 2003. Face recognition using boosted local features. MERL Tech. rep. TR2003-25. Mitsubishi Electric Research Laboratories, Cambridge, MA.
[22]
Kim, T. H., Park, S., and Shin, S. Y. 2003. Rhythmic-motion synthesis based on motion-beat analysis. In ACM Trans. Graph. 22, 3, 392--401.
[23]
Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion graphs. ACM Trans. Graph. 21, 3, 473--482.
[24]
Lee, J., Chai, J., Reitsma, P., Hodgins, J., and Pollard, N. 2002. Interactive control of avatars animated with human motion data. ACM Trans. Graph. 21, 3, 491--500.
[25]
Lee, J. and Shin, S. Y. 1999. A hierachical approach to interactive motion editing. In Proceedings of SIGGRAPH 1999. Computer Graphics Proceedings, Annual Conference Series. ACM Press, New York, NY, 39--48.
[26]
Leventon, M. E. and Freeman, W. T. 1998. Bayesian estimation of a 3D human motion from an image sequence. MERL Tech. rep. TR1998-06. Mitsubishi Electric Research Laboratories, Cambridge, MA.
[27]
Matusik, W., Buehler, C., and McMillan, L. 2001. Polyhedral visual hulls for real-time rendering. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques. Springer-Verlag, Berlin, Germany, 115--126.
[28]
Mikic, I., Triverdi, M., Hunter, E., and Cosman, P. 2001. Articulated body posture estimation from multicamera voxel data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 455--461.
[29]
Mori, G. and Malik, J. 2002. Estimating human body configurations using shape context matching. In Proceedings of the European Conference on Computer Vision. Vol. 3. 666--680.
[30]
Noser, H. and Thalmann, D. 1997. Sensor based synthetic actors in a tennis game simulation. In Proceedings of the 1997 Conference on Computer Graphics International. 189.
[31]
Point Grey Corporation. 2001. Dragonfly cameras. Available at http://www.ptgrey.com.
[32]
Ramanan, D. and Forsyth, D. A. 2004. Automatic annotation of everyday movements. In Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 1547--1554.
[33]
Rosales, R. and Sclaroff, S. 2000. Specialized mappings and the estimation of body pose from a single image. In Proceedings of the IEEE Human Motion Workshop. 19--24.
[34]
Rosales, R., Siddiqui, M., Alon, J., and Sclaroff, S. 2001. Estimating 3D body pose using uncalibrated cameras. In Boston University Computer Science Department Tech. rep. 2001-008. Boston University, Boston, MA.
[35]
Schapire, R. E. and Singer, Y. 1999. Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 37, 3, 297--336.
[36]
Shakhnarovich, G., Viola, P., and Darrell, T. 2003. Fast pose estimation with parameter-sensitive hashing. In Proceedings of the IEEE International Conference on Computer Vision. 750--757.
[37]
Shin, H. J., Lee, J., Shin, S. Y., and Gleicher, M. 2001. Computer puppetry: An importance-based approach. ACM Trans. Graph. 20, 2, 67--94.
[38]
Shipp, C. A. and Kuncheva, L. I. 2002. An investigation into how Adaboost affects classifier diversity. In Proceedings of 9th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems. 203--208.
[39]
Sidenbladh, H., Black, M. J., and Signal, L. 2002. Implicit probabilistic models of human motion for synthesis and tracking. In Proceedings of the European Conference on Computer Vision. 784--800.
[40]
Sminchisescu, C. and Triggs, B. 2003. Estimating articulated human motion with covariance scaled sampling. In Int. J. Robotics Res. 22, 6, 371--393.
[41]
Stenger, B., Thayananthan, A., Torr, P., and Cipolla, R. 2003. Filtering using a tree-based estimator. In Proceedings of the International Conference on Computer Vision. 1063--1070.
[42]
Vicon Motion Systems. 2002. Available at http://www.vicon.com/.
[43]
Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 511--518.
[44]
Viola, P., Jones, M. J., and Snow, D. 2003. Detecting pedestrians using patterns of motion and appearance. In Proceedings of the IEEE International Conference on Computer Vision. 734--741.
[45]
Wu, J., Rehg, J. M., and Mullin, M. D. 2004. Learning a rare event detection cascade by direct feature selection. In Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA. 1523--1530.
[46]
Yamamoto, M., Sato, A., Kawada, S., Kondo, T., and Osaki, Y. 1998. Incremental tracking of human actions from multiple views. In Proceedings of the IEEE International Conference on Computer Vision. 2--7.
[47]
Yin, K. and Pai, D. K. 2003. Footsee: An interactive animation system. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 329--338.

Cited By

View all
  • (2023)An Interactive Framework for Visually Realistic 3D Motion Synthesis using Evolutionarily-trained Spiking Neural NetworksProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/35855096:1(1-19)Online publication date: 16-May-2023
  • (2022)Spatial-Temporal Graph Convolutional Framework for Yoga Action Recognition and GradingComputational Intelligence and Neuroscience10.1155/2022/75005252022Online publication date: 1-Jan-2022
  • (2022)Fast all-focus image reconstruction method based on light field imagingITM Web of Conferences10.1051/itmconf/2022450103045(01030)Online publication date: 19-May-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 24, Issue 4
October 2005
244 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/1095878
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 October 2005
Published in TOG Volume 24, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Performance animation
  2. animation interface
  3. computer vision
  4. machine-learning
  5. motion capture
  6. motion control

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)4
Reflects downloads up to 13 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2023)An Interactive Framework for Visually Realistic 3D Motion Synthesis using Evolutionarily-trained Spiking Neural NetworksProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/35855096:1(1-19)Online publication date: 16-May-2023
  • (2022)Spatial-Temporal Graph Convolutional Framework for Yoga Action Recognition and GradingComputational Intelligence and Neuroscience10.1155/2022/75005252022Online publication date: 1-Jan-2022
  • (2022)Fast all-focus image reconstruction method based on light field imagingITM Web of Conferences10.1051/itmconf/2022450103045(01030)Online publication date: 19-May-2022
  • (2021)BibliographyHuman Motion Capture and Identification for Assistive Systems Design in Rehabilitation10.1002/9781119515104.biblio(207-230)Online publication date: 7-May-2021
  • (2019)Sparse over-complete patch matchingPattern Recognition Letters10.1016/j.patrec.2019.01.017122(1-6)Online publication date: May-2019
  • (2018)Cage-based performance captureACM SIGGRAPH 2018 Courses10.1145/3214834.3214836(1-72)Online publication date: 12-Aug-2018
  • (2018)Walking on Thin Air: Environment-Free Physics-Based Markerless Motion Capture2018 15th Conference on Computer and Robot Vision (CRV)10.1109/CRV.2018.00031(158-165)Online publication date: May-2018
  • (2018)ACM SIGGRAPH 2018 CoursesundefinedOnline publication date: 12-Aug-2018
  • (2017)Ballistic Shadow ArtProceedings of the 43rd Graphics Interface Conference10.5555/3141475.3141512(190-198)Online publication date: 1-Jun-2017
  • (2017)Movement Tracking from Monocular Video Based on the Particle Filter2017 IEEE Third International Conference on Multimedia Big Data (BigMM)10.1109/BigMM.2017.78(407-412)Online publication date: Apr-2017
  • Show More Cited By

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media