Article

Integrated learning for interactive synthetic characters

Authors:

Bruce Blumberg,

Michael Patrick Johnson,

Bill TomlinsonAuthors Info & Claims

SIGGRAPH '02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques

Pages 417 - 426

https://doi.org/10.1145/566570.566597

Published: 01 July 2002 Publication History

Abstract

The ability to learn is a potentially compelling and important quality for interactive synthetic characters. To that end, we describe a practical approach to real-time learning for synthetic characters. Our implementation is grounded in the techniques of reinforcement learning and informed by insights from animal training. It simplifies the learning task for characters by (a) enabling them to take advantage of predictable regularities in their world, (b) allowing them to make maximal use of any supervisory signals, and (c) making them easy to train by humans.We built an autonomous animated dog that can be trained with a technique used to train real dogs called "clicker training". Capabilities demonstrated include being trained to recognize and use acoustic patterns as cues for actions, as well as to synthesize new actions from novel paths through its motion space.A key contribution of this paper is to demonstrate that by addressing the three problems of state, action, and state-action space discovery at the same time, the solution for each becomes easier. Finally, we articulate heuristics and design principles that make learning practical for synthetic characters.

References

[1]

BALLARD, D. 1997. An Introduction to Natural Computation. MIT Press, Cambridge, MA.

Digital Library

[2]

BLUMBERG, B., AND GAYLEAN, T. 1995. Multi-level direction of autonomous creatures for real-time virtual environments. In Proceedings of SIGGRAPH 1995, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[3]

BURKE, R., ISLA, D., DOWNIE, M., IVANOV, Y., AND BLUMBERG, B. 2001. Creature smarts: The art and architecture of a virtual brain. In Proceedings of the Computer Game Developers Conference.

[4]

BURKE, R. 2001. Its about Time:Temporal Representation for Synthetic Characters. Master's thesis, The Media Lab, MIT.

[5]

COPPINGER, R., AND COPPINGER, L. 2001. Dogs: A Startling New Understanding of Canine Origin, Behavior, and Evolution. Scribner, New York, NY.

[6]

DOWNIE, M. 2000. behavior, animation, music: the music and movement of synthetic characters. Master's thesis, The Media Lab, MIT.

[7]

DRESCHER, G. 1991. Made-Up Minds:A Constructivist Approach to Artificial Intelligence. MIT Press, Cambridge MA.

Digital Library

[8]

EVANS, R. 2002. Varieties of learning. In AI Game Programming Wisdom, E. Rabin, Ed. Charles River Media, Hingham MA.

Digital Library

[9]

FALOUTSOS, P., VAN DE PANNE, M., AND TERZOPOLOUS, D. 2001. Composible controllers for physics-based character animation. In Proceedings of SIGGRAPH 2001, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[10]

FUNGE, J., TU, X., AND TERZOPOLOUS, D. 1999. Cognitive modeling: Knowledge, reasoning and planning for intelligent characters. In Proceedings of SIGGRAPH 1999, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[11]

GALLISTEL, C. R., AND GIBBON, J. 2000. Time, rate and conditioning. Psychological Review 107.

[12]

GLEICHER, M. 1998. Retargetting motion to new characters. In Proceedings of SIGGRAPH 1998, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[13]

GOULD, J., AND GOULD, C. 1999. The Animal Mind. W.H. Freeman, New York, NY.

[14]

GRAND, S., CLIFF, D., AND MALHOTRA, A. 1996. Creatures: Artificial life autonomous agents for home entertainment. In Proceedings of the Autonomous Agents '97 Conference.

Digital Library

[15]

GRZESZCZUK, R., AND TERZOPOULOS, D. 1995. Automated learning of muscle-actuated locomotion through control abstraction. In Proceedings of SIGGRAPH 1995, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[16]

GRZESZCZUK, R., TERZOPOULOS, D., AND HINTON, G. 1998. Neuroanimator: Fast neural network emulation and control of physics-based models. In Proceedings of SIGGRAPH 1998, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[17]

HODGINS, J., AND POLLARD, N. 1997. Adapting simulated behaviors for new characters. In Proceedings of SIGGRAPH 1997, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[18]

ISLA, D., BURKE, R., DOWNIE, M., AND BLUMBERG, B. 2001. A layered brain architecture for synthetic creatures. In Proceedings of The International Joint Conference on Artificial Intelligence.

Digital Library

[19]

ISLA, D. 2001. The Virtual Hippocampus: Spatial Common Sense for Synthetic Creatures. Master's thesis, The Media Lab, MIT.

[20]

IVANOV, Y., BLUMBERG, B., AND PENTLAND, A. 2001. Expectation maximization for weakly labeled data. In Proceedings of the 18th International Conference on Machine Learning.

Digital Library

[21]

IVANOV, Y. 2001. State Discovery for Autonomous Creatures. PhD thesis, The Media Lab, MIT.

[22]

KAELBLING, L. 1990. Learning in embedded systems. PhD thesis, Stanford University.

Digital Library

[23]

KAPLAN, F., OUDEYER, P.-Y., KUBINYI, E., AND MIKLOSI, A. 2001. Taming robots with clicker training : a solution for teaching complex behaviors. In Proceedings of the 9th European workshop on learning robots, LNAI, Springer, M. Quoy, P. Gaussier, and J. L. Wyatt, Eds.

[24]

LINDSAY, S. 2000. Applied Dog Behavior and Training. Iowa State University Press, Ames, IA.

[25]

LORENZ, K., AND LEYAHUSEN, P. 1973. Motivation of Human and Animal Behavior: An Ethological View. Van Nostrand Reinhold Co., New York, NY.

[26]

LORENZ, K. 1981. The Foundations of Ethology. Springer-Verlag, New York, NY.

[27]

MITCHELL, K. 1997. Machine Learning. McGraw Hill, New York, NY.

Digital Library

[28]

PERLIN, K., AND GOLDBERG, A. 1996. Improv: A system for scripting interactive actors in virtual worlds. In Proceedings of SIGGRAPH 1996, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[29]

PRYOR, K. 1999. Clicker Training for Dogs. Sunshine Books, Inc., Waltham, MA.

[30]

RABINER, L., AND JUANG, B.-H. 1993. Fundamentals of Speech Recognition. Prentice Hall, Englewood Cliffs, NJ.

Digital Library

[31]

RAMIREZ, K. 1999. Animal Training:Successful Animal Management Through Positive Reinforcement. Shedd Aquarium, Chicago, IL.

[32]

RESNER, B., STERN, A., AND FRANK, A. 1997. The truth about catz and dogz. In The Computer Games Developer Conference.

[33]

REYNOLDS, C. 1987. Flocks, herds and schools: A distributed behavioral model. In Proceedings of SIGGRAPH 1987, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[34]

ROSE, C., COHEN, M., AND BODENHEIMER, B. 1999. Verbs and adverbs: Multidimensional motion interpolation. IEEE Computer Graphics And Applications 18, 5.

Digital Library

[35]

SHETTLEWORTH, S. J. 1998. Cognition, Evolution and Behavior. Oxford University Press, New York, NY.

[36]

SUTTON, R., AND BARTO, A. 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge MA.

Digital Library

[37]

SUTTON, R. 1991. Reinforcement learning architectures for animats. In The First International Conference on Simulation of Adaptive Behavior, MIT Press, Paris, Fr.

Digital Library

[38]

THERRIEN, C. 1989. Decision Estimation and Classification: An Introduction to Pattern Recognition and Related Topics. John Wiley and Sons, New York, NY.

Digital Library

[39]

TOMLINSON, B., AND BLUMBERG, B. 2002. Alphawolf: Social learning, emotion and development in autonomous virtual agents. In First GSFC/JPL Workshop on Radical Agent Concepts.

[40]

TU, X., AND TERZOPOULOS, D. 1994. Artificial fishes: Physics, locomotion, perception, behavior. In Proceedings of SIGGRAPH 1994, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[41]

VAN DE PANNE, M., AND FIUME., E. 1993. Sensor-actuator networks. In Proceedings of SIGGRAPH 1993, ACM Press / ACM SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series, ACM.

Digital Library

[42]

VAN DE PANNE, M., KIM, R., AND FIUME., E. 1994. Synthesizing parameterized motions. In 5th Eurographics Workshop on Simulation and Animation.

[43]

WATKINS, C. J., AND DAYAN, P. 1992. Q-learning. Machine Learning 8.

Digital Library

[44]

WILKES, G. 1995. Click and Treat Training Kit. Click and Treat Inc., Mesa, AZ.

[45]

YOON, S., BLUMBERG, B., AND SCHNEIDER, G. 2000. Motivation-driven learning for interactive synthetic characters. In Proceedings of the Fourth International Conference on Autonomous Agents.

Digital Library

[46]

YOON, S., BURKE, R., AND BLUMBERG, B. 2000. Interactive training for synthetic characters. In Proceedings of AAAI 2000.

Digital Library

Cited By

Trick SHerbert FRothkopf CKoert D(2022)Interactive Reinforcement Learning With Bayesian Fusion of Multimodal AdviceIEEE Robotics and Automation Letters10.1109/LRA.2022.31821007:3(7558-7565)Online publication date: Jul-2022
https://doi.org/10.1109/LRA.2022.3182100
Chisari EWelschehold TBoedecker JBurgard WValada A(2022)Correct Me If I am Wrong: Interactive Learning for Robotic ManipulationIEEE Robotics and Automation Letters10.1109/LRA.2022.31455167:2(3695-3702)Online publication date: Apr-2022
https://doi.org/10.1109/LRA.2022.3145516
Lin JMa ZGomez RNakamura KHe BLi G(2020)A Review on Interactive Reinforcement Learning From Human Social FeedbackIEEE Access10.1109/ACCESS.2020.30062548(120757-120765)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3006254
Show More Cited By

Index Terms

Integrated learning for interactive synthetic characters
1. Computing methodologies
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Graphics input devices

Recommendations

Integrated learning for interactive synthetic characters

The ability to learn is a potentially compelling and important quality for interactive synthetic characters. To that end, we describe a practical approach to real-time learning for synthetic characters. Our implementation is grounded in the techniques ...
Automatic rigging and animation of 3D characters
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

Animating an articulated 3D character currently requires manual rigging to specify its internal skeletal structure and to define how the input motion deforms its surface. We present a method for animating characters automatically. Given a static ...
An art-directed wrinkle system for CG character clothing and skin

We present a kinematic system for creating art-directed clothing and skin wrinkles on CG characters used in the production of computer-animated feature films. This system employs a curve-based method for generating wrinkles on reference poses, which are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques

July 2002

574 pages

ISBN:1581135211

DOI:10.1145/566570

Conference Chair:
Tom Appolloni
Harris Corporation

Copyright © 2002 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2002

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGGRAPH02

Sponsor:

SIGGRAPH

SIGGRAPH02: The 29th International Conference on Computer Graphics and Interactive Techniques

July 23 - 26, 2002

Texas, San Antonio

Acceptance Rates

SIGGRAPH '02 Paper Acceptance Rate 67 of 358 submissions, 19%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

81
Total Citations
View Citations
1,808
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)1

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Trick SHerbert FRothkopf CKoert D(2022)Interactive Reinforcement Learning With Bayesian Fusion of Multimodal AdviceIEEE Robotics and Automation Letters10.1109/LRA.2022.31821007:3(7558-7565)Online publication date: Jul-2022
https://doi.org/10.1109/LRA.2022.3182100
Chisari EWelschehold TBoedecker JBurgard WValada A(2022)Correct Me If I am Wrong: Interactive Learning for Robotic ManipulationIEEE Robotics and Automation Letters10.1109/LRA.2022.31455167:2(3695-3702)Online publication date: Apr-2022
https://doi.org/10.1109/LRA.2022.3145516
Lin JMa ZGomez RNakamura KHe BLi G(2020)A Review on Interactive Reinforcement Learning From Human Social FeedbackIEEE Access10.1109/ACCESS.2020.30062548(120757-120765)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3006254
Li GWhiteson SKnox WHung H(2018)Social interaction for efficient agent learning from human rewardAutonomous Agents and Multi-Agent Systems10.1007/s10458-017-9374-832:1(1-25)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1007/s10458-017-9374-8
Phillips ESchaefer KBillings DJentsch FHancock P(2016)Human-animal teams as an analog for future human-robot teamsJournal of Human-Robot Interaction10.5898/JHRI.5.1.Phillips5:1(100-125)Online publication date: 23-Mar-2016
https://dl.acm.org/doi/10.5898/JHRI.5.1.Phillips
Chao CThomaz A(2016)Timed Petri nets for fluent turn-taking over multimodal interaction resources in human-robot collaborationInternational Journal of Robotics Research10.1177/027836491562729135:11(1330-1353)Online publication date: 1-Sep-2016
https://dl.acm.org/doi/10.1177/0278364915627291
Li GWhiteson SKnox WHung H(2016)Using informative behavior to increase engagement while learning from human rewardAutonomous Agents and Multi-Agent Systems10.1007/s10458-015-9308-230:5(826-848)Online publication date: 1-Sep-2016
https://dl.acm.org/doi/10.1007/s10458-015-9308-2
Harris TGittens C(2015)Modeling believable agents using a descriptive approachBiologically Inspired Cognitive Architectures10.1016/j.bica.2015.09.00414(10-21)Online publication date: Oct-2015
https://doi.org/10.1016/j.bica.2015.09.004
Yanik PManganelli JMerino JThreatt ABrooks JGreen KWalker I(2014)A Gesture Learning Interface for Simulated Robot Path Shaping With a Human TeacherIEEE Transactions on Human-Machine Systems10.1109/TSMC.2013.229171444:1(41-54)Online publication date: Feb-2014
https://doi.org/10.1109/TSMC.2013.2291714
Gray JBreazeal C(2014)Manipulating Mental States Through Physical ActionInternational Journal of Social Robotics10.1007/s12369-014-0234-26:3(315-327)Online publication date: 13-Apr-2014
https://doi.org/10.1007/s12369-014-0234-2
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents