research-article

Robots, Pancakes, and Computer Games: Designing Serious Games for Robot Imitation Learning

Authors:

Benjamin Walther-Franks,

Rainer MalakaAuthors Info & Claims

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

Pages 3623 - 3632

https://doi.org/10.1145/2702123.2702552

Published: 18 April 2015 Publication History

Abstract

Autonomous manipulation robots can be valuable aids as interactive agents in the home, yet it has proven extremely difficult to program their behavior. Imitation learning uses data on human demonstrations to build behavioral models for robots. In order to cover a wide range of action strategies, data from many individuals is needed. Acquiring such large amounts of data can be a challenge. Tools for data capturing in this domain must thus implement a good user experience. We propose to use human computation games in order to gather data on human manual behavior. We demonstrate the idea with a strategy game that is operated via a natural user interface. A comparison between using the game for action execution and demonstrating actions in a virtual environment shows that people interact longer and have a better experience when playing the game.

Supplementary Material

suppl.mov (pn2328-file3.mp4)

Supplemental video

Download
27.21 MB

MP4 File (p3623-walther-franks.mp4)

Download
136.18 MB

References

[1]

Aras, H., Krause, M., Haller, A., & Malaka, R. (2010). Webpardy: Harvesting QA by HC. In Proc. HCOMP 2010, (pp. 49--52). ACM.

Digital Library

[2]

Argall, B. D., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469--483.

Digital Library

[3]

Beetz, M., Klank, U., Kresse, I., Maldonado, A., Mosenlechner, L., Pangercic, D., Ruhr, T., & Tenorth, M. (2011). Robotic roommates making pancakes. In Proc. Humanoids 2011, (pp. 529--536). IEEE.

[4]

Billard, A., Calinon, S., Dillmann, R., & Schaal, S. (2008). Robot programming by demonstration. In B. Siciliano, & O. Khatib (Eds.) Springer Handbook of Robotics, chap. 60, (pp. 1371--1394). Springer.

[5]

Bollini, M., Tellex, S., Thompson, T., Roy, N., & Rus, D. (2013). Interpreting and executing recipes with a cooking robot. In J. P. Desai, G. Dudek, O. Khatib, & V. Kumar (Eds.) Experimental Robotics, vol. 88 of Springer Tracts in Advanced Robotics, (pp. 481--495). Springer.

[6]

Bonetta, L. (2009). New citizens for the life sciences. Cell, 138(6), 1043--1045.

[7]

Bowman, D. A., Kruijff, E., LaViola, J. J., & Poupyrev, I. (2004). 3D User Interfaces: Theory and Practice. Addison-Wesley.

Digital Library

[8]

Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: Defining "gamification". In Proc MindTrek 2011, (pp. 9--15). New York, NY, USA: ACM.

Digital Library

[9]

Flatla, D. R., Gutwin, C., Nacke, L. E., Bateman, S., & Mandryk, R. L. (2011). Calibration games: Making calibration tasks enjoyable by adding motivating game elements. In Proc. UIST 2011, (pp. 403--412). ACM.

Digital Library

[10]

Forbes, M., Chung, M. J., Cakmak, M., & Rao, R. P. N. (2014). Robot programming by demonstration with crowdsourced action fixes. In Proc. HCOMP 2014, (pp. 67--76). AAAI Press.

[11]

Haidu, A., Kohlsdorf, D., & Beetz, M. (2014). Learning task outcome prediction for robot control from interactive environments. In Proc. Intelligent Robots and Systems. IEEE/RSJ.

[12]

Hunicke, R., LeBlanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. In D. Fu, S. Henke, & J. Orkin (Eds.) AAAI Workshop Challenges in Game Artificial Intelligence. AAAI Press.

[13]

Kanda, T., Ishiguro, H., Imai, M., & Ono, T. (2004). Development and evaluation of interactive humanoid robots. Proceedings of the IEEE, 92(11), 1839--1850.

[14]

Krause, M., & Smeddinck, J. (2012). Human Computation: A New Aspect of Serious Games., (pp. 1027--1047). In Handbook of Research on Serious Games as Educational, Business and Research Tools: Development and Design. IGI Global.

[15]

Krause, M., Takhtamysheva, A., Wittstock, M., & Malaka, R. (2010). Frontiers of a paradigm: Exploring human computation with digital games. In Proc HCOMP 10, (pp. 22--25). ACM.

Digital Library

[16]

Kunze, L., Haidu, A., & Beetz, M. (2013). Acquiring task models for imitation learning through games with a purpose. In Proc. IROS, 2013, (pp. 102--107). IEEE.

[17]

Leap Motion, Inc. http://leapmotion.com (last visited on 2015-01-07).

[18]

Matyas, S., Matyas, C., Schlieder, C., Kiefer, P., Mitarai, H., & Kamata, M. (2008). Designing locationbased mobile games with a purpose: Collecting geospatial data with CityExplorer. In Proc. ACE 2008, (pp. 244--247). ACM.

Digital Library

[19]

McCarthy, J. (1968). Programs with common sense. In Semantic Information Processing, vol. 1, (pp. 403--418).

[20]

Pollard, N. S., Hodgins, J. K., Riley, M. J., & Atkeson, C. G. (2002). Adapting human motion for the control of a humanoid robot. In Proc. ICRA 2002, vol. 2, (pp. 1390--1397). IEEE.

[21]

Quinn, A. J., & Bederson, B. B. (2011). Human computation: A survey and taxonomy of a growing field. In Proc. CHI 2011, (pp. 1403--1412). ACM.

Digital Library

[22]

Ryan, R. M., Rigby, C. S., & Przybylski, A. (2006). The motivational pull of video games: A Self-Determination theory approach. Motivation and Emotion, 30(4), 344--360.

[23]

Schaal, S. (1999). Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6), 233--242.

[24]

Shamsuddin, S., Ismail, L. I., Yussof, H., Ismarrubie Zahari, N., Bahari, S., Hashim, H., & Jaffar, A. (2011). Humanoid robot NAO: Review of control and motion exploration. In Proc. ICCSCE, (pp. 511--516). IEEE.

[25]

Toris, R., Kent, D., & Chernova, S. (2014). The robot management system: A framework for conducting Human-Robot interaction studies through crowdsourcing. Journal of Human-Robot Interaction, 3(2), 25+.

[26]

Tuite, K., Snavely, N., Hsiao, D. Y., Smith, A. M., & Popović, Z. (2010). Reconstructing the world in 3D: Bringing games with a purpose outdoors. In Proc. FDG 2010, (pp. 232--239). ACM.

Digital Library

[27]

von Ahn, L., & Dabbish, L. (2008). Designing games with a purpose. Commun. ACM, 51(8), 58--67.

Digital Library

[28]

Yuen, M.-C., Chen, L.-J., & King, I. (2009). A survey of human computation systems. In Proc. CSE, vol. 4, (pp. 723--728). Los Alamitos, CA, USA: IEEE.

Digital Library

Cited By

Tyack AMekler E(2024)Self-Determination Theory and HCI Games Research: Unfulfilled Promises and Unquestioned ParadigmsACM Transactions on Computer-Human Interaction10.1145/367323031:3(1-74)Online publication date: 15-Jun-2024
https://dl.acm.org/doi/10.1145/3673230
Pappas GSiegel JVogiatzakis IPolitopoulos K(2022)Gamification and the Internet of Things in EducationHandbook on Intelligent Techniques in the Educational Process10.1007/978-3-031-04662-9_15(317-339)Online publication date: 16-Jun-2022
https://doi.org/10.1007/978-3-031-04662-9_15
Pappas GSiegel JPolitopoulos KSun Y(2021)A Gamified Simulator and Physical Platform for Self-Driving Algorithm Training and ValidationElectronics10.3390/electronics1009111210:9(1112)Online publication date: 8-May-2021
https://doi.org/10.3390/electronics10091112
Show More Cited By

Index Terms

Robots, Pancakes, and Computer Games: Designing Serious Games for Robot Imitation Learning

Recommendations

Reward Systems in Human Computation Games
CHI PLAY '16: Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play

Human computation games (HCGs) are games in which player interaction is used to solve problems intractable for computers. Most HCGs use simple reward mechanisms such as points or leaderboards, but in contrast, many mainstream games use more complex, and ...
Evaluating singleplayer and multiplayer in human computation games
FDG '17: Proceedings of the 12th International Conference on the Foundations of Digital Games

Human computation games (HCGs) can provide novel solutions to intractable computational problems, help enable scientific breakthroughs, and provide datasets for artificial intelligence. However, our knowledge about how to design and deploy HCGs that ...
A framework for exploring and evaluating mechanics in human computation games
FDG '17: Proceedings of the 12th International Conference on the Foundations of Digital Games

Human computation games (HCGs) are a crowdsourcing approach to solving computationally-intractable tasks using games. We outline a formal representation of the mechanics in HCGs, providing a structural breakdown to visualize, compare, and explore the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems

April 2015

4290 pages

ISBN:9781450331456

DOI:10.1145/2702123

General Chairs:
Bo Begole
Huawei, USA
,
Jinwoo Kim
Yonsei University, Korea
,
Program Chairs:
Kori Inkpen
Microsoft Research, USA
,
Woontack Woo
KAIST, Korea

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 April 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '15

Sponsor:

SIGCHI

CHI '15: CHI Conference on Human Factors in Computing Systems

April 18 - 23, 2015

Seoul, Republic of Korea

Acceptance Rates

CHI '15 Paper Acceptance Rate 486 of 2,120 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
438
Total Downloads

Downloads (Last 12 months)27
Downloads (Last 6 weeks)5

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tyack AMekler E(2024)Self-Determination Theory and HCI Games Research: Unfulfilled Promises and Unquestioned ParadigmsACM Transactions on Computer-Human Interaction10.1145/367323031:3(1-74)Online publication date: 15-Jun-2024
https://dl.acm.org/doi/10.1145/3673230
Pappas GSiegel JVogiatzakis IPolitopoulos K(2022)Gamification and the Internet of Things in EducationHandbook on Intelligent Techniques in the Educational Process10.1007/978-3-031-04662-9_15(317-339)Online publication date: 16-Jun-2022
https://doi.org/10.1007/978-3-031-04662-9_15
Pappas GSiegel JPolitopoulos KSun Y(2021)A Gamified Simulator and Physical Platform for Self-Driving Algorithm Training and ValidationElectronics10.3390/electronics1009111210:9(1112)Online publication date: 8-May-2021
https://doi.org/10.3390/electronics10091112
Tyack AMekler EBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Self-Determination Theory in HCI Games Research: Current Uses and Open QuestionsProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376723(1-22)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376723
Pfau JPorzel RPomarlan MCangalovic VGrudpan SHöffner SBateman JMalaka R(2019)Give MEANinGS to Robots with Kitchen Clash: A VR Human Computation Serious Game for World Knowledge AccumulationEntertainment Computing and Serious Games10.1007/978-3-030-34644-7_7(85-96)Online publication date: 4-Nov-2019
https://doi.org/10.1007/978-3-030-34644-7_7
Sarma HPorzel RMalaka R(2016)A Step Toward Automated Simulation in IndustryDynamics in Logistics10.1007/978-3-319-45117-6_9(99-105)Online publication date: 15-Sep-2016
https://doi.org/10.1007/978-3-319-45117-6_9
Haidu AKohlsdorf DBeetz M(2015)Learning action failure models from interactive physics-based simulations2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS.2015.7354136(5370-5375)Online publication date: Sep-2015
https://doi.org/10.1109/IROS.2015.7354136

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents