research-article

Evaluating CoBlox: A Comparative Study of Robotics Programming Environments for Adult Novices

Authors:

David Weintrop,

Patrick Francis,

David C. Shepherd,

Diana FranklinAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 366, Pages 1 - 12

https://doi.org/10.1145/3173574.3173940

Published: 21 April 2018 Publication History

Abstract

A new wave of collaborative robots designed to work alongside humans is bringing the automation historically seen in large-scale industrial settings to new, diverse contexts. However, the ability to program these machines often requires years of training, making them inaccessible or impractical for many. This paper rethinks what robot programming interfaces could be in order to make them accessible and intuitive for adult novice programmers. We created a block-based interface for programming a one-armed industrial robot and conducted a study with 67 adult novices comparing it to two programming approaches in widespread use in industry. The results show participants using the block-based interface successfully implemented robot programs faster with no loss in accuracy while reporting higher scores for usability, learnability, and overall satisfaction. The contribution of this work is showing the potential for using block-based programming to make powerful technologies accessible to a wider audience.

Supplementary Material

ZIP File (pn3222.zip)

Download
90.05 MB

References

[1]

Brenna D. Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. 2009. A survey of robot learning from demonstration. Robotics and autonomous systems 57, 5: 469--483.

Digital Library

[2]

A. Cory Bart, J. Tibau, D. Kafura, C. A. Shaffer, and E. Tilevich. 2017. Design and Evaluation of a Blockbased Environment with a Data Science Context. IEEE Transactions on Emerging Topics in Computing PP, 99: 1--1.

[3]

David Bau. 2015. Droplet, a blocks-based editor for text code. Journal of Computing Sciences in Colleges 30, 6: 138--144.

Digital Library

[4]

David Bau, Jeff Gray, Caitlin Kelleher, Josh Sheldon, and Franklyn Turbak. 2017. Learnable programming: blocks and beyond. Communications of the ACM 60, 6: 72--80.

Digital Library

[5]

A Begel and E Klopfer. 2007. Starlogo TNG: An introduction to game development. Journal of ELearning.

[6]

Geoffrey Biggs and Bruce MacDonald. 2003. A survey of robot programming systems. In Proceedings of the Australasian conference on robotics and automation, 1--10.

[7]

Aude Billard, Sylvain Calinon, Ruediger Dillmann, and Stefan Schaal. 2008. Robot programming by demonstration. In Springer handbook of robotics. Springer, 1371--1394.

[8]

Rainer Bischoff, Arif Kazi, and Markus Seyfarth. 2002. The MORPHA style guide for icon-based programming. In Robot and Human Interactive Communication, 2002. Proceedings. 11th IEEE International Workshop on, 482--487.

[9]

Douglas Blank, Deepak Kumar, Lisa Meeden, and Holly Yanco. 2006. The Pyro toolkit for AI and robotics. AI magazine 27, 1: 39.

[10]

Cynthia Breazeal and Brian Scassellati. 2002. Robots that imitate humans. Trends in cognitive sciences 6, 11: 481--487.

[11]

Brian Broll, Akos Lédeczi, Peter Volgyesi, Janos Sallai, Miklos Maroti, Alexia Carrillo, Stephanie L. Weeden-Wright, Chris Vanags, Joshua D. Swartz, and Melvin Lu. 2017. A Visual Programming Environment for Learning Distributed Programming. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE '17), 81--86.

Digital Library

[12]

J. Edward Colgate, J. Edward, Michael A. Peshkin, and Witaya Wannasuphoprasit. 1996. Cobots: Robots For Collaboration With Human Operators.

[13]

S. Cooper, W. Dann, and R. Pausch. 2000. Alice: a 3D tool for introductory programming concepts. Journal of Computing Sciences in Colleges 15, 5: 107--116.

Digital Library

[14]

C Duncan, T Bell, and S Tanimoto. 2014. Should Your 8-year-old Learn Coding? In Proceedings of the 9th Workshop in Primary and Secondary Computing Education (WiPSCE '14), 60--69.

Digital Library

[15]

Sarah Esper, Stephen R. Foster, and William G. Griswold. 2013. CodeSpells: embodying the metaphor of wizardry for programming. In Proceedings of the 18th ACM conference on Innovation and technology in computer science education, 249--254.

Digital Library

[16]

Annette Feng, Eli Tilevich, and Wu-chun Feng. 2015. Block-based programming abstractions for explicit parallel computing. In Blocks and Beyond Workshop (Blocks and Beyond), 2015 IEEE, 71--75.

Digital Library

[17]

D Franklin, G Skifstad, R Rolock, I Mehrotra, V Ding, A Hansen, D Weintrop, and D Harlow. 2017. Using Upper-Elementary Student Performance to Understand Conceptual Sequencing in a Blocks-based Curriculum. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE '17), 231--236.

Digital Library

[18]

N. Fraser. 2015. Ten things we've learned from Blockly. In 2015 IEEE Blocks and Beyond Workshop (Blocks and Beyond), 49--50.

Digital Library

[19]

E. Freund and B. Luedemann-Ravit. 2002. A system to automate the generation of program variants for industrial robot applications. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 1856--1861 vol.2.

[20]

Thomas A. Fuhlbrigge, Gregory Rossano, Hui Zhang, Jianjun Wang, and Zhongxue Gan. 2010. Method and apparatus for developing a metadata-infused software program for controlling a robot.

[21]

Shuchi Grover, Roy Pea, and Stephen Cooper. 2015. Designing for deeper learning in a blended computer science course for middle school students. Computer Science Education 25, 2: 199--237.

[22]

Martin Hägele, Walter Schaaf, and Evert Helms. 2002. Robot assistants at manual workplaces: Effective cooperation and safety aspects. In Proceedings of the 33rd ISR (International Symposium on Robotics), 7-- 11.

[23]

Robert Hopler and Martin Otter. 2001. A versatile C++ toolbox for model based, real time control systems of robotic manipulators. In Intelligent Robots and Systems, 2001. Proceedings. 2001 IEEE/RSJ International Conference on, 2208--2214.

[24]

M. S Horn, C Brady, A Hjorth, A Wagh, and U Wilensky. 2014. Frog pond: a codefirst learning environment on evolution and natural selection. In Proceedings of the 2014 conference on Interaction design and children, 357--360.

Digital Library

[25]

Andri Ioannidou, Alexander Repenning, and David C. Webb. 2009. AgentCubes: Incremental 3D end-user development. Journal of Visual Languages & Computing 20, 4: 236--251.

Digital Library

[26]

C. Kelleher and R. Pausch. 2005. Lowering the barriers to programming: A taxonomy of programming environments and languages for novice programmers. ACM Computing Surveys 37, 2: 83--137.

Digital Library

[27]

S. Kock, T. Vittor, B. Matthias, H. Jerregard, M. Källman, I. Lundberg, R. Mellander, and M. Hedelind. 2011. Robot concept for scalable, flexible assembly automation: A technology study on a harmless dualarmed robot. In 2011 IEEE International Symposium on Assembly and Manufacturing (ISAM), 1--5.

[28]

Daisuke Kushida, Masatoshi Nakamura, Satoru Goto, and Nobuhiro Kyura. 2001. Human direct teaching of industrial articulated robot arms based on force-free control. Artificial Life and Robotics 5, 1: 26--32.

[29]

Tom Lauwers and Illah Nourbakhsh. 2010. Designing the finch: Creating a robot aligned to computer science concepts. In AAAI Symposium on Educational Advances in Artificial Intelligence.

[30]

Lego Systems Inc. 2008. Lego Mindstorms NXT-G Invention System. Retrieved from http://mindstorms.lego.com

[31]

Makeblock Co., Ltd. 2017. mBot. Retrieved September 18, 2017 from http://www.makeblock.com/

[32]

J. H Maloney, M Resnick, N Rusk, B Silverman, and E Eastmond. 2010. The Scratch programming language and environment. ACM Transactions on Computing Education (TOCE) 10, 4: 16.

Digital Library

[33]

J Maloney, M Nagle, and J Mönig. 2017. GP: A General Purpose Blocks-Based Language. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE '17), 739--739.

Digital Library

[34]

Microbric Pty, Ltd. 2017. Edison Programmable Robot. Retrieved from https://meetedison.com/

[35]

Brad A. Myers. 1990. Taxonomies of visual programming and program visualization. Journal of Visual Languages & Computing 1, 1: 97--123.

Digital Library

[36]

Ozobot & Evollve, Inc. 2017. Ozobot. Retrieved from http://ozobot.com/

[37]

Zengxi Pan, Joseph Polden, Nathan Larkin, Stephen Van Duin, and John Norrish. 2012. Recent progress on programming methods for industrial robots. Robotics and Computer-Integrated Manufacturing 28, 2: 87--94.

Digital Library

[38]

J. N. Pires, K. Nilsson, and H. G. Petersen. 2005. Industrial robotics applications and industry-academia cooperation in Europe. IEEE Robotics Automation Magazine 12, 3: 5--6.

[39]

Mitchell Resnick, Brian Silverman, Yasmin Kafai, John Maloney, Andrés Monroy-Hernández, Natalie Rusk, Evelyn Eastmond, Karen Brennan, Amon Millner, Eric Rosenbaum, and Jay Silver. 2009. Scratch: Programming for all. Communications of the ACM 52, 11: 60.

Digital Library

[40]

Wolfgang Slany. 2014. Tinkering with Pocket Code, a Scratch-like programming app for your smartphone. In Proceedings of Constructionism 2014.

[41]

A. Strauss and J. Corbin. 1994. Grounded Theory Methodology: An Overview. In Strategies of Qualitative Inquiry. Sage Publications, Inc, Thousand Oaks, CA, 158--183.

[42]

D. Weintrop, D. C. Shepherd, P. Francis, and D. Franklin. 2017. Blockly goes to work: Block-based programming for industrial robots. In 2017 IEEE Blocks and Beyond Workshop, 29--36.

[43]

D Weintrop and U Wilensky. In Press. Comparing Blocks-based and Text-based Programming in High School Computer Science Classrooms. ACM Transactions on Computing Education (TOCE).

Digital Library

[44]

D. Weintrop and U. Wilensky. 2012. RoboBuilder: A program-to-play constructionist video game. In Proceedings of the Constructionism 2012 Conference.

[45]

D Weintrop and U. Wilensky. 2015. To Block or Not to Block, That is the Question: Students' Perceptions of Blocks-based Programming. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC '15), 199--208.

Digital Library

[46]

M. H. Wilkerson-Jerde and U. Wilensky. 2010. Restructuring Change, Interpreting Changes: The DeltaTick Modeling and Analysis Toolkit. In Proceedings of the Constructionism 2010 Conference.

[47]

David Wolber, Hal Abelson, Ellen Spertus, and Liz Looney. 2014. App Inventor 2: Create Your Own Android Apps. O'Reilly Media, Beijing.

Digital Library

[48]

Wonder Workshop, Inc. 2017. Dash & Dot. Retrieved from https://www.makewonder.com/

[49]

Y. F. Yong and M. C. Bonney. 1999. Off-Line Programming. In Handbook of Industrial Robotics, Shimon Y. Nof (ed.). John Wiley & Sons, Inc., 353-- 371.

[50]

Open Roberta. Retrieved September 18, 2017 from https://www.open-roberta.org/en/welcome/

Cited By

Kassem KSaad APascher MSchett MMichahelles F(2024)Push Me: Evaluating Usability and User Experience in Nudge-based Human-Robot Interaction through Embedded Force and Torque SensorsProceedings of Mensch und Computer 202410.1145/3670653.3677487(399-407)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3677487
Pereira DBarbosa FMorgado CMonga MLonati VBarendsen ESheard JPaterson J(2024)NextBlocks: An Interactive Block Programming PlatformProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653609(590-596)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653609
Gargioni LFogli DGrollman DBroadbent EJu WSoh HWilliams T(2024)Integrating ChatGPT with Blockly for End-User Development of Robot TasksCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640653(478-482)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640653
Show More Cited By

Index Terms

Evaluating CoBlox: A Comparative Study of Robotics Programming Environments for Adult Novices
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Software and its engineering
  1. Software notations and tools

Recommendations

Evaluating CoBlox: A Comparative Study of Robotics Programming Environments for Adult Novices
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems

A new wave of collaborative robots designed to work alongside humans is bringing the automation historically seen in large-scale industrial settings to new, diverse contexts. However, the ability to program these machines often requires years of ...
Experience Helps, but It Isn't Everything: Exploring Causes of Affective State in Novice Programmers
SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2

Affective state, referring to an individual's feeling, can impact students' confidence and retention in CS, particularly for novice programmers. However, little research has been conducted to examine how moments that occur during programming impact ...
Scratch and Google Blockly: How Girls' Programming Skills and Attitudes are Influenced
Koli Calling '19: Proceedings of the 19th Koli Calling International Conference on Computing Education Research

Block-based programming has become popular to teach programming to young students in introductory programming environments. Nevertheless, in most western countries, girls show a lack of interest in computer science, including programming. This paper ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

47
Total Citations
View Citations
881
Total Downloads

Downloads (Last 12 months)93
Downloads (Last 6 weeks)17

Reflects downloads up to 11 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kassem KSaad APascher MSchett MMichahelles F(2024)Push Me: Evaluating Usability and User Experience in Nudge-based Human-Robot Interaction through Embedded Force and Torque SensorsProceedings of Mensch und Computer 202410.1145/3670653.3677487(399-407)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3677487
Pereira DBarbosa FMorgado CMonga MLonati VBarendsen ESheard JPaterson J(2024)NextBlocks: An Interactive Block Programming PlatformProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653609(590-596)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653609
Gargioni LFogli DGrollman DBroadbent EJu WSoh HWilliams T(2024)Integrating ChatGPT with Blockly for End-User Development of Robot TasksCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640653(478-482)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610978.3640653
Karli UChen JAntony VHuang CGrollman DBroadbent EJu WSoh HWilliams T(2024)Alchemist: LLM-Aided End-User Development of Robot ApplicationsProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634969(361-370)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634969
Fronchetti FRitschel NSchorr LBarfield CChang GSpinola RHolmes RShepherd DRoychoudhury APaiva AAbreu RStorey M(2024)Block-based Programming for Two-Armed Robots: A Comparative StudyProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3623329(1-12)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3623329
Debloos JJost CArchambault D(2024)Programming Learning Difficulties: How Can Naive Users Create Human-Machine Interaction Scenarios?Computers Helping People with Special Needs10.1007/978-3-031-62849-8_30(241-248)Online publication date: 8-Jul-2024
https://dl.acm.org/doi/10.1007/978-3-031-62849-8_30
Aguzzi GCasadei RCerioni MViroli M(2024)ScaFi-Blocks: A Visual Aggregate Programming Environment for Low-Code Swarm DesignCoordination Models and Languages10.1007/978-3-031-62697-5_14(258-276)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-62697-5_14
Kassem KPavlova GSchlund SMichahelles F(2023)Build-A-Bot: Developing A Software Platform For A Modular Mobile RobotProceedings of the 13th International Conference on the Internet of Things10.1145/3627050.3627054(74-81)Online publication date: 7-Nov-2023
https://dl.acm.org/doi/10.1145/3627050.3627054
Mayr-Dorn CWinterer MSalomon CHohensinger DFürschuss H(2023)Assessing industrial end-user programming of robotic production cells: A controlled experimentJournal of Systems and Software10.1016/j.jss.2022.111547195(111547)Online publication date: Jan-2023
https://doi.org/10.1016/j.jss.2022.111547
Brouzos RPanayiotou KTsardoulias ESymeonidis A(2023)A Low-Code Approach for Connected RobotsJournal of Intelligent & Robotic Systems10.1007/s10846-023-01861-y108:2Online publication date: 19-Jun-2023
https://doi.org/10.1007/s10846-023-01861-y
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents