article

Free access

Graphical representation of programs in a demonstrational visual shell—an empirical evaluation

Editor: Dan R. Olson, Jr. Authors:

Francesmary Modugno,

Albert T. Corbett,

Brad A. MyersAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 4, Issue 3

Pages 276 - 308

https://doi.org/10.1145/264645.264659

Published: 01 September 1997 Publication History

PDF eReader

Abstract

An open question in the area of Programming by Demonstration (PBD) is how to best represent the inferred program. Without a way to view, edit, and share programs, PBD systems will never reach their full potential. We designed and implemented two graphical representation languages for a PBD desktop similar to the Apple Macintosh Finder. Although a user study showed that both languages enabled nonprogrammers to generate and comprehend programs, the study also revealed that the language that more closely reflected the desktop domain doubled users' abilities to accurately generate programs. Trends suggest that the same language was easier for users to comprehend. These findings suggest that it is possible for a PBD system to enable nonprogrammers to construct programs and that the form of the representation can impact the PBD system's effectiveness. A paper-and-pencil evaluation of the two versions of the PBD desktop prior to the study supported these finding and provided interesting feedback on the interaction between usability evaluations and user studies. In particular, the comparison of the paper-and-pencil evaluation with the empirical evaluation suggested that nonempirical evaluation techniques can provide guidance into how to interpret empirical data and, in particular, that PBD systems need to provide support for programming-strategy selection in order to be successful.

References

[1]

BORG, K. 1989. Visual programming and UNIX. In the IEEE Workshop on Visual Languages. IEEE, New York, 74-79.

Google Scholar

[2]

BOTZUM, K. 1995. An empirical study of shell programs. Tech. Rep., Bell Communications Research.

Google Scholar

[3]

CUNNIFF, N. AND TAYLOR, R. P. 1987. Graphical vs. textual representation: An empirical study of novices' program comprehension. In Empirical Studies of Programmers: 2nd Workshop (Washington, D.C., Dec.). G. M. Olson, S. Sheppard, and E. Soloway, Eds. 114-131.

Crossref

Google Scholar

[4]

CYPHER, A. 1991. EAGER: Programming repetitive tasks by example. In Proceedings of CHI '91 (New Orleans, La., Apr.). ACM, New York, 33-40.

Crossref

Google Scholar

[5]

CYPHER, A. 1993. Watch What I Do: Programming by Demonstration. The MIT Press, Cambridge, Mass.

Crossref

Google Scholar

[6]

GREEN, T. 1989. Cognitive dimensions of notations. In People and Computers V, A. Sutchiff and L. Macaulay, Eds. Cambridge University Press, Cambridge, Mass., 443-460.

Crossref

Google Scholar

[7]

GREEN, T. 1991. Describing information artifacts with cognitive dimensions and structure maps. In People and Computers VI, D. Diaper and N. Hammonds, Eds. Cambridge University Press, Cambridge, Mass., 297-316.

Google Scholar

[8]

GREEN, T., PETRE, M., AND BELLAMY, R. 1991. Comprehensibility of visual and textual programs: A test of superlativism against the 'Match-Mismatch' conjecture. In Empirical Studies of Programmers: 4th Workshop. Ablex, Norwood, N.J.

Google Scholar

[9]

GREEN, T. R. G. ANn PETRE, M. 1994. Cognitive dimensions as discussion tools for programming language design. Tech. Rep., M. R. C. Applied Psychology Unit, Cambridge, England.

Google Scholar

[10]

HAEBERLI, P. 1988. ConMan: A visual programming language for interactive graphics. In ACM SIGGRAPH (Atlanta, Ga.). ACM, New York, 103-111.

Crossref

Google Scholar

[11]

HALBERT, D.C. 1984. Programming by example. Ph.D. thesis, Computer Science Division, Univ. of California, Berkeley, Calif.

Crossref

Google Scholar

[12]

HENRY, T. R. ANn HUDSON, S. E. 1988. Squish: A graphical shell for UNIX. In Graphics Interface. 43-49.

Crossref

Google Scholar

[13]

JOVANOVIC, B. ANn FOLEY, J. D. 1986. A simple graphics interface to UNIX. Tech. Rep. GWU-IIST-86-23. Inst. for Information Science and Technology, The George Washington Univ., Washington, D.C.

Google Scholar

[14]

KURLANDER, D. ANn FEINER, S. 1988. Editable graphical histories. In the IEEE Workshop on Visual Languages (Pittsburgh, Pa., Oct.). IEEE, New York, 127-134.

Google Scholar

[15]

LIEBERMAN, H. 1992. Dominoes and storyboards: Beyond "Icons On Strings". In the IEEE Workshop on Visual Languages. IEEE, New York.

Google Scholar

[16]

MAULSBY, D. L., GREENBERG, S., AND MANDER, R. 1993. Prototyping an intelligent agent through Wizard of Oz. In Proceedings of InterCHI '93. 277-284.

Crossref

Google Scholar

[17]

MAULSBY, D. L. AND WITTEN, I. H. 1989. Inducing programs in a direct-manipulation environment. In Proceedings of CHI '89 (Austin, Tex., Apr.). ACM, New York, 57-62.

Crossref

Google Scholar

[18]

MODUGNO, F. 1995. Extending end-user programming in a visual shell with Programming by Demonstration and graphical language techniques. Ph.D. thesis, Tech. Rep. CMU-CS-95- 130. Carnegie Mellon Univ., Pittsburgh, Pa.

Crossref

Google Scholar

[19]

MODUGNO, F., GREEN, T., AND MYERS, B.A. 1994. Visual programming in a visual domain: A case study of cognitive dimensions. In Proceedings of Human Computer Interaction '94.

Crossref

Google Scholar

[20]

MOHER, T. B., MAK, D. C., BLUMENTHAL, B., AND LEVENTHAL, L.M. 1993. Comparing the comprehensibility of textual and graphical programs: The case of Petri Nets. In Empirical Studies of Programmers: 5th Workshop. Ablex, Norwood, N.J.

Google Scholar

[21]

MYERS, B.A. 1988. Creating User Interfaces by Demonstration. Academic Press, Boston, Mass.

Crossref

Google Scholar

[22]

MYERS, B.A. 1992. Demonstrational interfaces: A step beyond direct manipulation. IEEE Comput. 25, 8 (Aug.), 61-73.

Crossref

Google Scholar

[23]

MYERS, B. A., GUISE, D., DANNENBERG, R., VANDER ZANDEN, B., KOSBIE, D., PERVIN, E., MICKISH, A., AND MARCHAL, P. 1990. Garnet: Comprehensive support for graphical, highly-interactive user interfaces. IEEE Comput. 23, 11 (Nov.), 71-85.

Crossref

Google Scholar

[24]

PETRE, M. 1995. Why looking isn't always seeing: Readership skills and graphical programming. Commun. ACM 38, 6 (June), 33-44.

Crossref

Google Scholar

[25]

SHNEIDERMAN, B. 1983. Direct manipulation: A step beyond programming languages. Computer 16, 8 (Aug.), 57-69.

Google Scholar

[26]

SMITH, D., CYPHER, A., AND SPOHRER, J. 1994. Kidsim: Programming agents without a programming language. Commun. ACM 37, 7, 54-67.

Crossref

Google Scholar

[27]

SMITH, D. C., IRBY, C., KIMBALL, R., AND HARSLEM, E. 1982. Designing the Star user interface. Byte 7, 4 (Apr.), 242-287.

Google Scholar

Cited By

View all

Lee JGarduño LWalker EBurleson WMattern FSantini SCanny JLangheinrich MRekimoto J(2013)A tangible programming tool for creation of context-aware applicationsProceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing10.1145/2493432.2493483(391-400)Online publication date: 8-Sep-2013
https://dl.acm.org/doi/10.1145/2493432.2493483
Akama HYamamuro MMenjo TSatoh T(2011)Integrating multimedia data processing parts in cloud into folder programming environmentProceedings of the 13th International Conference on Information Integration and Web-based Applications and Services10.1145/2095536.2095560(130-137)Online publication date: 5-Dec-2011
https://dl.acm.org/doi/10.1145/2095536.2095560
Hundhausen CFarley SBrown J(2009)Can direct manipulation lower the barriers to computer programming and promote transfer of training?ACM Transactions on Computer-Human Interaction10.1145/1592440.159244216:3(1-40)Online publication date: 24-Sep-2009
https://dl.acm.org/doi/10.1145/1592440.1592442
Show More Cited By

Index Terms

Graphical representation of programs in a demonstrational visual shell—an empirical evaluation
1. Software and its engineering
  1. Software notations and tools
    1. Context specific languages
      1. Visual languages

Recommendations

Pavlov: an interface builder for designing animated interfaces

Conventional interface builders provide little support for interactive development of interfaces with application-specific graphics. Some Programming by Demonstration (PBD) systems do provide such support, but none provide full support for demonstrating ...
Adding rule-based reasoning to a demonstrational interface builder
UIST '92: Proceedings of the 5th annual ACM symposium on User interface software and technology

This paper presents a demonstrational interface builder with improved reasoning capabilities. The system is comprised of two major components: an interactive display manager and a rule-based reasoner. The display manager provides facilities to draw the ...
Visual design checklist for graphical user interface (GUI) evaluation
SAC '18: Proceedings of the 33rd Annual ACM Symposium on Applied Computing

The¹ Graphical User Interface (GUI) design is a major process within the software development lifecycle, because the GUI is the portion of the interface that will be in direct contact with the end-user. There are resources to help make the interface ...

Reviews

Reviewer: Hans J. Schneider

The purpose of this study was to see whether nonprogrammers would be able to generate and comprehend programs in a demonstrational visual shell. To construct a shell program using this system, the user executes concrete actions on actual data objects, and the system infers a general procedure. In designing Pursuit, a programming-by-demonstration system, the authors explored two equivalent program representation languages. The first language employs a comic-strip metaphor, and the second uses keywords for control constructs and for operations. The authors introduce these languages in s ection 2 by giving typical shell programs, such as a program to copy all files edited today and compress them. In the next section, they briefly review related work. Section 4 presents the details of the empirical evaluation. The study consisted of three parts: a tutorial introduction, construction of programs, and assessment of users' comprehension of the program representations. In generating programs, the comic-strip group was twice as successful as the keyword group. (In the text-based group, no user correctly solved the task that had a loop and a branch.) The test of comprehension was divided into asking for the right program, fixing a bug, and modifying a program. The only significant difference was in identifying the right solution to a problem: here the text-based group needed 25 percent less time to find the answer. A detailed discussion completes the paper. It includes some informal results gathered using questionnaires and a comparison with a previous pencil-and-paper evaluation. The paper is well written, self-contained, and easy to read. I recommend it to a general computer science audience as well as to experts in this field.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

ACM Transactions on Computer-Human Interaction Volume 4, Issue 3

Sept. 1997

112 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/264645

Editor:
Dan R. Olson
Brigham Young Univ, Polvo, UT

Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 1997

Published in TOCHI Volume 4, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
923
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)7

Reflects downloads up to 12 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lee JGarduño LWalker EBurleson WMattern FSantini SCanny JLangheinrich MRekimoto J(2013)A tangible programming tool for creation of context-aware applicationsProceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing10.1145/2493432.2493483(391-400)Online publication date: 8-Sep-2013
https://dl.acm.org/doi/10.1145/2493432.2493483
Akama HYamamuro MMenjo TSatoh T(2011)Integrating multimedia data processing parts in cloud into folder programming environmentProceedings of the 13th International Conference on Information Integration and Web-based Applications and Services10.1145/2095536.2095560(130-137)Online publication date: 5-Dec-2011
https://dl.acm.org/doi/10.1145/2095536.2095560
Hundhausen CFarley SBrown J(2009)Can direct manipulation lower the barriers to computer programming and promote transfer of training?ACM Transactions on Computer-Human Interaction10.1145/1592440.159244216:3(1-40)Online publication date: 24-Sep-2009
https://dl.acm.org/doi/10.1145/1592440.1592442
Metoyer RStumpf SNeumann CDodge JCao JSchnabel A(2009)Explaining How to Play Real-Time Strategy GamesResearch and Development in Intelligent Systems XXVI10.1007/978-1-84882-983-1_18(249-262)Online publication date: 19-Oct-2009
https://doi.org/10.1007/978-1-84882-983-1_18
Hupp DMiller RShen CJacob RBalakrishnan R(2007)Smart bookmarksProceedings of the 20th annual ACM symposium on User interface software and technology10.1145/1294211.1294226(81-90)Online publication date: 7-Oct-2007
https://dl.acm.org/doi/10.1145/1294211.1294226
Bolin MMiller R(2005)Naming page elements in end-user web automationProceedings of the first workshop on End-user software engineering10.1145/1083231.1083233(1-5)Online publication date: 21-May-2005
https://dl.acm.org/doi/10.1145/1083231.1083233
Bolin MMiller R(2005)Naming page elements in end-user web automationACM SIGSOFT Software Engineering Notes10.1145/1082983.108323330:4(1-5)Online publication date: 21-May-2005
https://dl.acm.org/doi/10.1145/1082983.1083233
(2002)Unix tools as visual programming components in a GUI-builder environmentSoftware—Practice & Experience10.1002/spe.42832:1(57-71)Online publication date: 14-Jan-2002
https://dl.acm.org/doi/10.1002/spe.428
Wolber DMyers B(2001)Stimulus-Response PBDYour Wish is My Command10.1016/B978-155860688-3/50017-8(321-XVI)Online publication date: 2001
https://doi.org/10.1016/B978-155860688-3/50017-8
Myers BMcDaniel R(2001)Demonstrational InterfacesYour Wish is My Command10.1016/B978-155860688-3/50004-X(45-III)Online publication date: 2001
https://doi.org/10.1016/B978-155860688-3/50004-X
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Pavlov: an interface builder for designing animated interfaces

Adding rule-based reasoning to a demonstrational interface builder

Visual design checklist for graphical user interface (GUI) evaluation

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations