research-article

Heuristic evaluation of programming language features: two parallel programming case studies

Authors:

Caitlin Sadowski and

Sri KurniawanAuthors Info & Claims

PLATEAU '11: Proceedings of the 3rd ACM SIGPLAN workshop on Evaluation and usability of programming languages and tools

October 2011

Pages 9 - 14

https://doi.org/10.1145/2089155.2089160

Published: 24 October 2011 Publication History

Abstract

Usability is an important feature for programming languages, and user evaluations can provide invaluable feedback on language design. However, user studies which compare programming languages or systems are often very expensive and typically inconclusive. In this paper, we posit that discount usability methods can be successfully applied to programming languages concepts such as language features. We give examples of useful feedback received from applying heuristic evaluation to two language features targeted at parallel programming.

References

[1]

K. Asanovic, R. Bodik, J. Demmel, T. Keaveny, K. Keutzer, J. Kubiatowicz, N. Morgan, D. Patterson, K. Sen, J. Wawrzynek, et al. A view of the parallel computing landscape. Communications of the ACM, 52(10):56--67, 2009.

Digital Library

[2]

A. Blackwell and T. Green. A Cognitive Dimensions questionnaire optimised for users. In Annual Meeting of the Psychology of Programming Interest Group, volume 12, pages 137--152, 2000.

[3]

T. Buxton, A. Tarrell, and A. Fruhling. Heuristic Evaluation of Mission-Critical Software Using a Large Team. International Conference on Human-Computer Interaction, pages 673--682, 2009.

Digital Library

[4]

S. Clarke. Evaluating a new programming language. In Workshop of the Psychology of Programming Interest Group, pages 275--289, 2001.

[5]

J. Dagit, J. Lawrance, C. Neumann, M. Burnett, R. Metoyer, and S. Adams. Using cognitive dimensions: advice from the trenches. Journal of Visual Languages & Computing, 17(4):302--327, 2006.

[6]

H. Desurvire, M. Caplan, and J. Toth. Using heuristics to evaluate the playability of games. In Conference on Human factors In computing systems (CHI), 2004. Extended Abstract.

Digital Library

[7]

C. Flanagan and S. Qadeer. A type and effect system for atomicity. In Conference on Programming Language Design and Implementation (PLDI), 2003.

Digital Library

[8]

W. Gray. Who Ya Gonna Call? You're on Your Own. IEEE Software, 14(4):26, 1997.

Digital Library

[9]

T. Green and M. Petre. Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework. Journal of Visual Languages and Computing, 7(2):131--174, 1996.

[10]

R. Jeffries, J. Miller, C. Wharton, and K. Uyeda. User interface evaluation in the real world: a comparison of four techniques. In Conference on Human factors In computing systems (CHI), pages 119--124. ACM, 1991.

Digital Library

[11]

S. Jones, A. Blackwell, and M. Burnett. A user-centred approach to functions in Excel. In International Conference on Functional Programming (ICFP). ACM, 2003.

Digital Library

[12]

D. Kane. Finding a place for discount usability engineering in agile development: throwing down the gauntlet. In Agile Development Conference (ADC), 2003.

Digital Library

[13]

S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. SIGPLAN Notices, 43(3):329--339, 2008.

Digital Library

[14]

J. Mankoff, A. Dey, G. Hsieh, J. Kientz, S. Lederer, and M. Ames. Heuristic evaluation of ambient displays. In Conference on Human factors In computing systems (CHI), 2003.

Digital Library

[15]

J. Nielsen. Usability Inspection Methods. Wiley, 1994.

Digital Library

[16]

J. Nielsen. Applying discount usability engineering. IEEE Software, 12(1):98--100, 1995.

Digital Library

[17]

J. Nielsen and R. Molich. Heuristic evaluation of user interfaces. In Conference on Human factors In computing systems (CHI). ACM, 1990.

Digital Library

[18]

J. Pane and B. Myers. Usability issues in the design of novice programming systems. Human-Computer Interaction Institute Technical Report Carnegie Mellon University-HCII-96-101, 1996.

[19]

J. Pane, B. Myers, and L. Miller. Using HCI techniques to design a more usable programming system. In Symposium on Human Centric Computing Languages and Environments, 2002.

Digital Library

[20]

D. Pinelle, N. Wong, and T. Stach. Heuristic evaluation for games: usability principles for video game design. In Conference on Human factors In computing systems (CHI), pages 1453--1462. ACM, 2008.

Digital Library

[21]

G. Sim, J. Read, and G. Cockton. Evidence based design of heuristics for computer assisted assessment. Human-Computer Interaction (INTERACT), 2009.

Digital Library

[22]

J. Yi and C. Flanagan. Effects for cooperable and serializable threads. In Workshop on Types in Language Design and Implementation (TLDI), 2010.

Digital Library

[23]

J. Yi, C. Sadowski, and C. Flanagan. Cooperative reasoning for preemptive execution. In Symposium on Principles and Practice of Parallel Programming (PPoPP), 2011.

Digital Library

Cited By

Ashmore RHowe AChilton RFaily S(2022)Programming Language Evaluation Criteria for Safety-Critical Software in the Air Domain2022 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)10.1109/ISSREW55968.2022.00072(230-237)Online publication date: Oct-2022
https://doi.org/10.1109/ISSREW55968.2022.00072
Damasio Oliveira JCampos MStangherlin Machado Paixão-Cortes V(2020)Usable and Accessible Robot Programming System for People Who Are Visually ImpairedUniversal Access in Human-Computer Interaction. Design Approaches and Supporting Technologies10.1007/978-3-030-49282-3_32(445-464)Online publication date: 10-Jul-2020
https://doi.org/10.1007/978-3-030-49282-3_32
Morales JRusu CBotella FQuinones D(2019)Programmer eXperience: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2019.29201247(71079-71094)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2920124
Show More Cited By

Index Terms

Heuristic evaluation of programming language features: two parallel programming case studies

Recommendations

Evaluation and usability of programming languages and tools: (PLATEAU)
Onward! 2011: Proceedings of the 10th SIGPLAN symposium on New ideas, new paradigms, and reflections on programming and software

Programming languages exist to enable programmers to develop software effectively. But how efficiently programmers can write software depends on the usability of the languages and tools that they develop with. The aim of this workshop is to discuss ...
Read More
Evaluation and usability of programming languages and tools (PLATEAU)
SPLASH '12: Proceedings of the 3rd annual conference on Systems, programming, and applications: software for humanity

Programming languages exist to enable programmers to develop software effectively. But how efficiently programmers can write software depends on the usability of the languages and tools that they develop with. The aim of this workshop is to discuss ...
Read More
Evaluation and usability of programming languages and tools (plateau)
OOPSLA '09: Proceedings of the 24th ACM SIGPLAN conference companion on Object oriented programming systems languages and applications

Programming languages exist to enable programmers to develop software effectively. But how efficiently programmers can write software depends on the usability of the languages and tools that they develop with. The aim of this workshop is to discuss ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PLATEAU '11: Proceedings of the 3rd ACM SIGPLAN workshop on Evaluation and usability of programming languages and tools

October 2011

90 pages

ISBN:9781450310246

DOI:10.1145/2089155

Program Chairs:
Craig Anslow
Victoria University of Wellington, New Zealand
,
Shane Markstrum
Google, USA
,
Emerson Murphy-Hill
North Carolina State University

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SPLASH '11

Sponsor:

SIGPLAN

SPLASH '11: Conference on Systems, Programming, and Applications: Software for Humanity

October 24, 2011

Oregon, Portland, USA

Acceptance Rates

Overall Acceptance Rate 5 of 8 submissions, 63%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
378
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Ashmore RHowe AChilton RFaily S(2022)Programming Language Evaluation Criteria for Safety-Critical Software in the Air Domain2022 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW)10.1109/ISSREW55968.2022.00072(230-237)Online publication date: Oct-2022
https://doi.org/10.1109/ISSREW55968.2022.00072
Damasio Oliveira JCampos MStangherlin Machado Paixão-Cortes V(2020)Usable and Accessible Robot Programming System for People Who Are Visually ImpairedUniversal Access in Human-Computer Interaction. Design Approaches and Supporting Technologies10.1007/978-3-030-49282-3_32(445-464)Online publication date: 10-Jul-2020
https://doi.org/10.1007/978-3-030-49282-3_32
Morales JRusu CBotella FQuinones D(2019)Programmer eXperience: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2019.29201247(71079-71094)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2920124
Herala AVanhala ENikula UKinnunen PSheard J(2015)Object-oriented programming course revisitedProceedings of the 15th Koli Calling Conference on Computing Education Research10.1145/2828959.2828974(23-32)Online publication date: 19-Nov-2015
https://dl.acm.org/doi/10.1145/2828959.2828974
Sadowski CYi JKim SLanza MDi Penta MXie T(2012)The evolution of data racesProceedings of the 9th IEEE Working Conference on Mining Software Repositories10.5555/2664446.2664473(171-174)Online publication date: 2-Jun-2012
https://dl.acm.org/doi/10.5555/2664446.2664473
Nishino HKonstan JChi EHöök K(2012)How can a DSL for expert end-users be designed for better usability?CHI '12 Extended Abstracts on Human Factors in Computing Systems10.1145/2212776.2223855(2673-2678)Online publication date: 5-May-2012
https://dl.acm.org/doi/10.1145/2212776.2223855
Sadowski CJaeheon Yi Sunghun Kim (2012)The evolution of data races2012 9th IEEE Working Conference on Mining Software Repositories (MSR)10.1109/MSR.2012.6224277(171-174)Online publication date: Jun-2012
https://doi.org/10.1109/MSR.2012.6224277

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents