article

Free access

On compiler error messages: what they say and what they mean

Author:

V. Javier TraverAuthors Info & Claims

Advances in Human-Computer Interaction, Volume 2010

Article No.: 3, Pages 1 - 26

https://doi.org/10.1155/2010/602570

Published: 01 January 2010 Publication History

PDF eReader Publisher Site

Abstract

Programmers often encounter cryptic compiler error messages that are difficult to understand and thus difficult to resolve. Unfortunately, most related disciplines, including compiler technology, have not paidmuch attention to this important aspect that affects programmers significantly, apparently because it is felt that programmers should adapt to compilers. In this article, however, this problem is studied from the perspective of the discipline of human-computer interaction to gain insight into why compiler errors messages make the work of programmers more difficult, and how this situation can be alleviated. Additionally, because poorly designed error messages affect novice programmers more adversely, the problems faced by computer science students while learning to program are analyzed, and the obstacles originated by compilers are identified. Examples of actual compiler error messages are provided and carefully commented. Finally, some possible measures that can be taken are outlined, and some principles for compiler error message design are included.

References

[1]

B. Beizer, "Software is different," Annals of Software Engineering, vol. 10, no. 1-4, pp. 293-310, 2000.

Digital Library

[2]

R. Pressman, Software Engineering: A Practitioner's Approach, McGraw-Hill, New York, NY, USA, 2000.

Digital Library

[3]

I. Sommerville, Software Engineering, Addison-Wesley, Reading, Mass, USA, 2001.

Digital Library

[4]

J. Rumbaugh, OMT Insights: Perspectives on Modelling from the Journal of Object-Oriented Programming, SIGS Books, New York, NY, USA, 1996.

[5]

J. Rumbaugh, I. Jacobson, and G. Booch, The UnifiedModeling Language Reference Manual, Addison-Wesley, Reading, Mass, USA, 1999.

[6]

W. S. Humphrey, Introduction to the Personal Software Process, Addison-Wesley, New York, NY, USA, 1997.

Digital Library

[7]

K. Beck, Extreme Programming Explained: Embrace Change, Addison-Wesley, Reading, Mass, USA, 2000.

Digital Library

[8]

D. Grune, H. E. Bal, C. J. H. Jacobs, and K. G. Langendoen, Modern Compiler Design, John Wiley & Sons, New York, NY, USA, 2000.

Digital Library

[9]

K. D. Cooper and L. Torczon, Engineering a Compiler, Morgan Kaufmann, San Francisco, Calif, USA, 2004.

[10]

S. S. Muchnick, Advanced Compiler Design and Implementation, Morgan Kaufmann, San Francisco, Calif, USA, 1997.

Digital Library

[11]

T. Schorsch, "CAP: an automated self-assessment tool to check pascal programs for syntax, logic and style errors," in Proceedings of the 26th SIGCSE Technical Symposium on Computer Science Education, pp. 168-172, ACM, 1995.

Digital Library

[12]

P. G. Moulton and M. E. Muller, "DITRAN--a compiler emphasizing diagnostics," Communications of the ACM, vol. 10, no. 1, pp. 45-52, 1967.

Digital Library

[13]

E. Kantorowitz and H. Laor, "Automatic generation of useful syntax error messages," Software: Practice and Experience, vol. 16, no. 7, pp. 627-640, 1986.

Digital Library

[14]

A. Alexandrescu, "Better template error messages," C/C++ Users Journal. March 1999. http://erdani.com/publications/ better template error messages.html.

[15]

T. K. Landauer, The Trouble with Computers: Usefulness, Usability, and Productivity, Person Educac, Person Educación, 2001.

[16]

V. J. Traver, "Sobre los mensajes de error de los compiladores," in Proceedings of the Actas del VII Congreso Internacional de Interacción Persona-Ordenador (Interacción '06), M. A. Redondo Duque, C. Bravo Santos, and M. Ortega Cantero, Eds., pp. 345-348, Puertollano, Spain, November 2006.

[17]

R. Brooks, "Towards a theory of the cognitive processes in computer programming," International Journal of Human Computer Studies, vol. 51, no. 2, pp. 197-211, 1999.

Digital Library

[18]

A. Ebrahimi, "Novice programmer errors: language constructs and plan composition," International Journal of Human-Computer Studies, vol. 41, no. 4, pp. 457-480, 1994.

Digital Library

[19]

T. Teitelbaum and T. Reps, "The Cornell program synthesizer: a syntax-directed programming environment," Communications of the ACM, vol. 24, no. 9, pp. 563-573, 1981.

Digital Library

[20]

M. Kölling and J. Rosenberg, "Blue--a language for teaching object-oriented programming," in Proceedings of the 27th SIGCSE Technical Symposium on Computer Science Education, pp. 190-194, ACM, Philadelphia, Pa, USA, March 1996.

Digital Library

[21]

M. Kölling, B. Quig, A. Patterson, and J. Rosenberg, "The BlueJ system and its pedagogy," Journal of Computer Science Education, vol. 13, no. 4, 2003.

[22]

BlueJ--the interactive Java environment. http://www.bluej .org/.

[23]

S. Cooper, W. Dann, and R. Pausch, "Teaching objects-first in introductory computer science," in Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education, pp. 191-195, Reno, Nev, USA, 2003.

Digital Library

[24]

"Alice: an educational software that teaches students computer programming in a 3D environment," http://www.alice.org.

[25]

A. Savidis, "Rapidly implementing languages to compile as C++ without crafting a compiler," Software--Practice and Experience, vol. 37, no. 15, pp. 1577-1620, 2007.

Digital Library

[26]

N. J. Coull, SNOOPIE: development of a learning support tool for novice programmers within a conceptual framework, Ph.D. thesis, School of Computer Science, University of St Andrews, St Andrews, Scotland, UK, 2008, http:// research-repository.st-andrews.ac.uk/handle/10023/522.

[27]

K. E. Gray and M. Flatt, "ProfessorJ: a gradual introduction to Java through language levels," in Proceedings of the 18th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, pp. 170-177, ACM, New York, NY, USA, 2003.

Digital Library

[28]

R. L. Shackelford and A. N. Badre, "Why can't smart students solve simple programming problems?" International Journal of Man-Machine Studies, vol. 38, no. 6, pp. 985-997, 1993.

Digital Library

[29]

R. Jeffries and J. R. Anderson, "Novice lisp errors: undetecte losses of information from working memory," Human-Computer Interaction, vol. 1, no. 2, pp. 107-131, 1985.

Digital Library

[30]

J. Bonar and E. Soloway, "Preprogramming knowledge: a major source of misconception in novice programmers," Human-Computer Interaction, vol. 1, no. 2, pp. 133-161, 1985.

Digital Library

[31]

J. J. Cañas, M. T. Bajo, and P. Gonzalvo, "Mental models and computer programming," The International Journal of Human-Computer Studies, vol. 40, no. 5, pp. 795-811, 1994.

Digital Library

[32]

M. C. Jadud, "Methods and tools for exploring novice compilation behaviour," in Proceedings of the 2nd International Computing Education Research Workshop (ICER '06), vol. 2006, pp. 73-84, ACM, New York, NY, USA, 2006.

Digital Library

[33]

C. Murphy, G. Kaiser, K. Loveland, and S. Hasan, "Retina: helping students and instructors based on observed programming activities," in Proceedings of the 40th ACM Technical Symposium on Computer Science Education (SIGCSE '09), pp. 178-182, Chattanooga, Tenn, USA, 2009.

Digital Library

[34]

M.-H. Nienaltowski, M. Pedroni, and B. Meyer, "Compiler error messages: what can help novices?" SIGCSE Bulletin, vol. 40, no. 1, pp. 168-172, 2008.

Digital Library

[35]

W. M. McKeeman, "Programming language design," in Compiler Construction: An Advanced Course, F. L. Bauer and J. Eickel, Eds., vol. 21 of Lecture Notes in Computer Science, pp. 515-519, Springer, Berlin, Germany, 1976.

Digital Library

[36]

J. F. Pane, C. Ratanamahatana, and B. A. Myers, "Studying the language and structure in non-programmers' solutions to programming problems," The International Journal ofHuman-Computer Studies, vol. 54, pp. 237-264, 2001.

Digital Library

[37]

J. F. Pane, B. A. Myers, and L. B. Miller, "Using HCI techniques to design a more usable programming system," in Proceedings of the IEEE Symposia on Human Centric Computing Languages and Environments (HCC '02), p. 198, IEEE Computer Society, Washington, DC, USA, 2002.

[38]

C. R. Litecky and G. B. Davis, "A study of errors, errorproneness, and error diagnosis in Cobol," Communications of the ACM, vol. 19, no. 1, pp. 33-37, 1976.

Digital Library

[39]

J. J. Horning, "What the compiler should tell the user," in Compiler Construction: an Advanced Course, F. L. Bauer and J. Eickel, Eds., vol. 21 of Lecture Notes in Computer Science, pp. 525-548, Springer, Berlin, Germany, 1976.

Digital Library

[40]

B. Shneiderman, "Designing computer system messages," Communications of the ACM, vol. 25, no. 9, pp. 610-611, 1982.

Digital Library

[41]

P. J. Brown, "Error messages: the neglected area of the man/machine interface," Communications of the ACM, vol. 26, no. 4, pp. 246-249, 1983.

Digital Library

[42]

T. Flowers, C. A. Carver, and J. Jackson, "Empowering students and building confidence in novice programmers through gauntlet," in Proceedings of the 34th ASEE/IEEE Frontiers in Education Conference (FIE '04), vol. 1, pp. T3H-10-T3H-13, October 2004.

[43]

J. Jackson, M. Cobb, and C. Carver, "Identifying top Java errors for novice programmers," in Proceedings of the 35th Annual Conference Frontiers in Education (FIE '05), pp. T4C-24-T4C- 27, October 2005.

[44]

B. Hartmann, D. MacDougall, J. Brandt, and S. R. Klemmer, "What would other programmers do? Suggesting solutions to error messages," in Proceedings of ACM Conference on Human Factors in Computing Systems (CHI '10), Atlanta, Ga, USA, April 2010.

Digital Library

[45]

C. Burrell and M. Melchert, "Augmenting compiler error reporting in the Karel++ Microworld," in Proceedings of the 20th Annual Conference of the National Advisory Committee on Computing Qualifications (NACCQ '07), pp. 41-46, New Zealand, 2007.

[46]

D. Grune and C. J. H. Jacobs, Parsing Techniques: a Practical Guide, Springer, Berlin, Germany, 2nd edition, 2008.

Digital Library

[47]

J. Scholtz and S.Wiedenbeck, "Using unfamiliar programming languages: the effects on expertise," Interacting with Computers, vol. 5, no. 1, pp. 13-30, 1993.

[48]

B. S. Lerner, M. Flower, D. Grossman, and C. Chambers, "Searching for type-errormessages," in Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI '07), pp. 425-434, San Diego, Calif, USA, 2007.

Digital Library

[49]

L. Zolman, STLFilt: an STL error message decryptor for C++. 2005. http://www.bdsoft.com/tools/stlfilt.html.

[50]

N. E. Boustani and J. Hage, "Improving type error messages for generic Java," in Proceedings of the ACM SIGPLAN Symposium on Partial Evaluation and Program Manipulation (PEPM '09), pp. 131-140, Savannah, Ga, USA, 2009.

Digital Library

[51]

C. L. Jeffery, "Generating LR syntax error messages from examples," ACM Transactions on Programming Languages and Systems, vol. 25, no. 5, pp. 631-640, 2003.

Digital Library

[52]

A. Dix, J. Finlay, A. Gregory, and R. Beale, Human-Computer Interaction, Prentice Hall, Upper Saddle River, NJ, USA, 2nd edition, 1998.

Digital Library

[53]

D. A.Norman, The Psychology of Everyday Things, BasicBooks, New York, NY, USA, 1988.

[54]

M. H. Ng Cheong Vee, B. Meyer, and K. L. Mannock, "Empirical study of novice errors and error paths," Tech. Rep., ETH Zurick, Zurick, Switzerland, 2005.

[55]

J. Preece, Y. Rogers, H. Sharp, D. Benyon, S. Holland, and T. Carey, Human-Computer Interaction, Addison Wesley, New York, NY, USA, 1994.

Digital Library

[56]

G. A. Miller, "The magical number seven, plus or minus two: some limits on our capacity for processing information," Psychological Review, vol. 63, no. 2, pp. 81-97, 1956.

[57]

L. Barfield, The User Interface: Concepts and Design, Addison-Wesley, Reading, Mass, USA, 1994.

[58]

A. V. Aho, R. Sethi, and J. D. Ullman, Compilers Principles, Techniques, and Tools, Addison-Wesley, Reading, Mass, USA, 1986.

Digital Library

[59]

L. Murphy, G. Lewandowski, R. McCauley, B. Simon, L. Thomas, and C. Zander, "Debugging: the good, the bad, and the quirky--a qualitative analysis of novices' strategies," in Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '08), pp. 163-167, ACM, 2008.

Digital Library

[60]

B. Reeves and C. Nass, The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places, Cambridge University Press, New York, NY, USA, 1996.

Digital Library

[61]

M.Hristova, A. Misra, M. Rutter, and R. Mercuri, "Identifying and correcting Java programming errors for introductory computer science students," in Proceedings of the 34th SIGCSE Technical Symposium on Computer Science Education, pp. 153- 156, Reno, Nev, USA, February 2003.

Digital Library

[62]

B. Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction, Addison-Wesley, Reading, Mass, USA, 1992.

Digital Library

[63]

G. M. Weinberg, The Psychology of Computer Programming, New York, NY, USA, Dorset House Publishing, 1998, Silver Anniversary Edition.

Digital Library

[64]

J. Nielsen, Ten usability heuristics. http://www.useit.com/ papers/heuristic/heuristic list.html.

[65]

J. Yang, "Explaining type errors by finding the source of a type conict," in Proceedings of the Scottish Functional Programming Workshop, P.W. Trinder, G. Michaelson, and H.-W. Loidl, Eds., vol. 1 of Trends in Functional Programming, pp. 59-67, 2000.

[66]

R.W. Picard, Affective Computing, The MIT Press, Cambridge, Mass, USA, 1997.

Digital Library

[67]

R. Navarro-Prieto and J. J. Cañas, "Are visual programming languages better? The role of imagery in program comprehension," The International Journal of Human-Computer Studies, vol. 54, no. 6, pp. 799-829, 1999.

Digital Library

[68]

D. A. Carrington, B. McEniery, and D. B. Johnston, "PSP in the large class," in Proceedings of the 14th Conference on Software Engineering Education and Training (CSEET '01), IEEE Computer Society, pp. 81-88, Charlotte, NC, USA, February 2001.

[69]

GCC, the GNU Compiler Collection. http://gcc.gnu.org/.

[70]

J. Nielsen, "Heuristic evaluaiton," in Usability Inspection Methods, J. Nielsen and R. L. Mack, Eds., John Wiley & Sons, New York, NY, USA, 1994.

[71]

S. Fitzgerald, G. Lewandowski, R. McCauley et al., "Debugging: finding, fixing and flailing, a multi-institutional study of novice debuggers," Computer Science Education, vol. 18, no. 2, pp. 93-116, 2008.

[72]

Bloodshed Software. MinGW C++ compiler. http://www.bloodshed. net/devcpp.html.

[73]

Microsoft Developer Network. Microsoft Visual C++. http:// msdn.microsoft.com/en-us/visualc.

[74]

Borland. C++ Builder. http://www.borland.com/.

Cited By

Ramos DLynce IManquinho VMartins RLe Goues C(2024)BatFix: Repairing language model-based transpilationACM Transactions on Software Engineering and Methodology10.1145/365866833:6(1-29)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3658668
Taylor AVassar ARenzella JPearce HStephenson BStone JBattestilli LRebelsky SShoop L(2024)dcc --help: Transforming the Role of the Compiler by Generating Context-Aware Error Explanations with Large Language ModelsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630822(1314-1320)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630822
Algaraibeh SJeffery CSoule T(2024)On Students’ Perception of Compiler Syntax Error Messages: A Human Factors ApproachHCI International 2024 – Late Breaking Papers10.1007/978-3-031-76815-6_1(3-16)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-76815-6_1
Show More Cited By

Index Terms

On compiler error messages: what they say and what they mean
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Software and its engineering
  1. Software notations and tools
    1. Compilers

Recommendations

An Effective Approach to Enhancing Compiler Error Messages
SIGCSE '16: Proceedings of the 47th ACM Technical Symposium on Computing Science Education

One of the many challenges novice programmers face from the time they write their first program is inadequate compiler error messages. These messages report details on errors the programmer has made and are the only feedback the programmer gets from the ...
Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research
ITiCSE-WGR '19: Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education

Diagnostic messages generated by compilers and interpreters such as syntax error messages have been researched for over half of a century. Unfortunately, these messages which include error, warning, and run-time messages, present substantial difficulty ...
Compiler error messages: what can help novices?
SIGCSE '08: Proceedings of the 39th SIGCSE technical symposium on Computer science education

Novices find it difficult to understand and use compiler error messages. It is useful to refine this observation and study the effect of different message styles on how well and quickly students identify errors in programs. For example, does an ...

Comments

Information & Contributors

Information

Published In

cover image Advances in Human-Computer Interaction

Advances in Human-Computer Interaction Volume 2010, Issue

January 2010

82 pages

ISSN:1687-5893

EISSN:1687-5907

Issue’s Table of Contents

Publisher

Hindawi Limited

London, United Kingdom

Publication History

Accepted: 01 June 2010

Revised: 16 March 2010

Published: 01 January 2010

Received: 20 July 2009

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
341
Total Downloads

Downloads (Last 12 months)75
Downloads (Last 6 weeks)5

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ramos DLynce IManquinho VMartins RLe Goues C(2024)BatFix: Repairing language model-based transpilationACM Transactions on Software Engineering and Methodology10.1145/365866833:6(1-29)Online publication date: 27-Jun-2024
https://dl.acm.org/doi/10.1145/3658668
Taylor AVassar ARenzella JPearce HStephenson BStone JBattestilli LRebelsky SShoop L(2024)dcc --help: Transforming the Role of the Compiler by Generating Context-Aware Error Explanations with Large Language ModelsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630822(1314-1320)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630822
Algaraibeh SJeffery CSoule T(2024)On Students’ Perception of Compiler Syntax Error Messages: A Human Factors ApproachHCI International 2024 – Late Breaking Papers10.1007/978-3-031-76815-6_1(3-16)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-76815-6_1
Dzhatdoyev SDes Rosiers JMorazán M(2024)Error Messages for Students Taught Using a Systematic Program Design CurriculumTrends in Functional Programming10.1007/978-3-031-74558-4_9(195-219)Online publication date: 8-Jan-2024
https://dl.acm.org/doi/10.1007/978-3-031-74558-4_9
Taipalus TGrahn H(2023)Framework for SQL Error Message Design: A Data-Driven ApproachACM Transactions on Software Engineering and Methodology10.1145/360718033:1(1-50)Online publication date: 23-Nov-2023
https://dl.acm.org/doi/10.1145/3607180
Tao YChen ZLiu YXuan JXu ZQin SSpinellis DGousios GChechik MDi Penta M(2021)Demystifying “bad” error messages in data science librariesProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3468560(818-829)Online publication date: 20-Aug-2021
https://dl.acm.org/doi/10.1145/3468264.3468560
Becker BDenny PPrather JPettit RNix RMooney C(2021)Towards Assessing the Readability of Programming Error MessagesProceedings of the 23rd Australasian Computing Education Conference10.1145/3441636.3442320(181-188)Online publication date: 2-Feb-2021
https://dl.acm.org/doi/10.1145/3441636.3442320
Denny PPrather JBecker BMooney CHomer JAlbrecht ZPowell GKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)On Designing Programming Error Messages for Novices: Readability and its Constituent FactorsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445696(1-15)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445696
Zhu ZKo HZhang YMartins PSaraiva JHu Z(2020)Unifying Parsing and Reflective Printing for Fully Disambiguated GrammarsNew Generation Computing10.1007/s00354-019-00082-y38:3(423-476)Online publication date: 1-Jul-2020
https://dl.acm.org/doi/10.1007/s00354-019-00082-y
Morales JRusu CBotella FQuiñones D(2020)Programmer eXperience: A Set of Heuristics for Programming EnvironmentsSocial Computing and Social Media. Participation, User Experience, Consumer Experience, and Applications of Social Computing10.1007/978-3-030-49576-3_15(205-216)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1007/978-3-030-49576-3_15
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Issue’s Table of Contents