Decoding Debugging Instruction: A Systematic Literature Review of Debugging Interventions
Authors: 
Stephanie Yang, Miles Baird, Eleanor O’Rourke, Karen Brennan, 
Bertrand SchneiderAuthors Info & Claims
Stephanie Yang, Miles Baird, Eleanor O’Rourke, Karen Brennan, Bertrand SchneiderAuthors Info & ClaimsArticle No.: 45, Pages 1 - 44
Abstract
Students learning computer science frequently struggle with debugging errors in their code. These struggles can have significant downstream effects—negatively influencing how students assess their programming ability and contributing to their decision to drop out of CS courses. However, debugging instruction is often an overlooked topic, and instructors report feeling unaware of effective approaches to teach debugging. Within the literature, research on the topic is sporadic, and though there are rigorous and insightful studies to be found, there is a need to synthesize instructional approaches for debugging. In this article, we review research from 2010 to 2022 on debugging interventions. We summarize the common pedagogical approaches for learning and categorize how these target specific cognitive and non-cognitive debugging skills, such as self-efficacy and emotion regulation. We also present a summary of assessment methods and their outcomes in order to discuss intervention efficacy and directions for further research. Our sample displays a diverse variety of debugging interventions and pedagogical approaches, ranging from games to unplugged activities. An evaluation of article results also presents encouraging findings, revealing several interventions that improved debugging accuracy and learning. Still, we notice gaps in interventions addressing non-cognitive debugging skills and observe limited success in guiding students toward adopting systematic debugging strategies. The review concludes with a discussion of future directions and implications for researchers and instructors in the field.
References
[1]
Fatima Abu Deeb and Timothy Hickey. 2021. Reflective debugging in Spinoza V3.0. In Proceedings of the Australasian Computing Education Conference (ACE ’21). ACM, New York, NY, 125–130. DOI:
[2]
Marzieh Ahmadzadeh, Dave Elliman, and Colin Higgins. 2005. An analysis of patterns of debugging among novice computer science students. SIGCSE Bull. 37, 3 (Jun. 2005), 84–88. DOI:
[3]
Umair Z. Ahmed, Nisheeth Srivastava, Renuka Sindhgatta, and Amey Karkare. 2020. Characterizing the pedagogical benefits of adaptive feedback for compilation errors by novice programmers. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET ’20). ACM, New York, NY, 139–150. DOI:
[4]
Junghyun Ahn, Woonhee Sung, and John B. Black. 2022. Unplugged debugging activities for developing young learners’ debugging skills, 421–437. DOI:
[5]
Vincent Aleven, Bruce McLaren, Ido Roll, and Kenneth Koedinger. 2006. Toward meta-cognitive tutoring: A model of help seeking with a cognitive tutor. Int. J. Artif. Intell. Educ. 16, 2 (Jan. 2006), 101–128.
[6]
Amal Alhadabi and Aryn C. Karpinski. 2020. Grit, self-efficacy, achievement orientation goals, and academic performance in University students. Int. J. Adolesc. Youth 25, 1 (Dec. 2020), 519–535. DOI:
[7]
Basma S. Alqadi and Jonathan I. Maletic. 2017. An empirical study of debugging patterns among novices programmers. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE ’17). ACM, New York, NY, 15–20. DOI:
[8]
Malek Alrashidi, Michael Gardner, and Vic Callaghan. 2017. Evaluating the use of pedagogical virtual machine with augmented reality to support learning embedded computing activity. In Proceedings of the 9th International Conference on Computer and Automation Engineering (ICCAE ’17). ACM, New York, NY, 44–50. DOI:
[9]
Apostolos Ampatzoglou, Stamatia Bibi, Paris Avgeriou, Marijn Verbeek, and Alexander Chatzigeorgiou. 2019. Identifying, categorizing and mitigating threats to validity in software engineering secondary studies. Inf. Softw. Technol. 106 (Feb. 2019), 201–230. DOI:
[10]
Pasquale Ardimento, Mario Luca Bernardi, Marta Cimitile, and Giuseppe De Ruvo. 2019. Reusing bugged source code to support novice programmers in debugging tasks. ACM Trans. Comput. Educ. 20, 1 (Nov. 2019), 1–24. DOI:
[11]
Ryan S. J. d. Baker, Sidney K. D’Mello, Ma Mercedes T. Rodrigo, and Arthur C. Graesser. 2010. Better to be frustrated than bored: The incidence, persistence, and impact of learners’ cognitive–affective states during interactions with three different computer-based learning environments. Int. J. Hum. Comput. Stud. 68, 4 (Apr. 2010), 223–241. DOI:
[12]
Albert Bandura. 1977. Self-efficacy: Toward a unifying theory of behavioral change. Psychol. Rev. 84, 2 (Mar. 1977), 191–215. DOI:
[13]
Albert Bandura. 1982. Self-efficacy mechanism in human agency. Am. Psychol. 37, 2 (1982), 122–147. DOI:
[14]
Albert Bandura. 1986. Social Foundations of Thought and Action: A Social Cognitive Theory. Prentice-Hall, Englewood Cliffs, NJ.
[15]
Lawrence W. Barsalou. 2008. Grounded cognition. Annu. Rev. Psychol. 59, 1 (Jan. 2008), 617–645. DOI:
[16]
Joseph E. Beck and Yue Gong. 2013. Wheel-spinning: Students who fail to master a skill. In Artificial Intelligence in Education. Springer, Berlin, 431–440. DOI:
[17]
B. A. Becker, P. Denny, R. Pettit, D. Bouchard, Dennis J. Bouvier, Brian Harrington, Amir Kamil, Amey Karkare, Chris McDonald, Peter-Michael Osera, Janice L. Pearce, and James Prather. 2019. Compiler error messages considered unhelpful: The landscape of text-based programming error message research. In Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education, 177–210.
[18]
Brett A. Becker, Kyle Goslin, and Graham Glanville. 2018. The effects of enhanced compiler error messages on a syntax error debugging test. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE ’18). ACM, New York, NY, 640–645. DOI:
[19]
Roman Bednarik, Carsten Schulte, Lea Budde, Birte Heinemann, and Hana Vrzakova. 2018. Eye-movement modeling examples in source code comprehension: A classroom study. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (Koli Calling ’18, Article 2). ACM, New York, NY, 1–8. DOI:
[20]
Susan Bergin, Ronan Reilly, and Desmond Traynor. 2005. Examining the role of self-regulated learning on introductory programming performance. In Proceedings of the 1st International Workshop on Computing Education Research (ICER ’05). ACM, New York, NY, 81–86. DOI:
[21]
Katerine Bielaczyc, Peter L. Pirolli, and Ann L. Brown. 1995. Training in self-explanation and self-regulation strategies: Investigating the effects of knowledge acquisition activities on problem solving. Cogn. Instr. 13, 2 (1995), 221–252.
[22]
Laura Bofferding, Sezai Kocabas, Mahtob Aqazade, Ana-Maria Haiduc, and Lizhen Chen. 2022. The effect of play and worked examples on first and third graders’ creating and debugging of programming algorithms. In Proceedings of the Computational Thinking in PreK-5: Empirical Evidence for Integration and Future Directions. ACM, New York, NY, 19–29. DOI:
[23]
Nigel Bosch and Sidney D’Mello. 2017. The affective experience of novice computer programmers. Int. J. Artif. Intell. Educ. 27, 1 (Mar. 2017), 181–206. DOI:
[24]
Axel Böttcher, Veronika Thurner, Kathrin Schlierkamp, and Daniela Zehetmeier. 2016. Debugging students’ debugging process. In Proceedings of the 2016 IEEE Frontiers in Education Conference (FIE ’16), 1–7. DOI:
[25]
Karen Brennan and Mitchel Resnick. 2012. New frameworks for studying and assessing the development of computational thinking. In Proceedings of the 2012 Annual Meeting of the American Educational Research Association, Vol. 1, 25.
[26]
Randal E. Bryant and David R. O’Hallaron. 2001. Introducing computer systems from a programmer's perspective. In Proceedings of the 32nd SIGCSE Technical Symposium on Computer Science Education (SIGCSE ’01). ACM, New York, NY, 90–94. DOI:
[27]
Sallyann Bryant, Pablo Romero, and Benedict du Boulay. 2008. Pair programming and the mysterious role of the navigator. Int. J. Hum. Comput. Stud. 66, 7 (Jul. 2008), 519–529. DOI:
[28]
Elizabeth Carter. 2015. Its debug: Practical results. J. Comput. Sci. Coll. 30, 3 (Jan. 2015), 9–15.
[29]
Mccoy Sharon Carver and Sally Clarke Risinger. 1987. Improving children's debugging skills. In Empirical Studies of Programmers: Second Workshop. Ablex Publishing Corp., 147–171.
[30]
Xingliang Chen, Antonija Mitrovic, and Moffat Mathews. 2020. Learning from Worked Examples, Erroneous Examples, and Problem Solving: Toward Adaptive Selection of Learning Activities. IEEE Trans. Learn. Technol. 13, 1 (2020), 135–149. DOI:
[31]
Ryan Chmiel and Michael C. Loui. 2004. Debugging: From novice to expert. In Proceedings of the 35th SIGCSE Technical Symposium on Computer Science Education (SIGCSE ’04). ACM, New York, NY, 17–21. DOI:
[32]
Cheng-Yu Chung and I.-Han Hsiao. 2020. Computational thinking in augmented reality: An investigation of collaborative debugging practices. In Proceedings of the 2020 6th International Conference of the Immersive Learning Research Network (iLRN ’20). ieeexplore.ieee.org, 54–61. DOI:
[33]
James Cross, Dean Hendrix, Larry Barowski, and David Umphress. 2014. Dynamic program visualizations: An experience report. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE ’14). ACM, New York, NY, 609–614. DOI:
[34]
Maggie Dahn, David Deliema, and Noel Enyedy. 2020. Art as a point of departure for understanding student experience in learning to code. Teach. Coll. Rec. 122, 8 (Aug. 2020), 1–42. DOI:
[35]
Sean Deitz and Ugo Buy. 2016. From video games to debugging code. In Proceedings of the 5th International Workshop on Games and Software Engineering (GAS ’16). ACM, New York, NY, 37–41. DOI:
[36]
David DeLiema, Maggie Dahn, Virginia J. Flood, and Francis F. Steen. 2019. Debugging as a context for fostering reflection on critical thinking and emotion. In Deeper Learning, Dialogic Learning, and Critical Thinking, 209–228. DOI:
[37]
Paul Denny, James Prather, and Brett A. Becker. 2020. Error message readability and novice debugging performance. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE ’20). ACM, New York, NY, 480–486. DOI:
[38]
Paul Denny, James Prather, Brett A Becker, James Finnie-Ansley, Arto Hellas, Juho Leinonen, Andrew Luxton-Reilly, Brent N Reeves, Eddie Antonio Santos, and Sami Sarsa. 2024. Computing education in the era of generative AI. Commun. ACM 67, 2 (Feb 2024), 56–67. DOI:
[39]
Sidney D’Mello and Art Graesser. 2012. Dynamics of affective states during complex learning. Learn. Instr. 22, 2 (Apr. 2012), 145–157. DOI:
[40]
M. Ducassé and A.-M. Emde. 1988. A review of automated debugging systems: knowledge, strategies and techniques. In Proceedings of the 10th international conference on Software engineering (ICSE ’88). IEEE Computer Society Press, Washington, DC, 162–171.
[41]
Angela L. Duckworth, Christopher Peterson, Michael D. Matthews, and Dennis R. Kelly. 2007. Grit: Perseverance and passion for long-term goals. J. Pers. Soc. Psychol. 92, 6 (Jun. 2007), 1087–1101. DOI:
[42]
Henry Duwe, Diane T. Rover, Phillip H. Jones, Nicholas D. Fila, and Mani Mina. 2022. Defining and supporting a debugging mindset in computer engineering courses. In Proceedings of the 2022 IEEE Frontiers in Education Conference (FIE ’22), 1–9. DOI:
[43]
C. S. Dweck. 1990. Self-theories and goals: Their role in motivation, personality, and development. Nebr. Symp. Motiv. 38 (1990), 199–235.
[44]
Carol S. Dweck. 2006. Mindset: The New Psychology of Success. Random House Publishing Group.
[45]
Carol S. Dweck, Gregory M. Walton, and Geoffrey L. Cohen. 2014. Academic Tenacity: Mindsets and Skills that Promote Long-Term Learning. Bill & Melinda Gates Foundation.
[46]
Bob Edmison and Stephen H. Edwards. 2020. Turn up the heat! using heat maps to visualize suspicious code to help students successfully complete programming problems faster. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET ’20). ACM, New York, NY, 34–44. DOI:
[47]
Kiran L. N. Eranki and Kannan M. Moudgalya. 2016. Program slicing technique: A novel approach to improve programming skills in novice learners. In Proceedings of the 17th Annual Conference on Information Technology Education (SIGITE ’16). ACM, New York, NY, 160–165. DOI:
[48]
K. Anders Ericsson. 2008. Deliberate practice and acquisition of expert performance: A general overview. Acad. Emerg. Med. 15, 11 (Nov. 2008), 988–994. DOI:
[49]
Richard Falk and Samuel S. Kim. 2019. The War System: An Interdisciplinary Approach. Routledge.
[50]
Zhiyu Fan, Xiang Gao, Martin Mirchev, Abhik Roychoudhury, and Shin Hwei Tan. 2023. Automated repair of programs from large language models. In Proceedings of the 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE ’23). IEEE, 1469–1481. DOI:
[51]
Joel Fenwick and Peter Sutton. 2012. Using quicksand to improve debugging practice in post-novice level students. In Proceedings of the Fourteenth Australasian Computing Education Conference 123 (2012), 141–146.
[52]
Deborah A. Fields, Yasmin B. Kafai, Luis Morales-Navarro, and Justice T. Walker. 2021. Debugging by design: A constructionist approach to high school students’ crafting and coding of electronic textiles as failure artefacts. Br. J. Educ. Technol. 52, 3 (May 2021), 1078–1092. DOI:
[53]
Sue Fitzgerald, Gary Lewandowski, Renée McCauley, Laurie Murphy, Beth Simon, Lynda Thomas, and Carol Zander. 2008. Debugging: Finding, fixing and flailing, a multi-institutional study of novice debuggers. Comput. Sci. Educ. 18, 2 (Jun. 2008), 93–116. DOI:
[54]
Rita Garcia, Chieh-Ju Liao, and Ariane Pearce. 2022. Read the debug manual: A debugging manual for CS1 students. In Proceedings of the 2022 IEEE Frontiers in Education Conference (FIE ’22). ieeexplore.ieee.org, 1–7.
[55]
Daniela Girardi, Nicole Novielli, Davide Fucci, and Filippo Lanubile. 2020. Recognizing developers’ emotions while programming. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering (ICSE ’20). ACM, New York, NY, 666–677. DOI:
[56]
Jamie Gorson, Kathryn Cunningham, Marcelo Worsley, and Eleanor O’Rourke. 2022. Using electrodermal activity measurements to understand student emotions while programming. In Proceedings of the 2022 ACM Conference on International Computing Education Research (ICER ’22), Vol. 1, ACM, New York, NY, 105–119. DOI:
[57]
Jamie Gorson and Eleanor O’Rourke. 2020. Why do CS1 students think they’re bad at programming? Investigating self-efficacy and self-assessments at three universities. In Proceedings of the 2020 ACM Conference on International Computing Education Research (ICER ’20). ACM, New York, NY, 170–181. DOI:
[58]
Luisa Greifenstein, Florian Obermueller, Ewald Wasmeier, Ute Heuer, and Gordon Fraser. 2021. Effects of hints on debugging scratch programs: An empirical study with primary school teachers in training. In Proceedings of the 16th Workshop in Primary and Secondary Computing Education (WiPSCE ’21, Article 3). ACM, New York, NY, 1–10. DOI:
[59]
Paul Gross and Kris Powers. 2005. Evaluating assessments of novice programming environments. In Proceedings of the 1st International Workshop on Computing Education Research (ICER ’05). ACM, New York, NY, 99–110. DOI:
[60]
L. Gugerty and G. Olson. 1986. Debugging by skilled and novice programmers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’86). ACM, New York, NY, 171–174. DOI:
[61]
Philip J. Guo. 2013. Online python tutor: Embeddable web-based program visualization for CS education. In Proceedings of the 44th ACM technical symposium on Computer science education (SIGCSE ’13). ACM, New York, NY, 579–584. DOI:
[62]
B. Hailpern and P. Santhanam. 2002. Software debugging, testing, and verification. IBM Syst. J. 41, 1 (2002), 4–12. DOI:
[63]
Brian Hanks, Sue Fitzgerald, Renée McCauley, Laurie Murphy, and Carol Zander. 2011. Pair programming in education: A literature review. Comput. Sci. Educ. 21, 2 (Jun. 2011), 135–173. DOI:
[64]
Björn Hartmann, Daniel MacDougall, Joel Brandt, and Scott R. Klemmer. 2010. What would other programmers do: Suggesting solutions to error messages. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10). ACM, New York, NY, 1019–1028. DOI:
[65]
Sami Heikkinen, Mohammed Saqr, Jonna Malmberg, and Matti Tedre. 2023. Supporting self-regulated learning with learning analytics interventions – A systematic literature review. Educ. Inf. Tech. 28, 3 (March 2023), 3059–3088. DOI:
[66]
R. L. Heilman and G. P. Ashby. 1971. Re-evaluation of debugging in the computer science curriculum. SIGCSE Bull. 3, 4 (Dec. 1971), 15–18. DOI:
[67]
Juha Helminen and Lauri Malmi. 2010. Jype - A program visualization and programming exercise tool for Python. In Proceedings of the 5th International Symposium on Software Visualization (SOFTVIS ’10). ACM, New York, NY, 153–162. DOI:
[68]
Matthew Hertz and Maria Jump. 2013. Trace-based teaching in early programming courses. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education (SIGCSE ’13). ACM, New York, NY, 561–566. DOI:
[69]
Mark A. Holliday and David Luginbuhl. 2004. CS1 assessment using memory diagrams. In Proceedings of the 35th SIGCSE Technical Symposium on Computer Science Education (SIGCSE ’04). ACM, New York, NY, 200–204. DOI:
[70]
Cruz Izu, Carsten Schulte, Ashish Aggarwal, and Renske Weeda. 2019. Fostering program comprehension in novice programmers - Learning activities and learning trajectories. In Innovation and Technology in Computer Science Education (ITiCSE ’19), 27–52. DOI:
[71]
Chaima Jemmali, Magy Seif El-Nasr, and Seth Cooper. 2022. The effects of adaptive procedural levels on engagement and performance in an educational programming game. In Proceedings of the 17th International Conference on the Foundations of Digital Games (FDG ’22), 1–12. DOI:
[72]
Mina C. Johnson-Glenberg, David A. Birchfield, Lisa Tolentino, and Tatyana Koziupa. 2014. Collaborative embodied learning in mixed reality motion-capture environments: Two science studies. J. Educ. Psychol. 106, 1 (Feb. 2014), 86–104. DOI:
[73]
David H. Jonassen and Woei Hung. 2006. Learning to troubleshoot: A new theory-based design architecture. Educ. Psychol. Rev. 18, 1 (Mar. 2006), 77–114. DOI:
[74]
Irvin R. Katz and John R. Anderson. 1987. Debugging: An analysis of bug-location strategies. Hum.–Comput. Interact. 3, 4 (Dec. 1987), 351–399.
[75]
Claudius M. Kessler and John R. Anderson. 1986. A model of novice debugging in LISP. In Papers Presented at the First Workshop on Empirical Studies of Programmers on Empirical Studies of Programmer. Ablex Publishing Corp., 198–212.
[76]
Päivi Kinnunen and Lauri Malmi. 2006. Why students drop out CS1 course? In Proceedings of the 2nd International Workshop on Computing Education Research (ICER ’06). ACM, New York, NY, 97–108. DOI:
[77]
Päivi Kinnunen and Beth Simon. 2012. My program is ok – Am I? Computing Freshmen's experiences of doing programming assignments. Comput. Sci. Educ. 22, 1 (Mar. 2012), 1–28. DOI:
[78]
David Klahr and Sharon Mccoy Carver. 1988. Cognitive objectives in a LOGO debugging curriculum: Instruction, learning, and transfer. Cogn. Psychol. 20, 3 (Jul. 1988), 362–404. DOI:
[79]
Donald E. Knuth. 1989. The errors of tex. Softw. Pract. Exp. 19, 7 (Jul. 1989), 607–685. DOI:
[80]
Amy J. Ko, Thomas D. LaToza, Stephen Hull, Ellen A. Ko, William Kwok, Jane Quichocho, Harshitha Akkaraju, and Rishin Pandit. 2019. Teaching explicit programming strategies to adolescents. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE ’19). ACM, New York, NY, 469–475. DOI:
[81]
Michael Kölling. 2008. Using BlueJ to introduce programming. In Reflections on the Teaching of Programming: Methods and Implementations. Jens Bennedsen, Michael E. Caspersen, and Michael Kölling (Eds.), Springer, Berlin, 98–115. DOI:
[82]
Donna Kotsopoulos, Lisa Floyd, Steven Khan, Immaculate Kizito Namukasa, Sowmya Somanath, Jessica Weber, and Chris Yiu. 2017. A pedagogical framework for computational thinking. Digital Experiences in Mathematics Education 3, 2 (Aug. 2017), 154–171.
[83]
Essi Lahtinen, Kirsti Ala-Mutka, and Hannu-Matti Järvinen. 2005. A study of the difficulties of novice programmers. SIGCSE Bull. 37, 3 (Jun. 2005), 14–18. DOI:
[84]
Thomas D. Latoza, Gina Venolia, and R. Deline. 2006. Maintaining mental models: A study of developer work habits. Int. Conf. Softw. Eng. (May 2006). DOI:
[85]
Michael J. Lee and Amy J. Ko. 2011. Personifying programming tool feedback improves novice programmers’ learning. In Proceedings of the 7th International Workshop on Computing Education Research (ICER ’11). ACM, New York, NY, 109–116. DOI:
[86]
V. C. S. Lee, Y. T. Yu, C. M. Tang, T. L. Wong, and C. K. Poon. 2018. ViDA: A virtual debugging advisor for supporting learning in computer programming courses. J. Comput. Assist. Learn. 34, 3 (Jun. 2018), 243–258. DOI:
[87]
J. Leinonen, A. Hellas, S. Sarsa, B. Reeves, P. Denny, J. Prather, and B. Becker. 2023. Using large language models to enhance programming error messages. Proceedings of the 54th ACM Technical Symposium on Computer Science Education 1 (2023), 563–569.
[88]
Chen Li, Emily Chan, Paul Denny, Andrew Luxton-Reilly, and Ewan Tempero. 2019. Towards a framework for teaching debugging. In Proceedings of the 21st Australasian Computing Education Conference (ACE ’19). ACM, New York, NY, 79–86. DOI:
[89]
Yu-Tzu Lin, Cheng-Chih Wu, Ting-Yun Hou, Yu-Chih Lin, Fang-Ying Yang, and Chia-Hu Chang. 2016. Tracking students’ cognitive processes during program debugging—An eye-movement approach. IEEE Trans. Educ. 59, 3 (Aug. 2016), 175–186. DOI:
[90]
A. Lishinski, A. Yadav, and R. Enbody. 2017. Students’ emotional reactions to programming projects in introduction to programming: Measurement approach and influence on learning outcomes. In Proceedings of the 2017 ACM Conference on International Computing Education Research (2017).
[91]
Zhongxiu Liu, Rui Zhi, Andrew Hicks, and Tiffany Barnes. 2017. Understanding problem solving behavior of 6–8 graders in a debugging game. Comput. Sci. Educ. 27, 1 (Jan. 2017), 1–29. DOI:
[92]
Dastyni Loksa and Amy J. Ko. 2016. The role of self-regulation in programming problem solving process and success. In Proceedings of the 2016 ACM Conference on International Computing Education Research (ICER ’16). ACM, New York, NY, 83–91. DOI:
[93]
Dastyni Loksa, Benjamin Xie, Harrison Kwik, and Amy J. Ko. 2020. Investigating novices’ in situ reflections on their programming process. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (SIGCSE ’20). ACM, New York, NY, 149–155. DOI:
[94]
R. Luckin, K. R. Koedinger, and J. Greer. 2007. Artificial Intelligence in Education: Building Technology Rich Learning Contexts that Work. IOS Press.
[95]
Andrew Luxton-Reilly, Emma McMillan, Elizabeth Stevenson, Ewan Tempero, and Paul Denny. 2018. Ladebug: An online tool to help novice programmers improve their debugging skills. In Proceedings of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE ’18). ACM, New York, NY, 159–164. DOI:
[96]
Andrew Luxton-Reilly, Simon, Ibrahim Albluwi, Brett A. Becker, Michail Giannakos, Amruth N. Kumar, Linda Ott, James Paterson, Michael James Scott, Judy Sheard, and Claudia Szabo. 2018. Introductory programming: A systematic literature review. In Proceedings Companion of the 23rd Annual ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE ’18 Companion). ACM, New York, NY, 55–106. DOI:
[97]
Nicholas Lytle, Mark Floryan, and Tiffany Barnes. 2019. Effects of a pathfinding program visualization on algorithm development. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (SIGCSE ’19). ACM, New York, NY, 225–231. DOI:
[98]
S. MacNeil, A. Tran, A. Hellas, J. Kim, S. Sarsa, Paul Denny, Seth Bernstein, and Juho Leinonen. 2023. Experiences from using code explanations generated by large language models in a web software development e-book. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education, Vol. 1, 931–937.
[99]
Katerina Mangaroska, Kshitij Sharma, Dragan Gašević, and Michail Giannakos. 2022. Exploring students’ cognitive and affective states during problem solving through multimodal data: Lessons learned from a programming activity. J. Comput. Assist. Learn. 38, 1 (Feb. 2022), 40–59. DOI:
[100]
Renée McCauley, Sue Fitzgerald, Gary Lewandowski, Laurie Murphy, Beth Simon, Lynda Thomas, and Carol Zander. 2008. Debugging: A review of the literature from an educational perspective. Comput. Sci. Educ. 18, 2 (Jun. 2008), 67–92. DOI:
[101]
Mary L. McHugh. 2012. Interrater reliability: The Kappa statistic. Biochem. Med. 22, 3 (2012), 276–282. DOI:
[102]
Debra K. Meyer and Julianne C. Turner. 2006. Re-conceptualizing emotion and motivation to learn in classroom contexts. Educ. Psychol. Rev. 18, 4 (Dec. 2006), 377–390. DOI:
[103]
Tilman Michaeli and Ralf Romeike. 2019. Current status and perspectives of debugging in the K12 classroom: A qualitative study. In Proceedings of the 2019 IEEE Global Engineering Education Conference (EDUCON ’19). IEEE. DOI:
[104]
Tilman Michaeli and Ralf Romeike. 2019. Improving debugging skills in the classroom: The effects of teaching a systematic debugging process. In Proceedings of the 14th Workshop in Primary and Secondary Computing Education (WiPSCE’19, Article 15). ACM, New York, NY, 1–7. DOI:
[105]
Michael A. Miljanovic and Jeremy S. Bradbury. 2017. RoboBUG: A serious game for learning debugging techniques. In Proceedings of the 2017 ACM Conference on International Computing Education Research (ICER ’17). ACM, New York, NY, 93–100. DOI:
[106]
Andrés Moreno, Niko Myller, Erkki Sutinen, and Mordechai Ben-Ari. 2004. Visualizing programs with Jeliot 3. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI ’04). ACM, New York, NY, 373–376. DOI:
[107]
Laurie Murphy, Sue Fitzgerald, Brian Hanks, and Renée McCauley. 2010. Pair debugging: A transactive discourse analysis. In Proceedings of the 6th International Workshop on Computing Education Research (ICER ’10). ACM, New York, NY, 51–58. DOI:
[108]
Laurie Murphy, Gary Lewandowski, Renée McCauley, Beth Simon, Lynda Thomas, and Carol Zander. 2008. Debugging: The good, the bad, and the quirky – A qualitative analysis of novices’ strategies. SIGCSE Bull. 40, 1 (Mar. 2008), 163–167. DOI:
[109]
Laurie Murphy and Lynda Thomas. 2008. Dangers of a fixed mindset: Implications of self-theories research for computer science education. In Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE ’08). ACM, New York, NY, 271–275. DOI:
[110]
Greg L. Nelson, Benjamin Xie, and Amy J. Ko. 2017. Comprehension first: Evaluating a novel pedagogy and tutoring system for program tracing in CS1. In Proceedings of the 2017 ACM Conference on International Computing Education Research (ICER ’17). ACM, New York, NY, 2–11. DOI:
[111]
Devon H. O’Dell. 2017. The debugging mindset: Understanding the psychology of learning strategies leads to effective problem-solving skills. Queueing Syst. 15, 1 (Feb. 2017), 71–90. DOI:
[112]
Paul W. Oman, Curtis, R. Cook, and Murthi Nanja. 1989. Effects of programming experience in debugging semantic errors. J. Syst. Softw. 9, 3 (Mar. 1989), 197–207. DOI:
[113]
Matthew J. Page, Joanne E. McKenzie, Patrick M. Bossuyt, Isabelle Boutron, Tammy C. Hoffmann, Cynthia D. Mulrow, Larissa Shamseer, Jennifer M. Tetzlaff, Elie A. Akl, Sue E. Brennan, Roger Chou, Julie Glanville, Jeremy M. Grimshaw, Asbjørn Hróbjartsson, Manoj M. Lalu, Tianjing Li, Elizabeth W. Loder, Evan Mayo-Wilson, Steve McDonald, Luke A. McGuinness, Lesley A. Stewart, James Thomas, Andrea C. Tricco, Vivian A. Welch, Penny Whiting, and David Moher. 2021. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Rev. Esp. Cardiol. 74, 9 (Sep. 2021), 790–799. DOI:
[114]
Ernesto Panadero, Anders Jonsson, and Juan Botella. 2017. Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educ. Res. Rev. 22 (Nov. 2017), 74–98. DOI:
[115]
Yulia Pechorina, Keith Anderson, and Paul Denny. 2023. Metacodenition: Scaffolding the problem-solving process for novice programmers. In Proceedings of the 25th Australasian Computing Education Conference (ACE ’23). ACM, New York, NY, 59–68. DOI:
[116]
D. N. Perkins and Fay Martin. 1986. Fragile knowledge and neglected strategies in novice programmers. In Papers Presented at the 1st Workshop on Empirical Studies of Programmers on Empirical Studies Of Programmers. Ablex Publishing Corp., 213–229.
[117]
Michael Perscheid, Benjamin Siegmund, Marcel Taeumel, and Robert Hirschfeld. 2017. Studying the advancement in debugging practice of professional software developers. Softw. Qual. Jo. 25, 1 (Mar. 2017), 83–110. DOI:
[118]
Chris Proctor. 2019. Measuring the computational in computational participation: Debugging interactive stories in middle school computer science. In 13th International Conference on Computer Supported Collaborative Learning (CSCL) 1 (2019), 104–111.
[119]
Adalbert Gerald Soosai Raj, Jignesh M. Patel, Richard Halverson, and Erica Rosenfeld Halverson. 2018. Role of live-coding in learning introductory programming. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (Koli Calling ’18, Article 13). ACM, New York, NY, 1–8. DOI:
[120]
Mitchel Resnick, John Maloney, Andrés Monroy-Hernández, Natalie Rusk, Evelyn Eastmond, Karen Brennan, Amon Millner, Eric Rosenbaum, Jay Silver, Brian Silverman, and Yasmin Kafai. 2009. Scratch. Commun. ACM 52, 11 (Nov. 2009), 60–67. DOI:
[121]
Dmitry Resnyansky, Mark Billinghurst, and Arindam Dey. 2019. An AR/TUI-supported debugging teaching environment. In Proceedings of the 31st Australian Conference on Human-Computer-Interaction (OZCHI ’19). ACM, New York, NY, 590–594. DOI:
[122]
Carol Rodgers. 2002. Defining reflection: Another look at John Dewey and reflective thinking. Teach. Coll. Rec. 104, 4 (Apr. 2002), 842–866. DOI:
[123]
André L. Santos. 2018. Enhancing visualizations in pedagogical debuggers by leveraging on code analysis. In Proceedings of the 18th Koli Calling International Conference on Computing Education Research (Koli Calling ’18, Article 11). ACM, New York, NY, 1–9. DOI:
[124]
Dale H. Schunk and Peggy A. Ertmer. 2000. Self-regulation and academic learning: Self-efficacy enhancing interventions. In Handbook of Self-Regulation. Monique Boekaerts, Paul R. Pintrich, and Moshe Zeidner (Eds.), Academic Press, San Diego, 631–649. DOI:
[125]
P. A. Schutz and R. E. Pekrun. 2007. Emotion in Education: A Volume in Educational Psychology, Vol.348. Academic Press, Elsevier, Cambridge, MA, 3–10.
[126]
Daniel L. Schwartz, Jessica M. Tsang, and Kristen P. Blair. 2016. The ABCs of How We Learn: 26 Scientifically Proven Approaches, How They Work, and When to Use Them. W. W. Norton & Company.
[127]
Michael James Scott and Gheorghita Ghinea. 2014. Measuring enrichment: The assembly and validation of an instrument to assess student self-beliefs in CS1. In Proceedings of the 10th Annual Conference on International Computing Education Research (ICER ’14). ACM, New York, NY, 123–130. DOI:
[128]
Valerie J. Shute, Matthew Ventura, and Fengfeng Ke. 2015. The power of play: The effects of portal 2 and lumosity on cognitive and noncognitive skills. Comput. Educ. 80 (Jan. 2015), 58–67. DOI:
[129]
Beth Simon, Dennis Bouvier, Tzu-Yi Chen, Gary Lewandowski, Robert McCartney, and Kate Sanders. 2008. Common sense computing (episode 4): Debugging. Comput. Sci. Educ. 18, 2 (Jun. 2008), 117–133. DOI:
[130]
Arnan Sipitakiat and Nusarin Nusen. 2012. Robo-blocks: Designing debugging abilities in a tangible programming system for early primary school children. In Proceedings of the 11th International Conference on Interaction Design and Children (IDC ’12). ACM, New York, NY, 98–105. DOI:
[131]
Chrysanthos Socratous and Andri Ioannou. 2020. Common errors, successful debugging, and engagement during block-based programming using educational robotics in elementary education (June 2020). In 14th International Conference of the Learning Sciences (ICLS) 2 (2020), 991–998.
[132]
Elliot Soloway, Kate Ehrlich, and Jeffrey Bonar. 1982. Tapping into tacit programming knowledge. In Proceedings of the 1982 Conference on Human Factors in Computing Systems (CHI ’82). ACM, New York, NY, 52–57. DOI:
[133]
Juha Sorva, Ville Karavirta, and Lauri Malmi. 2013. A review of generic program visualization systems for introductory programming education. ACM Trans. Comput. Educ. 13, 4 (Nov. 2013), 1–64. DOI:
[134]
Gerry Stahl. 2005. Group cognition in computer-assisted collaborative learning. J. Comput. Assist. Learn. 21, 2 (Apr. 2005), 79–90. DOI:
[135]
Randy Stein and Susan E. Brennan. 2004. Another person's eye gaze as a cue in solving programming problems. In Proceedings of the 6th International Conference on Multimodal Interfaces (ICMI ’04). ACM, New York, NY, 9–15. DOI:
[136]
Ryo Suzuki, Gustavo Soares, Andrew Head, Elena Glassman, Ruan Reis, Melina Mongiovi, Loris D’Antoni, and Björn Hartmann. 2017. TraceDiff: Debugging unexpected code behavior using trace divergences. In Proceedings of the 2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC ’17). IEEE, 107–115. DOI:
[137]
John Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cogn. Sci. 12, 2 (Apr. 1988), 257–285. DOI:
[138]
John Sweller, Jeroen J. G. van Merrienboer, and Fred G. W. C. Paas. 1998. Cognitive architecture and instructional design. Educ. Psychol. Rev. 10, 3 (Sep. 1998), 251–296. DOI:
[139]
Tamara van Gog and Nikol Rummel. 2010. Example-based learning: Integrating cognitive and social-cognitive research perspectives. Educ. Psychol. Rev. 22, 2 (Jun. 2010), 155–174. DOI:
[140]
Iris Vessey. 1985. Expertise in debugging computer systems: A process analysis. Int. J. Man. Mach. Stud. 23, 5 (Nov. 1985), 459–494. DOI:
[141]
Ioannis Vourletsis, Panagiotis Politis, and Ilias Karasavvidis. 2021. The effect of a computational thinking instructional intervention on students’ debugging proficiency level. In Res. E-Learn. ICT Educ. 15–34. DOI:
[142]
L. S. Vygotsky and Michael Cole. 1978. Mind in Society: Development of Higher Psychological Processes. Harvard University Press.
[143]
Jacqueline Whalley, Amber Settle, and Andrew Luxton-Reilly. 2021. Analysis of a process for introductory debugging. In Proceedings of the Australasian Computing Education Conference (ACE ’21). ACM, New York, NY, 11–20. DOI:
[144]
Jacqueline Whalley, Amber Settle, and Andrew Luxton-Reilly. 2021. Novice reflections on debugging. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (SIGCSE ’21). ACM, New York, NY, 73–79. DOI:
[145]
Claes Wohlin. 2014. Guidelines for snowballing in systematic literature studies and a replication in software engineering. In Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering (EASE ’14, Article 38). ACM, New York, NY, 1–10. DOI:
[146]
Gary K. W. Wong and Shan Jiang. 2018. Computational thinking education for children: Algorithmic thinking and debugging. In Proceedings of the 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE ’18). IEEE, 328–334. DOI:
[147]
Benjamin Xie, Greg L. Nelson, and Amy J. Ko. 2018. An explicit strategy to scaffold novice program tracing. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (SIGCSE ’18). ACM, New York, NY, 344–349. DOI:
[148]
Byung-Do Yoon and O. N. Garcia. 1998. Cognitive activities and support in debugging. In Proceedings of the 4th Annual Symposium on Human Interaction with Complex Systems. IEEE, 160–169. DOI:
[149]
Baichang Zhong and Tingting Li. 2020. Can pair learning improve students’ troubleshooting performance in robotics education? ACM J. Educ. Resour. Comput. 58, 1 (Mar. 2020), 220–248. DOI:
[150]
Barry J. Zimmerman. 1990. Self-regulated learning and academic achievement: An overview. Educ. Psychol. 25, 1 (1990), 3–17.
Index Terms
- Decoding Debugging Instruction: A Systematic Literature Review of Debugging Interventions
Recommendations
Improving Debugging Skills in the Classroom: The Effects of Teaching a Systematic Debugging Process
WiPSCE '19: Proceedings of the 14th Workshop in Primary and Secondary Computing EducationDebugging code is a central skill for students but also a considerable challenge when learning to program: helplessness and, in consequence, frustration when confronted with errors is a common phenomenon in the K12 classroom. Debugging is distinct from ...
Debugging Debugging
COMPSACW '11: Proceedings of the 2011 IEEE 35th Annual Computer Software and Applications Conference WorkshopsWhen a program fails to accomplish its intended task, debugging is conducted to identify and remove any bugs. The debugging operation itself is not immune to flaws. Empirical evidence suggests many bugs are found after shipping, which calls into ...
What constitutes debugging? An exploratory study of debugging episodes
AbstractWhen debugging, developers engage in activities such as navigating, editing, testing, and inspecting code. Despite being the building blocks of debugging, little is known about how they constitute debugging. To address this gap, we introduce the ...
Comments
Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 November 2024
Online AM: 05 September 2024
Accepted: 19 July 2024
Revised: 09 July 2024
Received: 04 January 2024
Published in TOCE Volume 24, Issue 4
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 503Total Downloads
- Downloads (Last 12 months)503
- Downloads (Last 6 weeks)112
Reflects downloads up to 02 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in