Article

An eye-tracking methodology for characterizing program comprehension processes

Authors:

Roman Bednarik,

Markku TukiainenAuthors Info & Claims

ETRA '06: Proceedings of the 2006 symposium on Eye tracking research & applications

Pages 125 - 132

https://doi.org/10.1145/1117309.1117356

Published: 27 March 2006 Publication History

Abstract

Program comprehension processes have previously been studied using methodologies such as think-aloud or comprehension summary analysis. Eye-tracking, however, has not been previously widely applied to studies of behavioral aspects of programming. We present a study in which program comprehension was investigated with a help of a remote eye-tracker. Novice and intermediate programmers used a program visualization tool to aid their comprehension while the location of fixations, fixation durations and attention switching between the areas of interest were recorded.In this paper 1) we propose an approach how to investigate trends in repeated-measures sparse-data of few cases captured by an eye-tracker and 2) using this technique, we characterize the development of program comprehension strategies during dynamic program visualization with help of eye-movement data.

References

[1]

Bednarik, R., and Tukiainen, M. 2004. Visual attention tracking during program debugging. In Proceedings of The Third Nordic Conference on Human-Computer Interaction (NordiCHI'04), ACM Press, New York, NY, USA, 331--334.

Digital Library

[2]

Bednarik, R., and Tukiainen, M. 2005. Effects of display blurring on the behavior of novices and experts during program debugging. In Proceedings of (CHI'05), ACM Press, Portland, OR, USA, Extended abstracts of the ACM Conference on Human Factors in Computing Systems, 1204--1207.

Digital Library

[3]

Bednarik, R., Myller, N., Sutinen, E., and Tukiainen, M. 2005. Effects of experience on gaze behaviour during program animation. In Proceedings of the 17th Annual Psychology of Programming Interest Group Workshop (PPIG'05), 49--61.

[4]

Bednarik, R., Myller, N., Sutinen, E., and Tukiainen, M. 2006. Analyzing individual differences in program comprehension with rich data. To appear in Technology, Instruction, Cognition and Learning.

[5]

Ben-Ari, M. 2001. Program visualization in theory and practice. Informatik/Informatique 2, 8--11.

[6]

Blackwell, A. F. 2002. First steps in programming: A rationale for attention investment models. In HCC '02: Proceedings of the IEEE 2002 Symposia on Human Centric Computing Languages and Environments (HCC'02), IEEE Computer Society, Washington, DC, USA, 2.

Digital Library

[7]

Branch, J. L. 2000. Investigating the information-seeking processes of adolescents: The value of using think alouds and think afters. Library and Information Science Research 22, 4, 371--392.

[8]

Brooks, R. 1983. Towards a theory of the comprehension of computer programs. International Journal of Man-Machine Studies 18, 543--554.

[9]

Burkhardt, J., Détienne, F., and Wiedenbeck, S. 2002. Object-oriented program comprehension: Effect of expertise, task and phase. Empirical Software Engineering 7, 2, 115--156.

Digital Library

[10]

Crosby, M. E., and Stelovsky, J. 1990. How do we read algorithms? A case study. IEEE Computer 23, 1, 24--35.

Digital Library

[11]

Ericsson, K. A., and Simon, H. A. 1984. Protocol analysis: Verbal reports as data. Braford Books/MIT Press, Cambridge, MA.

[12]

Good, J., and Brna, P. 2004. Program comprehension and authentic measurement: a scheme for analysing descriptions of programs. International Journal of Human Computer Studies 61, 2, 169--185.

Digital Library

[13]

Jansen, A. R., Blackwell, A. F., and Marriott, K. 2003. A tool for tracking visual attention: The Restricted Focus Viewer. Behavior Research Methods, Instruments, and Computers 35, 1, 57--69.

[14]

Just, M. A., and Carpenter, P. A. 1976. Eye fixations and cognitive processes. Cognitive Psychology 8, 441--480.

[15]

Ko, A. J., and Uttl, B. 2003. Individual differences in program comprehension strategies in unfamiliar programming systems. In IWPC '03: Proceedings of the 11th IEEE International Workshop on Program Comprehension, IEEE Computer Society, Washington, DC, USA, 175.

Digital Library

[16]

Letovsky, S. 1986. Cognitive processes in program comprehension. In Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers, Ablex Publishing Corp., Norwood, NJ, USA, 58--79.

Digital Library

[17]

Littman, D. C., Pinto, J., Letovsky, S., and Soloway, E. 1986. Mental models and software maintenance. In Papers presented at the first workshop on empirical studies of programmers on Empirical studies of programmers, Ablex Publishing Corp., Norwood, NJ, USA, 80--98.

Digital Library

[18]

Moreno, A., Myller, N., Sutinen, E., and Ben-Ari, M. 2004. Visualizing programs with Jeliot 3. In Proceedings of the Working Conference on Advance Visual Interfaces (AVI 2004), ACM, 373--376.

Digital Library

[19]

Nevalainen, S., and Sajaniemi, J. 2005. Short-term effects of graphical versus textual visualisation of variables on program perception. In Proceedings of the 17th Annual Psychology of Programming Interest Group Workshop (PPIG'05), 77--91.

[20]

Nielsen, J., Clemmensen, T., and Yssing, C. 2002. Getting access to what goes on in peoples heads? reflections on the think-aloud technique. In Proceedings of The Second Nordic Conference on Human-Computer Interaction (NordiCHI'02), ACM Press, New York, NY, USA, 101--110.

Digital Library

[21]

Petre, M. 1995. Why looking isn't always seeing: readership skills and graphical programming. Communications of ACM 38, 6, 33--44.

Digital Library

[22]

Rayner, K. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124, 372--422.

[23]

Romero, P., Lutz, R., Cox, R., and Du Boulay, B. 2002. Co-ordination of multiple external representations during Java program debugging. In HCC '02: Proceedings of the IEEE 2002 Symposia on Human Centric Computing Languages and Environments (HCC'02), IEEE Computer Society, Washington, DC, USA, 207.

Digital Library

[24]

Sajaniemi, J., and Kuittinen, M. 2003. Program animation based on the roles of variables. In SoftVis '03: Proceedings of the 2003 ACM symposium on Software visualization, ACM Press, New York, NY, USA, 7--16.

Digital Library

[25]

Soloway, E., Lampert, R., Letovsky, S., Littman, D., and Pinto, J. 1988. Designing documentation to compensate for delocalized plans. Communications of ACM 31, 11, 1259--1267.

Digital Library

[26]

Van Den Haak, M., Jong, M. D., and Schellens, P. J. 2003. Retrospective vs. concurrent think-aloud protocols: testing the usability of an online library catalogue. Behaviour and Information Technology 22, 5, 339--351.

[27]

Von Mayrhauser, A., and Vans, A. M. 1996. Identification of dynamic comprehension processes during large scale maintenance. IEEE Transactions on Software Engineering 22, 6, 424--437.

Digital Library

Cited By

Albeaino GJeelani IGheisari MIssa R(2025)Assessing proxemics impact on Human-Robot collaboration safety in construction: A virtual reality study with four-legged robotsSafety Science10.1016/j.ssci.2024.106682181(106682)Online publication date: Jan-2025
https://doi.org/10.1016/j.ssci.2024.106682
Weber TWiniker CMayer S(2024)An Investigation of How Software Developers Read Machine Learning CodeProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686678(165-176)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686678
Yang WObaidellah U(2024)Attention Dynamics in Programming: Eye Gaze Patterns of High- vs. Low-Ability Novice CodersProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655095(1-6)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655095
Show More Cited By

Index Terms

An eye-tracking methodology for characterizing program comprehension processes
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens

Recommendations

Analysis of code reading to gain more insight in program comprehension
Koli Calling '11: Proceedings of the 11th Koli Calling International Conference on Computing Education Research

Code reading, although an integral part of program comprehension, is rarely reflected. In this paper, we want to argue for a research approach and direction exploiting the potential that lies in the analysis of reading processes. Based on the vast ...
Eye tracking in computing education
ICER '14: Proceedings of the tenth annual conference on International computing education research

The methodology of eye tracking has been gradually making its way into various fields of science, assisted by the diminishing cost of the associated technology. In an international collaboration to open up the prospect of eye movement research for ...
Eye tracking analysis of computer program comprehension in programmers with dyslexia

This paper investigates the impact of dyslexia on the reading and comprehension of computer program code. Drawing upon work from the fields of program comprehension, eye tracking, dyslexia, models of reading and dyslexia gaze behaviour, a set of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '06: Proceedings of the 2006 symposium on Eye tracking research & applications

March 2006

175 pages

ISBN:1595933050

DOI:10.1145/1117309

Conference Chairs:
Kari-Jouko Räihä
University of Tampere
,
Andrew T. Duchowski
Clemson University

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 March 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ETRA06

Sponsor:

ETRA06: Eye Tracking Research and Applications

March 27 - 29, 2006

California, San Diego

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

112
Total Citations
View Citations
2,236
Total Downloads

Downloads (Last 12 months)117
Downloads (Last 6 weeks)16

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Albeaino GJeelani IGheisari MIssa R(2025)Assessing proxemics impact on Human-Robot collaboration safety in construction: A virtual reality study with four-legged robotsSafety Science10.1016/j.ssci.2024.106682181(106682)Online publication date: Jan-2025
https://doi.org/10.1016/j.ssci.2024.106682
Weber TWiniker CMayer S(2024)An Investigation of How Software Developers Read Machine Learning CodeProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3674805.3686678(165-176)Online publication date: 24-Oct-2024
https://dl.acm.org/doi/10.1145/3674805.3686678
Yang WObaidellah U(2024)Attention Dynamics in Programming: Eye Gaze Patterns of High- vs. Low-Ability Novice CodersProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655095(1-6)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655095
Hüsing SSparmann SSchulte CBolte M(2024)Identifying K-12 Students' Approaches to Using Worked Examples for Epistemic ProgrammingProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655094(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655094
Hauser FGrabinger LEzer TMottok JGruber H(2024)Analyzing and Interpreting Eye Movements in C++: Using Holistic Models of Image PerceptionProceedings of the 2024 Symposium on Eye Tracking Research and Applications10.1145/3649902.3655093(1-7)Online publication date: 4-Jun-2024
https://dl.acm.org/doi/10.1145/3649902.3655093
Tang NChen MNing ZBansal AHuang YMcMillan CLi T(2024)Developer Behaviors in Validating and Repairing LLM-Generated Code Using IDE and Eye Tracking2024 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL/HCC60511.2024.00015(40-46)Online publication date: 2-Sep-2024
https://doi.org/10.1109/VL/HCC60511.2024.00015
Cheng GZou DXie HWang F(2024)Exploring differences in self-regulated learning strategy use between high- and low-performing students in introductory programmingComputers & Education10.1016/j.compedu.2023.104948208:COnline publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1016/j.compedu.2023.104948
Grabinger LHauser FWolff CMottok J(2024)On Eye Tracking in Software EngineeringSN Computer Science10.1007/s42979-024-03045-35:6Online publication date: 26-Jul-2024
https://dl.acm.org/doi/10.1007/s42979-024-03045-3
Mansoor NPeterson CDodd MSharif B(2023)Assessing the Effect of Programming Language and Task Type On Eye Movements of Computer Science StudentsACM Transactions on Computing Education10.1145/3632530Online publication date: 14-Nov-2023
https://dl.acm.org/doi/10.1145/3632530
Wyrich MBogner JWagner S(2023)40 Years of Designing Code Comprehension Experiments: A Systematic Mapping StudyACM Computing Surveys10.1145/362652256:4(1-42)Online publication date: 9-Nov-2023
https://dl.acm.org/doi/10.1145/3626522
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents