research-article

Eye movements in code reading: relaxing the linear order

Authors:

Teresa Busjahn,

Roman Bednarik,

James H. Paterson,

Carsten Schulte,

Bonita Sharif, and

Sascha TammAuthors Info & Claims

ICPC '15: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension

May 2015

Pages 255 - 265

Published: 16 May 2015 Publication History

Abstract

Code reading is an important skill in programming. Inspired by the linearity that people exhibit while natural language text reading, we designed local and global gaze-based measures to characterize linearity (left-to-right and top-to-bottom) in reading source code. Unlike natural language text, source code is executable and requires a specific reading approach. To validate these measures, we compared the eye movements of novice and expert programmers who were asked to read and comprehend short snippets of natural language text and Java programs.

Our results show that novices read source code less linearly than natural language text. Moreover, experts read code less linearly than novices. These findings indicate that there are specific differences between reading natural language and source code, and suggest that non-linear reading skills increase with expertise. We discuss the implications for practitioners and educators.

References

[1]

V. R. Basili and H. D. Mills, "Understanding and documenting programs," IEEE Transactions on Software Engineering, vol. 18, pp. 270--283, 1982.

Digital Library

[2]

R. Bednarik, "Expertise-dependent visual attention strategies develop over time during debugging with multiple code representations," International Journal of Human-Computer Studies, vol. 70, no. 2, pp. 143--155, Feb. 2012.

Digital Library

[3]

R. Brooks, "Towards a theory of the comprehension of computer programs," International Journal of Man-Machine Studies, vol. 18, pp. 543--554, 1983.

[4]

R. P. L. Buse and W. R. Weimer, "Learning a metric for code readability," IEEE Transactions on Software Engineering, vol. 36, no. 4, pp. 546--558, Jul. 2010.

Digital Library

[5]

T. Busjahn, C. Schulte, and A. Busjahn, "Analysis of code reading to gain more insight in program comprehension," Koli, Finland: ACM, 2011, pp. 1--9. {Online}. Available: http://doi.acm.org/10.1145/2094131.2094133.

Digital Library

[6]

T. Busjahn, C. Schulte, B. Sharif, Simon, A. Begel, M. Hansen, R. Bednarik, P. Orlov, P. Ihantola, G. Shchekotova, and M. Antropova, "Eye tracking in computing education," in Proceedings of the Tenth Annual Conference on International Computing Education Research, Glasgow, Scotland, United Kingdom: ACM, 2014, pp. 3--10.

Digital Library

[7]

F. Cristino, S. Mathôt, J. Theeuwes, and I. D. Gilchrist, "ScanMatch: a novel method for comparing fixation sequences," Behavior Research Methods, vol. 42, no. 3, pp. 692--700, 2010.

[8]

M. E. Crosby, J. Scholtz, and S. Wiedenbeck, "The roles beacons play in comprehension for novice and expert programmers," in 14th Workshop of the Psychology of Programming Interest Group, 2002, pp. 58--73.

[9]

M. E. Crosby and J. Stelovsky, "How do we read algorithms? a case study," Computer, vol. 23, no. 1, pp. 25--35, 1990.

Digital Library

[10]

Q. Fan, "The effects of beacons, comments, and tasks on program comprehension process in software maintenance," PhD thesis, University of Maryland at Baltimore County, Catonsville, MD, USA, 2010.

Digital Library

[11]

T. Fritz, A. Begel, S. C. Müller, S. Yigit-Elliott, and M. Züger, "Using psycho-physiological measures to assess task difficulty in software development," in Proceedings of the 36th International Conference on Software Engineering, ser. ICSE 2014, Hyderabad, India: ACM, 2014, pp. 402--413.

Digital Library

[12]

M. E. Hansen, R. L. Goldstone, and A. Lumsdaine, "What makes code hard to understand?" CoRR, vol. abs/1304.5257, 2013.

[13]

J. Z. Jacobson and P. Dodwell, "Saccadic eye movements during reading," Brain and Language, vol. 8, no. 3, pp. 303--314, 1979.

[14]

A. Kennedy and W. Murray, "The components of reading time: eye movement patterns of good and poor readers," in Eye Movements: From Physiology to Cognition, New York, NY, USA: ACM, 1987, pp. 509--529.

[15]

W. Kintsch, Comprehension: A Paradigm for Cognition. Cambridge University Press, 1998.

[16]

S. Letovsky and E. Soloway, "Delocalized plans and program comprehension," IEEE Software, pp. 41--49, 1986.

Digital Library

[17]

B. Liblit, A. Begel, and E. Sweetser, "Cognitive perspectives on the role of naming in computer programs," in Proceedings of the 18th Annual Psychology of Programming Workshop, Brighton, England, United Kingdom, Sep. 2006.

[18]

R. Lister, E. S. Adams, S. Fitzgerald, W. Fone, J. Hamer, M. Lindholm, R. McCartney, J. E. Moström, K. Sanders, O. Seppälä, B. Simon, and L. Thomas, "A multi-national study of reading and tracing skills in novice programmers," in Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education, ser. ITiCSE-WGR '04, Leeds, United Kingdom: ACM, 2004, pp. 119--150.

Digital Library

[19]

K. Rayner, "Eye movements in reading and information processing: 20 years of research.," Psychological bulletin, vol. 124, no. 3, p. 372, 1998.

[20]

K. Rayner, S. P. Ardoin, and K. S. Binder, "Children's eye movements in reading: a commentary.," School Psychology Review, vol. 42, no. 2, pp. 223--233, 2013.

[21]

P. Rodeghero, C. McMillan, P. W. McBurney, N. Bosch, and S. D'Mello, "Improving automated source code summarization via an eye-tracking study of programmers," in Proceedings of the 36th International Conference on Software Engineering, ser. ICSE 2014, Hyderabad, India: ACM, 2014, pp. 390--401.

Digital Library

[22]

C. Schulte, "Block model: an educational model of program comprehension as a tool for a scholarly approach to teaching," in Proceedings of the Fourth International Workshop on Computing Education Research, ser. ICER '08, Sydney, Australia: ACM, 2008, pp. 149--160.

Digital Library

[23]

C. Schulte, T. Clear, A. Taherkhani, T. Busjahn, and J. H. Paterson, "An introduction to program comprehension for computer science educators," in Proceedings of the 2010 ITiCSE Working Group Reports, ser. ITiCSE-WGR '10, Ankara, Turkey: ACM, 2010, pp. 65--86.

Digital Library

[24]

B. Sharif, M. Falcone, and J. Maletic, "An eye-tracking study on the role of scan time in finding source code defects," in Proc. of the Symposium on Eye Tracking Research & Applications, Santa Barbara, CA: ACM, 2012, pp. 381--384.

Digital Library

[25]

J. Siegmund, C. Kästner, S. Apel, C. Parnin, A. Bethmann, T. Leich, G. Saake, and A. Brechmann, "Understanding understanding source code with functional magnetic resonance imaging," in Proceedings of the 36th International Conference on Software Engineering, ser. ICSE 2014, Hyderabad, India: ACM, 2014, pp. 378--389.

Digital Library

[26]

Simon, "Eye movements in programming education 2: analysing the novice's gaze," in Eye Movements in Programming Education II: Analyzing the Novice's Gaze, T. Busjahn, C. Schulte, S. Tamm, and R. Bednarik, Eds., Berlin, Germany, 2015, pp. 31--33.

[27]

E. Soloway, J. Pinto, S. Letovsky, D. Littman, and R. Lampert, "Designing documentation to compensate for delocalized plans," Communications of the ACM, vol. 31, pp. 1259--1267, 11 1988.

Digital Library

[28]

J. Sorva, "Notional machines and introductory programming education," ACM Transactions on Computing Education (TOCE), vol. 13, no. 2, p. 8, 2013.

Digital Library

[29]

R. Turner, M. Falcone, B. Sharif, and A. Lazar, "An eye-tracking study assessing the comprehension of C++ and Python source code," in Proc. of the Symposium on Eye Tracking Research & Applications, Safety Harbor, Florida: ACM, 2014, pp. 231--234.

Digital Library

[30]

H. Uwano, M. Nakamura, A. Monden, and K.-i. Matsumoto, "Analyzing individual performance of source code review using reviewers' eye movement," in Proc. of the Symposium on Eye Tracking Research & Applications, San Diego, California: ACM, 2006, pp. 133--140.

Digital Library

[31]

M. R. Wilson, S. J. Vine, E. Bright, R. S. Masters, D. Defriend, and J. S. McGrath, "Gaze training enhances laparoscopic technical skill acquisition and multi-tasking performance: a randomized, controlled study," Surgical Endoscopy, vol. 25, no. 12, pp. 3731--3739, 2011.

Cited By

Kather PDuran RVahrenhold J(2021)Through (Tracking) Their Eyes: Abstraction and Complexity in Program ComprehensionACM Transactions on Computing Education10.1145/348017122:2(1-33)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1145/3480171
Busjahn TTamm S(2021)A Deeper Analysis of AOI Coverage in Code ReadingACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3457422(1-7)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3457422
Harada HNakayama M(2021)Estimation of reading ability of program codes using features of eye movementsACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3457421(1-5)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3457421
Show More Cited By

Index Terms

Eye movements in code reading: relaxing the linear order

Recommendations

Eye Movements in Code Reading: Relaxing the Linear Order
ICPC '15: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension

Code reading is an important skill in programming. Inspired by the linearity that people exhibit while natural language text reading, we designed local and global gaze-based measures to characterize linearity (left-to-right and top-to-bottom) in reading ...
Read More
Eye movements during reading and reading assessment in swedish school children: a new window on reading difficulties
ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications

Research during the last decades has demonstrated that eye tracking methodology is an advantageous tool to study reading. A substantial amount of eye movement research has resulted in improved understanding of the reading process in skilled adult ...
Read More
Can the E-Z Reader Model Predict Eye Movements Over Code? Towards a Model of Eye Movements Over Source Code
ETRA '20 Short Papers: ACM Symposium on Eye Tracking Research and Applications

Studies of eye movements during source code reading have supported the idea that reading source code differs fundamentally from reading natural text. The paper analyzed an existing data set of natural language and source code eye movement data using the ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICPC '15: Proceedings of the 2015 IEEE 23rd International Conference on Program Comprehension

May 2015

325 pages

General Chair:
Andrea De Lucia
University of Salerno, Italy
,
Program Chairs:
Christian Bird
Microsoft Research
,
Rocco Oliveto
University of Molise, Italy

Sponsors

ACM: Association for Computing Machinery
SIGSOFT: ACM Special Interest Group on Software Engineering
IEEE-CS\DATC: IEEE Computer Society
TCSE: IEEE Computer Society's Tech. Council on Software Engin.

Publisher

IEEE Press

Publication History

Published: 16 May 2015

Check for updates

Qualifiers

Research-article

Conference

ICSE '15

Sponsor:

ACM
SIGSOFT
IEEE-CS\DATC
TCSE

ICSE '15: 37th International Conference on Software Engineering

May 16 - 24, 2015

Florence, Italy

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

44
Total Citations
View Citations
372
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)4

Other Metrics

View Author Metrics

Citations

Cited By

Kather PDuran RVahrenhold J(2021)Through (Tracking) Their Eyes: Abstraction and Complexity in Program ComprehensionACM Transactions on Computing Education10.1145/348017122:2(1-33)Online publication date: 1-Nov-2021
https://dl.acm.org/doi/10.1145/3480171
Busjahn TTamm S(2021)A Deeper Analysis of AOI Coverage in Code ReadingACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3457422(1-7)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3457422
Harada HNakayama M(2021)Estimation of reading ability of program codes using features of eye movementsACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3457421(1-5)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3457421
Fowler MZilles CSherriff MMerkle LCutter PMonge ASheard J(2021)Superficial Code-guise: Investigating the Impact of Surface Feature Changes on Students' Programming Question ScoresProceedings of the 52nd ACM Technical Symposium on Computer Science Education10.1145/3408877.3432413(3-9)Online publication date: 3-Mar-2021
https://dl.acm.org/doi/10.1145/3408877.3432413
Peitek NApel SParnin CBrechmann ASiegmund J(2021)Program Comprehension and Code Complexity MetricsProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00056(524-536)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00056
Talsma RBarendsen ESmetsers S(2020)Analyzing the influence of block highlighting on beginning programmers' reading behavior using eye trackingProceedings of the 9th Computer Science Education Research Conference10.1145/3442481.3442505(1-10)Online publication date: 19-Oct-2020
https://dl.acm.org/doi/10.1145/3442481.3442505
Ishida TUwano HIkutani Y(2020)Combining Biometric Data with Focused Document Types Classifies a Success of Program ComprehensionProceedings of the 28th International Conference on Program Comprehension10.1145/3387904.3389291(366-370)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.1145/3387904.3389291
Paudyal BCreed CFrutos-Pascual MWilliams IWakkary RAndersen KOdom WDesjardins APetersen M(2020)VoiceyeProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395553(21-33)Online publication date: 3-Jul-2020
https://dl.acm.org/doi/10.1145/3357236.3395553
Ishida TUwano H(2019)Time series analysis of programmer's EEG for debug state classificationCompanion Proceedings of the 3rd International Conference on the Art, Science, and Engineering of Programming10.1145/3328433.3328459(1-7)Online publication date: 1-Apr-2019
https://dl.acm.org/doi/10.1145/3328433.3328459
Chitalkina NKrejtz KSharif B(2019)When you don't see what you expectProceedings of the 11th ACM Symposium on Eye Tracking Research & Applications10.1145/3314111.3322866(1-3)Online publication date: 25-Jun-2019
https://dl.acm.org/doi/10.1145/3314111.3322866
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents