short-paper

Relating eye-tracking measures with changes in knowledge on search tasks

Authors:

Nilavra Bhattacharya,

Jacek GwizdkaAuthors Info & Claims

ETRA '18: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications

Article No.: 62, Pages 1 - 5

https://doi.org/10.1145/3204493.3204579

Published: 14 June 2018 Publication History

Abstract

We conducted an eye-tracking study where 30 participants performed searches on the web. We measured their topical knowledge before and after each task. Their eye-fixations were labelled as "reading" or "scanning". The series of reading fixations in a line, called "reading-sequences" were characterized by their length in pixels, fixation duration, and the number of fixations making up the sequence. We hypothesize that differences in knowledge-change of participants are reflected in their eye-tracking measures related to reading. Our results show that the participants with higher change in knowledge differ significantly in terms of their total reading-sequence-length, reading-sequence-duration, and number of reading fixations, when compared to participants with lower knowledge-change.

References

[1]

P. Borlund. 2003. The IIR Evaluation Model: A Framework for Evaluation of Interactive Information Retrieval Systems. Information Research 8, 3 (2003), paper no. 152.

[2]

Bertram C Brookes. 1980. The Foundations of Information Science. Part I. Philosophical Aspects. Information Scientist 2, 3-4 (1980), 125--133.

Digital Library

[3]

Michael J. Cole, Jacek Gwizdka, Chang Liu, Nicholas J. Belkin, and Xiangmin Zhang. 2013. Inferring User Knowledge Level from Eye Movement Patterns. Information Processing & Management 49, 5 (Sept. 2013), 1075--1091.

Digital Library

[4]

Roberto Colom, Francisco J Abad, Irene Rebollo, and Pei Chun Shih. 2005. Memory Span and General Intelligence: A Latent-Variable Approach. Intelligence 33, 6 (2005), 623--642.

[5]

Andrew RA Conway, Michael J Kane, and Randall W Engle. 2003. Working Memory Capacity and Its Relation to General Intelligence. Trends in cognitive sciences 7, 12 (2003), 547--552.

[6]

Elaine H Cooper and Allan J Pantle. 1967. The Total-Time Hypothesis in Verbal Learning. Psychological bulletin 68, 4 (1967), 221.

[7]

Shahram Eivazi, Roman Bednarik, Markku Tukiainen, Mikael von und zu Fraunberg, Ville Leinonen, and Juha E. Jääskeläinen. 2012. Gaze Behaviour of Expert and Novice Microneurosurgeons Differs During Observations of Tumor Removal Recordings. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '12). ACM, Santa Barbara, California, 377--380.

Digital Library

[8]

Edward Albert Feigenbaum. 1961. The Simulation of Verbal Learning Behavior. In Papers Presented at the May 9-11, 1961, Western Joint IRE-AIEE-ACM Computer Conference. ACM, 121--132.

Digital Library

[9]

Greg Francis, Ian Neath, and Daniel R. VanHorn. 2008. CogLab on a CD. Wadsworth/Thomson Learning. 00019.

[10]

Alex Franz and Thorsten Brants. 2006. All Our N-Gram Are Belong to You. Google Machine Translation Team 20 (2006).

[11]

Luanne Freund, Rick Kopak, and Heather O'Brien. 2016. The Effects of Textual Environment on Reading Comprehension: Implications for Searching as Learning. Journal of Information Science 42, 1 (Feb. 2016), 79--93. 00001.

Digital Library

[12]

Ujwal Gadiraju, Ran Yu, Stefan Dietze, and Peter Holtz. 2018. Analyzing Knowledge Gain of Users in Informational Search Sessions on the Web (CHIIR '18).

Digital Library

[13]

Jacek Gwizdka. 2014. Characterizing Relevance with Eye-Tracking Measures. In Proceedings of the 5th Information Interaction in Context Symposium (IIiX '14). ACM, New York, NY, USA, 58--67. 00028.

Digital Library

[14]

Jacek Gwizdka and Xueshu Chen. 2016. Towards Observable Indicators of Learning on Search. In Proceedings of the Second International Workshop on Search as Learning Co-Located with the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2016). CEUR, Pisa, Italy. 00000.

[15]

Atsushi Iwatsuki, Takatsugu Hirayama, Junya Morita, and Kenji Mase. 2016. Skilled Gaze Behavior Extraction Based on Dependency Analysis of Gaze Patterns on Video Scenes. In Proceedings of the Ninth Biennial ACM Symposium on Eye Tracking Research & Applications (ETRA '16). ACM, Charleston, South Carolina, 299--302.

Digital Library

[16]

Halszka Jarodzka, Katharina Scheiter, Peter Gerjets, and Tamara van Gog. 2010. In the Eyes of the Beholder: How Experts and Novices Interpret Dynamic Stimuli. Learning and Instruction 20, 2 (2010), 146--154. Eye tracking as a tool to study and enhance multimedia learning.

[17]

Marcel Adam Just and Patricia Ann Carpenter. 1987. The Psychology of Reading and Language Comprehension. Vol. x. Allyn & Bacon, Needham Heights, MA, US. 01267.

[18]

Samad Kardan and Cristina Conati. 2012. Exploring Gaze Data for Determining User Learning with an Interactive Simulation. In User Modeling, Adaptation, and Personalization, Judith Masthoff, Bamshad Mobasher, Michel Desmarais, and Roger Nkambou (Eds.). Lecture Notes in Computer Science, Vol. 7379. Springer Berlin / Heidelberg, 126--138.

Digital Library

[19]

Kurt Kraiger, J. Kevin Ford, and Eduardo Salas. 1993. Application of Cognitive, Skill-Based, and Affective Theories of Learning Outcomes to New Methods of Training Evaluation. Journal of Applied Psychology 78, 2 (1993), 311--328.

[20]

Jingjing Liu, Nicholas J. Belkin, Xiangmin Zhang, and Xiaojun Yuan. 2013. Examining Users' Knowledge Change in the Task Completion Process. Information Processing & Management 49, 5 (2013), 1058--1074. 00005.

Digital Library

[21]

M. Nivala, F. Hauser, J. Mottok, and H. Gruber. 2016. Developing Visual Expertise in Software Engineering: An Eye Tracking Study. In 2016 IEEE Global Engineering Education Conference (EDUCON). 613--620. ISSN:.

[22]

Lloyd Peterson and Margaret Jean Peterson. 1959. Short-Term Retention of Individual Verbal Items. Journal of experimental psychology 58, 3 (1959), 193.

[23]

Alexander Pollatsek, Barbara J Juhasz, Erik D Reichle, Debra Machacek, and Keith Rayner. 2008. Immediate and Delayed Effects of Word Frequency and Word Length on Eye Movements in Reading: A Reversed Delayed Effect of Word Length. Journal of Experimental Psychology: Human Perception and Performance 34, 3 (2008), 726.

[24]

Gary E. Raney and Keith Rayner. 1995. Word Frequency Effects and Eye Movements during Two Readings of a Text. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale 49, 2 (1995), 151--173.

[25]

Keith Rayner. 1998. Eye Movements in Reading and Information Processing: 20 Years of Research. Psychological bulletin 124, 3 (1998), 372.

[26]

Keith Rayner, Alexander Pollatsek, Jane Ashby, and Charles Clifton Jr. 2011. Psychology of Reading (2nd ed.). Psychology Press. 00000.

[27]

Toby Segaran and Jeff Hammerbacher. 2009. Beautiful Data: The Stories behind Elegant Data Solutions." O'Reilly Media, Inc.".

[28]

Hans Strasburger, Ingo Rentschler, and Martin Jüttner. 2011. Peripheral Vision and Pattern Recognition: A Review. Journal of vision 11, 5 (2011), 13--13.

[29]

William H Swanson and Gary E Fish. 1995. Color Matches in Diseased Eyes with Good Acuity: Detection of Deficits in Cone Optical Density and in Chromatic Discrimination. JOSA A 12, 10 (1995), 2230--2236.

[30]

Saeed Taki and Saeed Khazaei. 2011. Learning Vocabulary via Mobile Phone: Persian EFL Learners in Focus. Journal of Language Teaching & Research 2, 6 (2011).

[31]

Mathew J. Wilson and Max L. Wilson. 2013. A Comparison of Techniques for Measuring Sensemaking and Learning within Participant-Generated Summaries. Journal of the American Society for Information Science and Technology 64, 2 (2013), 291--306.

Digital Library

Cited By

Barz MKaragiannis PKildal JRivera Pinto Ade Munain JRosel JMadarieta MSalagianni KAivaliotis PMakris SSonntag D(2024)MASTER-XR: Mixed Reality Ecosystem for Teaching Robotics in ManufacturingIntegrated Systems: Data Driven Engineering10.1007/978-3-031-53652-6_10(167-182)Online publication date: 17-Sep-2024
https://doi.org/10.1007/978-3-031-53652-6_10
AlSalem TAlShamari M(2023)Assessing Interactive Web-Based Systems Using Behavioral Measurement TechniquesFuture Internet10.3390/fi1511036515:11(365)Online publication date: 11-Nov-2023
https://doi.org/10.3390/fi15110365
Dosso CTamine LPaubel PChevalier A(2023)Navigational and thematic exploration–exploitation trade-offs during web search: effects of prior domain knowledge, search contexts and strategies on search outcomeBehaviour & Information Technology10.1080/0144929X.2023.224251443:10(2232-2258)Online publication date: 10-Aug-2023
https://doi.org/10.1080/0144929X.2023.2242514
Show More Cited By

Index Terms

Relating eye-tracking measures with changes in knowledge on search tasks
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI

Recommendations

Measuring Learning During Search: Differences in Interactions, Eye-Gaze, and Semantic Similarity to Expert Knowledge
CHIIR '19: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval

We investigate the relationship between search behavior, eye -tracking measures, and learning. We conducted a user study where 30 participants performed searches on the web. We measured their verbal knowledge before and after each task in a content-...
Compensation of head movements in mobile eye-tracking data using an inertial measurement unit
UbiComp '14 Adjunct: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication

Analysis of eye movements recorded with a mobile eye-tracker is difficult since the eye-tracking data are severely affected by simultaneous head and body movements. Automatic analysis methods developed for remote-, and tower-mounted eye-trackers do not ...
Eye and pointer coordination in search and selection tasks
ETRA '10: Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications

Selecting a graphical item by pointing with a computer mouse is a ubiquitous task in many graphical user interfaces. Several techniques have been suggested to facilitate this task, for instance, by reducing the required movement distance. Here we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '18: Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications

June 2018

595 pages

ISBN:9781450357067

DOI:10.1145/3204493

Conference Chairs:
Bonita Sharif
Youngstown State University
,
Krzysztof Krejtz
University of Social Sciences and Humanities, Warsaw, Poland

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

Institute of Museum and Library Services

Conference

ETRA '18

Sponsor:

ETRA '18: 2018 Symposium on Eye Tracking Research and Applications

June 14 - 17, 2018

Warsaw, Poland

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
325
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)4

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Barz MKaragiannis PKildal JRivera Pinto Ade Munain JRosel JMadarieta MSalagianni KAivaliotis PMakris SSonntag D(2024)MASTER-XR: Mixed Reality Ecosystem for Teaching Robotics in ManufacturingIntegrated Systems: Data Driven Engineering10.1007/978-3-031-53652-6_10(167-182)Online publication date: 17-Sep-2024
https://doi.org/10.1007/978-3-031-53652-6_10
AlSalem TAlShamari M(2023)Assessing Interactive Web-Based Systems Using Behavioral Measurement TechniquesFuture Internet10.3390/fi1511036515:11(365)Online publication date: 11-Nov-2023
https://doi.org/10.3390/fi15110365
Dosso CTamine LPaubel PChevalier A(2023)Navigational and thematic exploration–exploitation trade-offs during web search: effects of prior domain knowledge, search contexts and strategies on search outcomeBehaviour & Information Technology10.1080/0144929X.2023.224251443:10(2232-2258)Online publication date: 10-Aug-2023
https://doi.org/10.1080/0144929X.2023.2242514
Barz MBhatti OSonntag D(2022)Implicit Estimation of Paragraph Relevance From Eye MovementsFrontiers in Computer Science10.3389/fcomp.2021.8085073Online publication date: 7-Jan-2022
https://doi.org/10.3389/fcomp.2021.808507
Guan XLei CFeng X(2022)The Predictive Effect of Eye-Movement Behavior on Reading Performance in Online Reading ContextsProceedings of the 14th International Conference on Education Technology and Computers10.1145/3572549.3572624(470-475)Online publication date: 28-Oct-2022
https://dl.acm.org/doi/10.1145/3572549.3572624
Labhishetty SZhai CCrestani FPasi GGaussier E(2022)PRE: A Precision-Recall-Effort Optimization Framework for Query SimulationProceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3539813.3545136(51-60)Online publication date: 23-Aug-2022
https://dl.acm.org/doi/10.1145/3539813.3545136
Bozkir EKasneci GUtz SKasneci E(2022)Regressive Saccadic Eye Movements on Fake News2022 Symposium on Eye Tracking Research and Applications10.1145/3517031.3529619(1-7)Online publication date: 8-Jun-2022
https://dl.acm.org/doi/10.1145/3517031.3529619
Chen YQiu YLin HZhao Y(2022)The effects of topic familiarity on college students' learning search processAslib Journal of Information Management10.1108/AJIM-09-2021-025274:6(1151-1173)Online publication date: 16-May-2022
https://doi.org/10.1108/AJIM-09-2021-0252
Chen JZhang LQian W(2022)Cognitive differences between readers attentive and inattentive to task-related information: an eye-tracking studyAslib Journal of Information Management10.1108/AJIM-01-2022-000775:5(917-939)Online publication date: 4-Oct-2022
https://doi.org/10.1108/AJIM-01-2022-0007
Gwizdka JTessmer RChan YRadhakrishnan KHenry M(2022)Eye-Gaze and Mouse-Movements on Web Search as Indicators of Cognitive ImpairmentInformation Systems and Neuroscience10.1007/978-3-031-13064-9_20(187-200)Online publication date: 3-Dec-2022
https://doi.org/10.1007/978-3-031-13064-9_20
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents