research-article

“The Best of Both Worlds!”: Integration of Web Page and Eye Tracking Data Driven Approaches for Automatic AOI Detection

Authors:

Yeliz Yesilada,

Simon HarperAuthors Info & Claims

ACM Transactions on the Web (TWEB), Volume 14, Issue 1

Article No.: 1, Pages 1 - 31

https://doi.org/10.1145/3372497

Published: 09 January 2020 Publication History

Abstract

Web pages are composed of different kinds of elements (menus, adverts, etc.). Segmenting pages into their elements has long been important in understanding how people experience those pages and in making those experiences “better.” Many approaches have been proposed that relate the resultant elements with the underlying source code; however, they do not consider users’ interactions. Another group of approaches analyses eye movements of users to discover areas that interest or attract them (i.e., areas of interest or AOIs). Although these approaches consider how users interact with web pages, they do not relate AOIs with the underlying source code. We propose a novel approach that integrates web page and eye tracking data driven approaches for automatic AOI detection. This approach segments an entire web page into its AOIs by considering users’ interactions and relates AOIs with the underlying source code. Based on the Adjusted Rand Index measure, our approach provides the most similar segmentation to the ground-truth segmentation compared to its individual components.

References

[1]

Hamed Ahmadi and Jun Kong. 2008. Efficient web browsing on small screens. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI’08). ACM, New York, NY, 23--30.

Digital Library

[2]

M. Elgin Akpınar and Yeliz Yesilada. 2013. Vision-based Page Segmentation Algorithm: Extended and Perceived Success. Springer International Publishing, Cham, 238--252.

[3]

Chieko Asakawa and Hironobu Takagi. 2000. Annotation-based transcoding for nonvisual web access. In Proceedings of the 4th International ACM Conference on Assistive Technologies (Assets’00). ACM, New York, NY, 172--179.

Digital Library

[4]

Lidong Bing, Rui Guo, Wai Lam, Zheng-Yu Niu, and Haifeng Wang. 2014. Web page segmentation with structured prediction and its application in web page classification. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’14). ACM, New York, NY, 767--776.

Digital Library

[5]

Paolo Boi, Gianni Fenu, Lucio Davide Spano, and Valentino Vargiu. 2016. Reconstructing user’s attention on the web through mouse movements and perception-based content identification. ACM Trans. Appl. Percept. 13, 3, Article 15 (May 2016), 21 pages.

Digital Library

[6]

Georg Buscher, Edward Cutrell, and Meredith Ringel Morris. 2009. What do you see when you’re surfing?: Using eye tracking to predict salient regions of web pages. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’09). ACM, New York, NY, 21--30.

Digital Library

[7]

Deng Cai, Shipeng Yu, Ji-Rong Wen, and Wei-Ying Ma. 2003. VIPS: A Vision-based Page Segmentation Algorithm. Technical Report. Microsoft. 28 pages. Retrieved from https://www.microsoft.com/en-us/research/publication/vips-a-vision-based-page-segmentation-algorithm/.

[8]

Ricardo J. G. B. Campello, Davoud Moulavi, and Joerg Sander. 2013. Density-based Clustering Based on Hierarchical Density Estimates. Springer, Berlin, 160--172.

[9]

Joyram Chakraborty and Michael P. McGuire. 2016. Directional scan path characterization of eye tracking sequences: A multi-scale approach. In Proceedings of the Future Technologies Conference (FTC’16). 51--61.

[10]

Dorin Comaniciu and Peter Meer. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 5 (May 2002), 603--619.

Digital Library

[11]

Tom N. Cornsweet. 1956. Determination of the stimuli for involuntary drifts and saccadic eye movements. J. Optical Soc. Amer. 46, 11 (1956), 987--988.

[12]

Gautier Drusch and J. M. Christian Bastien. 2012. Analyzing visual scanpaths on the web using the mean shift procedure and t-pattern detection: A bottom-up approach. In Proceedings of the Conference on Ergonomie Et Interaction Homme-machine (Ergo’IHM’12). ACM, New York, NY.

[13]

Sukru Eraslan and Yeliz Yesilada. 2015. Patterns in eyetracking scanpaths and the affecting factors. J. Web Eng. 14, 5--6 (Nov. 2015), 363--385.

[14]

Sukru Eraslan, Yeliz Yesilada, and Simon Harper. 2014. Identifying patterns in eyetracking scanpaths in terms of visual elements of web pages. In Web Engineering, Sven Casteleyn, Gustavo Rossi, and Marco Winckler (Eds.). Springer International Publishing, Cham, 163--180.

[15]

Sukru Eraslan, Yeliz Yesilada, and Simon Harper. 2016. Eye tracking scanpath analysis on web pages: How many users?. In Proceedings of the 9th Biennial ACM Symposium on Eye Tracking Research and Applications (ETRA’16). ACM, New York, NY, 103--110.

Digital Library

[16]

Sukru Eraslan, Yeliz Yesilada, and Simon Harper. 2016. Scanpath trend analysis on web pages: Clustering eye tracking scanpaths. ACM Trans. Web 10, 4 (Nov. 2016).

Digital Library

[17]

Sukru Eraslan, Yeliz Yesilada, and Simon Harper. 2017. Less users more confidence: How AOIs don’t affect scanpath trend analysis. J. Eye Move. Res. 10, 4 (Nov. 2017).

[18]

Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. A density-based algorithm for discovering clusters a density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining (KDD’96). AAAI Press, 226--231.

Digital Library

[19]

Hanyang Feng, Wenzhe Zhang, Hesheng Wu, and Chong-Jun Wang. 2016. Web page segmentation and its application for web information crawling. In Proceedings of the IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI’16). 598--605.

[20]

Brendan J. Frey and Delbert Dueck. 2007. Clustering by passing messages between data points. Science 315 (2007), 972--976.

[21]

Xiao-Dong Gu, Jinlin Chen, Wei-Ying Ma, and Guo-Liang Chen. 2002. Visual-based content understanding toward web adaptation. In Adaptive Hypermedia and Adaptive Web-based Systems, Paul De Bra, Peter Brusilovsky, and Ricardo Conejo (Eds.). Springer, Berlin, 164--173.

[22]

Tz-Huan Huang, Kai-Yin Cheng, and Yung-Yu Chuang. 2009. A collaborative benchmark for region of interest detection algorithms. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 296--303.

[23]

Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. J. Classif. 2, 1 (1 Dec 1985), 193--218.

[24]

Laurent Itti. 2005. Models of bottom-up attention and saliency. In Neurobiology of Attention, Laurent Itti, Geraint Rees, and John K. Tsotsos (Eds.). Academic Press, Burlington, 576--582.

[25]

Jeff Johnson. 2010. Designing with the Mind in Mind: Simple Guide to Understanding User Interface Design Rules. Morgan Kaufmann Publishers Inc., San Francisco, CA.

[26]

Christian Kohlschütter, Peter Fankhauser, and Wolfgang Nejdl. 2010. Boilerplate detection using shallow text features. In Proceedings of the 3rd ACM International Conference on Web Search and Data Mining (WSDM’10). ACM, New York, NY, 441--450.

Digital Library

[27]

Christian Kohlschütter and Wolfgang Nejdl. 2008. A densitometric approach to web page segmentation. In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM’08). ACM, New York, NY, 1173--1182.

Digital Library

[28]

Robert Kreuzer, Jurriaan Hage, and Ad Feelders. 2015. A Quantitative Comparison of Semantic Web Page Segmentation Approaches. Springer International Publishing, Cham, 374--391.

[29]

Aoqi Li, Yingxue Zhang, and Zhenzhong Chen. 2017. Scanpath mining of eye movement trajectories for visual attention analysis. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). 535--540.

[30]

James B. MacQueen. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics. University of California Press, Berkeley, CA, 281--297. https://projecteuclid.org/euclid.bsmsp/1200512992.

[31]

Florian Marchal, Sylvain Castagnos, and Anne Boyer. 2016. Tell me what you see, I will tell you what you remember. In Proceedings of the Conference on User Modeling Adaptation and Personalization (UMAP’16). ACM, New York, NY, 293--294.

Digital Library

[32]

Leland McInnes, John Healy, and Steve Astels. No date. Comparing Python Clustering Algorithms. Retrieved from http://hdbscan.readthedocs.io/en/latest/comparing_clustering_algorithms.html.

[33]

Eleni Michailidou, Simon Harper, and Sean Bechhofer. 2008. Visual complexity and aesthetic perception of web pages. In Proceedings of the 26th Annual ACM International Conference on Design of Communication (SIGDOC’08). ACM, New York, NY, 215--224.

Digital Library

[34]

Hadi Mohammadzadeh, Thomas Gottron, Franz Schweiggert, and Gholamreza Nakhaeizadeh. 2013. Extracting the Main Content of Web Documents Based on Character Encoding and a Naive Smoothing Method. Springer, Berlin, 217--236.

[35]

Anthony Nguyen, Vinod Chandran, and Sridha Sridharan. 2006. Gaze tracking for region of interest coding in JPEG 2000. Signal Process.: Image Commun. 21, 5 (2006), 359--377.

[36]

Jacob Nielson. 2000. Why You Only Need to Test with 5 Users. Retrieved from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/.

[37]

Pontus Olsson. 2007. Real-time and offline filters for eye tracking. Master’s thesis. KTH Royal Institute of Technology.

[38]

Bing Pan, Helene A. Hembrooke, Geri K. Gay, Laura A. Granka, Matthew K. Feusner, and Jill K. Newman. 2004. The determinants of web page viewing behavior: An eye-tracking study. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA’04). ACM, New York, NY, 147--154.

[39]

Kara Pernice and Jakob Nielsen. 2009. How to Conduct Eyetracking Studies. Technical Report. Nielsen Norman Group. Retrieved from https://media.nngroup.com/media/reports/free/How_to_Conduct_Eyetracking_Studies.pdf.

[40]

Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu, and Krzysztof Z. Gajos. 2013. Predicting users’ first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 2049--2058.

[41]

Andrés Sanoja and Stéphane Gançarski. 2014. Block-o-matic: A web page segmentation framework. In Proceedings of the International Conference on Multimedia Computing and Systems (ICMCS’14). 595--600.

[42]

Anthony Santella and Doug DeCarlo. 2004. Robust clustering of eye movement recordings for quantification of visual interest. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA’04). ACM, New York, NY, 27--34.

Digital Library

[43]

scikit-learn developers. [n.d.]. Clustering. Retrieved from http://scikit-learn.org/stable/modules/clustering.html#hierarchical-clustering.

[44]

scikit-learn developers. [n.d.]. sklearn.cluster.AgglomerativeClustering. Retrieved from http://scikit-learn.org/stable/modules/generated/sklearn.cluster.AgglomerativeClustering.html#sklearn.cluster.AgglomerativeClustering.

[45]

Lisa M. Sullivan, Janice Weinberg, and John F. Keaney. 2016. Common statistical pitfalls in basic science research. J. Amer. Heart Assoc. 5, 10 (2016), e004142.

[46]

Tobii Technology AB. 2010. Tobii Studio 2.X User Manual (Sep. 2010). Tobii Technology AB.

[47]

Geoffrey Underwood, Tom Foulsham, and Katherine Humphrey. 2009. Saliency and scan patterns in the inspection of real-world scenes: Eye movements during encoding and recognition. Vis. Cogn. 17, 6--7 (2009), 812--834.

[48]

Jessica M. Utts. 2014. Seeing Through Statistics (4th ed.). Cengage Learning, Stamford, CT.

[49]

Ou Wu, Yunfei Chen, Bing Li, and Weiming Hu. 2011. Evaluating the visual quality of web pages using a computational aesthetic approach. In Proceedings of the 4th ACM International Conference on Web Search and Data Mining (WSDM’11). ACM, New York, NY, 337--346.

Digital Library

[50]

Xin Yang and Yuanchun Shi. 2009. Enhanced gestalt theory guided web page segmentation for mobile browsing. In Proceedings of the IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, Vol. 3. 46--49.

Digital Library

[51]

Yeliz Yesilada. 2011. Web Page Segmentation: A Review. Technical Report. Middle East Technical University. Retrieved from http://emine.ncc.metu.edu.tr/deliverables/emine_D0.pdf.

[52]

Yeliz Yesilada, Simon Harper, and Sukru Eraslan. 2013. Experiential transcoding: An EyeTracking approach. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A’13). ACM, New York, NY.

Digital Library

[53]

Yeliz Yesilada, Caroline Jay, Robert Stevens, and Simon Harper. 2008. Validating the use and role of visual elements of web pages in navigation with an eye-tracking study. In Proceedings of the 17th International Conference on World Wide Web (WWW’08). ACM, New York, NY, 11--20.

Digital Library

[54]

Xinyi Yin and Wee Sun Lee. 2004. Using link analysis to improve layout on mobile devices. In Proceedings of the 13th International Conference on World Wide Web (WWW’04). ACM, New York, NY, 338--344.

Digital Library

[55]

Yu Chen, Xing Xie, Wei-Ying Ma, and Hong-Jiang Zhang. 2005. Adapting web pages for small-screen devices. IEEE Internet Comput. 9, 1 (Jan 2005), 50--56.

[56]

Jan Zeleny, Radek Burget, and Jaroslav Zendulka. 2017. Box clustering segmentation: A new method for vision-based web page preprocessing. Info. Process. Manage. 53, 3 (2017), 735--750.

Digital Library

Cited By

Lyu CChen HPeng XYe JWang H(2023)Optical Character Recognition (OCR)-Based and Gaussian Mixture Modeling-OCR-Based Slide-Level “With-Me-Ness”: Automated Measurement and Feedback of Learners’ Attention State during Video LecturesInternational Journal of Human–Computer Interaction10.1080/10447318.2023.2204272(1-20)Online publication date: 7-May-2023
https://doi.org/10.1080/10447318.2023.2204272
Jansen BJung SSalminen J(2022)Measuring user interactions with websites: A comparison of two industry standard analytics approaches using data of 86 websitesPLOS ONE10.1371/journal.pone.026821217:5(e0268212)Online publication date: 27-May-2022
https://doi.org/10.1371/journal.pone.0268212
Kirsh I(2022)Virtual Finger-Point Reading BehaviorsBig Data Research10.1016/j.bdr.2022.10032829:COnline publication date: 28-Aug-2022
https://dl.acm.org/doi/10.1016/j.bdr.2022.100328
Show More Cited By

Index Terms

“The Best of Both Worlds!”: Integration of Web Page and Eye Tracking Data Driven Approaches for Automatic AOI Detection
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interactive systems and tools
      1. User interface programming
2. Information systems
  1. World Wide Web
    1. Web interfaces

Recommendations

Area of interest adaption using feature importance
ETRA '23: Proceedings of the 2023 Symposium on Eye Tracking Research and Applications

In this paper, we present two approaches and algorithms that adapt areas of interest (AOI) or regions of interest (ROI), respectively, to the eye tracking data quality and classification task. The first approach uses feature importance in a greedy way ...
A robust approach for palmprint biometric recognition

Biometrics system uses an individual's physical or behavioural feature to recognise an individual. An easy-to-capture biometric modality that could work well with a commodity camera is palmprint. It has coarse lines which can be easily detected using a ...
Recent trends of ROI segmentation in iris biometrics: a survey

Segmentation in iris biometrics deals with the localisation of inner and outer boundaries of the iris and isolation of the region of interest (ROI) from the input eye image. The isolated ROI is further used to extract the meaningful features of iris for ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on the Web

ACM Transactions on the Web Volume 14, Issue 1

February 2020

133 pages

ISSN:1559-1131

EISSN:1559-114X

DOI:10.1145/3378674

Editor:
Brian D. Davison
Lehigh University, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 January 2020

Accepted: 01 November 2019

Revised: 01 September 2019

Received: 01 January 2019

Published in TWEB Volume 14, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
528
Total Downloads

Downloads (Last 12 months)83
Downloads (Last 6 weeks)6

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lyu CChen HPeng XYe JWang H(2023)Optical Character Recognition (OCR)-Based and Gaussian Mixture Modeling-OCR-Based Slide-Level “With-Me-Ness”: Automated Measurement and Feedback of Learners’ Attention State during Video LecturesInternational Journal of Human–Computer Interaction10.1080/10447318.2023.2204272(1-20)Online publication date: 7-May-2023
https://doi.org/10.1080/10447318.2023.2204272
Jansen BJung SSalminen J(2022)Measuring user interactions with websites: A comparison of two industry standard analytics approaches using data of 86 websitesPLOS ONE10.1371/journal.pone.026821217:5(e0268212)Online publication date: 27-May-2022
https://doi.org/10.1371/journal.pone.0268212
Kirsh I(2022)Virtual Finger-Point Reading BehaviorsBig Data Research10.1016/j.bdr.2022.10032829:COnline publication date: 28-Aug-2022
https://dl.acm.org/doi/10.1016/j.bdr.2022.100328
Kirsh IChbeir RManolopoulos YAkerkar RMizera-Pietraszko J(2020)Directions and Speeds of Mouse Movements on a Website and Reading PatternsProceedings of the 10th International Conference on Web Intelligence, Mining and Semantics10.1145/3405962.3405982(129-138)Online publication date: 30-Jun-2020
https://dl.acm.org/doi/10.1145/3405962.3405982
Michailidou EEraslan SYesilada YHarper S(2020)Automated Prediction of Visual Complexity of Web Pages: Tools and EvaluationsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2020.102523(102523)Online publication date: Aug-2020
https://doi.org/10.1016/j.ijhcs.2020.102523
Kirsh IJoy MKirsh Y(2020)Horizontal Mouse Movements (HMMs) on Web Pages as Indicators of User InterestHCI International 2020 – Late Breaking Posters10.1007/978-3-030-60700-5_53(416-423)Online publication date: 8-Nov-2020
https://doi.org/10.1007/978-3-030-60700-5_53
Kirsh IJoy M(2020)Exploring Pointer Assisted Reading (PAR): Using Mouse Movements to Analyze Web Users’ Reading Behaviors and PatternsHCI International 2020 - Late Breaking Papers: Multimodality and Intelligence10.1007/978-3-030-60117-1_12(156-173)Online publication date: 19-Jul-2020
https://dl.acm.org/doi/10.1007/978-3-030-60117-1_12
Cormier MCohen RMann RMoffatt KVogel DLiu MZheng S(undefined)Validation of an improved vision-based web page parsing pipelineACM Transactions on the Web10.1145/3580519
https://dl.acm.org/doi/10.1145/3580519

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents