Article

A comparison of LSA, wordNet and PMI-IR for predicting user click behavior

Authors:

Ishwinder Kaur,

Anthony J. HornofAuthors Info & Claims

CHI '05: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 51 - 60

https://doi.org/10.1145/1054972.1054980

Published: 02 April 2005 Publication History

Abstract

A predictive tool to simulate human visual search behavior would help interface designers inform and validate their design. Such a tool would benefit from a semantic component that would help predict search behavior even in the absence of exact textual matches between goal and target. This paper discusses a comparison of three semantic systems-LSA, WordNet and PMI-IR-to evaluate their performance in predicting the link that people would select given an information goal and a webpage. PMI-IR best predicted human performance as observed in a user study.

References

[1]

Ambroziak, J., & Woods, W. A. (1998). Natural Language Technology in Precision Content Retrieval. Proceedings of the International Conference on Natural Language Processing and Industrial Applications.]]

[2]

Banerjee, S., & Pedersen, T. (2003). Extended gloss over-laps as a measure of semantic relatedness. Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, 805--810.]]

Digital Library

[3]

Blackmon, M., Kitajima, M., & Polson, P. (2003). Repairing usability problems Identified by the Cognitive Walkthrough for the Web. Proceedings of ACM CHI 2003: Conference on Human Factors in Computing Systems, 497--504.]]

Digital Library

[4]

Brumby, D., & Howes, A. (2004). Good enough but I'll just check: Web-page search as attentional refocusing. Proceedings of the 6th International Conference on Cognitive Modeling, 46--51.]]

[5]

Budanitsky, A., & Hirst, G. (2001). Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. Workshop on WordNet and Other Lexical Resources, 2nd meeting of the North American Chapter of the Association for Computational Linguistics.]]

[6]

Byrne, M. D. (2001) ACT-R/PM and menu selection: Applying a cognitive architecture to HCI. International Journal of Human-Computer Studies. 55, 41--84.]]

Digital Library

[7]

Cai, Z., McNamara, D. S., Louwerse, M., Hu, X., Rowe, M., & Graesser, A. C. (2004). NLS: A Non-Latent Similarity Algorithm. Proceedings of the 26th Annual Meeting of the Cognitive Science Society. 180--185.]]

[8]

Chakravarthy, A. S., & Haase, K. B. (1995). NetSerf: Using semantic knowledge to find Internet information archives. Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, 4--11.]]

Digital Library

[9]

Chi, E. H., Pirolli, P., Chen, K., Pitkow. J. (2001) Using Information Scent to Model User Information Needs and Actions on the Web. Proceedings of ACM CHI 2001: Conference on Human Factors in Computing Systems, 490--497.]]

Digital Library

[10]

Chi, E. H., Rosien, A., Suppattanasiri, G., Williams, A., Royer, C., Chow, C., Robles, E., Dalal, B., Chen, J., Cousins, S. (2003) The Bloodhound Project: Automating Discovery of Web Usability Issues using the InfoScent(tm) Simulator. Proceedings of ACM CHI 2003: Conference on Human Factors in Computing Systems.]]

Digital Library

[11]

Church, K, Gale, W., Hanks, P., Hindle, D. (1991) Using Statistics in Lexical Analysis, in Zernik (ed.) Lexical Acquisition: Exploiting OnLine Resources to Build a Lexicon, 115--164, Lawrence Erlbaum Associates Publishers.]]

[12]

Dumais, S. T., Furnas, G., Landauer, T. K., Deerwester, S., & Harshman, R. (1988). Using Latent Semantic Analysis to improve access to textual information. Proceedings of ACM CHI '98: Conference on Human Factors in Computing Systems, 281--285.]]

Digital Library

[13]

Furnas, G., Landauer, T. K., Gomez, L. M., & Dumais, S. T. (1987). The vocabulary problem in human-system communication. Communications of the ACM, 30(11), 964--971.]]

Digital Library

[14]

Hornof, A. J. (2004) Cognitive Strategies for the Visual Search of Hierarchical Computer Displays. Human Computer Interaction. 19, 183--223.]]

Digital Library

[15]

Jiang, J. J., & Conrath, D. W. (1997). Semantic similarity based on corpus statistics and lexical taxonomy. Proceedings of the Conference on Research in Computational Linguistics.]]

[16]

Landauer, T. K., & Dumais, S. T. (1997). A solution to the Plato's Problem: The Latent Semantic Analysis theory of acquisition, induction and representation of knowledge. Psychological Review, 104, 211--240.]]

[17]

Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An Introduction to Latent Semantic Analysis. Discourse Processes, 25, 259--284.]]

[18]

Leacock, C., & Chodorow, M. (1998). Combining local context and WordNet similarity for word sense identification. In C. Fellbaum (Ed.), WordNet: an electronic lexical database (pp. 265--283).]]

[19]

Lenat, D. (1995) CYC: A Large-Scale Investment in Knowledge Infrastructure, Communications of the ACM, 38(11), 33--38.]]

Digital Library

[20]

Lieberman, H. (1995) Letizia: An Agent That Assists Web Browsing. Proceedings of the International Joint Conference on Artificial Intelligence.]]

Digital Library

[21]

Lin, D. (1998). An information-theoretic definition of similarity. Proceedings of International Conference on Machine Learning.]]

Digital Library

[22]

List of colleges and universities accessed at http://www.clas.ufl.edu/CLAS/american-universities.html]]

[23]

Lohse, G. L. (1993). A cognitive model for understanding graphical perception. Human Computer Interaction, 8, 353--388.]]

Digital Library

[24]

Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instrumentation, and Computers, 28, 203--208.]]

[25]

Lynch, G., Palmiter, S., & Tilt. C. (1999). The Max Model: A standard web site user model, Human Factors and the Web, downloadable at http://zing.ncsl.nist.gov/hfweb/proceedings/lynch/]]

[26]

Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 38(11), 39--41.]]

Digital Library

[27]

Olston, C. & Chi, E. H. (2003). ScentTrails: Integrating Browsing and Searching on the Web. ACM Transactions on Computer Human Interaction. 10(3), 177--197.]]

Digital Library

[28]

Pierce, B. J., Parkinson, S. R., & Sisson, N. (1992). Effects of semantic similarity, omission probability and number of alternatives on computer menu search. International Journal of Man-Machine Studies, 37(5), 653--677.]]

Digital Library

[29]

Pirolli, P., & Card, S. K. (1999). Information Foraging. Psychological Review, 106, 643--675.]]

[30]

Resnik, P. (1999). Semantic similarity in a taxonomy: An information based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research, 95--113.]]

[31]

Richardson, R., & Smeaton, A. F. (1995). Using WordNet in a Knowledge-Based Approach to Information Retrieval. Working Paper.]]

[32]

Sense tagged corpus available at http://multisemcor.itc.it/semcor.html]]

[33]

Sutcliffe, R. F. E., Boersma, P., Bon, A., Donker, T., Ferris, M. C., Hellwig, P., et al. (1995). Beyond keywords: Accurate retrieval from full text documents. Proceedings of the 2nd Language Engineering Convention.]]

[34]

Terra, E., & Clarke, C. L. A. (2003). Frequency Estimates for Statistical Word Similarity Measures. Proceedings of Human Language Technology Conference. North American chapter of the Association for Computational Linguistics annual meeting, 244--251.]]

Digital Library

[35]

The Chronicle of Higher education http://chronicle.com/]]

[36]

Tullis, T. S. (1988). A system for evaluating screen formats: research and application. In R. Hartson & D. Hix (Eds.), Advances in Human Computer Interaction. 2, 214--286.]]

Digital Library

[37]

Turney, P. (2001). Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL. Proceedings of the Twelfth European Conference on Machine Learning, 491--502.]]

Digital Library

[38]

WordNet Similarity CPAN reference http://search.cpan.org/dist/WordNet-Similarity/]]

[39]

Wu, Z., & Palmer, M. (1994). Verb semantics and lexical selection. Proceedings of 32nd Annual Meeting of the Association for Computational Linguistics, 133--138.]]

Digital Library

Cited By

Abri FGutierrez LKulkarni CNamin AJones K(2021)Toward Explainable Users: Using NLP to Enable AI to Understand Users’ Perceptions of Cyber Attacks2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC51774.2021.00254(1703-1710)Online publication date: Jul-2021
https://doi.org/10.1109/COMPSAC51774.2021.00254
Ahmed RAhmad TAlmutairi FQahtani AAlsufyani AAlmutiry O(2021)Fuzzy Semantic Classification of Multi-Domain E-Learning ConceptMobile Networks and Applications10.1007/s11036-021-01776-8Online publication date: 17-May-2021
https://doi.org/10.1007/s11036-021-01776-8
Rawashdeh ARawashdeh MAllu A(2020)An Empirical Study of Semantic Mining of Scholarly Papers Using Wordnet APIProceedings of the Future Technologies Conference (FTC) 2020, Volume 110.1007/978-3-030-63128-4_54(712-737)Online publication date: 31-Oct-2020
https://doi.org/10.1007/978-3-030-63128-4_54
Show More Cited By

Index Terms

Recommendations

Robust semantic text similarity using LSA, machine learning, and linguistic resources

Semantic textual similarity is a measure of the degree of semantic equivalence between two pieces of text. We describe the SemSim system and its performance in the *SEM 2013 and SemEval-2014 tasks on semantic textual similarity. At the core of our ...
Word Sense Determination using WordNet and Sense Co-occurrence
AINA '06: Proceedings of the 20th International Conference on Advanced Information Networking and Applications - Volume 01

This paper presents a method of word sense disambiguation that assigns a target word the sense that is most related to the senses of its neighbor words. We explore the use of measures of relatedness between word senses based on a novel hybrid approach. ...
Data-driven synset induction and disambiguation for wordnet development

Automatic methods for wordnet development in languages other than English generally exploit information found in Princeton WordNet (PWN) and translations extracted from parallel corpora. A common approach consists in preserving the structure of PWN and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '05: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2005

928 pages

ISBN:1581139985

DOI:10.1145/1054972

Conference Chairs:
Wendy Kellogg
IBM T. J. Watson Research Center
,
Shumin Zhai
IBM Almaden Research Center
,
General Chair:
Gerrit van der Veer
Vrije Universiteit, The Netherlands
,
Program Chair:
Carolyn Gale
Stanford University, USA

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 April 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CHI05

Sponsor:

CHI05: CHI 2005 Conference on Human Factors in Computing Systems

April 2 - 7, 2005

Oregon, Portland, USA

Acceptance Rates

CHI '05 Paper Acceptance Rate 93 of 372 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
1,315
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Abri FGutierrez LKulkarni CNamin AJones K(2021)Toward Explainable Users: Using NLP to Enable AI to Understand Users’ Perceptions of Cyber Attacks2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC51774.2021.00254(1703-1710)Online publication date: Jul-2021
https://doi.org/10.1109/COMPSAC51774.2021.00254
Ahmed RAhmad TAlmutairi FQahtani AAlsufyani AAlmutiry O(2021)Fuzzy Semantic Classification of Multi-Domain E-Learning ConceptMobile Networks and Applications10.1007/s11036-021-01776-8Online publication date: 17-May-2021
https://doi.org/10.1007/s11036-021-01776-8
Rawashdeh ARawashdeh MAllu A(2020)An Empirical Study of Semantic Mining of Scholarly Papers Using Wordnet APIProceedings of the Future Technologies Conference (FTC) 2020, Volume 110.1007/978-3-030-63128-4_54(712-737)Online publication date: 31-Oct-2020
https://doi.org/10.1007/978-3-030-63128-4_54
Kang HGong Y(2017)Developing a similarity searching module for patient safety event reporting system using semantic similarity measuresBMC Medical Informatics and Decision Making10.1186/s12911-017-0467-817:S2Online publication date: 5-Jul-2017
https://doi.org/10.1186/s12911-017-0467-8
Sharma SLather JDave M(2016)Semantic approach for Web service classification using machine learning and measures of semantic relatednessService Oriented Computing and Applications10.1007/s11761-015-0182-110:3(221-231)Online publication date: 1-Sep-2016
https://dl.acm.org/doi/10.1007/s11761-015-0182-1
Pozniak JBradford R(2015)Optimization of Cross-Lingual LSI Training DataComputer and Information Science 201510.1007/978-3-319-23467-0_5(57-73)Online publication date: 17-Oct-2015
https://doi.org/10.1007/978-3-319-23467-0_5
Singer PChung CBroder AShim KSuel T(2014)Understanding, leveraging and improving human navigation on the webProceedings of the 23rd International Conference on World Wide Web10.1145/2567948.2567956(27-32)Online publication date: 7-Apr-2014
https://dl.acm.org/doi/10.1145/2567948.2567956
Rezaei MFränti P(2014)Matching Similarity for Keyword-Based ClusteringProceedings of the Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition - Volume 862110.1007/978-3-662-44415-3_20(193-202)Online publication date: 20-Aug-2014
https://dl.acm.org/doi/10.1007/978-3-662-44415-3_20
Han LFinin TMcNamee PJoshi AYesha Y(2013)Improving Word Similarity by Augmenting PMI with Estimates of Word PolysemyIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2012.3025:6(1307-1322)Online publication date: 1-Jun-2013
https://dl.acm.org/doi/10.1109/TKDE.2012.30
Narock TZhou LYoon V(2013)Semantic similarity of ontology instances using polarity miningJournal of the American Society for Information Science and Technology10.1002/asi.2276964:2(416-427)Online publication date: 1-Feb-2013
https://dl.acm.org/doi/10.1002/asi.22769
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents