Article

Evaluating example-based search tools

Authors:

Pearl Huan Z. Pu,

Pratyush KumarAuthors Info & Claims

EC '04: Proceedings of the 5th ACM conference on Electronic commerce

Pages 208 - 217

https://doi.org/10.1145/988772.988804

Published: 17 May 2004 Publication History

Abstract

A crucial element in consumer electronic commerce is a catalog tool that not only finds the product for the user, but also convinces him that he has made the best choice. To do that, it is important to show him ample choices while keeping his interaction effort below an acceptable limit. Among the various interaction models used in operational e-commerce sites, ranked lists are by far the most popular tool for product navigation and selection. However, as the number of product features and the complexity of user's criteria increase, a ranked list's efficiency becomes less satisfactory. As an alternative, research groups from the intelligent user interface community have developed various example-based search tools, including SmartClient from our laboratory. These tools not only perform personalized search, but also support tradeoff analysis. However, despite the academic interest, example-based search paradigms have not been widely adopted in practice. We have examined the usability of such tools on a variety of tasks involving selection and tradeoff. The studies clearly show that example-based search is comparable to ranked lists on simple tasks, but significantly reduces the error rate and search time when complex tradeoffs are involved. This shows that such tools are likely to be useful particularly for extending the scope of consumer e-commerce to more complex products.

References

[1]

Bistarelli, S., Montanari, U., and Rossi, F. Semiring-based Constraint Solving and Optimization. Journal of the ACM, 44, 2 (Mar. 1997), 201--236.

Digital Library

[2]

Burke, R., Hammond, K., and Young, B. The FindMe Approach to Assisted Browsing. In IEEE Expert: Intelligent Systems and Their Applications, 12, 4 (Jul. 1997), 32--40.

Digital Library

[3]

Carenini G. and Poole D. Constructed Preferences and Value-focused Thinking: Implications for AI research on Preference Elicitation. AAAI-02 Workshop on Preferences in AI and CP: symbolic approaches, Edmonton, Canada, 2002.

[4]

L.F. Cranor, J. Reagle, and M.S. Ackerman. Beyond Concern: Understanding Net Users' Attitudes About Online Privacy. In Ingo Vogelsang and Benjamin M. Compaine (editors). The Internet Upheaval: Raising Questions, Seeking Answers in Communications Policy. The MIT Press, Cambridge, Massachusetts, 2000, 47--70.

[5]

Faltings, B., Torrens, M., and Pu, P. Solution Generation with Qualitative Models of Preferences, International Journal of Computational Intelligence and Applications, 2004. To appear in 2004.

[6]

Fink, J., and Kobsa, A. A Review and Analysis of Commercial User Modeling Servers for Personalization on the World Wide Web. User Modeling and User-Adapted Interaction, 10, 3-4 (2000), 209-249.

Digital Library

[7]

Keeney, R. Value-Focused Thinking: A Path to Creative Decision Making. Harvard University Press, 1992.

[8]

Linden, G., Hanks, S., and Lesh, N. Interactive assessment of user preference models: The automated travel assistant. Proceedings of User Modeling '97, 1997, 67--78.

[9]

Nguyen, H., and Haddawy, P. The Decision-Theoretic Video Advisor. In workshop notes, Recommender Systems, the Fifteenth National Conference on Artificial Intelligence (AAAI'98), 77--80.

[10]

Payne, J.W., Bettman, J.R., and Johnson, E.J., The Adaptive Decision Maker. Cambridge University Press, 1993.

[11]

Payne, J.W., Bettman, J.R., and Schkade, D.A. Measuring Constructed Preference: Towards a Building Code. Journal of Risk and Uncertainty, 19, 1-3 (1999), 243--270.

[12]

Pu, P., and Faltings, B. Enriching Buyers' experiences: the SmartClient Approach. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '00) (The Hague, The Netherlands, April 1-6, 2000). ACM Press, New York, NY, 2000, 289--296.

Digital Library

[13]

Pu, P., Faltings, B., and Torrens, M. User-Involved Preference Elicitation. In workshop notes, workshop on Configuration, the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI'03), 2003.

[14]

Pu, P., Kumar, P., and Faltings, B. User-Involved Tradeoff Analysis in Configuration Tasks. In workshop notes, the Third International Workshop on User-Interaction in Constraint Satisfaction, Ninth International Conference on Principles and Practice of Constraint Programming (CP2003).

[15]

Pu, P. and Faltings, B. User Preference Elicitation Via Tradeoff. Submitted to the International Journal of Constraints, December, 2003.

[16]

Shearin, S., and Lieberman, H. Intelligent Profiling by Example. Proceedings of the 6th international Conference on Intelligent User Interfaces (Santa Fe, New Mexico, USA, 2001) ACM Press, New York, NY, 2001, 145--151.

Digital Library

[17]

Shimazu, H. ExpertClerk: Navigating Shoppers' Buying Process with the Combination of Asking and Proposing. In Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI'01) (Seattle, Washington, USA, August 4-10, 2001).

Digital Library

[18]

S. Spiekermann and C. Paraschiv. Motivating Human-Agent Interaction: Transferring Insights from Behavioral Marketing to Agent Design. Electronic Commerce Research, Kluwer Academic Publishers, the Netherlands, (2002), 255--285.

Digital Library

[19]

Stolze, M. Comparative Study of Analytical Product Selection Support Mechanisms, Proceedings of INTERACT 99, Edinborough, UK, August 30 - September 3, 1999.

[20]

M. Torrens, R. Weigel and B. Faltings. Java Constraint Library: bringing constraints technology on the Internet using the Java language. In workshop notes, Constraints and Agents, the Fourteenth National Conference on Artificial Intelligence (AAAI'97), July 1997.

[21]

M. Torrens and B. Faltings. SmartClients: Constraint satisfaction as a paradigm for scaleable intelligent information systems. In workshop notes, Artificial Intelligence for Electronic Commerce, the Sixteenth National Conference on Artificial Intelligence (AAAI'99), July 1999, 10--15.

[22]

Torrens, M., Faltings, B., and Pu, P., SmartClients: Constraint Satisfaction as a Paradigm for Scaleable Intelligent Information Systems. International Journal of Constraints, 7, 1 (Jan. 2002), 49--69.

Digital Library

Cited By

Rana ASanner SBouadjenek MDicarlantonio RFarmaner G(2023)User Experience and The Role of Personalization in Critiquing-Based Conversational RecommendationACM Transactions on the Web10.1145/3597499Online publication date: 18-May-2023
https://dl.acm.org/doi/10.1145/3597499
Rana ABridge DPaternò FOliver NConati CSpano LTintarev N(2020)Navigation-by-preferenceProceedings of the 25th International Conference on Intelligent User Interfaces10.1145/3377325.3377496(155-165)Online publication date: 17-Mar-2020
https://dl.acm.org/doi/10.1145/3377325.3377496
Holeňa MPulc PKopp MHoleňa MPulc PKopp M(2020)Important Internet Applications of ClassificationClassification Methods for Internet Applications10.1007/978-3-030-36962-0_1(1-68)Online publication date: 30-Jan-2020
https://doi.org/10.1007/978-3-030-36962-0_1
Show More Cited By

Index Terms

Evaluating example-based search tools
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
  2. Interaction design
    1. Interaction design process and methods
      1. User centered design

Recommendations

Evaluating leading web search engines on children's queries
HCII'11: Proceedings of the 14th international conference on Human-computer interaction: users and applications - Volume Part IV

This study compared retrieved results, relevance ranking, and overlap across Google, Yahoo!, Bing, Yahoo Kids!, and Ask Kids on 15 queries constructed by middle school children. Queries included one word, two words, and multiple words/phrases/natural ...
Evaluating the performance and neutrality/bias of search engines
VALUETOOLS 2019: Proceedings of the 12th EAI International Conference on Performance Evaluation Methodologies and Tools

Different search engines provide different outputs for the same keyword. This may be due to different definitions of relevance, to different ranking aggregation methods, and/or to different knowledge/anticipation of users' preferences, but rankings are ...
Implementing and evaluating phrasal query suggestions for proximity search

This paper describes and evaluates a unified approach to phrasal query suggestions in the context of a high-precision search engine. The search engine performs ranked extended-Boolean searches with the proximity operator near being the default ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

EC '04: Proceedings of the 5th ACM conference on Electronic commerce

May 2004

278 pages

ISBN:1581137710

DOI:10.1145/988772

General Chair:
Jack Breese
Microsoft
,
Program Chairs:
Joan Feigenbaum
Yale University
,
Margo Seltzer
Harvard University

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 May 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

EC04

Sponsor:

EC04: Fifth ACM Conference on Electronic Commerce 2004 ( held in conjunction with WWW'04: The 2004 World Wide Web Conference )

May 17 - 20, 2004

NY, New York, USA

Acceptance Rates

EC '04 Paper Acceptance Rate 24 of 146 submissions, 16%;

Overall Acceptance Rate 664 of 2,389 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

52
Total Citations
View Citations
813
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rana ASanner SBouadjenek MDicarlantonio RFarmaner G(2023)User Experience and The Role of Personalization in Critiquing-Based Conversational RecommendationACM Transactions on the Web10.1145/3597499Online publication date: 18-May-2023
https://dl.acm.org/doi/10.1145/3597499
Rana ABridge DPaternò FOliver NConati CSpano LTintarev N(2020)Navigation-by-preferenceProceedings of the 25th International Conference on Intelligent User Interfaces10.1145/3377325.3377496(155-165)Online publication date: 17-Mar-2020
https://dl.acm.org/doi/10.1145/3377325.3377496
Holeňa MPulc PKopp MHoleňa MPulc PKopp M(2020)Important Internet Applications of ClassificationClassification Methods for Internet Applications10.1007/978-3-030-36962-0_1(1-68)Online publication date: 30-Jan-2020
https://doi.org/10.1007/978-3-030-36962-0_1
Kern Dvan Hoek WHienert DBratteteig TSandnes FBerget G(2018)Evaluation of a search interface for preference-based rankingProceedings of the 10th Nordic Conference on Human-Computer Interaction10.1145/3240167.3240170(184-194)Online publication date: 29-Sep-2018
https://dl.acm.org/doi/10.1145/3240167.3240170
Huaiqing HHongrui DHaohan L(2016)Overview and Investigation of the Visualization Methods in Recommendation Systems2016 8th International Conference on Computational Intelligence and Communication Networks (CICN)10.1109/CICN.2016.123(601-605)Online publication date: Dec-2016
https://doi.org/10.1109/CICN.2016.123
Grasch PFelfernig AReinfrank FYang QKing ILi QPu PKarypis G(2013)ReCommentProceedings of the 7th ACM conference on Recommender systems10.1145/2507157.2507161(157-164)Online publication date: 12-Oct-2013
https://dl.acm.org/doi/10.1145/2507157.2507161
Oku KKotera RKitayama DSumiya K(2012)Geographical Recommender System Using User Interest Model Based on Map Operation and Category SelectionInternational Journal of Handheld Computing Research10.4018/jhcr.20120701013:3(1-16)Online publication date: 1-Jul-2012
https://dl.acm.org/doi/10.4018/jhcr.2012070101
Pu PChen LHu R(2012)Evaluating recommender systems from the user's perspectiveUser Modeling and User-Adapted Interaction10.1007/s11257-011-9115-722:4-5(317-355)Online publication date: 1-Oct-2012
https://dl.acm.org/doi/10.1007/s11257-011-9115-7
Chen LPu P(2012)Critiquing-based recommendersUser Modeling and User-Adapted Interaction10.1007/s11257-011-9108-622:1-2(125-150)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.1007/s11257-011-9108-6
Knijnenburg BWillemsen MMynatt ESchoner DFitzpatrick GHudson SEdwards KRodden T(2010)The effect of preference elicitation methods on the user experience of a recommender systemCHI '10 Extended Abstracts on Human Factors in Computing Systems10.1145/1753846.1754001(3457-3462)Online publication date: 9-Apr-2010
https://dl.acm.org/doi/10.1145/1753846.1754001
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents