research-article

Getting the Message?: A Study of Explanation Interfaces for Microblog Data Analysis

Authors:

James Schaffer,

Prasanna Giridhar,

Tobias Höllerer,

Tarek Abdelzaher,

John O'DonovanAuthors Info & Claims

IUI '15: Proceedings of the 20th International Conference on Intelligent User Interfaces

Pages 345 - 356

https://doi.org/10.1145/2678025.2701406

Published: 18 March 2015 Publication History

Abstract

In many of today's online applications that facilitate data exploration, results from information filters such as recommender systems are displayed alongside traditional search tools. However, the effect of prediction algorithms on users who are performing open-ended data exploration tasks through a search interface is not well understood. This paper describes a study of three interface variations of a tool for analyzing commuter traffic anomalies in the San Francisco Bay Area. The system supports novel interaction between a prediction algorithm and a human analyst, and is designed to explore the boundaries, limitations and synergies of both. The degree of explanation of underlying data and algorithmic process was varied experimentally across each interface. The experiment (N=197) was performed to assess the impact of algorithm transparency/explanation on data analysis tasks in terms of search success, general insight into the underlying data set and user experience. Results show that 1) presence of recommendations in the user interface produced a significant improvement in recall of anomalies, 2) participants were able to detect anomalies in the data that were missed by the algorithm, 3) participants who used the prediction algorithm performed significantly better when estimating quantities in the data, and 4) participants in the most explanatory condition were the least biased by the algorithm's predictions when estimating quantities.

References

[1]

Ahn, J.-w., and Brusilovsky, P. Adaptive visualization for exploratory information retrieval. Information Processing & Management 49, 5 (2013), 1139--1164.

Digital Library

[2]

Amar, R. A., and Stasko, J. T. Knowledge precepts for design and evaluation of information visualizations. Visualization and Computer Graphics, IEEE Transactions on 11, 4 (2005), 432--442.

Digital Library

[3]

Arnold, V., Clark, N., Collier, P. A., Leech, S. A., and Sutton, S. G. The differential use and effect of knowledge-based system explanations in novice and expert judgment decisions. Mis Quarterly (2006), 79--97.

Digital Library

[4]

Bilgic, M., and Mooney, R. J. Explaining recommendations: Satisfaction vs. promotion. In Beyond Personalization Workshop, IUI, vol. 5 (2005).

[5]

Bostandjiev, S., O'Donovan, J., and Höllerer, T. Tasteweights: a visual interactive hybrid recommender system. In RecSys, P. Cunningham, N. J. Hurley, I. Guy, and S. S. Anand, Eds., ACM (2012), 35--42.

Digital Library

[6]

Buhrmester, M., Kwang, T., and Gosling, S. D. Amazon's mechanical turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science 6, 1 (2011), 3--5.

[7]

Chang, R., Ziemkiewicz, C., Green, T. M., and Ribarsky, W. Defining insight for visual analytics. Computer Graphics and Applications, IEEE 29, 2 (2009), 14--17.

Digital Library

[8]

Daly, E. M., Lecue, F., and Bicer, V. Westland row why so slow?: fusing social media and linked data sources for understanding real-time traffic conditions. In Proceedings of the 2013 international conference on Intelligent user interfaces, ACM (2013), 203--212.

Digital Library

[9]

Fleder, D. M., and Hosanagar, K. Recommender systems and their impact on sales diversity. In Proceedings of the 8th ACM conference on Electronic commerce, ACM (2007), 192--199.

Digital Library

[10]

Giridhar, P., Amin, M. T. A., Abdelzaher, T. F., Kaplan, L. M., George, J., and Ganti, R. K. Clarisense: Clarifying sensor anomalies using social network feeds. In 2014 IEEE International Conference on Pervasive Computing and Communication Workshops, PerCom 2014 Workshops, Budapest, Hungary, March 24-28, 2014 (2014), 395--400.

[11]

Gratzl, S., Gehlenborg, N., Lex, A., Pfister, H., and Streit, M. Domino: Extracting, comparing, and manipulating subsets across multiple tabular datasets. Visualization and Computer Graphics, IEEE Transactions on 20, 12 (Dec 2014), 2023--2032.

[12]

Gregor, S., and Benbasat, I. Explanations from intelligent systems: Theoretical foundations and implications for practice. MIS quarterly (1999), 497--530.

Digital Library

[13]

Herlocker, J. L., Konstan, J. A., and Riedl, J. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work, CSCW '00, ACM (New York, NY, USA, 2000), 241--250.

Digital Library

[14]

Herlocker, J. L., Konstan, J. A., and Riedl, J. Explaining collaborative filtering recommendations. In Proceedings of ACM CSCW'00 Conference on Computer-Supported Cooperative Work (2000), 241--250.

Digital Library

[15]

Knijnenburg, B. P., Bostandjiev, S., O'Donovan, J., and Kobsa, A. Inspectability and control in social recommenders. In Proceedings of the sixth ACM conference on Recommender systems, ACM (2012), 43--50.

Digital Library

[16]

McCarthy, K., Reilly, J., McGinty, L., and Smyth, B. Experiments in dynamic critiquing. In Proceedings of the 10th International Conference on Intelligent User Interfaces, IUI '05, ACM (New York, NY, USA, 2005), 175--182.

Digital Library

[17]

North, C. Toward measuring visualization insight. Computer Graphics and Applications, IEEE 26, 3 (2006), 6--9.

Digital Library

[18]

O'Donovan, J., Smyth, B., Gretarsson, B., Bostandjiev, S., and Höllerer, T. Peerchooser: visual interactive recommendation. In CHI '08: Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, ACM (New York, NY, USA, 2008), 1085--1088.

Digital Library

[19]

Oppenheimer, D. M., Meyvis, T., and Davidenko, N. Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology 45, 4 (2009), 867--872.

[20]

Paolacci, G., Chandler, J., and Ipeirotis, P. G. Running experiments on amazon mechanical turk. Judgment and Decision Making 5 (2010), 411--419.

[21]

Parra, D., Brusilovsky, P., and Trattner, C. See what you want to see: visual user-driven approach for hybrid recommendation. In IUI'14 19th International Conference on Intelligent User Interfaces, IUI'14, Haifa, Israel, February 24-27, 2014 (2014), 235--240.

Digital Library

[22]

Ren, D., Hollerer, T., and Yuan, X. ivisdesigner: Expressive interactive design of information visualizations. Visualization and Computer Graphics, IEEE Transactions on 20, 12 (Dec 2014), 2092--2101.

[23]

Schaffer, J., Abdelzaher, T., Jones, D., Hollerer, T., Gonzalez, C., Harman, J., and O'Donovan, J. Truth, lies, and data: Credibility representation in data analysis. In Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), 2014 IEEE International Inter-Disciplinary Conference on, IEEE (2014), 28--34.

[24]

Sinha, R., and Swearingen, K. The role of transparency in recommender systems. In CHI'02 extended abstracts on Human factors in computing systems, ACM (2002), 830--831.

Digital Library

[25]

Tintarev, N., and Masthoff, J. A survey of explanations in recommender systems. In Data Engineering Workshop, 2007 IEEE 23rd International Conference on, IEEE (2007), 801--810.

Digital Library

[26]

Tintarev, N., O'Donovan, J., Brusilovsky, P., Felfernig, A., Semeraro, G., and Lops, P. Recsys' 14 joint workshop on interfaces and human decision making for recommender systems. In Proceedings of the 8th ACM Conference on Recommender systems, ACM (2014), 383--384.

Digital Library

[27]

Yi, J. S., ah Kang, Y., Stasko, J. T., and Jacko, J. A. Toward a deeper understanding of the role of interaction in information visualization. Visualization and Computer Graphics, IEEE Transactions on 13, 6 (2007), 1224--1231.

Digital Library

[28]

Yi, J. S., Kang, Y.-a., Stasko, J. T., and Jacko, J. A. Understanding and characterizing insights: how do people gain insights using information visualization? In Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization, ACM (2008), 4.

Digital Library

[29]

Zhang, J., and Pu, P. A comparative study of compound critique generation in conversational recommender systems. In In Proceedings of the 4th International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH2006, Springer (2006), 234--243.

Digital Library

[30]

Zhang, J., Wang, Y., and Vassileva, J. Socconnect: A personalized social network aggregator and recommender. Information Processing and Management 49, 3 (2013), 721--737.

Digital Library

Cited By

Amin RHammer PButz A(2024)Using Machine Learning to Improve Interactive Visualizations for Large Collected Traffic Detector DataProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645177(803-816)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645177
Cappuccio EFadda DLanzilotti RRinzivillo S(2023)Demo: an Interactive Visualization Combining Rule-Based and Feature Importance ExplanationsProceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter10.1145/3605390.3610811(1-4)Online publication date: 20-Sep-2023
https://dl.acm.org/doi/10.1145/3605390.3610811
Brachman MPan QDo HDugan CChaudhary AJohnson JRai PChakraborti TGschwind TLaredo JMiksovic CScotton PTalamadupula KThomas G(2023)Follow the Successful Herd: Towards Explanations for Improved Use and Mental Models of Natural Language SystemsProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584088(220-239)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584088
Show More Cited By

Index Terms

Getting the Message?: A Study of Explanation Interfaces for Microblog Data Analysis
1. Human-centered computing

Recommendations

Exploring a Design Space of Graphical Adaptive Menus: Normal vs. Small Screens
Special Issue on IUI 2018

Graphical Adaptive Menus are Graphical User Interface menus whose predicted items of immediate use can be automatically rendered in a prediction window. Rendering this prediction window is a key question for adaptivity to enable the end-user to ...
User Interfaces for Counteracting Decision Manipulation in Group Recommender Systems
UMAP'19 Adjunct: Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization

In group recommender systems,decision manipulation refers to an attack in which a group member makes attempts to push his/her favorite options. In this paper, we propose user interfaces to counteract decision manipulation in group recommender systems. ...
Towards the design of user-centric strategy recommendation systems for collaborative Human–AI tasks
Abstract
Artificial Intelligence is being employed by humans to collaboratively solve complicated tasks for search and rescue, manufacturing, etc. Efficient teamwork can be achieved by understanding user preferences and recommending different strategies ...
Graphical abstract

Display Omitted
Highlights
- We study Strategy Recommendation Systems across modalities: single, similar, diverse, all.
- Single strategy recommendations are more useful than similar strategy recommendations.
- Agreeable participants experience higher workloads.

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '15: Proceedings of the 20th International Conference on Intelligent User Interfaces

March 2015

480 pages

ISBN:9781450333061

DOI:10.1145/2678025

General Chairs:
Oliver Brdiczka
Vectra Networks, Inc.
,
Polo Chau
Georgia Tech
,
Program Chairs:
Giuseppe Carenini
University of British Columbia
,
Shimei Pan
University of Maryland
,
Per Ola Kristensson
University of Cambridge

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 March 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

U.S. Army Research Laboratory

Conference

IUI'15

Sponsor:

IUI'15: IUI'15 20th International Conference on Intelligent User Interfaces

March 29 - April 1, 2015

Georgia, Atlanta, USA

Acceptance Rates

IUI '15 Paper Acceptance Rate 47 of 205 submissions, 23%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
391
Total Downloads

Downloads (Last 12 months)34
Downloads (Last 6 weeks)4

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Amin RHammer PButz A(2024)Using Machine Learning to Improve Interactive Visualizations for Large Collected Traffic Detector DataProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645177(803-816)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3640543.3645177
Cappuccio EFadda DLanzilotti RRinzivillo S(2023)Demo: an Interactive Visualization Combining Rule-Based and Feature Importance ExplanationsProceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter10.1145/3605390.3610811(1-4)Online publication date: 20-Sep-2023
https://dl.acm.org/doi/10.1145/3605390.3610811
Brachman MPan QDo HDugan CChaudhary AJohnson JRai PChakraborti TGschwind TLaredo JMiksovic CScotton PTalamadupula KThomas G(2023)Follow the Successful Herd: Towards Explanations for Improved Use and Mental Models of Natural Language SystemsProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584088(220-239)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584088
Ma SSun MMa X(2023)Modeling Adaptive Expression of Robot Learning Engagement and Exploring Its Effects on Human TeachersACM Transactions on Computer-Human Interaction10.1145/357181330:5(1-48)Online publication date: 23-Sep-2023
https://dl.acm.org/doi/10.1145/3571813
Bertrand AViard TBelloum REagan JMaxwell W(2023)On Selective, Mutable and Dialogic XAI: a Review of What Users Say about Different Types of Interactive ExplanationsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581314(1-21)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581314
Shang RFeng KShah C(2022)Why Am I Not Seeing It? Understanding Users’ Needs for Counterfactual Explanations in Everyday RecommendationsProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency10.1145/3531146.3533189(1330-1340)Online publication date: 21-Jun-2022
https://dl.acm.org/doi/10.1145/3531146.3533189
Quispe G DRajabiyazdi FJamieson G(2020)A Machine Learning-Based Micro-World Platform for Condition-Based Maintenance2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC42975.2020.9283448(288-295)Online publication date: 11-Oct-2020
https://doi.org/10.1109/SMC42975.2020.9283448
Naiseh MJiang NMa JAli R(2020)Explainable Recommendations in Intelligent Systems: Delivery Methods, Modalities and RisksResearch Challenges in Information Science10.1007/978-3-030-50316-1_13(212-228)Online publication date: 25-Jun-2020
https://doi.org/10.1007/978-3-030-50316-1_13
Schneider H(2019)Designing for empowerment – An investigation and critical reflectionit - Information Technology10.1515/itit-2018-003661:1(59-65)Online publication date: 4-Jan-2019
https://doi.org/10.1515/itit-2018-0036
Tsai CBrusilovsky PFu WPan SBrdiczka OChau PCalvary G(2019)Explaining recommendations in an interactive hybrid social recommenderProceedings of the 24th International Conference on Intelligent User Interfaces10.1145/3301275.3302318(391-396)Online publication date: 17-Mar-2019
https://dl.acm.org/doi/10.1145/3301275.3302318
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents