research-article

Surfacing Visualization Mirages

Authors:

Gordon Kindlmann,

Michael CorrellAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 16

https://doi.org/10.1145/3313831.3376420

Published: 23 April 2020 Publication History

Abstract

Dirty data and deceptive design practices can undermine, invert, or invalidate the purported messages of charts and graphs. These failures can arise silently: a conclusion derived from a particular visualization may look plausible unless the analyst looks closer and discovers an issue with the backing data, visual specification, or their own assumptions. We term such silent but significant failures . We describe a conceptual model of mirages and show how they can be generated at every stage of the visual analytics process. We adapt a methodology from software testing, as a way of automatically surfacing potential mirages at the visual encoding stage of analysis through modifications to the underlying data and chart specification. We show that metamorphic testing can reliably identify mirages across a variety of chart types with relatively little prior knowledge of the data or the domain.

Supplementary Material

ZIP File (pn4636aux.zip)

A pdf supplement containing an expanded table of mirage types and figures detailing our metamorphic operations.

Download
216.55 KB

MP4 File (a293-mcnutt-presentation.mp4)

Download
42.94 MB

References

[1]

Anushka Anand and Justin Talbot. 2015. Automatic Selection of Partitioning Variables for Small Multiple Displays. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2015), 669--677.

Digital Library

[2]

Anonymous. 2019. Glitchart: When charts attack. https://glitch-chart.tumblr.com/. (2019). Accessed: 2019-08--13.

[3]

Zan Armstrong and Martin Wattenberg. 2014. Visualizing Statistical Mix Effects and Simpson's Paradox. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2132--2141.

[4]

Daniel W. Barowy, Emery D. Berger, and Benjamin Zorn. 2018. ExceLint: Automatically Finding Spreadsheet Formula Errors. Proceedings of ACM Programming Languages 2, OOPSLA, Article 148 (Oct. 2018), 26 pages.

Digital Library

[5]

Daniel W Barowy, Dimitar Gochev, and Emery D Berger. 2014. CheckCell: Data Debugging for Spreadsheets. Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications OOPSLA 49, 10 (2014), 507--523.

Digital Library

[6]

Earl T Barr, Mark Harman, Phil McMinn, Muzammil Shahbaz, and Shin Yoo. 2014. The Oracle Problem in Software Testing: A Survey. IEEE Transactions on Software Engineering 41, 5 (2014), 507--525.

Digital Library

[7]

Carsten Binnig, Lorenzo De Stefani, Tim Kraska, Eli Upfal, Emanuel Zgraggen, and Zheguang Zhao. 2017. Toward Sustainable Insights, or Why Polygamy is Bad for You. In CIDR 8th Biennial Conference on Innovative Data Systems Research. www.cidrdb.org. http://cidrdb.org/cidr2017/index.html

[8]

David Borland, Wenyuan Wang, and David Gotz. 2018. Contextual Visualization. IEEE Computer Graphics and Applications 38, 6 (2018), 17--23.

Digital Library

[9]

Mike Brachmann, Carlos Bautista, Sonia Castelo, Su Feng, Juliana Freire, Boris Glavic, Oliver Kennedy, Heiko Müeller, Rémi Rampin, William Spoth, and others. 2019. Data Debugging and Exploration with Vizier. In Proceedings of the 2019 International Conference on Management of Data. ACM, 1877--1880.

Digital Library

[10]

Sabrina Bresciani and Martin J Eppler. 2009. The Risks of Visualization. Identität und Vielfalt der Kommunikations-wissenschaft (2009), 165--178.

[11]

Sabrina Bresciani and Martin J Eppler. 2015. The Pitfalls of Visual Representations: A Review and Classification of Common Errors Made While Designing and Interpreting Visualizations. Sage Open 5, 4 (2015).

[12]

Alberto Cairo. 2015. Graphics Lies, Misleading Visuals. In New Challenges for Data Design. Springer, 103--116.

[13]

Alberto Cairo. 2019. How Charts Lie. WW Norton & Company.

[14]

Ed Huai-hsin Chi. 2000. A Taxonomy of Visualization Techniques Using the Data State Reference Model. In IEEE Symposium on Information Visualization 2000. INFOVIS 2000. Proceedings. IEEE, 69--75.

[15]

Charisee Chiw, Gordon Kindlmann, and John Reppy. 2017. DATm: Diderot's Automated Testing Model. In IEEE/ACM 12th International Workshop on Automation of Software Testing (AST). IEEE, 45--51.

Digital Library

[16]

William S Cleveland, Persi Diaconis, and Robert McGill. 1982. Variables on Scatterplots Look More Highly Correlated When the Scales are Increased. Science 216, 4550 (1982), 1138--1141.

[17]

Andy Cockburn, Carl Gutwin, and Alan Dix. 2018. Hark No More: on the Preregistration of CHI Experiments. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 141.

Digital Library

[18]

Michael Correll. 2019. Ethical Dimensions of Visualization Research. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 188.

Digital Library

[19]

Michael Correll, Enrico Bertini, and Steven Franconeri. 2019. Truncating the Y-Axis: Threat or Menace? CoRR abs/1907.02035 (2019). http://arxiv.org/abs/1907.02035

[20]

Michael Correll and Jeffrey Heer. 2016. Surprise! Bayesian Weighting for De-Biasing Thematic Maps. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 651--660.

Digital Library

[21]

Michael Correll and Jeffrey Heer. 2017. Black Hat Visualization. In Workshop on Dealing with Cognitive Biases in Visualisations (DECISIVe), IEEE VIS.

[22]

Michael Correll, Mingwei Li, Gordon Kindlmann, and Carlos Scheidegger. 2018. Looks Good To Me: Visualizations As Sanity Checks. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 830--839.

Digital Library

[23]

Alexandra Diehl, Alfie Abdul-Rahman, Mennatallah El-Assady, Benjamin Bach, Daniel Keim, and Min Chen. 2018. VisGuides: A Forum for Discussing Visualization Guidelines. In Eurographics Conference on Visualization, EuroVis 2018, Short Papers. Eurographics Association, 61--65.

[24]

Catherine D'Ignazio and Lauren Klein. 2016. Feminist Data Visualization. In IEEE VIS: Workshop on Visualization for the Digital Humanities (VIS4DH).

[25]

Catherine D'Ignazio and Lauren Klein. 2019. Data Feminism. MIT Press. 2018 Draft.

[26]

Evanthia Dimara, Steven Franconeri, Catherine Plaisant, Anastasia Bezerianos, and Pierre Dragicevic. 2018. A Task-based Taxonomy of Cognitive Biases for Information Visualization. IEEE Transactions on Visualization and Computer Graphics (2018).

[27]

Alastair F. Donaldson, Hugues Evrard, Andrei Lascu, and Paul Thomson. 2017. Automated Testing of Graphics Shader Compilers. Proceedings of ACM Programming Languages 1, OOPSLA (Oct. 2017), 93:1--93:29.

Digital Library

[28]

Marian Dörk, Patrick Feng, Christopher Collins, and Sheelagh Carpendale. 2013. Critical InfoVis: exploring the politics of visualization. In 2013 ACM SIGCHI Conference on Human Factors in Computing Systems, Extended Abstracts. ACM, 2189--2198.

Digital Library

[29]

Pierre Dragicevic, Yvonne Jansen, Abhraneel Sarma, Matthew Kay, and Fanny Chevalier. 2019. Increasing the Transparency of Research Papers with Explorable Multiverse Analyses. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 65.

Digital Library

[30]

Bradley Efron. 1992. Bootstrap Methods: Another Look at the Jackknife. In Breakthroughs in statistics. Springer, 569--593.

[31]

Stephen Few. 2019. The Data Loom: Weaving Understanding by Thinking Critically and Scientifically with Data. Analytics Press.

[32]

Velitchko Andreev Filipov, Alessio Arleo, Paolo Federico, and Silvia Miksch. 2019. CV3: Visual Exploration, Assessment, and Comparison of CVs. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 107--118.

[33]

Brian Ford. 2014. write-good: Naive linter for English prose. https://github.com/btford/write-good. (2014). Accessed: 2018-06--28.

[34]

Andrew Gelman and Eric Loken. 2013. The garden of forking paths: Why multiple comparisons can be a problem, even when there is no "fishing expedition" or "p-hacking" and the research hypothesis was posited ahead of time. Department of Statistics, Columbia University (2013).

[35]

David Gotz, Wenyuan Wang, Annie T Chen, and David Borland. 2019. Visualization Model Validation via Inline Replication. Information Visualization 18, 4 (2019).

[36]

World Bank Group. 2019. World Development Indicators. http://datatopics.worldbank.org/ world-development-indicators/. (2019).

[37]

Ralph Guderlei and Johannes Mayer. 2007. Statistical Metamorphic Testing Testing Programs with Random Output by Means of Statistical Hypothesis Tests and Metamorphic Testing. In Seventh International Conference on Quality Software. IEEE, 404--409.

[38]

Yue Guo, Carsten Binnig, and Tim Kraska. 2017. What you see is not what you get!: Detecting Simpson's Paradoxes during Data Exploration. In ACM SIGMOD Workshop on Human-In-the-Loop Data Analytics (HILDA). ACM, 2:1--2:5.

Digital Library

[39]

Donna Haraway. 1988. Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective. Feminist studies 14, 3 (1988), 575--599.

[40]

Jeffrey Heer. 2019a. Agency plus automation: Designing artificial intelligence into interactive systems. Proceedings of the National Academy of Sciences 116, 6 (2019), 1844--1850.

[41]

Jeffery Heer. 2019b. Visualization is Not Enough. (2019). https://homes.cs.washington.edu/~jheer/talks/ EuroVis2019-Capstone.pdf EuroVis Capstone.

[42]

Jeffrey Heer and Maneesh Agrawala. 2006. Multi-Scale Banking to 45 Degrees. IEEE Transactions on Visualization and Computer Graphics 12, 5 (2006), 701--708.

Digital Library

[43]

William L Hibbard, Charles R Dyer, and Brian E Paul. 1994. A Lattice Model for Data Display. In Proceedings of the Conference on Visualization. IEEE, 310--317.

[44]

Heike Hofmann, Lendie Follett, Mahbubul Majumder, and Dianne Cook. 2012. Graphical Tests for Power Comparison of Competing Designs. IEEE Transactions on Visualization and Computer Graphics 18, 12 (2012), 2441--2448.

Digital Library

[45]

Darrell Huff. 1993. How to Lie with Statistics. WW Norton & Company.

Digital Library

[46]

Jessica Hullman and Nick Diakopoulos. 2011. Visualization Rhetoric: Framing Effects in Narrative Visualization. IEEE Transactions on Visualization and Computer Graphics 17, 12 (2011), 2231--2240.

Digital Library

[47]

Nick Hynes, D Sculley, and Michael Terry. 2017. The Data Linter: Lightweight, Automated Sanity Checking for ML Data Sets. In NIPS: Workshop on Systems for ML and Open Source Software.

[48]

Tobias Isenberg, Petra Isenberg, Jian Chen, Michael Sedlmair, and Torsten Möller. 2013. A Systematic Review on the Practice of Evaluating Visualization. IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2818--2827.

Digital Library

[49]

Hassan M Jannah. 2014. MetaReader: A Dataset Meta-Exploration and Documentation Tool. https://github.com/jannah/MetaReader. (2014). Accessed: 2019-08--13.

[50]

Stephen C Johnson. 1977. Lint, a C Program Checker. Citeseer.

[51]

Sean Kandel, Jeffrey Heer, Catherine Plaisant, Jessie Kennedy, Frank Van Ham, Nathalie Henry Riche, Chris Weaver, Bongshin Lee, Dominique Brodbeck, and Paolo Buono. 2011. Research directions in data wrangling: Visualizations and transformations for usable and credible data. Information Visualization 10, 4 (2011), 271--288.

Digital Library

[52]

Sean Kandel, Ravi Parikh, Andreas Paepcke, Joseph M Hellerstein, and Jeffrey Heer. 2012. Profiler: Integrated Statistical Analysis and Visualization for Data Quality Assessment. In Proceedings of the International Working Conference on Advanced Visual Interfaces. ACM, 547--554.

Digital Library

[53]

Won Kim, Byoung-Ju Choi, Eui-Kyeong Hong, Soo-Kyung Kim, and Doheon Lee. 2003. A Taxonomy of Dirty Data. Data Mining and Knowledge Discovery 7, 1 (2003), 81--99.

Digital Library

[54]

Gordon Kindlmann and Carlos Scheidegger. 2014. An Algebraic Process for Visualization Design. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2181--2190.

[55]

Gordon Kindlmann and Carlos Scheidegger. 2016. Algebraic Visualization Design for Pedagogy. IEEE VIS Workshop on Pedagogy of Data Visualization. (Oct. 2016).

[56]

Robert M Kirby and Cláudio T Silva. 2008. The Need for Verifiable Visualization. IEEE Computer Graphics and Applications 28, 5 (2008), 78--83.

Digital Library

[57]

Ha-Kyung Kong, Zhicheng Liu, and Karrie Karahalios. 2018. Frames and Slants in Titles of Visualizations on Controversial Topics. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, ACM, 438.

Digital Library

[58]

Ha-Kyung Kong, Zhicheng Liu, and Karrie Karahalios. 2019. Trust and Recall of Information across Varying Degrees of Title-Visualization Misalignment. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 346.

Digital Library

[59]

Amperser Labs. 2019. proselint: A linter for prose. http://proselint.com/. (2019). Accessed: 2019-08-01.

[60]

Sam Lavigne, Brian Clifton, and Francis Tseng. 2017. Predicting Financial Crime: Augmenting the Predictive Policing Arsenal. CoRR abs/1704.07826 (2017). http://arxiv.org/abs/1704.07826

[61]

Doris Jung Lin Lee, Himel Dev, Huizi Hu, Hazem Elmeleegy, and Aditya G Parameswaran. 2019. Avoiding Drill-down Fallacies with VisPilot: Assisted Exploration of Data Subsets. In Proceedings of the 24th International Conference on Intelligent User Interfaces IUI. ACM, 186--196.

Digital Library

[62]

Alan Lundgard, Crystal Lee, and Arvind Satyanarayan. 2019. Sociotechnical Considerations for Accessible Visualization Design. In IEEE VGTC Conference on Visualization: Short Papers. IEEE.

[63]

Aran Lunzer and Amelia McNamara. 2014. It Ain't Necessarily So: Checking Charts for Robustness. IEEE VisWeek Poster Proceedings (2014).

[64]

Giorgia Lupi. 2017. Data Humanism: the Revolutionary Future of Data Visualization. Print Magazine 30 (2017).

[65]

Jock Mackinlay. 1986. Automating the Design of Graphical Presentations of Relational Information. ACM Transactions On Graphics 5, 2 (1986), 110--141.

Digital Library

[66]

Jock Mackinlay, Pat Hanrahan, and Chris Stolte. 2007. Show Me: Automatic Presentation for Visual Analysis. IEEE Transactions on Visualization and Computer Graphics 13, 6 (2007), 1137--1144.

Digital Library

[67]

Justin Matejka and George Fitzmaurice. 2017. Same Stats, Different Graphs: Generating Datasets with Varied Appearance and Identical Statistics through Simulated Annealing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 1290--1294.

Digital Library

[68]

Adrian Mayorga and Michael Gleicher. 2013. Splatterplots: Overcoming Overdraw in Scatter Plots. IEEE Transactions on Visualization and Computer Graphics 19, 9 (2013), 1526--1538.

Digital Library

[69]

Eva Mayr, Nicole Hynek, Saminu Salisu, and Florian Windhager. 2019. Trust in Information Visualization. In EuroVis Workshop on Trustworthy Visualization (TrustVis), Robert Kosara, Kai Lawonn, Lars Linsen, and Noeska Smit (Eds.). The Eurographics Association.

[70]

Andrew McNutt and Gordon Kindlmann. 2018. Linting for Visualization: Towards a Practical Automated Visualization Guidance System. In VisGuides: 2nd Workshop on the Creation, Curation, Critique and Conditioning of Principles and Guidelines in Visualization.

[71]

Elijah Meeks. 2017. Linting Rules for Complex Data Visualization. (2017). https://www.youtube.com/watch?v=_KEl-Spdaz0 PlotCon.

[72]

Luana Micallef, Gregorio Palmas, Antti Oulasvirta, and Tino Weinkauf. 2017. Towards Perceptual Optimization of the Visual Design of Scatterplots. IEEE Transactions on Visualization and Computer Graphics 23, 6 (2017), 1588--1599.

Digital Library

[73]

Andrew Vande Moere. 2007. Towards Designing Persuasive Ambient Visualization. In Proceedings of the 1st International Workshop on Ambient Information Systems, Colocated at Pervasive 2007, Toronto, Canada, May 13, 2007 (CEUR Workshop Proceedings), William R. Hazlewood, Lorcan Coyle, and Sunny Consolvo (Eds.), Vol. 254. CEUR-WS.org. http://ceur-ws.org/Vol-254/paper10.pdf

[74]

Dominik Moritz, Chenglong Wang, Greg L Nelson, Halden Lin, Adam M Smith, Bill Howe, and Jeffrey Heer. 2019. Formalizing Visualization Design Knowledge as Constraints: Actionable and Extensible Models in Draco. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2019), 438--448.

Digital Library

[75]

Kvanç Mu¸ slu, Yuriy Brun, and Alexandra Meliou. 2015. Preventing data errors with continuous testing. In Proceedings of the 2015 International Symposium on Software Testing and Analysis ISSTA. ACM, 373--384.

Digital Library

[76]

George E Newman and Brian J Scholl. 2012. Bar graphs depicting averages are perceptually misinterpreted: The within-the-bar bias. Psychonomic Bulletin & Review 19, 4 (2012), 601--607.

[77]

Mimi Onuoha. 2018. On Missing Data Sets. https://github.com/MimiOnuoha/missing-datasets. (2018). Accessed: 2019-08--13.

[78]

Anshul Vikram Pandey, Katharina Rall, Margaret L Satterthwaite, Oded Nov, and Enrico Bertini. 2015. How Deceptive are Deceptive Visualizations?: An Empirical Analysis of Common Distortion Techniques. In Proceedings of the 2015 CHI Conference on Human Factors in Computing Systems. ACM, 1469--1478.

Digital Library

[79]

Peter Pirolli and Stuart Card. 2005. The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis. In Proceedings of International Conference on Intelligence Analysis, Vol. 5. McLean, VA, USA, 2--4.

[80]

Catherine Plaisant. 2005. Information Visualization and the Challenge of Universal Usability. In Exploring Geovisualization. Elsevier, 53--82.

[81]

Xiaoying Pu and Matthew Kay. 2018. The Garden of Forking Paths in Visualization: A Design Space for Reliable Exploratory Visual Analytics: Position Paper. In IEEE VIS: Evaluation and Beyond-Methodological Approaches for Visualization (BELIV). IEEE, 37--45.

[82]

Zening Qu and Jessica Hullman. 2017. Keeping Multiple Views Consistent: Constraints, Validations, and Exceptions in Visualization Authoring. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2017), 468--477.

[83]

Vijayshankar Raman and Joseph M Hellerstein. 2001. Potter's Wheel: An Interactive Data Cleaning System. In International Conference on Very Large Data Bases VLDB, Vol. 1. 381--390. http://www.vldb.org/conf/2001/P381.pdf

[84]

Stephen Redmond. 2019. Visual Cues in Estimation of Part-To-Whole Comparisons. (2019).

[85]

Jacob Ritchie, Daniel Wigdor, and Fanny Chevalier. 2019. A Lie Reveals the Truth: Quasimodes for Task-Aligned Data Presentation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 193.

Digital Library

[86]

Bernice E. Rogowitz and Alan D. Kalvin. 2001. The "Which Blair Project": A Quick Visual Method for Evaluating Perceptual Color Maps. In IEEE Visualization 2001, Proceedings. IEEE, 183--190.

[87]

Bernice E Rogowitz, Lloyd A Treinish, and Steve Bryson. 1996. How Not to Lie with Visualization. Computers in Physics 10, 3 (1996), 268--273.

Digital Library

[88]

Hans Rosling and Zhongxing Zhang. 2011. Health advocacy with Gapminder animated statistics. Journal of Epidemiology and Global Health 1, 1 (2011), 11--14.

[89]

Dominik Sacha, Hansi Senaratne, Bum Chul Kwon, Geoffrey Ellis, and Daniel A Keim. 2015. The Role of Uncertainty, Awareness, and Trust in Visual Analytics. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2015), 240--249.

Digital Library

[90]

Babak Salimi, Johannes Gehrke, and Dan Suciu. 2018. Bias in OLAP Queries: Detection, Explanation, and Removal. In Proceedings of the 2018 International Conference on Management of Data. ACM, 1021--1035.

Digital Library

[91]

Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, and Jeffrey Heer. 2016. Vega-Lite: A Grammar of Interactive Graphics. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 341--350.

Digital Library

[92]

Sergio Segura, Gordon Fraser, Ana B Sanchez, and Antonio Ruiz-Cortés. 2016. A Survey on Metamorphic Testing. IEEE Transactions on Software Engineering 42, 9 (2016), 805--824.

[93]

Hayeong Song and Danielle Albers Szafir. 2018. Where's My Data? Evaluating Visualizations with Missing Data. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 914--924.

Digital Library

[94]

Arjun Srinivasan, Steven M Drucker, Alex Endert, and John Stasko. 2018. Augmenting Visualizations with Interactive Data Facts to Facilitate Interpretation and Communication. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 672--681.

Digital Library

[95]

Michael Stonebraker, Daniel Bruckner, Ihab F Ilyas, George Beskales, Mitch Cherniack, Stanley B Zdonik, Alexander Pagan, and Shan Xu. 2013. Data Curation at Scale: The Data Tamer System. In CIDR Sixth Biennial Conference on Innovative Data Systems Research. www.cidrdb.org. http://cidrdb.org/cidr2013/index.html

[96]

Danielle Albers Szafir. 2018. The Good, the Bad, and the Biased: Five Ways Visualizations Can Mislead (and How to Fix Them). ACM Interactions 25, 4 (2018), 26--33.

Digital Library

[97]

Tableau. 2019. Tableau Prep. https://www.tableau.com/products/prep. (2019).

[98]

Nan Tang, Eugene Wu, and Guoliang Li. 2019. Towards Democratizing Relational Data Visualization. In Proceedings of the 2019 International Conference on Management of Data. ACM, 2025--2030.

Digital Library

[99]

Trifacta. 2019. Trifacta. https://www.trifacta.com/. (2019).

[100]

Trulia. 2019. New York Real Estate Market Overview. https://www.trulia.com/real_estate/New_York-New_York/. (2019). Accessed: 2019-08--19.

[101]

André Caldero Valdez, Martina Ziefle, and Michael Sedlmair. 2017. A Framework for Studying Biases in Visualization Research. (2017).

[102]

Jarke J. van Wijk. 2005. The Value of Visualization. In 16th IEEE Visualization Conference, VIS. IEEE, 79--86.

[103]

Jacob VanderPlas, Brian E Granger, Jeffrey Heer, Dominik Moritz, Kanit Wongsuphasawat, Arvind Satyanarayan, Eitan Lees, Ilia Timofeev, Ben Welsh, and Scott Sievert. 2018. Altair: Interactive Statistical Visualizations for Python. J. Open Source Software 3, 32 (2018), 1057.

[104]

Rafael Veras and Christopher Collins. 2020. Discriminability Tests for Visualization Effectiveness and Scalability. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 749--758.

[105]

Paul Vickers, Joe Faith, and Nick Rossiter. 2012. Understanding Visualization: A Formal Approach Using Category Theory and Semiotics. IEEE Transactions on Visualization and Computer Graphics 19, 6 (2012), 1048--1061.

Digital Library

[106]

Howard Wainer. 1984. How to Display Data Badly. The American Statistician 38, 2 (1984), 137--147.

[107]

Emily Wall, Leslie M Blaha, Lyndsey Franklin, and Alex Endert. 2017. Warning, Bias May Occur: A Proposed Approach to Detecting Cognitive Bias in Interactive Visual Analytics. In 2017 IEEE Conference on Visual Analytics Science and Technology (VAST). IEEE, 104--115.

[108]

Emily Wall, John Stasko, and Alex Endert. 2019. Toward a Design Space for Mitigating Cognitive Bias in Vis. In IEEE VGTC Conference on Visualization: Short Papers. IEEE.

[109]

Pei Wang and Yeye He. 2019. Uni-Detect: A Unified Approach to Automated Error Detection in Tables. In Proceedings of the 2019 International Conference on Management of Data SIGMOD. ACM, 811--828.

Digital Library

[110]

Brian Whitworth. 2005. Polite Computing. Behaviour & Information Technology 24, 5 (2005), 353--363.

[111]

Hadley Wickham, Dianne Cook, Heike Hofmann, and Andreas Buja. 2010. Graphical inference for infovis. IEEE Transactions on Visualization and Computer Graphics 16, 6 (2010), 973--979.

Digital Library

[112]

Jo Wood, Alexander Kachkaev, and Jason Dykes. 2018. Design Exposition with Literate Visualization. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 759--768.

Digital Library

[113]

Eugene Wu and Samuel Madden. 2013. Scorpion: Explaining Away Outliers in Aggregate Queries. Proceedings of the VLDB Endowment 6, 8 (2013), 553--564.

Digital Library

[114]

Keke Wu, Shea Tanis, and Danielle Szafir. 2019. Designing Communicative Visualization for People with Intellectual Developmental Disabilities. (Aug 2019).

[115]

Cindy Xiong, Joel Shapiro, Jessica Hullman, and Steven Franconeri. 2020. Illusion of Causality in Visualized Data. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 853--862.

[116]

Cindy Xiong, Lisanne van Weelden, and Steven Franconeri. 2019. The Curse of Knowledge in Visual Data Communication. IEEE Transactions on Visualization and Computer Graphics (2019), 1--1.

[117]

Emanuel Zgraggen, Zheguang Zhao, Robert Zeleznik, and Tim Kraska. 2018. Investigating the Effect of the Multiple Comparisons Problem in Visual Analysis. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 479.

Digital Library

[118]

Zheguang Zhao, Lorenzo De Stefani, Emanuel Zgraggen, Carsten Binnig, Eli Upfal, and Tim Kraska. 2017. Controlling False Discoveries During Interactive Data Exploration. In Proceedings of the 2017 ACM International Conference on Management of Data SIGMOD. ACM, 527--540.

Digital Library

[119]

Zhi Quan Zhou and Liqun Sun. 2019. Metamorphic Testing of Driverless Cars. Commun. ACM 62, 3 (Feb. 2019), 61--67.

Digital Library

[120]

Caroline Ziemkiewicz and Robert Kosara. 2009. Embedding Information Visualization within Visual Representation. In Advances in Information and Intelligent Systems. Vol. 251. Springer, 307--326.

Cited By

Lan XLiu Y(2025)“I Came Across a Junk”: Understanding Design Flaws of Data Visualization from the Public's PerspectiveIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345634131:1(393-403)Online publication date: Jan-2025
https://doi.org/10.1109/TVCG.2024.3456341
Milesi MMartinez-Maldonado R(2024)Data Storytelling in Learning Analytics? A Qualitative Investigation into Educators’ Perceptions of Benefits and RisksProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636865(167-177)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3636555.3636865
Harrison GBryson KBamba ADovichi LBinion ABorem AUr B(2024)JupyterLab in Retrograde: Contextual Notifications That Highlight Fairness and Bias Issues for Data ScientistsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642755(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642755
Show More Cited By

Index Terms

Surfacing Visualization Mirages
1. Human-centered computing
  1. Visualization
    1. Visualization application domains
      1. Information visualization

Recommendations

Extreme visualization: squeezing a billion records into a million pixels
SIGMOD '08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data

Database searches are usually performed with query languages and form fill in templates, with results displayed in tabular lists. However, excitement is building around dynamic queries sliders and other graphical selectors for query specification, with ...
Polyarchy visualization: visualizing multiple intersecting hierarchies
CHI '02: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We describe a new information structure composed of multiple intersecting hierarchies, which we call Polyarchies. Visualizing polyarchies enables use of novel views for discovery of relationships which are very difficult using existing hierarchy ...
Animated visualization of multiple intersecting hierarchies

The authors describe a new information structure composed of multiple intersecting hierarchies, which we call a Polyarchy. Visualizing polyarchies enables use of novel views for discovery of relationships which are very difficult using existing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
1,251
Total Downloads

Downloads (Last 12 months)157
Downloads (Last 6 weeks)15

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lan XLiu Y(2025)“I Came Across a Junk”: Understanding Design Flaws of Data Visualization from the Public's PerspectiveIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345634131:1(393-403)Online publication date: Jan-2025
https://doi.org/10.1109/TVCG.2024.3456341
Milesi MMartinez-Maldonado R(2024)Data Storytelling in Learning Analytics? A Qualitative Investigation into Educators’ Perceptions of Benefits and RisksProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636865(167-177)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1145/3636555.3636865
Harrison GBryson KBamba ADovichi LBinion ABorem AUr B(2024)JupyterLab in Retrograde: Contextual Notifications That Highlight Fairness and Bias Issues for Data ScientistsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642755(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642755
Ge LEasterday MKay MDimara ECheng PFranconeri S(2024)V-FRAMER: Visualization Framework for Mitigating Reasoning Errors in Public PolicyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642750(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642750
Lisnic MLex AKogan M(2024)"Yeah, this graph doesn't show that": Analysis of Online Engagement with Misleading Data VisualizationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642448(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642448
Sarma APu XCui YCorrell MBrown EKay M(2024)Odds and Insights: Decision Quality in Exploratory Data Analysis Under UncertaintyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641995(1-14)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641995
Gu KGrunde-McLaughlin MMcNutt AHeer JAlthoff T(2024)How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz StudyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641891(1-22)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641891
Sultanum NBromley DCorrell M(2024)Data Guards: Challenges and Solutions for Fostering Trust in Data2024 IEEE Visualization and Visual Analytics (VIS)10.1109/VIS55277.2024.00019(56-60)Online publication date: 13-Oct-2024
https://doi.org/10.1109/VIS55277.2024.00019
McNutt AStone MHeer J(2024)Mixing Linters with GUIs: A Color Palette Design ProbeIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345631731:1(327-337)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456317
Crisan AShang MBrochu E(2024)Eliciting Model Steering Interactions From Users via Data and Visual Design ProbesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332289830:9(6005-6019)Online publication date: Sep-2024
https://doi.org/10.1109/TVCG.2023.3322898
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents