research-article

Parallel embeddings: a visualization technique for contrasting learned representations

Authors:

Dustin L. Arendt,

Wenwen DouAuthors Info & Claims

IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

Pages 259 - 274

https://doi.org/10.1145/3377325.3377514

Published: 17 March 2020 Publication History

Abstract

We introduce "Parallel Embeddings", a new technique that generalizes the classical Parallel Coordinates visualization technique to sequences of learned representations. This visualization technique is designed for concept-oriented "model comparison" tasks, allowing data scientists to understand qualitative differences in how models interpret input data. We compare user performance with our tool against Tensor Board Embedding Projector for understanding model accuracy and qualitative model differences. With our tool, users were more accurate and learned strategies for the tasks more quickly. Furthermore, users' analytical process in the comparison condition was positively influenced by using our tool beforehand.

References

[1]

Eric Alexander and Michael Gleicher. 2015. Task-driven comparison of topic models. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2015), 320--329.

Digital Library

[2]

Saleema Amershi, Max Chickering, Steven M Drucker, Bongshin Lee, Patrice Simard, and Jina Suh. 2015. Modeltracker: Redesigning performance analysis tools for machine learning. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 337--346.

Digital Library

[3]

Dustin Arendt and Meg Pirrung. 2017. The "y" of it Matters, Even for Storyline Visualization. In 2017 IEEE Conference on Visual Analytics Science and Technology (VAST). IEEE, 81--91.

[4]

David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6541--6549.

[5]

James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, Feb (2012), 281--305.

Digital Library

[6]

Pierre Dragicevic. 2016. Fair statistical communication in HCI. In Modern Statistical Methods for HCI. Springer, 291--330.

[7]

Tim Dwyer and Lev Nachmanson. 2009. Fast edge-routing for large graphs. In International Symposium on Graph Drawing. Springer, 147--158.

[8]

Niklas Elmqvist and Jean-Daniel Fekete. 2009. Hierarchical aggregation for information visualization: Overview, techniques, and design guidelines. IEEE Transactions on Visualization and Computer Graphics 16, 3 (2009), 439--454.

Digital Library

[9]

Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and Robust Automated Machine Learning. In Advances in Neural Information Processing Systems 28, C. Cortes, N.D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett (Eds.). Curran Associates, Inc., 2962--2970.

Digital Library

[10]

Kārlis Freivalds. 2001. Curved edge routing. In International Symposium on Fundamentals of Computation Theory. Springer, 126--137.

[11]

Sandra G Hart and Lowell E Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in Psychology. Vol. 52. Elsevier, 139--183.

[12]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 770--778.

[13]

Julian Heinrich and Daniel Weiskopf. 2013. State of the Art of Parallel Coordinates. In Eurographics (STARs). 95--116.

[14]

Fred Matthew Hohman, Minsuk Kahng, Robert Pienta, and Duen Horng Chau. 2018. Visual analytics in deep learning: An interrogative survey for the next frontiers. IEEE Transactions on Visualization and Computer Graphics (2018).

Digital Library

[15]

Max Jaderberg, Wojciech M Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castaneda, Charles Beattie, Neil C Rabinowitz, Ari S Morcos, Avraham Ruderman, et al. 2018. Human-level performance in first-person multi-player games with population-based deep reinforcement learning. arXiv preprint arXiv:1807.01281 (2018).

[16]

Jimmy Johansson, Patric Ljung, Mikael Jern, and Matthew Cooper. 2005. Revealing structure within clustered parallel coordinates displays. In IEEE Symposium on Information Visualization, 2005. INFOVIS 2005. IEEE, 125--132.

[17]

Minsuk Kahng, Pierre Y Andrews, Aditya Kalro, and Duen Horng Polo Chau. 2017. ActiVis: Visual exploration of industry-scale deep neural network models. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2017), 88--97.

[18]

Minsuk Kahng, Dezhi Fang, and Duen Horng Polo Chau. 2016. Visual exploration of machine learning results using data cube analysis. In Proceedings of the Workshop on Human-In-the-Loop Data Analytics. ACM, 1.

Digital Library

[19]

Wiebke Köpp and Tino Weinkauf. 2018. Temporal treemaps: Static visualization of evolving trees. IEEE transactions on visualization and computer graphics 25, 1 (2018), 534--543.

[20]

Antoine Lhuillier, Christophe Hurter, and Alexandru Telea. 2017. State of the art in edge and trail bundling techniques. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 619--645.

[21]

Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of Psychology (1932).

[22]

Dongyu Liu, Weiwei Cui, Kai Jin, Yuxiao Guo, and Huamin Qu. 2018. Deep-tracker: Visualizing the training process of convolutional neural networks. ACM Transactions on Intelligent Systems and Technology (TIST) 10, 1 (2018), 6.

[23]

Mengchen Liu, Jiaxin Shi, Zhen Li, Chongxuan Li, Jun Zhu, and Shixia Liu. 2016. Towards better analysis of deep convolutional neural networks. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 91--100.

Digital Library

[24]

Jonas Lukasczyk, Gunther Weber, Ross Maciejewski, Christoph Garth, and Heike Leitte. 2017. Nested tracking graphs. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 12--22.

[25]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, Nov (2008), 2579--2605.

[26]

Kevin T McDonnell and Klaus Mueller. 2008. Illustrative parallel coordinates. In Computer Graphics Forum, Vol. 27. Wiley Online Library, 1031--1038.

[27]

Leland McInnes, John Healy, and James Melville. 2018. UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018).

[28]

H Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. 2013. Ad click prediction: a view from the trenches. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data mining. ACM, 1222--1230.

Digital Library

[29]

Paul Mihas. 2019. Qualitative data analysis. In Oxford Research Encyclopedia of Education.

[30]

Paulo E Rauber, Samuel G Fadel, Alexandre X Falcao, and Alexandru C Telea. 2016. Visualizing the hidden activity of artificial neural networks. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 101--110.

Digital Library

[31]

Khairi Reda, Chayant Tantipathananandh, Andrew Johnson, Jason Leigh, and Tanya Berger-Wolf. 2011. Visualizing the evolution of community structures in dynamic social networks. In Computer Graphics Forum, Vol. 30. Wiley Online Library, 1061--1070.

[32]

Hans Rosling and Zhongxing Zhang. 2011. Health advocacy with Gapminder animated statistics. Journal of epidemiology and global health 1, 1 (2011), 11--14.

[33]

Martin Rosvall and Carl T Bergstrom. 2010. Mapping change in large networks. PloS one 5, 1 (2010), e8694.

[34]

Emily Saldanha, Brenda Praggastis, Todd Billow, and Dustin L. Arendt. 2019. ReLVis: Visual Analytics for Situational Awareness During Reinforcement Learning Experimentation. In EuroVis 2019 - Short Papers, Jimmy Johansson, Filip Sadlo, and G. Elisabeta Marai (Eds.). The Eurographics Association.

[35]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[36]

Daniel Smilkov, Nikhil Thorat, Charles Nicholson, Emily Reif, Fernanda B Viégas, and Martin Wattenberg. 2016. Embedding projector: Interactive visualization and interpretation of embeddings. In Proc. Neural Inf. Process. Syst. Workshop Interpretable ML Complex Syst.

[37]

Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, and Alexander M Rush. 2018. Seq-2-seq-vis: A visual debugging tool for sequence-to-sequence models. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 353--363.

Digital Library

[38]

C. J. Van Rijsbergen. 1979. Information Retrieval (2nd ed.). Butterworth-Heinemann.

[39]

Corinna Vehlow, Fabian Beck, and Daniel Weiskopf. 2017. Visualizing group structures in graphs: A survey. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 201--225.

[40]

Junpeng Wang, Liang Gou, Han-Wei Shen, and Hao Yang. 2018. Dqnviz: A visual analytics approach to understand deep q-networks. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 288--298.

Digital Library

[41]

John Wenskovitch, Ian Crandell, Naren Ramakrishnan, Leanna House, and Chris North. 2017. Towards a systematic combination of dimension reduction and clustering in visual analytics. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2017), 131--141.

[42]

Wathsala Widanagamaachchi, Cameron Christensen, Valerio Pascucci, and Peer-Timo Bremer. 2012. Interactive exploration of large-scale time-varying data using dynamic tracking graphs. In IEEE Symposium on Large Data Analysis and Visualization (LDAV). IEEE, 9--17.

[43]

Moritz Wittenhagen, Christian Cherek, and Jan Borchers. 2016. Chronicler: Interactive exploration of source code history. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 3522--3532.

Digital Library

[44]

Wei Yu, Kuiyuan Yang, Yalong Bai, Hongxun Yao, and Yong Rui. 2014. Visualizing and comparing convolutional neural networks. arXiv preprint arXiv:1412.6631 (2014).

[45]

Tom Zahavy, Nir Ben-Zrihem, and Shie Mannor. 2016. Graying the black box: Understanding DQNs. (2016), 1899--1908.

[46]

Haipeng Zeng, Hammad Haleem, Xavier Plantaz, Nan Cao, and Huamin Qu. 2017. CNNComparator: Comparative analytics of convolutional neural networks. arXiv preprint arXiv:1710.05285 (2017).

[47]

Jiawei Zhang, Yang Wang, Piero Molino, Lezhi Li, and David S Ebert. 2018. Manifold: A model-agnostic framework for interpretation and diagnosis of machine learning models. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 364--373.

Digital Library

Cited By

Yeh CChen YWu AChen CViégas FWattenberg M(2024)AttentionViz: A Global View of Transformer AttentionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332716330:1(262-272)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3327163
Rong YLeemann TNguyen TFiedler LQian PUnhelkar VSeidel TKasneci GKasneci E(2024)Towards Human-Centered Explainable AI: A Survey of User Studies for Model ExplanationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.333184646:4(2104-2122)Online publication date: Apr-2024
https://doi.org/10.1109/TPAMI.2023.3331846
Nagahisarchoghaei MNur NCummins LNur NKarimi MNandanwar SBhattacharyya SRahimi S(2023)An Empirical Survey on Explainable AI Technologies: Recent Trends, Use-Cases, and Categories from Technical and Application PerspectivesElectronics10.3390/electronics1205109212:5(1092)Online publication date: 22-Feb-2023
https://doi.org/10.3390/electronics12051092
Show More Cited By

Index Terms

Parallel embeddings: a visualization technique for contrasting learned representations
1. Computing methodologies
  1. Machine learning
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools
  2. Visualization
    1. Visualization techniques

Recommendations

Generic tool for visualization of model differences
IWMCP '10: Proceedings of the 1st International Workshop on Model Comparison in Practice

Model comparison includes three major concerns: presentation, calculation, and visualization of model differences. In this paper we address the concern of visualization of model differences in the context of model configuration management systems. Since ...
Polyarchy visualization: visualizing multiple intersecting hierarchies
CHI '02: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We describe a new information structure composed of multiple intersecting hierarchies, which we call Polyarchies. Visualizing polyarchies enables use of novel views for discovery of relationships which are very difficult using existing hierarchy ...
Animated visualization of multiple intersecting hierarchies

The authors describe a new information structure composed of multiple intersecting hierarchies, which we call a Polyarchy. Visualizing polyarchies enables use of novel views for discovery of relationships which are very difficult using existing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

March 2020

607 pages

ISBN:9781450371186

DOI:10.1145/3377325

General Chairs:
Fabio Paternò,
Nuria Oliver,
Program Chairs:
Cristina Conati,
Lucio Davide Spano,
Nava Tintarev

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI '20

Sponsor:

IUI '20: 25th International Conference on Intelligent User Interfaces

March 17 - 20, 2020

Cagliari, Italy

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
444
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)2

Reflects downloads up to 09 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yeh CChen YWu AChen CViégas FWattenberg M(2024)AttentionViz: A Global View of Transformer AttentionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332716330:1(262-272)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3327163
Rong YLeemann TNguyen TFiedler LQian PUnhelkar VSeidel TKasneci GKasneci E(2024)Towards Human-Centered Explainable AI: A Survey of User Studies for Model ExplanationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.333184646:4(2104-2122)Online publication date: Apr-2024
https://doi.org/10.1109/TPAMI.2023.3331846
Nagahisarchoghaei MNur NCummins LNur NKarimi MNandanwar SBhattacharyya SRahimi S(2023)An Empirical Survey on Explainable AI Technologies: Recent Trends, Use-Cases, and Categories from Technical and Application PerspectivesElectronics10.3390/electronics1205109212:5(1092)Online publication date: 22-Feb-2023
https://doi.org/10.3390/electronics12051092
Eckelt KHinterreiter AAdelberger PWalchshofer CDhanoa VHumer CHeckmann MSteinparz CStreit M(2023)Visual Exploration of Relationships and Structure in Low-Dimensional EmbeddingsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.315676029:7(3312-3326)Online publication date: 1-Jul-2023
https://doi.org/10.1109/TVCG.2022.3156760
Saeed WOmlin C(2023)Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunitiesKnowledge-Based Systems10.1016/j.knosys.2023.110273263(110273)Online publication date: Mar-2023
https://doi.org/10.1016/j.knosys.2023.110273
Sivaraman VWu YPerer A(2022)Emblaze: Illuminating Machine Learning Representations through Interactive Comparison of Embedding SpacesProceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490099.3511137(418-432)Online publication date: 22-Mar-2022
https://dl.acm.org/doi/10.1145/3490099.3511137
Boggust ACarter BSatyanarayan A(2022)Embedding Comparator: Visualizing Differences in Global Structure and Local Neighborhoods via Small MultiplesProceedings of the 27th International Conference on Intelligent User Interfaces10.1145/3490099.3511122(746-766)Online publication date: 22-Mar-2022
https://dl.acm.org/doi/10.1145/3490099.3511122
Sevastjanova RCakmak ERavfogel SCotterell REl-Assady M(2022)Visual Comparison of Language Model AdaptationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.3209458(1-11)Online publication date: 2022
https://doi.org/10.1109/TVCG.2022.3209458
Xuan XZhang XKwon OMa K(2022)VAC-CNN: A Visual Analytics System for Comparative Studies of Deep Convolutional Neural NetworksIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316534728:6(2326-2337)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1109/TVCG.2022.3165347
Heimerl FKralj CMöller TGleicher M(2022)embComp: Visual Interactive Comparison of Vector EmbeddingsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.304591828:8(2953-2969)Online publication date: 1-Aug-2022
https://dl.acm.org/doi/10.1109/TVCG.2020.3045918
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents