article

Characterizing users' visual analytic activity for insight provenance

Authors:

Michelle X. ZhouAuthors Info & Claims

Information Visualization, Volume 8, Issue 1

Pages 42 - 55

https://doi.org/10.1057/ivs.2008.31

Published: 01 January 2009 Publication History

Abstract

Insight provenance - a historical record of the process and rationale by which an insight is derived - is an essential requirement in many visual analytics applications. Although work in this area has relied on either manually recorded provenance (for example, user notes) or automaticalily recorded event-based insight provenance (for example, clicks, drags and key-presses), both approaches have fundamental limitations. Our aim is to develop a new approach that combines the benefits of both approaches while avoiding their deficiencies. Toward this goal, we characterize users' visual analytic activity at multiple levels of granularity. Moreover, we identify a critical level of abstraction, Actions, that can be used to represent visual analytic activity, with a set of general but semantically meaningful behavior types. In turn, the action types can be used as the semantic building blocks for insight provenance. We present a catalog of common actions identified through observations of several different visual analytic systems. In addition, we define a taxonomy to categorize actions into three major classes based on their semantic intent. The concept of actions has been integrated into our lab's prototype visual analytic system, HARVEST, as the basis for its insight provenance capabilities.

References

[1]

Thomas, J.J. and Cook, K.A. (2005) Illuminating the Path: The Research and Development Agenda for Visual Analytics. IEEE Press, New York.

[2]

Bavoil, L., Callahan, S. P., Crossno, P. J., Freire, J., Scheidegger, C. E., Silva, C. T. and Vo, H. T. (2005) Vistrails: Enabling interactive multiple-view visualizations. In: C.T. Silva, E. Groller, and H. Rushmeier (eds.) Proceedings of IEEE Visualization; IEEE Computer Society: Minneopolis, MN.

[3]

Eccles, R., Kapler, T., Harper, R. and Wright, W. (2007) Stories in geotime. In: W. Ribarsky, and J. Dill (eds.) Proceedings of the IEEE Symposium on Visual Analytics Science and Technology; IEEE Computer Society: Sacramento, CA.

[4]

i2 Incorporated. Analyst's Notebook. www.i2inc.com.

[5]

Jankun-Kelly, T., Ma, K-L. and Gertz, M. (2002) A model for the visualization exploration process. In: Proceedings of IEEE Visualization; IEEE Computer Society: Boston, MA.

[6]

Kreuseler, M., Nocke, T. and Schumann, H. (2004) A history mechanism for visual data mining. In: Proceedings of IEEE Information Visualization; IEEE Computer Society: Austin, TX.

[7]

Lankenau, R. A., Eick, M. A., Decherd, A., Khalio, M., Paris, P. and Fugitt, J. (2006) Vast 2006 contest second place, corporate category: Decide. In: P.C. Wong, and D. Keim (eds.) Contest Results for IEEE Symposium on Visual Analytics Science and Technology; IEEE Computer Society: Baltimore, MD.

[8]

Nardi, B. A. (1996) Context and Consciousness. Cambridge, MA: The MIT Press.

[9]

Gotz, D. and Zhou, M.X. (2008a) An Empirical Study of User Interaction Behavior During Visual Analysis. IBM Research. Technical Report RC24525.

[10]

Gotz, D. and Zhou, M.X. (2008b) Characterizing users' visual analytic activity for insight provenance. In: D. Ebert and T. Ertl (eds.) Proceedings of IEEE Symposium on Visual Analytics Science and Technology; IEEE Computer Society: Columbus, OH.

[11]

Leont'ev, A. N. (1978) Activity, Consciousness, Personality. Englewood Cliffs, NJ: Prentice-Hall.

[12]

Gotz, D., Zhou, M. X. and Wen, Z. (2006) A study of information gathering and result processing in intelligence analysis. In: M.X. Zhou, and M.T. Maybury (eds.) Proceedings of Intelligent Users Interfaces Workshop on Intelligent User Interface for Intelligent Analysis; ACM: Sydney, Australia.

[13]

Patterson, E. S., Woods, D. D., Tinapple, D. and Roth, E. M. (2001) Using cognitive task analysis (CTA) to seed design concepts for intelligence analysts under data overload. In: Proceedings of the Human Factors and Ergonomics Society's 45th Annual Meeting; Minneapolis, MN.

[14]

Pirolli, P. and Card, S. (2005) The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proceedings of the International Conference on Intelligence Analysis; McLean, VA.

[15]

Hampson, F. and Cowley, P. (2005) Instrumenting the intelligence analysis process. In: Proceedings of the International Conference on Intelligence Analysis; McLean, VA.

[16]

Scholtz, J., Morse, E. and Hewett, T. (2004) In depth observational studies of professional intelligence analysts. In: D.A. Vincenzi, M. Mouloua, and P.A. Hancock (eds.) Proceedings of the Human Performance, Situation Awareness, and Automation Conference; Routledge: USA.

[17]

Saraiya, P., North, C., Lam, V. and Duca, K. A. (2006) An insight-based longitudinal study of visual analytics. IEEE Transactions on Visualization and Computer Graphics, 12(6), 1511-1522.

Digital Library

[18]

Groth, D. P. and Streefkerk, K. (2006) Provenance and annotation for visual exploration systems. IEEE Transactions on Visualization and Computer Graphics, 12(6), 1500-1510.

Digital Library

[19]

Shrinivasan, Y. B. and van WijK, J. J. (2008) Supporting the analytical reasoning process in information visualization, In: M. Czerwinski, A. Lund, and D. Tan (eds.) Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; ACM: Florence, Italy.

[20]

Chi, E. H. (2000) A taxonomy of visualization techniques using the data state reference model. In: S. Roth, and D. Keim (eds.) Proceedings of IEEE Information Visualization; IEEE Computer Society: Salt Lake City, Utah.

[21]

Chi, E.H. and Riedl, J. T. (1998) An operator interaction framework for visualization systems. In: J. Dill, and G. Wills (eds.) Proceedings of IEEE Information Visualization; IEEE Computer Society: Research Triangle Park, NC.

[22]

Chuah, M. C. and Roth, S. F. (1996) On the semantics of interactive visualizations. In: S. Card, and S.G. Eick (eds.) Proceedings of IEEE Information Visualization; IEEE Computer Society: San Francisco, CA.

[23]

Wehrend, S. and Lewis, C. (1990) A problem-oriented classification of visualization techniques. In: Proceedings of IEEE Visualization; IEEE Computer Society: San Francisco, CA.

[24]

Zhou, M. X. and Feiner, S. K. (1998) Visual task characterization for automated visual discourse synthesis. In: M.E. Atwood, C-M. Karat, A. Lund, J. Coutaz, and J. Karat (eds.) Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; ACM Press/Addison-Wesley Publishing Co.: Los Angeles, CA.

[25]

Amar, R., Eagan, J. and Stasko, J. (2005) Low-level components of analytic activity in information visualization. In: J. Stasko, and M. Ward (eds.) Proceedings of IEEE Information Visualization; IEEE Computer Society: Minneopolis, MN.

[26]

Amar, R. and Stasko, J. (2004) A knowledge task-based framework for design and evaluation of information visualizations. In: M. Ward, and T. Munzner (eds.) Proceedings of IEEE Information Visualization; IEEE Computer Society: Austin, TX.

[27]

Gotz, D., Zhou, M. X. and Aggarwal, V. (2006) Interactive visual synthesis of analytic knowledge. In: P.C. Wong, and D. Keim (eds.) Proceedings of the IEEE Symposium on Visual Analytics Science and Technology; IEEE Computer Society: Baltimore, MD.

[28]

Shneiderman, B. (1996) The eyes have it: a task by data type taxonomy for information visualizations. In: M. Burnett, and W. Citrin (eds.) Proceedings of the IEEE Symposium on Visual Languages; IEEE Computer Society: Boulder, CO.

[29]

Wattenberg, M. (1998) Map of the market, http://www. smartmoney.com/marketmap/.

[30]

Viegas, F. B., Wattenberg, M., van Ham, F., Kriss, J. and McKeon, M. (2007) Many eyes: A site for visualization at internet scale. In: M. Chen, C. Hansen, C. North, A. Pang, and J. van Wijk (eds.) Proceedings of IEEE Information Visualization; IEEE Computer Society: Sacramento, MA.

Cited By

Bressa NLouis JWillett WHuron S(2024)Input Visualization: Collecting and Modifying Data with Visual RepresentationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642808(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642808
Fan DFay Siu ARao HKim GVazquez XGreco LO'Modhrain SFollmer S(2023)The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19ACM Transactions on Accessible Computing10.1145/355789916:1(1-29)Online publication date: 29-Mar-2023
https://dl.acm.org/doi/10.1145/3557899
Battle LOttley A(2023)What Do We Mean When We Say “Insight”? A Formal Synthesis of Existing TheoryIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332669830:9(6075-6088)Online publication date: 24-Oct-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3326698
Show More Cited By

Recommendations

Analytic provenance: process+interaction+insight
CHI EA '11: CHI '11 Extended Abstracts on Human Factors in Computing Systems

Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces. One key aspect that separates visual analytics from other related fields (InfoVis, SciVis, HCI) is the focus on analytical reasoning. While the final ...
Sharing insight provenance in collaborative visual analytics
Modeling and Leveraging Analytic Focus During Exploratory Visual Analysis
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Visual analytics systems enable highly interactive exploratory data analysis. Across a range of fields, these technologies have been successfully employed to help users learn from complex data. However, these same exploratory visualization techniques ...

Comments

Information & Contributors

Information

Published In

cover image Information Visualization

Information Visualization Volume 8, Issue 1

January 2009

84 pages

ISSN:1473-8716

Issue’s Table of Contents

Publisher

Palgrave Macmillan

Publication History

Published: 01 January 2009

Accepted: 16 November 2008

Received: 07 November 2008

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

51
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bressa NLouis JWillett WHuron S(2024)Input Visualization: Collecting and Modifying Data with Visual RepresentationsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642808(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642808
Fan DFay Siu ARao HKim GVazquez XGreco LO'Modhrain SFollmer S(2023)The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19ACM Transactions on Accessible Computing10.1145/355789916:1(1-29)Online publication date: 29-Mar-2023
https://dl.acm.org/doi/10.1145/3557899
Battle LOttley A(2023)What Do We Mean When We Say “Insight”? A Formal Synthesis of Existing TheoryIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332669830:9(6075-6088)Online publication date: 24-Oct-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3326698
Chakhchoukh MBoukhelifa NBezerianos A(2023)Understanding How In-Visualization Provenance Can Support Trade-Off AnalysisIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317107429:9(3758-3774)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TVCG.2022.3171074
Domova VVrotsou K(2023)A Model for Types and Levels of Automation in Visual Analytics: A Survey, a Taxonomy, and ExamplesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316376529:8(3550-3568)Online publication date: 1-Aug-2023
https://dl.acm.org/doi/10.1109/TVCG.2022.3163765
Walchshofer CHinterreiter AXu KStitz HStreit M(2023)Provectories: Embedding-Based Analysis of Interaction Provenance DataIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.313569729:12(4816-4831)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TVCG.2021.3135697
Wang XWu ZHuang WWei YHuang ZXu MChen W(2023)VIS+AI: integrating visualization with artificial intelligence for efficient data analysisFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-023-2691-y17:6Online publication date: 8-Jun-2023
https://dl.acm.org/doi/10.1007/s11704-023-2691-y
Rae IZhou FBilsing MBunge P(2022)Understanding Visual Investigation Patterns Through Digital “Field” ObservationsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517445(1-16)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517445
Siu AFan DKim GRao HVazquez XO'Modhrain SFollmer SVazquez SDrake TAhmetovic DYaneva V(2021)COVID-19 highlights the issues facing blind and visually impaired people in accessing data on the webProceedings of the 18th International Web for All Conference10.1145/3430263.3452432(1-15)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3430263.3452432
Yan JGu ZRzeszotarski JKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Tessera: Discretizing Data Analysis Workflows on a Task LevelProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445728(1-15)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445728
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents