research-article

An application-independent system for visualizing user operation history

Authors:

Toshio Nakamura,

Takeo IgarashiAuthors Info & Claims

UIST '08: Proceedings of the 21st annual ACM symposium on User interface software and technology

Pages 23 - 32

https://doi.org/10.1145/1449715.1449721

Published: 19 October 2008 Publication History

Abstract

A history-of-user-operations function helps make applications easier to use. For example, users may have access to an operation history list in an application to undo or redo a past operation. To provide an overview of a long operation history and help users find target interactions or application states quickly, visual representations of operation history have been proposed. However, most previous systems are tightly integrated with target applications and difficult to apply to new applications. We propose an application-independent method that can visualize the operation history of arbitrary GUI applications by monitoring the input and output GUI events from outside of the target application. We implemented a prototype system that visualizes operation sequences of generic Java Awt/Swing applications using an annotated comic strip metaphor. We tested the system with various applications and present results from a user study.

Supplementary Material

JPG File (p23-igarashi.jpg)

Download
14.73 KB

MOV File (p23-igarashi.mov)

Download
30.81 MB

References

[1]

Agrawala, M., Phan, D., Heiser, J., Haymaker, J., Klingner, J., Hanrahan, P., and Tversky, B. Designing Effective Step-By-Step Assembly Instructions. In Proceedings of SIGGRAPH, pp. 828--837, 2003.

Digital Library

[2]

Agrawala, M., and Stolte, C. Rendering Effective Route Maps: Improving Usability Through Generalization. In Proceedings of SIGGRAPH, pp. 241--249, 2001.

Digital Library

[3]

Assa, J., Caspi, Y., and Cohen-Or, D. Action Synopsis: Pose Selection and Illustration. In Proceedings of SIGGRAPH, pp. 667--676, 2005.

Digital Library

[4]

Baudisch, P., Cutrell, E., and Robertson, G. High-Density Cursor: A Visualization Technique that Helps Users Keep Track of Fast-Moving Mouse Cursors. In Proceedings of INTERACT, pp. 236--243, 2003.

[5]

Baudisch, P., Tan, D., Collomb, M., Robbins, D., Hinckley, K., Agrawala, M., Zhao, S., and Ramos, G. Phosphor: Explaining Transitions in the User Interface Using Afterglow Effects. In Proceedings of UIST, pp.169--178, 2006.

Digital Library

[6]

Bergman, L., Castelli, V., Lau, T., and Oblinger, D. DocWizards: a system for authoring follow-me documentation wizards. In Proceedings of UIST, pp. 191--200, 2005.

Digital Library

[7]

Berlage, T. A Selective Undo Mechanism for Graphical User Interfaces Based On Command Objects. In Proceedings of CHI, pp. 269--294, 1994.

Digital Library

[8]

Bezerianos, A., Dragicevic, P., and Balakrishnan, R. Mnemonic Rendering: An Image-Based Approach for Exposing Hidden Changes in Dynamic Displays. In Proceedings of UIST, pp. 159--168, 2006.

Digital Library

[9]

Freeman, W.T., and Zhang, H. Shape-Time Photography. In Proceedings of CVPR, pp. 151--157, 2003.

[10]

Goldman, D.B., Curless, B., Salesin, D., and Seitz, S.M. Schematic Storyboarding for Video Visualization and Editing. In Proceedings of SIGGRAPH, pp. 862--871, 2006.

Digital Library

[11]

Guimbretie're, F., Dixon, M., and Hinckley, K. Experi-Scope: an analysis tool for interaction data. In Proceedings of CHI, pp.1333--1342, 2007.

Digital Library

[12]

Igarashi, T., Matsuoka, S., and Tanaka, T. Teddy: A Sketching Interface for 3D Freeform Design. In Proceedings of SIGGRAPH, pp. 409--416, 1999.

Digital Library

[13]

Irani, M., and Anandan, P. Video Indexing Based on Mosaic Representations. IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 86, No. 5, pp. 905--921, 1998.

[14]

Kaptelinin, V., Mantyla, T., and Astrom, J. Transient Visual Cues for Scrolling: An Empirical Study. In CHI '02 Extended Abstracts, pp. 620--621, 2002.

Digital Library

[15]

Kawagishi, Y., Hatsuyama, K., and Kondo, K. Cartoon Blur: Non-Photorealistic Motion Blur. In Proceedings of CGI, pp. 276--281, 2003.

[16]

Kim, B., and Essa, I. Video-based Nonphotorealistic and Expressive Illustration of Motion. In Proceedings of the CGI, pp. 32--35, 2005.

Digital Library

[17]

Kurlander, D. and Feiner, S. A history-based macro by example system. In Proceedings of UIST, pages 99--106, 1992.

Digital Library

[18]

Lieberman, H. Mondrian: A Teachable Graphical Editor. In Watch What I Do: Programming by Demonstration, pp. 341--358, 1993.

Digital Library

[19]

Little, G., Lau, T. A., Cypher, A., Lin, J., Haber, E. M., and Kandogan, E., Koala: capture, share, automate, personalize business processes on the web, In Proceedings of CHI, pp. 943--946, 2007.

Digital Library

[20]

Massey, M., and Bender, W. Salient Stills: Process and Practice. IBM Systems Journal, Vol. 35, No.3--4, pp. 557--573, 1996.

Digital Library

[21]

Masuch, M., Schlechtweg, S., and Schulz, R. Speedlines: Depicting Motion in Motionless Pictures. In SIGGRAPH '99 Conference Abstracts and Applications, pp. 277, 1999.

Digital Library

[22]

Meng, C., Yasue, M., Imamiya, A., and Mao, X. Visualizing Histories for Selective Undo and Redo. In Proceedings of APCHI, pp. 459, 1998.

Digital Library

[23]

Modugno, F. and Myers, B. A. Pursuit: graphically representing programs in a demonstrational visual shell. In Conference Companion of CHI, pp.455--456, 1994.

Digital Library

[24]

Myers, B.A., McDaniel, R.G., Miller, R.C., Ferrency, A.S., Faulring, A., Kyle, B.D., Mickish, A., Klimovitski, A., and Doane, P. The Amulet Environment: New Models for Effective User Interface Software Development. IEEE Transaction on Software Engineering, Vol. 23, No. 6, pp. 347--365, 1997.

Digital Library

[25]

Mynatt, E.D., Igarashi, T., Edwards, W.K., and La-Marca, A. Flatland: new dimensions in office whiteboards. In Proceedings of CHI, pp. 346--353, 1999.

Digital Library

[26]

Rekimoto, J. Time-Machine Computing: A Time-Centric Approach for the Information Environment. In Proceedings of UIST, pp. 45--54, 1999.

Digital Library

[27]

Su, S. Visualizing, Editing, and Inferring Structure in 2D Graphics, UIST 2007 Doctoral Symposium, 2007.

[28]

Taniguchi, Y., Akutsu, A., and Tonomura, Y. PanoramaExcerpts: Extracting and Packing Panoramas for Video Browsing. In Proceedings of MULTIMEDIA, pp. 427--436, 1997.

Digital Library

[29]

Teodosio, L., and Bender, W. Salient Video Stills: Content and Context Preserved. In Proceedings of MULTIMEDIA, pp. 39--46, 1993.

Digital Library

Cited By

Veuskens TRamakers RLeen DLuyten K(2023)History in Motion: Interactive 3D Animated Visualizations for Understanding and Exploring the Modeling History of 3D CAD DesignsProceedings of the 8th ACM Symposium on Computational Fabrication10.1145/3623263.3623358(1-13)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3623263.3623358
Zhang LAgrawal AOney SGuo A(2023)VRGit: A Version Control System for Collaborative Content Creation in Virtual RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581136(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581136
SUNAYAMA WNAKAE TNISHIHARA YHATANAKA Y(2020)Presentation System of Operating Histories to Support Choice and Operation of Tools for Beginners of Text Mining Software TETDMTETDMを用いたテキストマイニング初心者のツールの選択と操作を助ける操作履歴の提示Journal of Japan Society for Fuzzy Theory and Intelligent Informatics10.3156/jsoft.32.5_84132:5(841-850)Online publication date: 15-Oct-2020
https://doi.org/10.3156/jsoft.32.5_841
Show More Cited By

Index Terms

An application-independent system for visualizing user operation history
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Graphical user interfaces

Recommendations

User interface history
CHI EA '08: CHI '08 Extended Abstracts on Human Factors in Computing Systems

User Interfaces have been around as long as computers have existed, even well before the field of Human-Computer Interaction was established. Over the years, some papers on the history of Human-Computer Interaction and User Interfaces have appeared, ...
Application-independent user interface design using an application exposure
Taking stock of user interface history
NordiCHI '08: Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges

The user interface is coming of age and papers adressing UI history have appeared in fair amounts in the last 25 years. Most of them address particular aspects such as an innovative interface paradigm or the contribution of a visionary or a research ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '08: Proceedings of the 21st annual ACM symposium on User interface software and technology

October 2008

308 pages

ISBN:9781595939753

DOI:10.1145/1449715

General Chair:
Steve Cousins
Willow Garage, USA
,
Program Chair:
Michel Beaudouin-Lafon
Université Paris-Sud, France

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST08

Sponsor:

UIST08: The 21st Annual ACM Symposium on User Interface Software and Technology

October 19 - 22, 2008

CA, Monterey, USA

Acceptance Rates

Overall Acceptance Rate 842 of 3,967 submissions, 21%

Upcoming Conference

UIST '24

Sponsor:
sigchi
sigchi

The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
1,394
Total Downloads

Downloads (Last 12 months)24
Downloads (Last 6 weeks)1

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Veuskens TRamakers RLeen DLuyten K(2023)History in Motion: Interactive 3D Animated Visualizations for Understanding and Exploring the Modeling History of 3D CAD DesignsProceedings of the 8th ACM Symposium on Computational Fabrication10.1145/3623263.3623358(1-13)Online publication date: 8-Oct-2023
https://dl.acm.org/doi/10.1145/3623263.3623358
Zhang LAgrawal AOney SGuo A(2023)VRGit: A Version Control System for Collaborative Content Creation in Virtual RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581136(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581136
SUNAYAMA WNAKAE TNISHIHARA YHATANAKA Y(2020)Presentation System of Operating Histories to Support Choice and Operation of Tools for Beginners of Text Mining Software TETDMTETDMを用いたテキストマイニング初心者のツールの選択と操作を助ける操作履歴の提示Journal of Japan Society for Fuzzy Theory and Intelligent Informatics10.3156/jsoft.32.5_84132:5(841-850)Online publication date: 15-Oct-2020
https://doi.org/10.3156/jsoft.32.5_841
Liu ZLiu ZMunzner TBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Data-driven Multi-level Segmentation of Image Editing LogsProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376152(1-12)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376152
Ajisaka SHara SMatsuchi MLuo SYoshida SXie HMiyata K(2020)Learning Rubik's Cube through User Operation History2020 Nicograph International (NicoInt)10.1109/NicoInt50878.2020.00015(43-46)Online publication date: Jun-2020
https://doi.org/10.1109/NicoInt50878.2020.00015
Thoravi Kumaravel BNguyen CDiVerdi SHartmann BBrewster SFitzpatrick GCox AKostakos V(2019)TutoriVRProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300514(1-12)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300514
Bao LXing ZXia XLo D(2019)VT-Revolution: Interactive Programming Video Tutorial Authoring and Watching SystemIEEE Transactions on Software Engineering10.1109/TSE.2018.280291645:8(823-838)Online publication date: 1-Aug-2019
https://doi.org/10.1109/TSE.2018.2802916
Bao LXing ZXia XLo DLi SLeavens GGarcia APăsăreanu C(2018)VT-revolution: interactive programming tutorials made possibleProceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3236024.3264587(924-927)Online publication date: 26-Oct-2018
https://dl.acm.org/doi/10.1145/3236024.3264587
Vatavu R(2017)Smart-PocketsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2017.01.005103:C(1-21)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.1016/j.ijhcs.2017.01.005
Chen HWei LHartmann BAgrawala MWyman CYuksel C(2016)Data-driven adaptive history for image editingProceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games10.1145/2856400.2856417(103-111)Online publication date: 27-Feb-2016
https://dl.acm.org/doi/10.1145/2856400.2856417
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents