research-article

Open access

mage: Fluid Moves Between Code and Graphical Work in Computational Notebooks

Authors:

Mary Beth Kery,

Dominik Moritz,

Kanit Wongsuphasawat,

Kayur PatelAuthors Info & Claims

UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

Pages 140 - 151

https://doi.org/10.1145/3379337.3415842

Published: 20 October 2020 Publication History

Abstract

We aim to increase the flexibility at which a data worker can choose the right tool for the job, regardless of whether the tool is a code library or an interactive graphical user interface (GUI). To achieve this flexibility, we extend computational notebooks with a new API mage, which supports tools that can represent themselves as both code and GUI as needed. We discuss the design of mage as well as design opportunities in the space of flexible code/GUI tools for data work. To understand tooling needs, we conduct a study with nine professional practitioners and elicit their feedback on mage and potential areas for flexible code/GUI tooling. We then implement six client tools for mage that illustrate the main themes of our study findings. Finally, we discuss open challenges in providing flexible code/GUI interactions for data workers.

Supplementary Material

VTT File (3379337.3415842.vtt)

Download
5.99 KB

MP4 File (ufp4581pv.mp4)

Preview video

Download
8.74 MB

MP4 File (ufp4581vf.mp4)

Video figure

Download
14.82 MB

MP4 File (3379337.3415842.mp4)

Presentation Video

Download
40.29 MB

References

[1]

Alex Bigelow, Steven Drucker, Danyel Fisher, and Miriah Meyer. 2016. Iterating between tools to create and edit visualizations. IEEE Transactions on Visualization and Computer Graphics (2016). http://dx.doi.org/10.1109/TVCG.2016.2598609

[2]

Joshua Bloch. 2006. How to design a good API and why it matters. In Companion to the 21st ACM SIGPLAN symposium on Object-oriented programming systems, languages, and applications.

[3]

Ángel Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, and Duen Horng Chau. 2019. FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning. IEEE Conference on Visual Analytics Science and Technology (VAST) (2019). http://dx.doi.org/10.1109/VAST47406.2019.8986948

[4]

Carrie J Cai and Philip J Guo. 2019. Software Developers Learning Machine Learning: Motivations, Hurdles, and Desires. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE.

[5]

Scott Davidoff, Min Kyung Lee, Anind K Dey, and John Zimmerman. 2007. Rapidly exploring application design through speed dating. In International Conference on Ubiquitous Computing. Springer.

[6]

Ian Drosos, Titus Barik, Philip J Guo, Robert DeLine, and Sumit Gulwani. Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists. (????).

[7]

Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. (2017). http://archive.ics.uci.edu/ml

[8]

Gregory Piatetsky for KDnuggets. 2018. Python eats away at R: Top Software for Analytics, Data Science, Machine Learning in 2018: Trends and Analysis. (2018). https://www.kdnuggets.com/2018/05/poll-tools-analytics-data-science-machine-learning-results.html.

[9]

Sumit Gulwani. 2011. Automating string processing in spreadsheets using input-output examples. ACM Sigplan Notices (2011).

[10]

Björn Hartmann, Loren Yu, Abel Allison, Yeonsoo Yang, and Scott R Klemmer. 2008. Design as exploration: creating interface alternatives through parallel authoring and runtime tuning. In Proceedings of the 21st annual ACM symposium on User interface software and technology. ACM. http://dx.doi.org/10.1145/1449715.1449732

Digital Library

[11]

Brian Hempel, Justin Lubin, and Ravi Chugh. 2019. Sketch-n-Sketch: Output-Directed Programming for SVG. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. http://dx.doi.org/10.1145/3332165.3347925

Digital Library

[12]

Edwin L Hutchins, James D Hollan, and Donald A Norman. 1985. Direct manipulation interfaces. Human-computer interaction (1985). http://dx.doi.org/10.1207/s15327051hci0104_2

[13]

Kaggle. 2017. Kaggle Machine Learning & Data Science Survey 2017. Dataset. (2017). Available from Kaggle https://www.kaggle.com/kaggle/kaggle-survey-2017.

[14]

Kaggle. 2018. 2018 Kaggle ML & DS Survey. Dataset. (2018). Available from Kaggle https://www.kaggle.com/kaggle/kaggle-survey-2018.

[15]

Sean Kandel, Andreas Paepcke, Joseph Hellerstein, and Jeffrey Heer. 2011. Wrangler: Interactive visual specification of data transformation scripts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. http://dx.doi.org/10.1145/1978942.1979444

Digital Library

[16]

Mary Beth Kery, Amber Horvath, and Brad Myers. 2017. Variolite: Supporting Exploratory Programming by Data Scientists. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA. http://dx.doi.org/10.1145/3025453.3025626

Digital Library

[17]

Thomas Kluyver, Benjamin Ragan-Kelley, Fernando Pérez, Brian Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica Hamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Damián Avila, Safia Abdalla, and Carol Willing. 2016. Jupyter Notebooks -- a publishing format for reproducible computational workflows. In Positioning and Power in Academic Publishing: Players, Agents and Agendas, F. Loizides and B. Schmidt (Eds.). IOS Press.

[18]

Kayur Patel, Naomi Bancroft, Steven M Drucker, James Fogarty, Andrew J Ko, and James Landay. 2010. Gestalt: integrated support for implementation and analysis in machine learning. In Proceedings of the 23nd annual ACM symposium on User interface software and technology. ACM. http://dx.doi.org/10.1145/1866029.1866038

Digital Library

[19]

Kayur Patel, James Fogarty, James A Landay, and Beverly Harrison. 2008. Investigating statistical machine learning as a tool for software development. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. http://dx.doi.org/10.1145/1357054.1357160

Digital Library

[20]

Roger D Peng. 2011. Reproducible research in computational science. Science (2011).

[21]

Jeffrey M Perkel. 2018. Why Jupyter is data scientists' computational notebook of choice. Nature (2018). http://dx.doi.org/10.1038/d41586-018-07196--1

[22]

Stephen G Powell, Kenneth R Baker, and Barry Lawson. 2008. A critical review of the literature on spreadsheet errors. Decision Support Systems (2008).

[23]

Donghao Ren, Saleema Amershi, Bongshin Lee, Jina Suh, and Jason D Williams. 2016. Squares: Supporting interactive performance analysis for multiclass classifiers. IEEE transactions on visualization and computer graphics (2016). http://dx.doi.org/10.1109/TVCG.2016.2598828

Digital Library

[24]

Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, and Jeffrey Heer. 2016. Vega-lite: A grammar of interactive graphics. IEEE transactions on visualization and computer graphics (2016). http://dx.doi.org/10.1109/TVCG.2016.2599030

Digital Library

[25]

Helen Shen. 2014. Interactive notebooks: Sharing the code. Nature News (2014).

[26]

Chris Stolte, Diane Tang, and Pat Hanrahan. 2002. Polaris: A system for query, analysis, and visualization of multidimensional relational databases. IEEE Transactions on Visualization and Computer Graphics (2002). http://dx.doi.org/10.1145/1400214.1400234

[27]

Jacob VanderPlas, Brian Granger, Jeffrey Heer, Dominik Moritz, Kanit Wongsuphasawat, Arvind Satyanarayan, Eitan Lees, Ilia Timofeev, Ben Welsh, and Scott Sievert. 2018. Altair: Interactive Statistical Visualizations for Python. Journal of Open Source Software (2018). http://dx.doi.org/10.21105/joss.01057

[28]

Kanit Wongsuphasawat, Dominik Moritz, Anushka Anand, Jock Mackinlay, Bill Howe, and Jeffrey Heer. 2015. Voyager: Exploratory analysis via faceted browsing of visualization recommendations. IEEE transactions on visualization and computer graphics (2015). http://dx.doi.org/10.1109/TVCG.2015.2467191

[29]

Qian Yang, Jina Suh, Nan-Chen Chen, and Gonzalo Ramos. 2018. Grounding interactive machine learning tool design in how non-experts actually build models. In Proceedings of the 2018 Designing Interactive Systems Conference. http://dx.doi.org/10.1145/3196709.3196729

Digital Library

Cited By

Weber TMayer S(2024)Usability and Adoption of Graphical Tools for Data-Driven DevelopmentProceedings of Mensch und Computer 202410.1145/3670653.3670658(231-241)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3670658
Weber TEhe JMayer S(2024)Extending Jupyter with Multi-Paradigm EditorsProceedings of the ACM on Human-Computer Interaction10.1145/36602478:EICS(1-22)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3660247
De Santana TNeto PDe Almeida EAhmed I(2024)Bug Analysis in Jupyter Notebook Projects: An Empirical StudyACM Transactions on Software Engineering and Methodology10.1145/364153933:4(1-34)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3641539
Show More Cited By

Index Terms

mage: Fluid Moves Between Code and Graphical Work in Computational Notebooks
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. Interaction devices
      1. Haptic devices

Recommendations

Managing Messes in Computational Notebooks
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Data analysts use computational notebooks to write code for analyzing and visualizing data. Notebooks help analysts iteratively write analysis code by letting them interleave code with output, and selectively execute cells. However, as analysis ...
The Future of Notebook Programming Is Fluid
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems

A new kind of widget has begun appearing in the data science notebook programming community that can fluidly switch its own appearance between two representations: a graphical user interface (GUI) tool and plain textual code. Data scientists of all ...
StickyLand: Breaking the Linear Presentation of Computational Notebooks
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

How can we better organize code in computational notebooks? Notebooks have become a popular tool among data scientists, as they seamlessly weave text and code together, supporting users to rapidly iterate and document code experiments. However, it is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

October 2020

1297 pages

ISBN:9781450375146

DOI:10.1145/3379337

General Chairs:
Shamsi Iqbal
Microsoft Research, USA
,
Karon MacLean
University of British Columbia, Canada
,
Program Chairs:
Fanny Chevalier
University of Toronto, Canada
,
Stefanie Mueller
MIT CSAIL, USA

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UIST '20

Sponsor:

UIST '20: The 33rd Annual ACM Symposium on User Interface Software and Technology

October 20 - 23, 2020

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 842 of 3,967 submissions, 21%

Upcoming Conference

UIST '24

Sponsor:
sigchi
sigchi

The 37th Annual ACM Symposium on User Interface Software and Technology

October 13 - 16, 2024

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

46
Total Citations
View Citations
1,227
Total Downloads

Downloads (Last 12 months)280
Downloads (Last 6 weeks)40

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Weber TMayer S(2024)Usability and Adoption of Graphical Tools for Data-Driven DevelopmentProceedings of Mensch und Computer 202410.1145/3670653.3670658(231-241)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1145/3670653.3670658
Weber TEhe JMayer S(2024)Extending Jupyter with Multi-Paradigm EditorsProceedings of the ACM on Human-Computer Interaction10.1145/36602478:EICS(1-22)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3660247
De Santana TNeto PDe Almeida EAhmed I(2024)Bug Analysis in Jupyter Notebook Projects: An Empirical StudyACM Transactions on Software Engineering and Methodology10.1145/364153933:4(1-34)Online publication date: 18-Apr-2024
https://dl.acm.org/doi/10.1145/3641539
Wang ZMunechika DLee SChau D(2024)SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational NotebooksExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650848(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650848
Harden JWang AFaust RIsaacs KKirshenbaum NWenskovitch JZhao JNorth C(2024)Human-Notebook Interactions: The CHI of Computational NotebooksExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636318(1-6)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636318
Wang FLin YYang LLi HGu MZhu MQu H(2024)OutlineSpark: Igniting AI-powered Presentation Slides Creation from Computational Notebooks through OutlinesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642865(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642865
Harrison GBryson KBamba ADovichi LBinion ABorem AUr B(2024)JupyterLab in Retrograde: Contextual Notifications That Highlight Fairness and Bias Issues for Data ScientistsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642755(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642755
Hohman FWang CLee JGörtler JMoritz DBigham JRen ZForet CShan QZhang X(2024)Talaria: Interactively Optimizing Machine Learning Models for Efficient InferenceProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642628(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642628
Gu KShang RAlthoff TWang CDrucker S(2024)How Do Analysts Understand and Verify AI-Assisted Data Analyses?Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642497(1-22)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642497
Gadhave KCutler ZLex A(2024)Persist: Persistent and Reusable Interactions in Computational NotebooksComputer Graphics Forum10.1111/cgf.1509243:3Online publication date: 10-Jun-2024
https://doi.org/10.1111/cgf.15092
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents