Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3472749.3474731acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Idyll Studio: A Structured Editor for Authoring Interactive & Data-Driven Articles

Published: 12 October 2021 Publication History

Abstract

Interactive articles are an effective medium of communication in education, journalism, and scientific publishing, yet are created using complex general-purpose programming tools. We present Idyll Studio, a structured editor for authoring and publishing interactive and data-driven articles. We extend the Idyll framework to support reflective documents, which can inspect and modify their underlying program at runtime, and show how this functionality can be used to reify the constituent parts of a reactive document model—components, text, state, and styles—in an expressive, interoperable, and easy-to-learn graphical interface. In a study with 18 diverse participants, all could perform basic editing and composition, use datasets and variables, and specify relationships between components. Most could choreograph interactive visualizations and dynamic text, although some struggled with advanced uses requiring unstructured code editing. Our findings suggest Idyll Studio lowers the threshold for non-experts to create interactive articles and allows experts to rapidly specify a wide range of article designs.

References

[1]
Fereshteh Amini, Nathalie Henry Riche, Bongshin Lee, Christophe Hurter, and Pourang Irani. 2015. Understanding data videos: Looking at narrative visualization through the cinematography lens. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. 1459–1468.
[2]
Anders Andersen. 1998. A note on reflection in Python 1.5. Lancaster University 46(1998).
[3]
B Atkinson. 1987. HyperCard [computer program]. Cupertino, CA: Apple Computer(1987).
[4]
Benjamin Bach, Natalie Kerracher, Kyle Wm Hall, Sheelagh Carpendale, Jessie Kennedy, and Nathalie Henry Riche. 2016. Telling stories about dynamic networks with graph comics. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 3670–3682.
[5]
Benjamin Bach, Zezhong Wang, Matteo Farinella, Dave Murray-Rust, and Nathalie Henry Riche. 2018. Design patterns for data comics. In Proceedings of the 2018 chi conference on human factors in computing systems. 1–12.
[6]
Sriram Karthik Badam, Andreas Mathisen, Roman Rädle, Clemens N Klokmose, and Niklas Elmqvist. 2018. Vistrates: A component model for ubiquitous analytics. IEEE transactions on visualization and computer graphics 25, 1(2018), 586–596.
[7]
Michel Beaudouin-Lafon. 2000. Instrumental interaction: an interaction model for designing post-WIMP user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems. 446–453.
[8]
Michel Beaudouin-Lafon and Wendy E Mackay. 2000. Reification, polymorphism and reuse: three principles for designing visual interfaces. In Proceedings of the working conference on Advanced visual interfaces. 102–109.
[9]
Michael Bostock, Vadim Ogievetsky, and Jeffrey Heer. 2011. D3: Data-Driven Documents. IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis) (2011). http://idl.cs.washington.edu/papers/d3
[10]
Judd D Bradbury and Rosanna E Guadagno. 2020. Documentary narrative visualization: Features and modes of documentary film in narrative visualization. Information Visualization 19, 4 (2020), 339–352.
[11]
Bill Burdick. 2011. leisure. https://github.com/zot/Leisure.
[12]
Jordi Cabot. 2018. WordPress: A content management system to democratize publishing. IEEE Software 35, 3 (2018), 89–92.
[13]
Matthew Conlen. 2017. Using Apparatus with Idyll. Retrieved March 1, 2021 from https://mathisonian.com/writing/apparatus
[14]
Matthew Conlen and Jeffrey Heer. 2018. Idyll: A markup language for authoring and publishing interactive articles on the web. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. 977–989.
[15]
Matthew Conlen and Fred Hohman. 2018. The Beginner’s Guide to Dimensionality Reduction. Workshop on Visualization for AI Explainability (VISxAI) at IEEE VIS (2018). https://idyll.pub/post/dimensionality-reduction-293e465c2a3443e8941b016d/
[16]
Matthew Conlen and Fred Hohman. 2019. Launching the parametric press. (2019).
[17]
Matthew Conlen, Alex Kale, and Jeffrey Heer. 2019. Capture & analysis of active reading behaviors for interactive articles on the web. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 687–698.
[18]
Morgan Dixon and James Fogarty. 2010. Prefab: Implementing Advanced Behaviors Using Pixel-Based Reverse Engineering of Interface Structure. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 1525–1534. https://doi.org/10.1145/1753326.1753554
[19]
Morgan Dixon, James Fogarty, and Jacob Wobbrock. 2012. A General-Purpose Target-Aware Pointing Enhancement Using Pixel-Level Analysis of Graphical Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 3167–3176. https://doi.org/10.1145/2207676.2208734
[20]
Pierre Dragicevic, Yvonne Jansen, Abhraneel Sarma, Matthew Kay, and Fanny Chevalier. 2019. Increasing the transparency of research papers with explorable multiverse analyses. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–15.
[21]
James R Eagan, Michel Beaudouin-Lafon, and Wendy E Mackay. 2011. Cracking the cocoa nut: user interface programming at runtime. In Proceedings of the 24th annual ACM symposium on User interface software and technology. 225–234.
[22]
Leah Findlater, Alex Jansen, Kristen Shinohara, Morgan Dixon, Peter Kamb, Joshua Rakita, and Jacob O Wobbrock. 2010. Enhanced area cursors: reducing fine pointing demands for people with motor impairments. In Proceedings of the 23nd annual ACM symposium on User interface software and technology. 153–162.
[23]
Marsha E Fonteyn, Benjamin Kuipers, and Susan J Grobe. 1993. A description of think aloud method and protocol analysis. Qualitative health research 3, 4 (1993), 430–441.
[24]
Ira R Forman, Nate Forman, and John Vlissides Ibm. 2004. Java reflection in action. (2004).
[25]
Krzysztof Z Gajos, Daniel S Weld, and Jacob O Wobbrock. 2010. Automatically generating personalized user interfaces with Supple. Artificial Intelligence 174, 12-13 (2010), 910–950.
[26]
Nahum Gershon and Ward Page. 2001. What storytelling can do for information visualization. Commun. ACM 44, 8 (2001), 31–37.
[27]
Mohamed G Gouda and Ted Herman. 1991. Adaptive programming. IEEE Transactions on Software Engineering 17, 9 (1991), 911–921.
[28]
Esther Greussing and Hajo G Boomgaarden. 2019. Simply bells and whistles? Cognitive effects of visual aesthetics in digital longforms. Digital Journalism 7, 2 (2019), 273–293.
[29]
Brian Hempel and Ravi Chugh. 2016. Semi-automated svg programming via direct manipulation. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. 379–390.
[30]
Brian Hempel, Justin Lubin, Grace Lu, and Ravi Chugh. 2018. Deuce: a lightweight user interface for structured editing. In Proceedings of the 40th International Conference on Software Engineering. 654–664.
[31]
Fred Hohman, Matthew Conlen, Jeffrey Heer, and Duen Horng Polo Chau. 2020. Communicating with interactive articles. Distill 5, 9 (2020), e28.
[32]
Jessica Hullman and Nick Diakopoulos. 2011. Visualization rhetoric: Framing effects in narrative visualization. IEEE transactions on visualization and computer graphics 17, 12(2011), 2231–2240.
[33]
Jessica Hullman, Steven Drucker, Nathalie Henry Riche, Bongshin Lee, Danyel Fisher, and Eytan Adar. 2013. A deeper understanding of sequence in narrative visualization. IEEE Transactions on visualization and computer graphics 19, 12(2013), 2406–2415.
[34]
Rubaiat Habib Kazi, Fanny Chevalier, Tovi Grossman, and George Fitzmaurice. 2014. Kitty: Sketching Dynamic and Interactive Illustrations. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST ’14). Association for Computing Machinery, New York, NY, USA, 395–405. https://doi.org/10.1145/2642918.2647375
[35]
Holger M Kienle. 2010. It’s about time to take JavaScript (more) seriously. IEEE software 27, 3 (2010), 60–62.
[36]
Yea-Seul Kim, Katharina Reinecke, and Jessica Hullman. 2017. Explaining the gap: Visualizing one’s predictions improves recall and comprehension of data. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 1375–1386.
[37]
Matthew G Kirschenbaum. 2016. Track changes: A literary history of word processing. Harvard University Press.
[38]
Clemens N Klokmose, James R Eagan, Siemen Baader, Wendy Mackay, and Michel Beaudouin-Lafon. 2015. Webstrates: Shareable dynamic media. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. 280–290.
[39]
Donald Ervin Knuth. 1984. Literate programming. Comput. J. 27, 2 (1984), 97–111.
[40]
Amy Ko, Htet Htet Aung, and Brad A Myers. 2005. Design requirements for more flexible structured editors from a study of programmers’ text editing. In CHI’05 extended abstracts on human factors in computing systems. 1557–1560.
[41]
Amy Ko and Brad A Myers. 2006. Barista: An implementation framework for enabling new tools, interaction techniques and views in code editors. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 387–396.
[42]
Sam Lau and Philip J Guo. 2020. Data Theater: A Live Programming Environment for Prototyping Data-Driven Explorable Explanations. In Workshop on Live Programming (LIVE).
[43]
Bongshin Lee, Nathalie Henry Riche, Petra Isenberg, and Sheelagh Carpendale. 2015. More than telling a story: Transforming data into visually shared stories. IEEE computer graphics and applications 35, 5 (2015), 84–90.
[44]
Pattie Maes. 1987. Concepts and experiments in computational reflection. ACM Sigplan Notices 22, 12 (1987), 147–155.
[45]
Richard E Mayer. 2002. Multimedia learning. In Psychology of learning and motivation. Vol. 41. Elsevier, 85–139.
[46]
Sean McKenna, N Henry Riche, Bongshin Lee, Jeremy Boy, and Miriah Meyer. 2017. Visual narrative flow: Exploring factors shaping data visualization story reading experiences. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 377–387.
[47]
Philip K McKinley, Seyed Masoud Sadjadi, Eric P Kasten, and Betty HC Cheng. 2004. Composing adaptive software. Computer 37, 7 (2004), 56–64.
[48]
Graham McNeill and S Hale. 2019. Viz-Blocks: Building Visualizations and Documents in the Browser. (2019).
[49]
Jens Monig, Yoshiki Ohshima, and John Maloney. 2015. Blocks at your fingertips: Blurring the line between blocks and text in GP. In 2015 IEEE Blocks and Beyond Workshop (Blocks and Beyond). IEEE, 51–53.
[50]
Brad Myers, Scott E Hudson, and Randy Pausch. 2000. Past, present, and future of user interface software tools. ACM Transactions on Computer-Human Interaction (TOCHI) 7, 1(2000), 3–28.
[51]
Ted Nelson. 1967. Stretchtext – hypertext note #8. Project Xanadu (1967).
[52]
Observable 2018. Observable. https://observablehq.com/.
[53]
Yoshiki Ohshima, Aran Lunzer, Bert Freudenberg, and Ted Kaehler. 2013. KScript and KSWorld: A time-aware and mostly declarative language and interactive GUI framework. In Proceedings of the 2013 ACM international symposium on New ideas, new paradigms, and reflections on programming & software. 117–134.
[54]
Dan R. Olsen, Scott E. Hudson, Thom Verratti, Jeremy M. Heiner, and Matt Phelps. 1999. Implementing Interface Attachments Based on Surface Representations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI ’99). Association for Computing Machinery, New York, NY, USA, 191–198. https://doi.org/10.1145/302979.303038
[55]
Fernando Pérez and Brian E Granger. 2007. IPython: a system for interactive scientific computing. Computing in science & engineering 9, 3 (2007), 21–29.
[56]
Jeffrey M Perkel. 2018. Why Jupyter is data scientists’ computational notebook of choice.Nature 563, 7732 (2018), 145–147.
[57]
Vincent Quint and Irene Vatton. 1986. Grif: An interactive system for structured document manipulation. In Text Processing and Document Manipulation, Proceedings of the International Conference. 200–312.
[58]
Roman Rädle, Midas Nouwens, Kristian Antonsen, James R Eagan, and Clemens N Klokmose. 2017. Codestrates: Literate computing with webstrates. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 715–725.
[59]
Casey Reas and Ben Fry. 2007. Processing: a programming handbook for visual designers and artists. Mit Press.
[60]
Adam Rule, Aurélien Tabard, and James D Hollan. 2018. Exploration and explanation in computational notebooks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.
[61]
Arvind Satyanarayan and Jeffrey Heer. 2014. Authoring narrative visualizations with ellipsis. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 361–370.
[62]
Arvind Satyanarayan and Jeffrey Heer. 2014. Lyra: An interactive visualization design environment. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 351–360.
[63]
Arvind Satyanarayan, Dominik Moritz, Kanit Wongsuphasawat, and Jeffrey Heer. 2017. Vega-Lite: A Grammar of Interactive Graphics. IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis) (2017). http://idl.cs.washington.edu/papers/vega-lite
[64]
Arvind Satyanarayan, Ryan Russell, Jane Hoffswell, and Jeffrey Heer. 2016. Reactive Vega: A Streaming Dataflow Architecture for Declarative Interactive Visualization. IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis) (2016). http://idl.cs.washington.edu/papers/reactive-vega-architecture
[65]
Toby Schachman and Joshua Horowitz. 2016. Apparatus. https://github.com/cdglabs.
[66]
Pascal Schneiders. 2020. What remains in mind? Effectiveness and efficiency of explainers at conveying information. Media and Communication 8, 1 (2020), 218–231.
[67]
Edward Segel and Jeffrey Heer. 2010. Narrative visualization: Telling stories with data. IEEE transactions on visualization and computer graphics 16, 6(2010), 1139–1148.
[68]
Brian Smith. 1982. Reflection and semantics in a procedural language. Technical Report, TR-272, MIT Laboratory for Computer Science (1982).
[69]
Charles D Stolper, Bongshin Lee, N Henry Riche, and John Stasko. 2016. Emerging and recurring data-driven storytelling techniques: Analysis of a curated collection of recent stories. Microsoft Research, Washington, USA(2016).
[70]
John Sweller. 2011. Cognitive load theory. In Psychology of learning and motivation. Vol. 55. Elsevier, 37–76.
[71]
Tim Teitelbaum and Thomas Reps. 1981. The Cornell program synthesizer: a syntax-directed programming environment. Commun. ACM 24, 9 (1981), 563–573.
[72]
Andries Van Dam. 1997. Post-WIMP user interfaces. Commun. ACM 40, 2 (1997), 63–67.
[73]
Bret Victor. 2011. Explorable Explorations. Retrieved March 1, 2021 from http://worrydream.com/ExplorableExplanations/
[74]
Bret Victor. 2011. Tangle: a JavaScript library for reactive documents. Retrieved August 1, 2017 from http://worrydream.com/Tangle/
[75]
Lev Semenovich Vygotsky. 1980. Mind in society: The development of higher psychological processes. Harvard university press.
[76]
April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How data scientists use computational notebooks for real-time collaboration. Proceedings of the ACM on Human-Computer Interaction 3, CSCW(2019), 1–30.
[77]
Qiyu Zhi, Alvitta Ottley, and Ronald Metoyer. 2019. Linking and layout: Exploring the integration of text and visualization in storytelling. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 675–685.
[78]
Jonathan Zong, Dhiraj Barnwal, Rupayan Neogy, and Arvind Satyanarayan. 2020. Lyra 2: Designing interactive visualizations by demonstration. IEEE Transactions on Visualization and Computer Graphics (2020).
[79]
Douglas Zongker. 2003. Creating animation for presentations. Ph.D. Dissertation. University of Washington.

Cited By

View all
  • (2024)A Design Language for Prototyping and Storyboarding Data-Driven StoriesApplied Sciences10.3390/app1404138714:4(1387)Online publication date: 8-Feb-2024
  • (2024)Augmented Physics: Creating Interactive and Embedded Physics Simulations from Static Textbook DiagramsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676392(1-12)Online publication date: 13-Oct-2024
  • (2024)Explorable Explainable AI: Improving AI Understanding for Community Health Workers in IndiaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642733(1-21)Online publication date: 11-May-2024
  • Show More Cited By
  1. Idyll Studio: A Structured Editor for Authoring Interactive & Data-Driven Articles

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '21: The 34th Annual ACM Symposium on User Interface Software and Technology
    October 2021
    1357 pages
    ISBN:9781450386357
    DOI:10.1145/3472749
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 October 2021

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. computational media
    2. explorable explanations
    3. interactive articles

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    UIST '21

    Acceptance Rates

    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)120
    • Downloads (Last 6 weeks)10
    Reflects downloads up to 01 Mar 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)A Design Language for Prototyping and Storyboarding Data-Driven StoriesApplied Sciences10.3390/app1404138714:4(1387)Online publication date: 8-Feb-2024
    • (2024)Augmented Physics: Creating Interactive and Embedded Physics Simulations from Static Textbook DiagramsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676392(1-12)Online publication date: 13-Oct-2024
    • (2024)Explorable Explainable AI: Improving AI Understanding for Community Health Workers in IndiaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642733(1-21)Online publication date: 11-May-2024
    • (2024)Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI CollaborationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642726(1-19)Online publication date: 11-May-2024
    • (2024)More Than Data Stories: Broadening the Role of Visualization in Contemporary JournalismIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328758530:8(5240-5259)Online publication date: Aug-2024
    • (2024)MaugVLink: Augmenting Mathematical Formulas with Visual Links2024 IEEE 17th Pacific Visualization Conference (PacificVis)10.1109/PacificVis60374.2024.00048(337-342)Online publication date: 23-Apr-2024
    • (2024)WebWriter: Authoring and Remixing ExplorablesTechnology Enhanced Learning for Inclusive and Equitable Quality Education10.1007/978-3-031-72312-4_35(247-253)Online publication date: 13-Sep-2024
    • (2023)Augmented Math: Authoring AR-Based Explorable Explanations by Augmenting Static Math TextbooksProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606827(1-16)Online publication date: 29-Oct-2023
    • (2023)Living Papers: A Language Toolkit for Augmented Scholarly CommunicationProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606791(1-13)Online publication date: 29-Oct-2023
    • (2023)Notable: On-the-fly Assistant for Data Storytelling in Computational NotebooksProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580965(1-16)Online publication date: 19-Apr-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media