Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3613904.3642726acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration

Published: 11 May 2024 Publication History

Abstract

Data storytelling is powerful for communicating data insights, but it requires diverse skills and considerable effort from human creators. Recent research has widely explored the potential for artificial intelligence (AI) to support and augment humans in data storytelling. However, there lacks a systematic review to understand data storytelling tools from the perspective of human-AI collaboration, which hinders researchers from reflecting on the existing collaborative tool designs that promote humans’ and AI’s advantages and mitigate their shortcomings. This paper investigated existing tools with a framework from two perspectives: the stages in the storytelling workflow where a tool serves, including analysis, planning, implementation, and communication, and the roles of humans and AI in each stage, such as creators, assistants, optimizers, and reviewers. Through our analysis, we recognize the common collaboration patterns in existing tools, summarize lessons learned from these patterns, and further illustrate research opportunities for human-AI collaboration in data storytelling.

Supplemental Material

MP4 File - Video Presentation
Video Presentation
Transcript for: Video Presentation
ZIP File - Supplementary material
Supplementary material for "Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration" - 1-coded_paper_list.csv: The file includes the data storytelling tools coded with our framework about stages and humans' and AI's roles in their collaboration. The data is used for plotting Figures 2, 3, 4, 5, 7, and 8. It is also shown in Table 2. We further provide an interactive browser of the data at https://human-ai-universe.github.io/data-storytelling. - 2-implementation_stage_output.csv: The file includes data used for plotting Figure 6, where we show the development of tool output between 2010 and 2022. - 3-tasks_formats.pdf: The file summarizes humans' and AI's tasks and data story formats of all surveyed tools.

References

[1]
Hiroshi Akiba, Chaoli Wang, and Kwan-Liu Ma. 2009. AniViz: A Template-Based Animation Tool for Volume Visualization. IEEE Computer Graphics and Applications 30, 5 (2009), 61–71.
[2]
Saleema Amershi, Daniel S. Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi T. Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for Human-AI Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.
[3]
Fereshteh Amini, Nathalie Henry Riche, Bongshin Lee, Andres Monroy-Hernandez, and Pourang Irani. 2016. Authoring Data-Driven Videos with DataClips. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 501–510.
[4]
Leilani Battle and Alvitta Ottley. 2022. A Programmatic Definition of Visualization Insights, Objectives, and Tasks. arXiv preprint arXiv:2206.04767 (2022).
[5]
Leilani Battle and Alvitta Ottley. 2023. What Exactly is an Insight? A Literature Review. arXiv preprint arXiv:2307.06551 (2023).
[6]
Matthew Brehmer and Robert Kosara. 2021. From Jam Session to Recital: Synchronous Communication and Collaboration Around Data in Organizations. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 1139–1149.
[7]
Matthew Brehmer, Bongshin Lee, Nathalie Henry Riche, David Tittsworth, Kate Lytvynets, Darren Edge, and Christopher White. 2019. Timeline Storyteller: The Design & Deployment of an Interactive Authoring Tool for Expressive Timeline Narratives. In Proceedings of the Computation+Journalism Symposium, Vol. 2.
[8]
Chris Bryan, Kwan-Liu Ma, and Jonathan Woodring. 2016. Temporal Summary Images: An Approach to Narrative Visualization via Interactive Annotation Generation and Placement. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 511–520.
[9]
Yining Cao, Jane L E, Zhutian Chen, and Haijun Xia. 2023. DataParticles: Block-based and Language-oriented Authoring of Animated Unit Visualizations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15.
[10]
Tara Capel and Margot Brereton. 2023. What is Human-Centered about Human-Centered AI? A Map of the Research Landscape. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–23.
[11]
Qing Chen, Shixiong Cao, Jiazhe Wang, and Nan Cao. 2023. How Does Automation Shape the Process of Narrative Visualization: A Survey of Tools. IEEE Transactions on Visualization and Computer Graphics (2023).
[12]
Qing Chen, Fuling Sun, Xinyue Xu, Zui Chen, Jiazhe Wang, and Nan Cao. 2021. VizLinter: A Linter and Fixer Framework for Data Visualization. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 206–216.
[13]
Yang Chen, Scott Barlowe, and Jing Yang. 2010. Click2Annotate: Automated Insight Externalization with Rich Semantics. In Proceedings of 2010 IEEE Symposium on Visual Analytics Science and Technology. IEEE, 155–162.
[14]
Yang Chen, Jing Yang, Scott Barlowe, and Dong H Jeong. 2010. Touch2Annotate: Generating Better Annotations with Less Human Effort on Multi-touch Interfaces. In Extended Abstracts of the 2010 CHI Conference on Human Factors in Computing Systems. 3703–3708.
[15]
Zhutian Chen, Yun Wang, Qianwen Wang, Yong Wang, and Huamin Qu. 2019. Towards Automated Infographic Design: Deep Learning-based Auto-Extraction of Extensible Timeline. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 917–926.
[16]
Zhutian Chen, Qisen Yang, Xiao Xie, Johanna Beyer, Haijun Xia, Yingcai Wu, and Hanspeter Pfister. 2022. Sporthesia: Augmenting Sports Videos Using Natural Language. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 918–928.
[17]
Zhutian Chen, Shuainan Ye, Xiangtong Chu, Haijun Xia, Hui Zhang, Huamin Qu, and Yingcai Wu. 2021. Augmenting Sports Videos with VisCommentator. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 824–834.
[18]
Hao Cheng, Junhong Wang, Yun Wang, Bongshin Lee, Haidong Zhang, and Dongmei Zhang. 2022. Investigating the Role and Interplay of Narrations and Animations in Data Videos. Computer Graphics Forum 41, 3 (2022), 527–539.
[19]
Fanny Chevalier, Nathalie Henry Riche, Catherine Plaisant, Amira Chalbi, and Christophe Hurter. 2016. Animations 25 Years Later: New Roles and Opportunities. In Proceedings of the International Working Conference on Advanced Visual Interfaces. 280–287.
[20]
Fanny Chevalier, Melanie Tory, Bongshin Lee, Jarke van Wijk, Giuseppe Santucci, Marian Dörk, and Jessica Hullman. 2018. From Analysis to Communication: Supporting the Lifecycle of a Story. In Data-Driven Storytelling. AK Peters/CRC Press, 151–183.
[21]
Noptanit Chotisarn, Junhua Lu, Libinzi Ma, Jingli Xu, Linhao Meng, Bingru Lin, Ying Xu, Xiaonan Luo, and Wei Chen. 2021. Bubble Storytelling with Automated Animation: A Brexit Hashtag Activism Case Study. Journal of Visualization 24 (2021), 101–115.
[22]
Darius Coelho and Klaus Mueller. 2020. Infomages: Embedding Data into Thematic Images. Computer Graphics Forum 39, 3 (2020), 593–606.
[23]
Matthew Conlen, Megan Vo, Alan Tan, and Jeffrey Heer. 2021. Idyll Studio: A Structured Editor for Authoring Interactive & Data-Driven Articles. In Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology. 1–12.
[24]
Anamaria Crisan and Brittany Fiore-Gartland. 2021. Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
[25]
Anamaria Crisan, Brittany Fiore-Gartland, and Melanie Tory. 2020. Passing the Data Baton: A Retrospective Analysis on Data Science Work and Workers. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1860–1870.
[26]
Weiwei Cui, Jinpeng Wang, He Huang, Yun Wang, Chin-Yew Lin, Haidong Zhang, and Dongmei Zhang. 2021. A Mixed-Initiative Approach to Reusing Infographic Charts. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 173–183.
[27]
Weiwei Cui, Xiaoyu Zhang, Yun Wang, He Huang, Bei Chen, Lei Fang, Haidong Zhang, Jian-Guan Lou, and Dongmei Zhang. 2019. Text-to-Viz: Automatic Generation of Infographics from Proportion-Related Natural Language Statements. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 906–916.
[28]
Sebastian Deterding, Jonathan Hook, Rebecca Fiebrink, Marco Gillies, Jeremy Gow, Memo Akten, Gillian Smith, Antonios Liapis, and Kate Compton. 2017. Mixed-Initiative Creative Interfaces. In Extended Abstracts of the 2017 CHI Conference on Human Factors in Computing Systems. 628–635.
[29]
Victor Dibia. 2023. LIDA: A Tool for Automatic Generation of Grammar-Agnostic Visualizations and Infographics using Large Language Models. arXiv preprint arXiv:2303.02927 (2023).
[30]
Rui Ding, Shi Han, Yong Xu, Haidong Zhang, and Dongmei Zhang. 2019. QuickInsights: Quick and Automatic Discovery of Insights from Multi-Dimensional Data. In Proceedings of the 2019 ACM International Conference on Management of Data. 317–332.
[31]
Veronika Domova and Katerina Vrotsou. 2022. A Model for Types and Levels of Automation in Visual Analytics: A Survey, a Taxonomy, and Examples. IEEE Transactions on Visualization and Computer Graphics (2022).
[32]
Christina Elmer, Jonathan Schwabish, and Benjamin Wiederkehr. 2018. Organizing the Work of Data-Driven Visual Storytelling. In Data-Driven Storytelling. AK Peters/CRC Press, 185–208.
[33]
Mica R Endsley. 1987. The Application of Human Factors to the Development of Expert Systems for Advanced Cockpits. In Proceedings of the Human Factors Society Annual Meeting, Vol. 31. SAGE Publications Sage CA: Los Angeles, CA, 1388–1392.
[34]
Arlen Fan, Yuxin Ma, Michelle Mancenido, and Ross Maciejewski. 2022. Annotating Line Charts for Addressing Deception. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–12.
[35]
Zezheng Feng, Huamin Qu, Shuang-Hua Yang, Yulong Ding, and Jie Song. 2022. A Survey of Visual Analytics in Urban Area. Expert Systems 39, 9 (2022), e13065.
[36]
Xin Fu, Yun Wang, Haoyu Dong, Weiwei Cui, and Haidong Zhang. 2019. Visualization Assessment: A Machine Learning Approach. In Proceedings of the 2019 IEEE Visualization Conference. IEEE, 126–130.
[37]
Tong Gao, Jessica R Hullman, Eytan Adar, Brent Hecht, and Nicholas Diakopoulos. 2014. NewsViews: An Automated Pipeline for Creating Custom Geovisualizations for News. In Proceedings of the 2014 CHI Conference on Human Factors in Computing Systems. 3005–3014.
[38]
Tong Ge, Bongshin Lee, and Yunhai Wang. 2021. CAST: Authoring Data-Driven Chart Animations. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
[39]
Samuel Gratzl, Alexander Lex, Nils Gehlenborg, Nicola Cosgrove, and Marc Streit. 2016. From Visual Exploration to Storytelling and Back Again. Computer Graphics Forum 35, 3 (2016), 491–500.
[40]
Brian D Hall, Lyn Bartram, and Matthew Brehmer. 2022. Augmented Chironomia for Presenting Data to Remote Audiences. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–14.
[41]
Jeffrey Heer. 2019. Agency Plus Automation: Designing Artificial Intelligence into Interactive Systems. Proceedings of the National Academy of Sciences 116, 6 (2019), 1844–1850.
[42]
Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proceedings of the 1999 CHI Conference on Human Factors in Computing Systems. 159–166.
[43]
Youyang Hou and Dakuo Wang. 2017. Hacking with NPOs: Collaborative Analytics and Broker Roles in Civic Data Hackathons. Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 1–16.
[44]
Kevin Hu, Michiel A Bakker, Stephen Li, Tim Kraska, and César Hidalgo. 2019. VizML: A Machine Learning Approach to Visualization Recommendation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
[45]
Jessica Hullman, Nicholas Diakopoulos, and Eytan Adar. 2013. Contextifier: Automatic Generation of Annotated Stock Visualizations. In Proceedings of the 2013 CHI Conference on Human Factors in Computing Systems. 2707–2716.
[46]
Jessica Hullman, Steven Drucker, Nathalie Henry Riche, Bongshin Lee, Danyel Fisher, and Eytan Adar. 2013. A Deeper Understanding of Sequence in Narrative Visualization. IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2406–2415.
[47]
Jialun Aaron Jiang, Kandrea Wade, Casey Fiesler, and Jed R Brubaker. 2021. Supporting Serendipity: Opportunities and Challenges for Human-AI Collaboration in Qualitative Analysis. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–23.
[48]
DaYe Kang, Tony Ho, Nicolai Marquardt, Bilge Mutlu, and Andrea Bianchi. 2021. ToonNote: Improving Communication in Computational Notebooks Using Interactive Data Comics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
[49]
Nam Wook Kim, Nathalie Henry Riche, Benjamin Bach, Guanpeng Xu, Matthew Brehmer, Ken Hinckley, Michel Pahud, Haijun Xia, Michael J McGuffin, and Hanspeter Pfister. 2019. DataToon: Drawing Dynamic Network Comics With Pen + Touch Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
[50]
Nam Wook Kim, Hyejin Im, Nathalie Henry Riche, Alicia Wang, Krzysztof Gajos, and Hanspeter Pfister. 2019. DataSelfie: Empowering People to Design Personalized Visuals to Represent Their Data. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
[51]
Nam Wook Kim, Eston Schweickart, Zhicheng Liu, Mira Dontcheva, Wilmot Li, Jovan Popovic, and Hanspeter Pfister. 2016. Data-Driven Guides: Supporting Expressive Design for Information Graphics. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2016), 491–500.
[52]
Younghoon Kim and Jeffrey Heer. 2021. Gemini2: Generating Keyframe-Oriented Animated Transitions Between Statistical Graphics. In Proceedings of the 2021 IEEE Visualization Conference. IEEE, 201–205.
[53]
Younghoon Kim, Kanit Wongsuphasawat, Jessica Hullman, and Jeffrey Heer. 2017. GraphScape: A Model for Automated Reasoning about Visualization Similarity and Sequencing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2628–2638.
[54]
David Kouril, Ondrej Strnad, Peter Mindek, Sarkis Halladjian, Tobias Isenberg, Eduard Groeller, and Ivan Viola. 2021. Molecumentary: Adaptable Narrated Documentaries Using Molecular Visualization. IEEE Transactions on Visualization and Computer Graphics (2021).
[55]
Xiaoyan Kui, Naiming Liu, Qiang Liu, Jingwei Liu, Xiaoqian Zeng, and Chao Zhang. 2022. A Survey of Visual Analytics Techniques for Online Education. Visual Informatics (2022).
[56]
Vivian Lai, Chacha Chen, Alison Smith-Renner, Q Vera Liao, and Chenhao Tan. 2023. Towards a Science of Human-AI Decision Making: An Overview of Design Space in Empirical Human-Subject Studies. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 1369–1385.
[57]
Shahid Latif, Zheng Zhou, Yoon Kim, Fabian Beck, and Nam Wook Kim. 2021. Kori: Interactive Synthesis of Text and Charts in Data Documents. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 184–194.
[58]
Bongshin Lee, Rubaiat Habib Kazi, and Greg Smith. 2013. SketchStory: Telling More Engaging Stories with Data through Freeform Sketching. IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2416–2425.
[59]
Bongshin Lee, Nathalie Henry Riche, Petra Isenberg, and Sheelagh Carpendale. 2015. More Than Telling a Story: Transforming Data into Visually Shared Stories. IEEE Computer Graphics and Applications 35, 5 (2015), 84–90.
[60]
Peter Lee, Sebastien Bubeck, and Joseph Petro. 2023. Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine. New England Journal of Medicine 388, 13 (2023), 1233–1239.
[61]
Haotian Li, Yun Wang, Q Vera Liao, and Huamin Qu. 2023. Why is AI not a Panacea for Data Workers? An Interview Study on Human-AI Collaboration in Data Storytelling. arXiv preprint arXiv:2304.08366 (2023).
[62]
Haotian Li, Yong Wang, Aoyu Wu, Huan Wei, and Huamin Qu. 2022. Structure-aware Visualization Retrieval. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–14.
[63]
Haotian Li, Min Xu, Yong Wang, Huan Wei, and Huamin Qu. 2021. A Visual Analytics Approach to Facilitate the Proctoring of Online Exams. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–17.
[64]
Haotian Li, Lu Ying, Haidong Zhang, Yingcai Wu, Huamin Qu, and Yun Wang. 2023. Notable: On-the-fly Assistant for Data Storytelling in Computational Notebooks. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–16.
[65]
Lucy Li and David Bamman. 2021. Gender and Representation Bias in GPT-3 Generated Stories. In Proceedings of the Third Workshop on Narrative Understanding. 48–55.
[66]
Wenchao Li, Sarah Schöttler, James Scott-Brown, Yun Wang, Siming Chen, Huamin Qu, and Benjamin Bach. 2023. NetworkNarratives: Data Tours for Visual Network Exploration and Analysis. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15.
[67]
Wenchao Li, Zhan Wang, Yun Wang, Di Weng, Liwenhan Xie, Siming Chen, Haidong Zhang, and Huamin Qu. 2023. GeoCamera: Telling Stories in Geographic Visualizations with Camera Movements. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–15.
[68]
Joseph CR Licklider. 1960. Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics1 (1960), 4–11.
[69]
Yanna Lin, Haotian Li, Aoyu Wu, Yong Wang, and Huamin Qu. 2023. DMiner: Dashboard Design Mining and Recommendation. IEEE Transactions on Visualization and Computer Graphics (2023).
[70]
Yanna Lin, Haotian Li, Leni Yang, Aoyu Wu, and Huamin Qu. 2023. InkSight: Leveraging Sketch Interaction for Documenting Chart Findings in Computational Notebooks. arXiv preprint arXiv:2307.07922 (2023).
[71]
Dongyu Liu, Di Weng, Yuhong Li, Jie Bao, Yu Zheng, Huamin Qu, and Yingcai Wu. 2017. SmartAdP: Visual Analytics of Large-scale Taxi Trajectories for Selecting Billboard Locations. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2017), 1–10.
[72]
Shuqi Liu, Mingtian Tao, Yifei Huang, Changbo Wang, and Chenhui Li. 2022. Image-Driven Harmonious Color Palette Generation for Diverse Information Visualization. IEEE Transactions on Visualization and Computer Graphics (2022).
[73]
Paul Benjamin Lowry, Aaron Curtis, and Michelle René Lowry. 2004. Building a Taxonomy and Nomenclature of Collaborative Writing to Improve Interdisciplinary Research and Practice. The Journal of Business Communication 41, 1 (2004), 66–99.
[74]
Junhua Lu, Wei Chen, Hui Ye, Jie Wang, Honghui Mei, Yuhui Gu, Yingcai Wu, Xiaolong Luke Zhang, and Kwan-Liu Ma. 2021. Automatic Generation of Unit Visualization-based Scrollytelling for Impromptu Data Facts Delivery. In Proceedings of the 2021 IEEE Pacific Visualization Symposium. IEEE, 21–30.
[75]
Junhua Lu, Jie Wang, Hui Ye, Yuhui Gu, Zhiyu Ding, Mingliang Xu, and Wei Chen. 2020. Illustrating Changes in Time-Series Data With Data Video. IEEE Computer Graphics and Applications 40, 2 (2020), 18–31.
[76]
Todd Lubart. 2005. How can Computers be Partners in the Creative Process: Classification and Commentary on the Special Issue. International Journal of Human-Computer Studies 63, 4-5 (2005), 365–369.
[77]
Yaoli Mao, Dakuo Wang, Michael Muller, Kush R Varshney, Ioana Baldini, Casey Dugan, and Aleksandra Mojsilović. 2019. How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?Proceedings of the ACM on Human-Computer Interaction 3, GROUP (2019), 1–23.
[78]
Andreas Mathisen, Tom Horak, Clemens Nylandsted Klokmose, Kaj Grønbæk, and Niklas Elmqvist. 2019. InsideInsights: Integrating Data-Driven Reporting in Collaborative Visual Analytics. Computer Graphics Forum 38, 3 (2019), 649–661.
[79]
Liam McNabb and Robert S Laramee. 2017. Survey of Surveys (SoS) - Mapping The Landscape of Survey Papers in Information Visualization. Computer Graphics Forum 36, 3 (2017), 589–617.
[80]
Ronald Metoyer, Qiyu Zhi, Bart Janczuk, and Walter Scheirer. 2018. Coupling Story to Visualization: Using Textual Analysis as a Bridge Between Data and Interpretation. In Proceedings of the 23rd International Conference on Intelligent User Interfaces. 503–507.
[81]
Yao Ming, Huamin Qu, and Enrico Bertini. 2018. RuleMatrix: Visualizing and Understanding Classifiers with Rules. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 342–352.
[82]
Eric Mörth, Stefan Bruckner, and Noeska N Smit. 2022. ScrollyVis: Interactive Visual Authoring of Guided Dynamic Narratives for Scientific Scrollytelling. IEEE Transactions on Visualization and Computer Graphics (2022).
[83]
Humphrey O Obie, Caslon Chua, Iman Avazpour, Mohamed Abdelrazek, John Grundy, and Tomasz Bednarz. 2020. Authoring Logically Sequenced Visual Data Stories with Gravity. Journal of Computer Languages 58 (2020), 100961.
[84]
Humphrey O Obie, Dac Thanh Chuong Ho, Iman Avazpour, John Grundy, Mohamed Abdelrazek, Tomasz Bednarz, and Caslon Chua. 2022. Gravity++: A Graph-based Framework for Constructing Interactive Visualization Narratives. Journal of Computer Languages 71 (2022), 101125.
[85]
OpenAI. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 (2023).
[86]
Raja Parasuraman, Thomas B Sheridan, and Christopher D Wickens. 2000. A Model for Types and Levels of Human Interaction with Automation. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 30, 3 (2000), 286–297.
[87]
Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative Agents: Interactive Simulacra of Human Behavior. arXiv preprint arXiv:2304.03442 (2023).
[88]
Ilona R Posner and Ronald M Baecker. 1992. How People Write Together (Groupware). In Proceedings of the 25th Hawaii International Conference on System Sciences, Vol. 4. IEEE, 127–138.
[89]
Xiaoying Pu, Sean Kross, Jake M Hofman, and Daniel G Goldstein. 2021. Datamations: Animated Explanations of Data Analysis Pipelines. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
[90]
Chunyao Qian, Shizhao Sun, Weiwei Cui, Jian-Guang Lou, Haidong Zhang, and Dongmei Zhang. 2020. Retrieve-Then-Adapt: Example-based Automatic Generation for Proportion-related Infographics. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 443–452.
[91]
Xuedi Qin, Yuyu Luo, Nan Tang, and Guoliang Li. 2020. Making Data Visualization More Efficient and Effective: A Survey. The VLDB Journal 29 (2020), 93–117.
[92]
Donghao Ren, Matthew Brehmer, Bongshin Lee, Tobias Höllerer, and Eun Kyoung Choe. 2017. ChartAccent: Annotation for Data-Driven Storytelling. In Proceedings of the 2017 IEEE Pacific Visualization Symposium. 230–239.
[93]
Donghao Ren, Bongshin Lee, and Matthew Brehmer. 2018. Charticulator: Interactive Construction of Bespoke Chart Layouts. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 789–799.
[94]
Pengkun Ren, Yi Wang, and Fan Zhao. 2023. Re-understanding of Data Storytelling Tools from a Narrative Perspective. Visual Intelligence 1, 1 (2023), 11.
[95]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684–10695.
[96]
Arvind Satyanarayan and Jeffrey Heer. 2014. Authoring Narrative Visualizations with Ellipsis. Computer Graphics Forum 33, 3 (2014), 361–370.
[97]
Edward Segel and Jeffrey Heer. 2010. Narrative Visualization: Telling Stories with Data. IEEE Transactions on Visualization and Computer Graphics 16, 6 (2010), 1139–1148.
[98]
Rita Sevastjanova, Mennatallah El-Assady, Adam Bradley, Christopher Collins, Miriam Butt, and Daniel Keim. 2021. VisInReport: Complementing Visual Discourse Analytics Through Personalized Insight Reports. IEEE Transactions on Visualization and Computer Graphics 28, 12 (2021), 4757–4769.
[99]
Leixian Shen, Yizhi Zhang, Haidong Zhang, and Yun Wang. 2023. Data Player: Automatic Generation of Data Videos with Narration-Animation Interplay. arXiv preprint arXiv:2308.04703 (2023).
[100]
Danqing Shi, Fuling Sun, Xinyue Xu, Xingyu Lan, David Gotz, and Nan Cao. 2021. AutoClips: An Automatic Approach to Video Generation from Data Facts. Computer Graphics Forum 40, 3 (2021), 495–505.
[101]
Danqing Shi, Xinyue Xu, Fuling Sun, Yang Shi, and Nan Cao. 2020. Calliope: Automatic Visual Data Story Generation from a Spreadsheet. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 453–463.
[102]
Yang Shi, Tian Gao, Xiaohan Jiao, and Nan Cao. 2023. Understanding Design Collaboration Between Designers and Artificial Intelligence: A Systematic Literature Review. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1–35.
[103]
Yang Shi, Pei Liu, Siji Chen, Mengdi Sun, and Nan Cao. 2022. Supporting Expressive and Faithful Pictorial Visualization Design with Visual Style Transfer. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 236–246.
[104]
Minjeong Shin, Joohee Kim, Yunha Han, Lexing Xie, Mitchell Whitelaw, Bum Chul Kwon, Sungahn Ko, and Niklas Elmqvist. 2022. Roslingifier: Semi-Automated Storytelling for Animated Scatterplots. IEEE Transactions on Visualization and Computer Graphics (2022).
[105]
Ben Shneiderman. 2007. Creativity Support Tools: Accelerating Discovery and Innovation. Commun. ACM 50, 12 (2007), 20–32.
[106]
Ben Shneiderman and Pattie Maes. 1997. Direct Manipulation vs. Interface Agents. Interactions 4, 6 (1997), 42–61.
[107]
Xinhuan Shu, Aoyu Wu, Junxiu Tang, Benjamin Bach, Yingcai Wu, and Huamin Qu. 2020. What Makes a Data-GIF Understandable?IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1492–1502.
[108]
Arjun Srinivasan, Steven M Drucker, Alex Endert, and John Stasko. 2018. Augmenting Visualizations with Interactive Data Facts to Facilitate Interpretation and Communication. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 672–681.
[109]
Manuel Stein, Halldor Janetzko, Andreas Lamprecht, Thorsten Breitkreutz, Philipp Zimmermann, Bastian Goldlücke, Tobias Schreck, Gennady Andrienko, Michael Grossniklaus, and Daniel A Keim. 2017. Bring It to the Pitch: Combining Video and Movement Data to Enhance Team Sport Analysis. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2017), 13–22.
[110]
Hariharan Subramonyam and Jessica Hullman. 2023. Are We Closing the Loop Yet? Gaps in the Generalizability of VIS4ML Research. arXiv preprint arXiv:2308.06290 (2023).
[111]
Nicole Sultanum, Fanny Chevalier, Zoya Bylinskii, and Zhicheng Liu. 2021. Leveraging Text-Chart Links to Support Authoring of Data-Driven Articles with VizFlow. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–17.
[112]
Nicole Sultanum and Arjun Srinivasan. 2023. DataTales: Investigating the use of Large Language Models for Authoring Data-Driven Articles. arXiv preprint arXiv:2308.04076 (2023).
[113]
Mengdi Sun, Ligan Cai, Weiwei Cui, Yanqiu Wu, Yang Shi, and Nan Cao. 2022. Erato: Cooperative Data Story Editing via Fact Interpolation. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 983–993.
[114]
Bo Tang, Shi Han, Man Lung Yiu, Rui Ding, and Dongmei Zhang. 2017. Extracting Top-K Insights from Multi-dimensional Data. In Proceedings of the 2017 ACM International Conference on Management of Data. 1509–1524.
[115]
Tan Tang, Junxiu Tang, Jiewen Lai, Lu Ying, Yingcai Wu, Lingyun Yu, and Peiran Ren. 2022. SmartShots: An Optimization Approach for Generating Videos with Data Visualizations Embedded. ACM Transactions on Interactive Intelligent Systems 12, 1 (2022), 1–21.
[116]
John R Thompson, Zhicheng Liu, and John Stasko. 2021. Data Animator: Authoring Expressive Animated Data Graphics. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–18.
[117]
Alice Thudt, Jagoda Walny, Theresia Gschwandtner, Jason Dykes, and John Stasko. 2018. Exploration and Explanation in Data-Driven Storytelling. In Data-Driven Storytelling. AK Peters/CRC Press, 59–83.
[118]
Chao Tong, Richard C. Roberts, Rita Borgo, Sean P. Walton, Robert S. Laramee, Kodzo Wegba, Aidong Lu, Yun Wang, Huamin Qu, Qiong Luo, and Xiaojuan Ma. 2018. Storytelling and Visualization: An Extended Survey. Information 9, 3 (2018), 65.
[119]
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288 (2023).
[120]
Anjul Tyagi, Jian Zhao, Pushkar Patel, Swasti Khurana, and Klaus Mueller. 2022. Infographics Wizard: Flexible Infographics Authoring and Design Exploration. Computer Graphics Forum 41, 3 (2022), 121–132.
[121]
Marialena Vagia, Aksel A Transeth, and Sigurd A Fjerdingen. 2016. A Literature Review on the Levels of Automation during the Years. What are the Different Taxonomies that have been Proposed?Applied Ergonomics 53 (2016), 190–202.
[122]
Dakuo Wang, Elizabeth Churchill, Pattie Maes, Xiangmin Fan, Ben Shneiderman, Yuanchun Shi, and Qianying Wang. 2020. From Human-Human Collaboration to Human-AI Collaboration: Designing AI Systems That Can Work Together with People. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–6.
[123]
Dakuo Wang, Justin D Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019. Human-AI Collaboration in Data Science: Exploring Data Scientists’ Perceptions of Automated AI. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–24.
[124]
Fengjie Wang, Xuye Liu, Oujing Liu, Ali Neshati, Tengfei Ma, Min Zhu, and Jian Zhao. 2023. Slide4N: Creating Presentation Slides from Computational Notebooks with Human-AI Collaboration. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–18.
[125]
Junpeng Wang, Shixia Liu, and Wei Zhang. 2023. Visual Analytics For Machine Learning: A Data Perspective Survey. arXiv preprint arXiv:2307.07712 (2023).
[126]
Qianwen Wang, Zhutian Chen, Yong Wang, and Huamin Qu. 2021. A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization. IEEE Transactions on Visualization and Computer Graphics 28, 12 (2021), 5134–5153.
[127]
Xumeng Wang, Ziliang Wu, Wenqi Huang, Yating Wei, Zhaosong Huang, Mingliang Xu, and Wei Chen. 2023. VIS+AI: Integrating Visualization with Artificial Intelligence for Efficient Data Analysis. Frontiers of Computer Science 17, 6 (2023), 1–12.
[128]
Xingbo Wang, Haipeng Zeng, Yong Wang, Aoyu Wu, Zhida Sun, Xiaojuan Ma, and Huamin Qu. 2020. VoiceCoach: Interactive Evidence-based Training for Voice Modulation Skills in Public Speaking. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
[129]
Yun Wang, Yi Gao, Ray Huang, Weiwei Cui, Haidong Zhang, and Dongmei Zhang. 2021. Animated Presentation of Static Infographics with InfoMotion. Computer Graphics Forum 40, 3 (2021), 507–518.
[130]
Yun Wang, Leixian Shen, Zhengxin You, Xinhuan Shu, Bongshin Lee, John Thompson, Haidong Zhang, and Dongmei Zhang. 2023. WonderFlow: Narration-Centric Design of Animated Data Videos. arXiv preprint arXiv:2308.04040 (2023).
[131]
Yun Wang, Zhida Sun, Haidong Zhang, Weiwei Cui, Ke Xu, Xiaojuan Ma, and Dongmei Zhang. 2019. DataShot: Automatic Generation of Fact Sheets from Tabular Data. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 895–905.
[132]
Yun Wang, Haidong Zhang, He Huang, Xi Chen, Qiufeng Yin, Zhitao Hou, Dongmei Zhang, Qiong Luo, and Huamin Qu. 2018. InfoNice: Easy Creation of Information Graphics. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.
[133]
Zijie J Wang, Alex Kale, Harsha Nori, Peter Stella, Mark E Nunnally, Duen Horng Chau, Mihaela Vorvoreanu, Jennifer Wortman Vaughan, and Rich Caruana. 2022. Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4132–4142.
[134]
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned Language Models are Zero-Shot Learners. In Proceedings of the International Conference on Learning Representations.
[135]
James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda Viégas, and Jimbo Wilson. 2019. The What-If Tool: Interactive Probing of Machine Learning Models. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 56–65.
[136]
Florian Wolf and Edward Gibson. 2005. Representing Discourse Coherence: A Corpus-Based Study. Computational Linguistics 31, 2 (2005), 249–287.
[137]
Kanit Wongsuphasawat, Zening Qu, Dominik Moritz, Riley Chang, Felix Ouk, Anushka Anand, Jock Mackinlay, Bill Howe, and Jeffrey Heer. 2017. Voyager 2: Augmenting Visual Analysis with Partial View Specifications. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2648–2659.
[138]
Aoyu Wu, Yun Wang, Xinhuan Shu, Dominik Moritz, Weiwei Cui, Haidong Zhang, Dongmei Zhang, and Huamin Qu. 2021. AI4VIS: Survey on Artificial Intelligence Approaches for Data Visualization. IEEE Transactions on Visualization and Computer Graphics (2021).
[139]
Haijun Xia, Nathalie Henry Riche, Fanny Chevalier, Bruno De Araujo, and Daniel Wigdor. 2018. DataInk: Direct and Creative Data-Oriented Drawing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.
[140]
Leni Yang, Cindy Xiong, Jason K Wong, Aoyu Wu, and Huamin Qu. 2021. Explaining with Examples: Lessons Learned from Crowdsourced Introductory Description of Information Visualizations. IEEE Transactions on Visualization and Computer Graphics (2021).
[141]
Leni Yang, Xian Xu, XingYu Lan, Ziyan Liu, Shunan Guo, Yang Shi, Huamin Qu, and Nan Cao. 2021. A Design Space for Applying the Freytag’s Pyramid Structure to Data Stories. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 922–932.
[142]
Yilin Ye, Rong Huang, and Wei Zeng. 2022. VISAtlas: An Image-based Exploration and Query System for Large Visualization Collections via Neural Image Embedding. IEEE Transactions on Visualization and Computer Graphics (2022).
[143]
Lu Ying, Yun Wang, Haotian Li, Shuguang Dou, Haidong Zhang, Xinyang Jiang, Huamin Qu, and Yingcai Wu. 2023. Reviving Static Charts into Live Charts. arXiv preprint arXiv:2309.02967 (2023).
[144]
Lin-Ping Yuan, Ziqi Zhou, Jian Zhao, Yiqiu Guo, Fan Du, and Huamin Qu. 2021. InfoColorizer: Interactive Recommendation of Color Palettes for Infographics. IEEE Transactions on Visualization and Computer Graphics 28, 12 (2021), 4252–4266.
[145]
Jiayi Eris Zhang, Nicole Sultanum, Anastasia Bezerianos, and Fanny Chevalier. 2020. DataQuilt: Extracting Visual Elements from Images to Craft Pictorial Visualizations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
[146]
Jian Zhao, Shenyu Xu, Senthil Chandrasegaran, Chris Bryan, Fan Du, Aditi Mishra, Xin Qian, Yiran Li, and Kwan-Liu Ma. 2021. ChartStory: Automated Partitioning, Layout, and Captioning of Charts into Comic-Style Narratives. IEEE Transactions on Visualization and Computer Graphics 29, 2 (2021), 1384–1399.
[147]
Zhenpeng Zhao and Niklas Elmqvist. 2023. The Stories We Tell About Data: Surveying Data-Driven Storytelling Using Visualization. IEEE Computer Graphics and Applications (2023).
[148]
Chengbo Zheng, Dakuo Wang, April Yi Wang, and Xiaojuan Ma. 2022. Telling Stories from Computational Notebooks: AI-Assisted Presentation Slides Creation for Presenting Data Science Work. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–20.
[149]
Sujia Zhu, Guodao Sun, Qi Jiang, Meng Zha, and Ronghua Liang. 2020. A Survey on Automatic Infographics and Visualization Recommendations. Visual Informatics 4, 3 (2020), 24–40.
[150]
Jonathan Zong, Josh Pollock, Dylan Wootton, and Arvind Satyanarayan. 2022. Animated Vega-Lite: Unifying Animation with a Grammar of Interactive Graphics. IEEE Transactions on Visualization and Computer Graphics 29, 1 (2022), 149–159.

Cited By

View all
  • (2024)Human+Human or Human+AI: A Comparative Analysis of a Narrative Visualization Design GuideProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702070(1-13)Online publication date: 7-Oct-2024
  • (2024)Personalized Generative Storytelling with AI-Visual Illustrations for the Promotion of Knowledge in Cultural Heritage TourismProceedings of the 6th workshop on the analySis, Understanding and proMotion of heritAge Contents10.1145/3689094.3689465(28-32)Online publication date: 28-Oct-2024
  • (2024)Exploring the Impact of AI-generated Image Tools on Professional and Non-professional Users in the Art and Design FieldsCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681890(451-458)Online publication date: 11-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
May 2024
18961 pages
ISBN:9798400703300
DOI:10.1145/3613904
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. Data storytelling
  2. human-AI collaboration

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

CHI '24

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2,176
  • Downloads (Last 6 weeks)343
Reflects downloads up to 09 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Human+Human or Human+AI: A Comparative Analysis of a Narrative Visualization Design GuideProceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems10.1145/3702038.3702070(1-13)Online publication date: 7-Oct-2024
  • (2024)Personalized Generative Storytelling with AI-Visual Illustrations for the Promotion of Knowledge in Cultural Heritage TourismProceedings of the 6th workshop on the analySis, Understanding and proMotion of heritAge Contents10.1145/3689094.3689465(28-32)Online publication date: 28-Oct-2024
  • (2024)Exploring the Impact of AI-generated Image Tools on Professional and Non-professional Users in the Art and Design FieldsCompanion Publication of the 2024 Conference on Computer-Supported Cooperative Work and Social Computing10.1145/3678884.3681890(451-458)Online publication date: 11-Nov-2024
  • (2024)“I Came Across a Junk”: Understanding Design Flaws of Data Visualization from the Public's PerspectiveIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345634131:1(393-403)Online publication date: 10-Sep-2024
  • (2024)CoInsight: Visual Storytelling for Hierarchical Tables With Connected InsightsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.338855330:6(3049-3061)Online publication date: 15-Apr-2024
  • (2024)Hierarchically Recognizing Vector Graphics and A New Chart-Based Vector Graphics DatasetIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.339429846:12(7556-7573)Online publication date: Dec-2024
  • (2024)From Data to Story: Towards Automatic Animated Data Video Creation with LLM-Based Multi-Agent Systems2024 IEEE VIS Workshop on Data Storytelling in an Era of Generative AI (GEN4DS)10.1109/GEN4DS63889.2024.00008(20-27)Online publication date: 13-Oct-2024
  • (2024)Constraint Representation Towards Precise Data-Driven Storytelling2024 IEEE VIS Workshop on Data Storytelling in an Era of Generative AI (GEN4DS)10.1109/GEN4DS63889.2024.00006(4-12)Online publication date: 13-Oct-2024
  • (2024)Teaching Data Storytelling in a Computational Thinking Course for Students in Communications CareersInteractive Storytelling10.1007/978-3-031-78450-7_11(175-185)Online publication date: 19-Dec-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Full Text

View this article in Full Text.

Full Text

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media