research-article

A Mixed-Initiative Approach to Reusing Infographic Charts

Authors: Weiwei Cui, Jinpeng Wang, He Huang, Yun Wang, Chin-Yew Lin, Haidong Zhang, Dongmei ZhangAuthors Info & Claims

IEEE Transactions on Visualization and Computer Graphics, Volume 28, Issue 1

Pages 173 - 183

https://doi.org/10.1109/TVCG.2021.3114856

Published: 01 January 2022 Publication History

Abstract

Infographic bar charts have been widely adopted for communicating numerical information because of their attractiveness and memorability. However, these infographics are often created manually with general tools, such as PowerPoint and Adobe Illustrator, and merely composed of primitive visual elements, such as text blocks and shapes. With the absence of chart models, updating or reusing these infographics requires tedious and error-prone manual edits. In this paper, we propose a mixed-initiative approach to mitigate this pain point. On one hand, machines are adopted to perform precise and trivial operations, such as mapping numerical values to shape attributes and aligning shapes. On the other hand, we rely on humans to perform subjective and creative tasks, such as changing embellishments or approving the edits made by machines. We encapsulate our technique in a PowerPoint add-in prototype and demonstrate the effectiveness by applying our technique on a diverse set of infographic bar chart examples.

References

[1]

R. A. Al-Zaidy and C. L. Giles. Automatic extraction of data from bar charts. In Proceedings of the 8th international conference on knowledge capture, pages 1–4, 2015.

Digital Library

[2]

S. Bateman, R. L. Mandryk, C. Gutwin, A. Genest, D. McDine, and C. Brooks. Useful junk?: the effects of visual embellishment on comprehension and memorability of charts. In In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 2573–2582. ACM, 2010.

[3]

L. Battle, P. Duan, Z. Miranda, D. Mukusheva, R. Chang, and M. Stonebraker. Beagle: Automated extraction and interpretation of visualizations from the web. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 1–8, 2018.

[4]

M. A. Borkin, Z. Bylinskii, N. W. Kim, C. M. Bainbridge, C. S. Yeh, D. Borkin, H. Pfister, and A. Oliva. Beyond memorability: Visualization recognition and recall. IEEE transactions on visualization and computer graphics, 22 (1): pp. 519–528, 2016.

Digital Library

[5]

J. Brosz, M. A. Nacenta, R. Pusch, S. Carpendale, and C. Hurter. Transmogrification: causal manipulation of visualizations. In Proceedings of the 26th annual ACM symposium on User interface software and technology, pages 97–106, 2013.

[6]

Z. Bylinskii, S. Alsheikh, S. Madan, A. Recasens, K. Zhong, H. Pfister, F. Durand, and A. Oliva. Understanding infographics through textual and visual tag prediction. arXiv preprint arXiv:, 2017.

[7]

Z. Chen, Y. Wang, Q. Wang, Y. Wang, and H. Qu. Towards automated infographic design: Deep learning-based auto-extraction of extensible timeline. IEEE transactions on visualization and computer graphics, 26 (1): pp. 917–926, 2019.

[8]

J. Choi, S. Jung, D. G. Park, J. Choo, and N. Elmqvist. “Visualizing for the non-visual: Enabling the visually impaired to use visualization”. In Computer Graphics Forum, volume 38, pages 249–260. Wiley Online Library, 2019.

[9]

W. Cui, X. Zhang, Y. Wang, H. Huang, B. Chen, L. Fang, H. Zhang, J.-G. Lou, and D. Zhang. Text-to-Viz: Automatic generation of infographics from proportion-related natural language statements. IEEE transactions on visualization and computer graphics, 26 (1): pp. 906–916, 2019.

[10]

S. Elzer, S. Carberry, and I. Zukerman. The automated understanding of simple bar charts. Artificial Intelligence, 175 (2): pp. 526–555, 2011.

Digital Library

[11]

Free power point templates. [Online]. Available: https://www.free-power-point-templates.com/. Accessed: 2021-06-20.

[12]

T. Ge, Y. Zhao, B. Lee, D. Ren, B. Chen, and Y. Wang. “Canis: A high-level language for data-driven chart animations”. In Computer Graphics Forum, volume 39, pages 607–617. Wiley Online Library, 2020.

[13]

J. Harper and M. Agrawala. Deconstructing and restyling d3 visualizations. In Proceedings of the 27th annual ACM symposium on User interface software and technology, pages 253–262, 2014.

Digital Library

[14]

J. Harper and M. Agrawala. Converting basic d3 charts into reusable style templates. IEEE transactions on visualization and computer graphics, 24 (3): pp. 1274–1286, 2017.

[15]

R. L. Harris. Information graphics: A comprehensive illustrated reference. Oxford University Press, 2000.

Digital Library

[16]

L. Harrison, K. Reinecke, and R. Chang. Infographic aesthetics: Designing for the first impression. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 1187–1190. ACM, 2015.

[17]

E. Hoque and M. Agrawala. Searching the visual style and structure of d3 visualizations. IEEE transactions on visualization and computer graphics, 26 (1): pp. 1236–1245, 2019.

[18]

S. C. Johnson. Hierarchical clustering schemes. Psychometrika, 32 (3): pp. 241–254, 1967.

[19]

D. Jung, W. Kim, H. Song, J.-I. Hwang, B. Lee, B. Kim, and J. Seo. ChartSense: Interactive data extraction from chart images. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 6706–6717, 2017.

[20]

G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in neural information processing systems, pages 3146–3154, 2017.

[21]

N. W. Kim, E. Schweickart, Z. Liu, M. Dontcheva, W. Li, J. Popovic, and H. Pfister. Data-driven guides: Supporting expressive design for information graphics. IEEE transactions on visualization and computer graphics, 23 (1): pp. 491–500, 2017.

Digital Library

[22]

Y. Kim and J. Heer. Gemini: A grammar and recommender system for animated transitions in statistical graphics. IEEE Transactions on Visualization and Computer Graphics, 27 (2): pp. 485–494, 2020.

[23]

N. Kong and M. Agrawala. Graphical overlays: Using layered elements to aid chart reading. IEEE transactions on visualization and computer graphics, 18 (12): pp. 2631–2638, 2012.

Digital Library

[24]

H. W. Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2 (1–2): pp. 83–97, 1955.

[25]

Z. Liu, J. Thompson, A. Wilson, M. Dontcheva, J. Delorey, S. Grigg, B. Kerr, and J. Stasko. Data illustrator: Augmenting vector design tools with lazy data binding for expressive visualization authoring. In Proceedings of the CHI Conference on Human Factors in Computing Systems, page 123. ACM, 2018.

[26]

M. Lu, C. Wang, J. Lanir, N. Zhao, H. Pfister, D. Cohen-Or, and H. Huang. Exploring visual information flows in infographics. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 1–12, 2020.

[27]

S. Madan, Z. Bylinskii, M. Tancik, A. Recasens, K. Zhong, S. Al-sheikh, H. Pfister, A. Oliva, and F. Durand. Synthetically trained icon proposals for parsing and summarizing infographics. arXiv preprint arXiv:, 2018.

[28]

G. G. Méndez, M. A. Nacenta, and S. Vandenheste. iVoLVER: Interactive visual language for visualization extraction and reconstruction. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 4073–4085. ACM, 2016.

[29]

A. V. Moere and H. Purchase. On the role of design in information visualization. Information Visualization, 10 (4): pp. 356–371, 2011.

Digital Library

[30]

J. Poco and J. Heer. “Reverse-engineering visualizations: Recovering visual encodings from chart images”. In Computer Graphics Forum, volume 36, pages 353–363. Wiley Online Library, 2017.

Digital Library

[31]

J. Poco, A. Mayhua, and J. Heer. Extracting and retargeting color mappings from bitmap images of visualizations. IEEE transactions on visualization and computer graphics, 24 (1): pp. 637–646, 2017.

[32]

C. Qian, S. Sun, W. Cui, J.-G. Lou, H. Zhang, and D. Zhang. Retrieve-Then-Adapt: Example-based automatic generation for proportion-related infographics. IEEE Transactions on Visualization and Computer Graphics, 27 (2): pp. 443–452, 2020.

[33]

D. Ren, T. Höllerer, and X. Yuan. iVisDesigner: Expressive interactive design of information visualizations. IEEE transactions on visualization and computer graphics, 20 (12): pp. 2092–2101, 2014.

[34]

D. Ren, B. Lee, and M. Brehmer. Charticulator: Interactive construction of bespoke chart layouts. IEEE transactions on visualization and computer graphics, 25 (1): pp. 789–799, 2019.

Digital Library

[35]

A. Satyanarayan and J. Heer. “Lyra: An interactive visualization design environment”. In Computer Graphics Forum, volume 33, pages 351–360. Wiley Online Library, 2014.

[36]

M. Savva, N. Kong, A. Chhajta, L. Fei-Fei, M. Agrawala, and J. Heer. ReVision: Automated classification, analysis and redesign of chart images. In Proceedings of the 24th annual ACM symposium on User interface software and technology, pages 393–402, 2011.

[37]

D. Shi, X. Xu, F. Sun, Y. Shi, and N. Cao. Calliope: Automatic visual data story generation from a spreadsheet. IEEE Transactions on Visualization and Computer Graphics, 27 (2): pp. 453–463, 2020.

[38]

N. Siegel, Z. Horvitz, R. Levin, S. Divvala, and A. Farhadi. FigureSeer: Parsing result-figures in research papers. In European Conference on Computer Vision, pages 664–680. Springer, 2016.

[39]

D. Skau and R. Kosara. Readability and precision in pictorial bar charts. In Proceedings of the Eurographics/IEEE VGTC Conference on Visualization: Short Papers, pages 91–95. Eurographics Association, 2017.

Digital Library

[40]

Slides carnival. [Online]. Available: https://www.slidescarnival.com/. Accessed: 2021-06-20.

[41]

Slidesgo. [Online]. Available: https://slidesgo.com/. Accessed: 2021-06-20.

[42]

Y. Wang, Z. Sun, H. Zhang, W. Cui, K. Xu, X. Ma, and D. Zhang. DataShot: Automatic generation of fact sheets from tabular data. IEEE Transactions on Visualization and Computer Graphics, 26 (1): pp. 895–905. 2019.

[43]

Y. Wang, H. Zhang, H. Huang, X. Chen, Q. Yin, Z. Hou, D. Zhang, Q. Luo, and H. Qu. Infonice: Easy creation of information graphics. In Proceedings of the CHI Conference on Human Factors in Computing Systems, page 335. ACM, 2018.

[44]

A. Wu, W. Tong, T. Dwyer, B. Lee, P. Isenberg, and H. Qu. Mobilevisfixer: Tailoring web visualizations for mobile phones leveraging an explainable reinforcement learning framework. IEEE Transactions on Visualization and Computer Graphics, 27 (2): pp. 464–474, 2020.

[45]

A. Wu, Y. Wang, X. Shu, D. Moritz, W. Cui, H. Zhang, D. Zhang, and H. Qu. Survey on artificial intelligence approaches for visualization data. arXiv preprint arXiv:, 2021.

[46]

A. Wu, L. Xie, B. Lee, Y. Wang, W. Cui, and H. Qu. Learning to automate chart layout configurations using crowdsourced paired comparison. In Proceedings of the CHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2021. Association for Computing Machinery.

[47]

H. Xia, N. Henry Riche, F. Chevalier, B. De Araujo, and D. Wigdor. Dataink: Direct and creative data-oriented drawing. In Proceedings of the CHI Conference on Human Factors in Computing Systems, page 223. ACM, 2018.

[48]

J. E. Zhang, N. Sultanum, A. Bezerianos, and F. Chevalier. Dataquilt: Extracting visual elements from images to craft pictorial visualizations. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pages 1–13, 2020.

Cited By

Li HWang YQu H(2024)Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI CollaborationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642726(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642726
Liu ZChen CHooker J(2024)Manipulable Semantic Components: A Computational Representation of Data Visualization ScenesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345629631:1(732-742)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456296
Chen CLee BWang YChang YLiu Z(2024)Mystique: Deconstructing SVG Charts for Layout ReuseIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332735430:1(447-457)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3327354
Show More Cited By

Recommendations

EXCEL 2007 CHARTS MADE EASY
An improved process monitoring by mixed multivariate memory control charts: An application in wind turbine field
Highlights
- Mixed multivariate memory control charts are proposed for process mean vector.
- ...
Abstract
Memory control chart such as multivariate CUSUM (MCUSUM) and multivariate EWMA (MEWMA) control charts are considered superior for the detection of small-to-moderate variation in the process mean vector. In this article, we have ...
Hierarchical Pixel Bar Charts

Simple presentation graphics are intuitive and easy-to-use, but only show highly aggregated data. Bar charts, for example, only show a rather small number of data values and x-y-plots often have a high degree of overlap. Presentation techniques are ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Visualization and Computer Graphics

IEEE Transactions on Visualization and Computer Graphics Volume 28, Issue 1

Jan. 2022

1190 pages

ISSN:1077-2626

Issue’s Table of Contents

1077-2626 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Educational Activities Department

United States

Publication History

Published: 01 January 2022

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li HWang YQu H(2024)Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI CollaborationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642726(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642726
Liu ZChen CHooker J(2024)Manipulable Semantic Components: A Computational Representation of Data Visualization ScenesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345629631:1(732-742)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456296
Chen CLee BWang YChang YLiu Z(2024)Mystique: Deconstructing SVG Charts for Layout ReuseIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332735430:1(447-457)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3327354
Kim HRossi RHullman JHoffswell J(2024)Dupo: A Mixed-Initiative Authoring Tool for Responsive VisualizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332658330:1(934-943)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3326583
Lu JWang GCai YWu X(2024)Towards Automated Infographic Authoring From Natural Language Statement With Multiple Proportional FactsIEEE Transactions on Multimedia10.1109/TMM.2024.336072226(7101-7113)Online publication date: 31-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3360722
He SChen YXia YLi YLiang HYu L(2024)Visual harmony: text-visual interplay in circular infographicsJournal of Visualization10.1007/s12650-024-00957-327:2(255-271)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s12650-024-00957-3
He L(2022)Industry 4.0 Oriented Distributed Infographic DesignMobile Information Systems10.1155/2022/47432162022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/4743216

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents