research-article

Data Quality, Mismatched Expectations, and Moving Requirements: The Challenges of User-Centred Dashboard Design

Authors:

Mohammed Alhamadi,

Markel VigoAuthors Info & Claims

NordiCHI '22: Nordic Human-Computer Interaction Conference

Article No.: 11, Pages 1 - 14

https://doi.org/10.1145/3546155.3546708

Published: 08 October 2022 Publication History

Abstract

Interactive information dashboards can help both specialists and the general public understand complex datasets; but interacting with these dashboards often presents users with challenges such as understanding and verifying the presented information. To overcome these challenges, developers first need to acquire a thorough understanding of user perspectives, including strategies that users take when presented with problematic dashboards. We interviewed seventeen dashboard developers to establish (i) their understanding of user problems, (ii) the adaptations introduced as a result, and (iii) whether user-tailored dashboards can cater for users’ individual differences. We find that users’ literacy does not typically align with that required to use dashboards, while dashboard developers struggle with keeping up with changing requirements. We also find that developers are able to propose solutions to most users’ problems but not all. Encouragingly, our findings also highlight that tailoring dashboards to individual user needs is not only desirable, but also feasible. These findings inform future dashboard design recommendations that can mitigate the identified challenges including recommendations for data presentation and visual literacy.

References

[1]

Iosif Alvertis, Sotiris Koussouris, Dimitris Papaspyros, Evangelos Arvanitakis, Spiros Mouzakitis, Sebastian Franken, Sabine Kolvenbach, and Wolgang Prinz. 2016. User involvement in software development processes. Procedia Computer Science 97 (2016), 73–83.

[2]

Humberto Lidio Antonelli, Sandra Souza Rodrigues, Willian Massami Watanabe, and Renata Pontin de Mattos Fortes. 2018. A survey on accessibility awareness of Brazilian web developers. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion. 71–79.

Digital Library

[3]

Amy Barnett, Michelle Winning, Stephen Canaris, Michael Cleary, Andrew Staib, and Clair Sullivan. 2019. Digital transformation of hospital quality and safety: real-time data for real-time action. Australian Health Review 43, 6 (2019), 656–661.

[4]

Ben Bausili and Mat Hughes. 2016. Designing Efficient Production Dashboards. Technical Report. Tableau. Retrieved February 10, 2022 from https://www.tableau.com/sites/default/files/2021-10/Designing-Efficient-Workbooks-2021-Interworks_0.pdf

[5]

Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.

[6]

Peter Brusilovski, Alfred Kobsa, and Wolfgang Nejdl. 2007. The adaptive web: methods and strategies of web personalization. Vol. 4321. Springer Science & Business Media.

[7]

Sandra C Buttigieg, Adriana Pace, and Cheryl Rathert. 2017. Hospital performance dashboards: a literature review. Journal of health organization and management (2017).

[8]

CDC. 2020. COVIDView. Retrieved April 10, 2020 from cdc.gov/coronavirus/2019-ncov/covid-data/covidview/

[9]

Sven Charleer, Kathrin Gerling, Francisco Gutiérrez, Hans Cauwenbergh, Bram Luycx, and Katrien Verbert. 2018. Real-time dashboards to support esports spectating. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play. 59–71.

Digital Library

[10]

Eun Kyoung Choe, Bongshin Lee, Haining Zhu, Nathalie Henry Riche, and Dominikus Baur. 2017. Understanding self-reflection: how people reflect on personal data through visual data exploration. In Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare. 173–182.

Digital Library

[11]

L Cohen. 2017. Impacts of business intelligence on population health: a systematic literature review. In Proceedings of the South African Institute of Computer Scientists and Information Technologists. 1–9.

Digital Library

[12]

Ashley Colley, Kirsi Halttu, Marja Harjumaa, and Harri Oinas-Kukkonen. 2016. Insights from the design and evaluation of a personal health dashboard. In 2016 49th Hawaii International Conference on System Sciences (HICSS). IEEE, 3483–3492.

Digital Library

[13]

Inès Dabbebi, Sébastien Iksal, Jean-Marie Gilliot, Madeth May, and Serge Garlatti. 2017. Towards adaptive dashboards for learning analytic: An approach for conceptual design and implementation.

[14]

Joan Morris DiMicco and Nancy Mann. 2016. User research to inform product design: turning failure into small successes. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 872–879.

Digital Library

[15]

Ensheng Dong, Hongru Du, and Lauren Gardner. 2020. An interactive web-based dashboard to track COVID-19 in real time. The Lancet infectious diseases 20, 5 (2020), 533–534.

[16]

Dawn Dowding, Jacqueline A Merrill, Nicole Onorato, Yolanda Barrón, Robert J Rosati, and David Russell. 2018. The impact of home care nurses’ numeracy and graph literacy on comprehension of visual display information: implications for dashboard design. Journal of the American Medical Informatics Association 25, 2(2018), 175–182.

[17]

Dawn Dowding, Rebecca Randell, Peter Gardner, Geraldine Fitzpatrick, Patricia Dykes, Jesus Favela, Susan Hamer, Zac Whitewood-Moores, Nicholas Hardiker, Elizabeth Borycki, 2015. Dashboards for improving patient care: review of the literature. International journal of medical informatics 84, 2(2015), 87–100.

[18]

Vanessa Echeverria, Roberto Martinez-Maldonado, Roger Granda, Katherine Chiluiza, Cristina Conati, and Simon Buckingham Shum. 2018. Driving data storytelling from learning design. In Proceedings of the 8th international conference on learning analytics and knowledge. 131–140.

Digital Library

[19]

Wayne W Eckerson. 2010. Performance dashboards: measuring, monitoring, and managing your business. John Wiley & Sons.

[20]

Micheline Elias and Anastasia Bezerianos. 2012. Annotating bi visualization dashboards: Needs & challenges. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1641–1650.

Digital Library

[21]

Stephen Few. 2006. Information dashboard design: The effective visual communication of data. O’Reilly Media, Inc.

[22]

Howard Forman, Cary Gross, Ayotomiwa Ojo, Sarah Pitafi, Hannah Todd, Eric Mayberry, Shivaji Kumar, and Patrick Hansen. 2021. We Rate Covid Dashboards. https://www.ratecoviddashboard.com/

[23]

Maria-Elena Froese and Melanie Tory. 2016. Lessons learned from designing visualization dashboards. IEEE computer graphics and applications 36, 2 (2016), 83–89.

Digital Library

[24]

Mirta Galesic and Rocio Garcia-Retamero. 2011. Graph literacy: A cross-cultural comparison. Medical Decision Making 31, 3 (2011), 444–457.

[25]

Jean-Marie Gilliot, Sébastien Iksal, Daniel Magloire Medou, and Inès Dabbebi. 2018. Participatory design of learning analytics dashboards. In Proceedings of the 30th Conference on l’Interaction Homme-Machine. 119–127.

Digital Library

[26]

Danielle Hagood, Cynthia Carter Ching, and Sara Schaefer. 2016. Integrating physical activity data in videogames with user-centered dashboards. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge. 530–531.

Digital Library

[27]

Ross Haupt, Brenda Scholtz, and Andre Calitz. 2015. Using business intelligence to support strategic sustainability information management. In Proceedings of the 2015 Annual Research Conference on South African Institute of Computer Scientists and Information Technologists. 1–11.

Digital Library

[28]

Eelco Herder and Olaf van Maaren. 2020. Privacy dashboards: the impact of the type of personal data and user control on trust and perceived risk. In Adjunct publication of the 28th ACM conference on user modeling, adaptation and personalization. 169–174.

[29]

Morten Hertzum, Torkil Clemmensen, Kasper Hornbæk, Jyoti Kumar, Qingxin Shi, and Pradeep Yammiyavar. 2011. Personal usability constructs: How people construe usability across nationalities and stakeholder groups. International Journal of Human-Computer Interaction 27, 8(2011), 729–761.

[30]

Ioana Jivet, Maren Scheffel, Marcus Specht, and Hendrik Drachsler. 2018. License to evaluate: Preparing learning analytics dashboards for educational practice. In Proceedings of the 8th international conference on learning analytics and knowledge. 31–40.

Digital Library

[31]

Simon L Jones. 2015. Exploring correlational information in aggregated quantified self data dashboards. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers. 1075–1080.

Digital Library

[32]

Fatemeh Salehian Kia, Stephanie D Teasley, Marek Hatala, Stuart A Karabenick, and Matthew Kay. 2020. How patterns of students dashboard use are related to their achievement and self-regulatory engagement. In Proceedings of the tenth international conference on learning analytics & knowledge. 340–349.

Digital Library

[33]

Rob Kitchin, Tracey P Lauriault, and Gavin McArdle. 2015. Knowing and governing cities through urban indicators, city benchmarking and real-time dashboards. Regional Studies, Regional Science 2, 1 (2015), 6–28.

[34]

Richelle J Koopman, Karl M Kochendorfer, Joi L Moore, David R Mehr, Douglas S Wakefield, Borchuluun Yadamsuren, Jared S Coberly, Robin L Kruse, Bonnie J Wakefield, and Jeffery L Belden. 2011. A diabetes dashboard and physician efficiency and accuracy in accessing data needed for high-quality diabetes care. The Annals of Family Medicine 9, 5 (2011), 398–405.

[35]

Annakaisa Kultima. 2015. Developers’ perspectives on iteration in game development. In Proceedings of the 19th International academic Mindtrek conference. 26–32.

Digital Library

[36]

David Lee, Jesus Ricardo Alvarez Felix, Shan He, Dietmar Offenhuber, and Carlo Ratti. 2015. CityEye: Real-time visual dashboard for managing urban services and citizen feedback loops. In Proceedings of the 14th International Conference on Computing in Urban Planning and Urban Management (CUPUM), Cambridge, MA, USA. 7–10.

[37]

Andreia Filipa Martins, Anabela C Alves, and Celina P Leão. 2017. Development and implementation of dashboards for operational monitoring using participatory design in a lean context. In International Symposium on Qualitative Research. Springer, 237–249.

[38]

Suvodeep Mazumdar, Daniela Petrelli, and Fabio Ciravegna. 2014. Exploring user and system requirements of linked data visualization through a visual dashboard approach. Semantic Web 5, 3 (2014), 203–220.

Digital Library

[39]

Saudi MOH. 2020. Saudi COVID-19 Dashboard. Retrieved May 21, 2020 from covid19.moh.gov.sa/

[40]

NHS. 2020. UK COVID-19. Retrieved April 10, 2020 from www.gov.uk/government/publications/covid-19-track-coronavirus-cases

[41]

Apurva Pathak, Bidyut Kr Patra, Arnab Chakraborty, and Abhishek Agarwal. 2015. A city traffic dashboard using social network data. In Proceedings of the 2nd IKDD Conference on Data Sciences. 1–4.

Digital Library

[42]

Firaz Peer. 2019. Community Indicator Data Dashboards as Infrastructures for Data Literacy. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion. 109–112.

Digital Library

[43]

Bernhard Peischl and Sandra Lang. 2013. What Can We Learn from In-process Metrics on Issue Management?–Insights from an Industrial Case Study. In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops. IEEE, 124–125.

Digital Library

[44]

James Richardson, Kurt Schlegel, Rita Sallam, Austin Kronz, and Julian Sun. 2021. Magic Quadrant for Analytics and Business Intelligence Platforms. Technical Report. Gartner.

[45]

Lynne D Roberts, Joel A Howell, and Kristen Seaman. 2017. Give me a customizable dashboard: Personalized learning analytics dashboards in higher education. Technology, Knowledge and Learning 22, 3 (2017), 317–333.

[46]

Elizabeth Rojas, Viviana Bastidas, and Christian Cabrera. 2020. Cities-Board: A Framework to Automate the Development of Smart Cities Dashboards. IEEE Internet of Things Journal 7, 10 (2020), 10128–10136.

[47]

Alper Sarikaya, Michael Correll, Lyn Bartram, Melanie Tory, and Danyel Fisher. 2018. What do we talk about when we talk about dashboards?IEEE transactions on visualization and computer graphics 25, 1(2018), 682–692.

[48]

Beat A Schwendimann, Maria Jesus Rodriguez-Triana, Andrii Vozniuk, Luis P Prieto, Mina Shirvani Boroujeni, Adrian Holzer, Denis Gillet, and Pierre Dillenbourg. 2016. Perceiving learning at a glance: A systematic literature review of learning dashboard research. IEEE Transactions on Learning Technologies 10, 1 (2016), 30–41.

Digital Library

[49]

Ben Shneiderman. 1997. Designing the User Interface: Strategies for Effective Human-Computer Interaction (3rd ed.). Addison-Wesley Longman Publishing Co., Inc., USA.

Digital Library

[50]

Ben Shneiderman. 2003. The eyes have it: A task by data type taxonomy for information visualizations. In The craft of information visualization. Elsevier, 364–371.

[51]

Arjun Srinivasan, Matthew Brehmer, Bongshin Lee, and Steven M Drucker. 2018. What’s the Difference? Evaluating Variations of Multi-Series Bar Charts for Visual Comparison Tasks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–12.

Digital Library

[52]

Kamonphop Srisopha, Daniel Link, and Barry Boehm. 2021. How should developers respond to app reviews? features predicting the success of developer responses. In Evaluation and Assessment in Software Engineering. 119–128.

[53]

Kaiwen Sun, Abraham H Mhaidli, Sonakshi Watel, Christopher A Brooks, and Florian Schaub. 2019. It’s my data! Tensions among stakeholders of a learning analytics dashboard. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–14.

Digital Library

[54]

Helena Tendedez, Roisin McNaney, Maria-Angela Ferrario, and Jon Whittle. 2018. Scoping the Design Space for Data Supported Decision-Making Tools in Respiratory Care: Needs, Barriers and Future Aspirations. In Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare. 217–226.

Digital Library

[55]

Henri Tokola, Christoph Gröger, Eeva Järvenpää, and Esko Niemi. 2016. Designing manufacturing dashboards on the basis of a Key Performance Indicator survey. Procedia CIRP 57(2016), 619–624.

[56]

Melanie Tory, Lyn Bartram, Brittany Fiore-Gartland, and Anamaria Crisan. 2021. Finding Their Data Voice: Practices and Challenges of Dashboard Users. IEEE Computer Graphics and Applications(2021).

[57]

Andrea Vázquez-Ingelmo, Francisco José García-Peñalvo, and Roberto Therón. 2019. Tailored information dashboards: A systematic mapping of the literature. In Proceedings of the XX International Conference on Human Computer Interaction. 1–8.

Digital Library

[58]

Andrea Vázquez-Ingelmo, Francisco José García-Peñalvo, Roberto Therón, and Miguel Ángel Conde. 2020. Representing Data Visualization Goals and Tasks Through Meta-Modeling to Tailor Information Dashboards. Applied Sciences 10, 7 (2020), 2306.

[59]

Andrea Vázquez-Ingelmo, Francisco José García-Peñalvo, Roberto Theron, Daniel Amo Filvà, and David Fonseca Escudero. 2019. Connecting domain-specific features to source code: Towards the automatization of dashboard generation. Cluster Computing (2019), 1–14.

[60]

Heike Vornhagen, Brian Davis, and Manel Zarrouk. 2018. Sensemaking of complex sociotechnical systems: the case of governance dashboards. In Proceedings of the 19th Annual International Conference on Digital Government Research: Governance in the Data Age. 1–2.

Digital Library

[61]

Simon Wakeling, Paul Clough, James Wyper, and Amy Balmain. 2015. Graph literacy and business intelligence: Investigating user understanding of dashboard data visualizations. Business Intelligence Journal 20, 4 (2015), 8–19.

[62]

Anne Marie JWM Weggelaar-Jansen, Damien SE Broekharst, and Martine De Bruijne. 2018. Developing a hospital-wide quality and safety dashboard: a qualitative research study. BMJ quality & safety 27, 12 (2018), 1000–1007.

[63]

WHO. 2020. WHO Health Emergency Dashboard. Retrieved April 10, 2020 from covid19.who.int/

[64]

Ainhoa Yera, Javier Muguerza, Olatz Arbelaitz, Iñigo Perona, Richard N Keers, Darren M Ashcroft, Richard Williams, Niels Peek, Caroline Jay, and Markel Vigo. 2019. Modelling the interactive behaviour of users with a medication safety dashboard in a primary care setting. International journal of medical informatics 129 (2019), 395–403.

[65]

Ogan M Yigitbasioglu and Oana Velcu. 2012. A review of dashboards in performance management: Implications for design and research. International Journal of Accounting Information Systems 13, 1(2012), 41–59.

[66]

Dong Whi Yoo and Munmun De Choudhury. 2019. Designing dashboard for campus stakeholders to support college student mental health. In Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare. 61–70.

Digital Library

Cited By

Nguyen NLithander MÖstlund CKarunaratne TJobe W(2024)TEADASH: Implementing and Evaluating a Teacher-Facing Dashboard Using Design Science ResearchInformatics10.3390/informatics1103006111:3(61)Online publication date: 26-Aug-2024
https://doi.org/10.3390/informatics11030061
Dolatabadi SGatial EBudinská IBalogh Z(2024)Integrating Human-Computer Interaction Principles in User-Centered Dashboard Design: Insights from Maintenance Management2024 IEEE 28th International Conference on Intelligent Engineering Systems (INES)10.1109/INES63318.2024.10629098(000219-000224)Online publication date: 17-Jul-2024
https://doi.org/10.1109/INES63318.2024.10629098

Recommendations

Exploring user and system requirements of linked data visualization through a visual dashboard approach

One of the open problems in Semantic Web research is which tools should be provided to users to explore linked data. This is even more urgent now that massive amount of linked data is being released by governments worldwide. The development of single ...
A Wearable User's Dashboard and Turn Signals

Wearable computers should provide users with constant, useful, and usable information that they can refer to when they want andthat lets them do common tasks simply. This is akin to a car's dashboard and turn signals. The author discusses how to display ...
Towards Unified Information Dashboard Design - Common Principles, Practices, and Challenges
Abstract
Information dashboards consolidate large amounts of data into a single visual display, providing instant insights to decision-makers. However, their development is challenging due to limited study on a unified methodology for designing effective ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

NordiCHI '22: Nordic Human-Computer Interaction Conference

October 2022

1091 pages

ISBN:9781450396998

DOI:10.1145/3546155

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

NordiCHI '22

NordiCHI '22: Nordic Human-Computer Interaction Conference

October 8 - 12, 2022

Aarhus, Denmark

Acceptance Rates

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
291
Total Downloads

Downloads (Last 12 months)139
Downloads (Last 6 weeks)21

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nguyen NLithander MÖstlund CKarunaratne TJobe W(2024)TEADASH: Implementing and Evaluating a Teacher-Facing Dashboard Using Design Science ResearchInformatics10.3390/informatics1103006111:3(61)Online publication date: 26-Aug-2024
https://doi.org/10.3390/informatics11030061
Dolatabadi SGatial EBudinská IBalogh Z(2024)Integrating Human-Computer Interaction Principles in User-Centered Dashboard Design: Insights from Maintenance Management2024 IEEE 28th International Conference on Intelligent Engineering Systems (INES)10.1109/INES63318.2024.10629098(000219-000224)Online publication date: 17-Jul-2024
https://doi.org/10.1109/INES63318.2024.10629098

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents