Usability Evaluation of Online Questionnaires on Mobile Devices
Pages 521 - 526
Abstract
Participants increasingly use mobile devices, especially smartphones, to fill out online questionnaires. However, standard questionnaire templates are often not optimized for presentation on smartphones, raising the question of whether such layouts may lead to an unfavorable usability assessment. In this article, we compared a questionnaire template specially developed for mobile devices with two standard layouts. For this purpose, online questionnaires with identical content were implemented using the three templates. The participants (N=402) were randomly assigned to one of the three layout conditions. For comparison, the participants processed a comprehensive questionnaire and then evaluated the usability of the respective questionnaire design with the System Usability Scale (SUS). The results reveal that the respondents tend to rate the layout optimized for smartphones better.
References
[1]
Ioannis Andreadis. 2015. Web Surveys Optimized for Smartphones: Are there Differences Between Computer and Smartphone Users? methods, data, analyses 9, 2 (2015), 16.
[2]
John Brooke. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.
[3]
Mario Callegaro. 2010. Do you know which device your respondent has used to take your online survey. Survey Practice 3, 6 (2010), 1--12.
[4]
Mick P Couper and Gregg J Peterson. 2017. Why do web surveys take longer on smartphones? Social Science Computer Review 35, 3 (2017), 357--377.
[5]
Marika De Bruijne and Arnaud Wijnant. 2013. Can mobile web surveys be taken on computers? A discussion on a multi-device survey design. Survey Practice 6, 4 (2013), 1--8.
[6]
Marika De Bruijne and Arnaud Wijnant. 2013. Comparing survey results obtained via mobile devices and computers: An experiment with a mobile web survey on a heterogeneous group of mobile devices versus a computer-assisted web survey. Social Science Computer Review 31, 4 (2013), 482--504.
[7]
EN ISO DIN. 2011. 9241-210: Ergonomie der Mensch-System-Interaktion-Teil 210: Prozess zur Gestaltung gebrauchstauglicher interaktiver Systeme. Berlin: Beuth (2011).
[8]
Emily Geisen and Jennifer Romano Bergstrom. 2017. Usability testing for survey research. Morgan Kaufmann.
[9]
Luminita Giurgiu and Ilie Gligorea. 2017. Responsive Web Design Techniques. In International conference KNOWLEDGE-BASED ORGANIZATION, Vol. 23. De Gruyter Open, 37--42.
[10]
Samuel D Gosling, Peter J Rentfrow, and William B Swann Jr. 2003. A very brief measure of the Big-Five personality domains. Journal of Research in personality 37, 6 (2003), 504--528.
[11]
Reetta Heinonen, Riitta Luoto, Pirjo Lindfors, and Clas-Håkan Nygård. 2012. Usability and feasibility of mobile phone diaries in an experimental physical exercise study. Telemedicine and e-Health 18, 2 (2012), 115--119.
[12]
Rachel Horwitz. 2014. Usability of the ACS Internet Instrument on Mobile Devices 1. In 2014 Proceedings of Statistics Canada Symposium.
[13]
Azham Hussain, Emmanuel OC Mkpojiogu, and Zakaria Hussain. 2015. Usability evaluation of a web-based health awareness portal on Smartphone devices using ISO 9241-11 model. Jurnal Teknologi 77, 4 (2015).
[14]
Jeff Sauro. 2011. Measuring Usability with the System Usability Scale (SUS). https://measuringu.com/sus/ letzter Zugriff: 15.04.2019.
[15]
Philip Kortum and Mary Sorber. 2015. Measuring the usability of mobile applications for phones and tablets. International Journal of Human-Computer Interaction 31, 8 (2015), 518--529.
[16]
Jennie W Lai, Lorelle Vanno, Michael W Link, Jennie Pearson, Hala Makowska, Karen Benezra, and Mark Green. 2010. Life360: Usability of Mobile Devices for Time Use Surveys. Survey Practice 3, 1 (2010), 3022.
[17]
Amber D Lambert and Angie L Miller. 2015. Living with smartphones: Does completion device affect survey responses? Research in Higher Education 56, 2 (2015), 166--177.
[18]
Bettina Laugwitz, Ulf Schubert, Waltraud Ilmberger, Nina Tamm, Theo Held, and Martin Schrepp. 2009. Subjektive Benutzerzufriedenheit quantitativ erfassen: Erfahrungen mit dem User Experience Questionnaire UEQ. In Tagungsband UP09, Henning Brau, Sarah Diefenbach, Marc Hassenzahl, Kirstin Kohler, Franz Koller, Matthias Peissner, Kostanija Petrovic, Meinald Thielsch, Daniel Ullrich, and Dirk Zimmermann (Eds.). Fraunhofer Verlag, Stuttgart, 220--225.
[19]
Limesurvey. 2019. Open Source Survey Application Limesurvey. https://www.limesurvey.org
[20]
Peter Lugtig, Vera Toepoel, and Alerk Amin. 2016. Mobile-only web survey respondents. Survey Practice 9, 3 (2016), 1--8.
[21]
Aigul Mavletova. 2013. Data quality in PC and mobile web surveys. Social Science Computer Review 31, 6 (2013), 725--743.
[22]
Aigul Mavletova and Mick P Couper. 2014. Mobile web survey design: scrolling versus paging, SMS versus e-mail invitations. Journal of Survey Statistics and Methodology 2, 4 (2014), 498--518.
[23]
Kailashkumar V Natda. 2013. Responsive web design. Eduvantage 1, 1 (2013).
[24]
Helge Nissen and Monique Janneck. 2018. Designempfehlungen für Fragebogen auf mobilen Endgeräten. In Wissensgemeinschaften in Wirtschaft, Wissenschaft und öffentlicher Verwaltung, Thomas Köhler, Eric Schoop, and Nina Kahnwald (Eds.). TUDPress, 261--270.
[25]
Helge Nissen and Monique Janneck. 2018. Einfluss des verwendeten Endgeräts auf das Nutzungsverhalten in Online-Befragungen. In Mensch und Computer 2018 - Tagungsband, Raimund Dachselt and Gerhard Weber (Eds.). Gesellschaft für Informatik e.V., Bonn.
[26]
Erica L Olmsted-Hawala, Elizabeth M Nichols, Temika Holland, and Marylisa Gareau. 2016. Results of Usability Testing of the 2014 American Community Survey on Smartphones and Tablets Phase I: Before Optimization for Mobile Devices. Survey Methodology (2016), 03.
[27]
Vidal Díaz de Rada and Juan Antonio Domínguez-Álvarez. 2014. Response quality of self-administered questionnaires: A comparison between paper and web questionnaires. Social Science Computer Review 32, 2 (2014), 256--269.
[28]
Shimon Sarraf, Jennifer Brooks, and J Cole. 2014. Taking surveys with smartphones: A look at usage among college students. In AAPOR Annual Conference, Anaheim, CA.
[29]
Meinald T Thielsch, Timo Lenzner, and Torsten Melles. 2012. Wie gestalte ich gute Items und Interviewfragen. Praxis der Wirtschaftspsychologie II: Themen und Fallbeispiele für Studium und Praxis (2012), 221--240.
[30]
Jenifer Tidwell. 2010. Designing Interfaces. O'Reilly Media, Inc.
[31]
Gustavo F Tondello, Rina R Wehbe, Lisa Diamond, Marc Busch, Andrzej Marczewski, and Lennart E Nacke. 2016. The gamification user types hexad scale. In Proceedings of the 2016 annual symposium on computer-human interaction in play. ACM, 229--243.
Index Terms
- Usability Evaluation of Online Questionnaires on Mobile Devices
Recommendations
Reliability and validity of the mobile phone usability questionnaire (MPUQ)
This study was a follow-up to determine the psychometric quality of the usability questionnaire items derived from a previous study (Ryu and Smith-Jackson, 2005), and to find a subset of items that represents a higher measure of reliability and ...
Comparing the usability of grid-bag and constraint-based layouts
OzCHI '12: Proceedings of the 24th Australian Computer-Human Interaction ConferenceWhile the usability of GUI design methods has been studied in general, the usability of layout specification methods is largely unexplored.
In this paper we provide an empirical comparison of two popular GUI layout models, grid-bag layout and constraint-...
Decision models for comparative usability evaluation of mobile phones using the mobile phone usability questionnaire (MPUQ)
A comparative usability evaluation was performed using various subjective evaluation methods, including Mobile Phone Usability Questionnaire (MPUQ). Further, decision-making models using Analytic Hierarchy Process (AHP) and multiple linear regression ...
Comments
Information & Contributors
Information
Published In
September 2019
863 pages
Copyright © 2019 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 September 2019
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
MuC'19
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 146Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)0
Reflects downloads up to 24 Dec 2024
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in