Article

Crowdsourced web site evaluation with crowdstudy

Authors:

Michael Nebeling,

Maximilian Speicher,

Michael Grossniklaus,

Moira C. NorrieAuthors Info & Claims

ICWE'12: Proceedings of the 12th international conference on Web Engineering

Pages 494 - 497

https://doi.org/10.1007/978-3-642-31753-8_52

Published: 23 July 2012 Publication History

Publisher Site

Abstract

Many different automatic usability evaluation tools have been specifically developed for web sites and web-based services, but they usually cannot replace user testing. At the same time, traditional usability evaluation methods can be both expensive and time consuming. We will demonstrate CrowdStudy, a toolkit for crowdsourced testing of web interfaces that allows, not only to efficiently recruit larger amounts of test users, but also to evaluate web sites under many different conditions.

References

[1]

Matera, M., Rizzo, F., Carughi, G.: Web Usability: Principles and Evaluation Methods. Web Engineering (2006)

Google Scholar

[2]

Insfran, E., Fernandez, A.: A Systematic Review of Usability Evaluation in Web Development. In: Hartmann, S., Zhou, X., Kirchberg, M. (eds.) WISE 2008. LNCS, vol. 5176, pp. 81-91. Springer, Heidelberg (2008)

Digital Library

Google Scholar

[3]

Atterer, R., Wnuk, M., Schmidt, A.: Knowing the User's EveryMove - User Activity Tracking for Website Usability Evaluation and Implicit Interaction. In: Proc. WWW (2006)

Digital Library

Google Scholar

[4]

Hong, J. I., Heer, J., Waterson, S., Landay, J. A.:WebQuilt: A Proxy-based Approach to Remote Web Usability Testing. TOIS 19(3) (2001)

Digital Library

Google Scholar

[5]

Kittur, A., Chi, E. H., Suh, B.: Crowdsourcing User Studies With Mechanical Turk. In: Proc. CHI (2008)

Digital Library

Google Scholar

[6]

Nebeling, M., Norrie, M.C.: Tools and Architectural Support for Crowdsourced Adaptation of Web Interfaces. In: Auer, S., Díaz, O., Papadopoulos, G. A. (eds.) ICWE 2011. LNCS, vol. 6757, pp. 243-257. Springer, Heidelberg (2011)

Digital Library

Google Scholar

Cited By

View all

Kazemi SLiebchen GCetinkaya D(2022)A Survey on the Usability and User Experience of the Open Community Web PortalsHCI International 2022 - Late Breaking Papers. Design, User Experience and Interaction10.1007/978-3-031-17615-9_28(409-423)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-17615-9_28
Choi YMonserrat TPark JShin HLee NKim J(2021)ProtoChatProceedings of the ACM on Human-Computer Interaction10.1145/34329244:CSCW3(1-27)Online publication date: 5-Jan-2021
https://dl.acm.org/doi/10.1145/3432924
Chen YPandey MSong JLasecki WOney SBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Improving Crowd-Supported GUI Testing with Structural GuidanceProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376835(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376835
Show More Cited By

Recommendations

CrowdStudy: general toolkit for crowdsourced evaluation of web interfaces
EICS '13: Proceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems

While traditional usability testing methods can be both time consuming and expensive, tools for automated usability evaluation tend to oversimplify the problem by limiting themselves to supporting only certain evaluation criteria, settings, tasks and ...
Automated Web Site Evaluation - An Approach Based on Ranking SVM
WI-IAT '09: Proceedings of the 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology - Volume 03

This paper proposes an automated web site evaluation approach using machine learning to cope with ranking problems. Evaluating web sites is a significant task for web service because evaluated web sites provide useful information for users to estimate ...
Korean Web Site Usability for Disabled People
APCHI '08: Proceedings of the 8th Asia-Pacific conference on Computer-Human Interaction

In this paper, we evaluated 10 Korean web sites whether these sites have enough usability for the disabled people. Theoretically, there must be no difference using web content to an ordinary user and to a user with disabilities. However, under practical ...

Comments

Information & Contributors

Information

Published In

ICWE'12: Proceedings of the 12th international conference on Web Engineering

July 2012

511 pages

ISBN:9783642317521

Editors:
Marco Brambilla
Dipartimento di Elettronica e Informazione, Politecnico di Milano, Via Ponzio 34/5, Milano, Italy
,
Takehiro Tokuda
Department of Computer Science, Tokyo Institute of Technology, 2-12-1 Oookayama, Tokyo, Japan
,
Robert Tolksdorf
Institut für Informatik, Freie Universität Berlin, Königin-Luise-Strasse 24-26, Berlin, Germany

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 23 July 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Kazemi SLiebchen GCetinkaya D(2022)A Survey on the Usability and User Experience of the Open Community Web PortalsHCI International 2022 - Late Breaking Papers. Design, User Experience and Interaction10.1007/978-3-031-17615-9_28(409-423)Online publication date: 26-Jun-2022
https://dl.acm.org/doi/10.1007/978-3-031-17615-9_28
Choi YMonserrat TPark JShin HLee NKim J(2021)ProtoChatProceedings of the ACM on Human-Computer Interaction10.1145/34329244:CSCW3(1-27)Online publication date: 5-Jan-2021
https://dl.acm.org/doi/10.1145/3432924
Chen YPandey MSong JLasecki WOney SBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Improving Crowd-Supported GUI Testing with Structural GuidanceProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376835(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376835
Chen XJiang HChen ZHe TNie L(2019)Automatic test report augmentation to assist crowdsourced testingFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-018-7308-513:5(943-959)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s11704-018-7308-5
Jiang HChen XHe TChen ZLi X(2018)Fuzzy Clustering of Crowdsourced Test Reports for AppsACM Transactions on Internet Technology10.1145/310616418:2(1-28)Online publication date: 2-Feb-2018
https://dl.acm.org/doi/10.1145/3106164
Feng YJones JChen ZFang CLo DApel SKhurshid S(2016)Multi-objective test report prioritization using image understandingProceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering10.1145/2970276.2970367(202-213)Online publication date: 25-Aug-2016
https://dl.acm.org/doi/10.1145/2970276.2970367
Morishima AFukusumi SKitagawa H(2016)CyLog/Game aspectInformation Systems10.1016/j.is.2016.01.00462:C(170-184)Online publication date: 1-Dec-2016
https://dl.acm.org/doi/10.1016/j.is.2016.01.004
Feng YChen ZJones JFang CXu BDi Nitto EHarman MHeymans P(2015)Test report prioritization to assist crowdsourced testingProceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering10.1145/2786805.2786862(225-236)Online publication date: 30-Aug-2015
https://dl.acm.org/doi/10.1145/2786805.2786862
Nebeling MSpeicher MNorrie MForbrig PDewan PHarrison MLuyten K(2013)CrowdStudyProceedings of the 5th ACM SIGCHI symposium on Engineering interactive computing systems10.1145/2494603.2480303(255-264)Online publication date: 24-Jun-2013
https://dl.acm.org/doi/10.1145/2494603.2480303
Nebeling MLeone SNorrie M(2012)Crowdsourced web engineering and designProceedings of the 12th international conference on Web Engineering10.1007/978-3-642-31753-8_3(31-45)Online publication date: 23-Jul-2012
https://dl.acm.org/doi/10.1007/978-3-642-31753-8_3

Abstract

References

Cited By

Recommendations

CrowdStudy: general toolkit for crowdsourced evaluation of web interfaces

Automated Web Site Evaluation - An Approach Based on Ranking SVM

Korean Web Site Usability for Disabled People

Comments

Information

Published In

Sponsors

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations