column

To Replicate or Not to Replicate?

Author:

Nikola BanovicAuthors Info & Claims

GetMobile: Mobile Computing and Communications, Volume 19, Issue 4

Pages 23 - 27

https://doi.org/10.1145/2904337.2904346

Published: 16 March 2016 Publication History

Abstract

To replicate or not to replicate experiments and studies seems to be a burning question in the Human-Computer Interaction (HCI) community. This question is equally relevant to the field of mobile computing, where researchers face unique challenges when attempting to replicate studies of mobile device usage in the field. Some of those challenges are inherent in conducting such studies to understand complex socio-technical systems, which involve different technologies and diverse populations of users from different social, economic, and cultural backgrounds. However, those same challenges point to a great opportunity to expand our knowledge of how people use and interact with their mobile devices and understand how those interactions evolve over time.

References

[1]

Nikola Banovic, Christina Brant, Jennifer Mankoff, and Anind Dey. 2014. Proactive Tasks: the short of mobile device use sessions. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services (MobileHCI '14). ACM, New York, NY, USA, 243--252.

[2]

Susanne Boll, Niels Henze, Martin Pielot, Benjamin Poppinga, and Torben Schinke. 2011. My App is an Experiment: Experience from User Studies in Mobile App Stores. Int. J. Mob. Hum. Comput. Interact. 3, 4 (October 2011), 71--91.

Digital Library

[3]

Matthias Bohmer, Brent Hecht, Johannes Schoning, Antonio Kruger, and Gernot Bauer. 2011. Falling asleep with Angry Birds, Facebook and Kindle: a large-scale study on mobile application usage. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI'11). ACM, New York, NY, USA, 47--56.

Digital Library

[4]

Barry Brown, Stuart Reeves, and Scott Sherwood. 2011. Into the wild: challenges and opportunities for field trial methods. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 1657--1666.

Digital Library

[5]

Juan Pablo Carrascal and Karen Church. 2015. An In-Situ Study of Mobile App & Mobile Search Interactions. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2739--2748.

Digital Library

[6]

Karen Church, Denzil Ferreira, Nikola Banovic, and Kent Lyons. 2015. Understanding the Challenges of Mobile Phone Usage Data. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). ACM, New York, NY, USA, 504--514.

Digital Library

[7]

David Dearman, Alex Varshavsky, Eyal De Lara, and Khai N. Truong. 2007. An exploration of location error estimation. In Proceedings of the 9th international conference on Ubiquitous computing (UbiComp '07), John Krumm, Gregory D. Abowd, Aruna Seneviratne, and Thomas Strang (Eds.). Springer-Verlag, Berlin, Heidelberg, 181--198.

[8]

Chris Drummond. 2009. Replicability is not reproducibility: nor is it good science. In Proceedings of the Evaluation Methods for Machine Learning Workshop at ICML.

[9]

Denzil Ferreira, Eija Ferreira, Jorge Goncalves, Vassilis Kostakos, and Anind K. Dey. 2013. Revisiting human-battery interaction with an interactive battery interface. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing (UbiComp'13. ACM, New York, NY, USA, 563--572.

Digital Library

[10]

Denzil Ferreira, Jorge Goncalves, Vassilis Kostakos, Louise Barkhuus, and Anind K. Dey. 2014. Contextual experience sampling of mobile application micro-usage. In Proceedings of the 16th international conference on Humancomputer interaction with mobile devices & services (MobileHCI '14). ACM, New York, NY, USA, 91--100.

Digital Library

[11]

Denzil Ferreira, Vassilis Kostakos, Alastair R. Beresford, Janne Lindqvist, and Anind K. Dey. 2015. Securacy: an empirical investigation of Android applications' network usage, privacy and security. In Proceedings of the 8th ACM Conference on Security & Privacy in Wireless and Mobile Networks (WiSec '15). ACM, New York, NY, USA, Article 11, 11 pages.

Digital Library

[12]

Jee Hyun Ha, Bumsu Chin, Doo-Heum Park, Seung-Ho Ryu, and Jaehak Yu. 2008. Characteristics of excessive cellular phone use in Korean adolescents. In CyberPsychology & Behavior 11, no. 6, 783--784.

[13]

Khai N. Truong, Thariq Shihipar, and Daniel J. Wigdor. 2014. Slide to X: unlocking the potential of smartphone unlocking. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3635--3644.

Digital Library

[14]

Rajan Vaish, Keith Wyngarden, Jingshu Chen, Brandon Cheung, and Michael S. Bernstein. 2014. Twitch crowdsourcing: crowd contributions in short bursts of time. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3645--3654.

Digital Library

[15]

Daniel T. Wagner, Andrew Rice, and Alastair R. Beresford. 2014. Device analyzer: large-scale mobile data collection. SIGMETRICS Perform. Eval. Rev. 41, 4 (April 2014), 53--56.

Digital Library

[16]

Max L. Wilson, Wendy Mackay, Ed Chi, Michael Bernstein, Dan Russell, and Harold Thimbleby. 2011. RepliCHI -- CHI should be replicating and validating results more: discuss. In CHI '11 Extended Abstracts on Human Factors in Computing Systems (CHI EA '11). ACM, New York, NY, USA, 463--466.

Digital Library

Cited By

Hidaka TSei YNishida NYamanaka SShizuki B(2023)Advanced Investigation of Steering Performance with Error-Accepting DelaysInternational Journal of Human–Computer Interaction10.1080/10447318.2023.219258640:14(3622-3635)Online publication date: 3-Apr-2023
https://doi.org/10.1080/10447318.2023.2192586
Galster MWeyns D(2023)Empirical research in software architecture — Perceptions of the communityJournal of Systems and Software10.1016/j.jss.2023.111684202(111684)Online publication date: Aug-2023
https://doi.org/10.1016/j.jss.2023.111684
Komninos A(2022)The Value of Open Data in HCI: A Case Report from Mobile Text Entry ResearchMultimodal Technologies and Interaction10.3390/mti60900716:9(71)Online publication date: 23-Aug-2022
https://doi.org/10.3390/mti6090071
Show More Cited By

Recommendations

Replicate, Replicate, Replicate
RESER '11: Proceedings of the 2011 Second International Workshop on Replication in Empirical Software Engineering Research

Replication is a standard part of scientific experimentation. Unfortunately, in software engineering, replication of experiments is often considered an inferior type of research, or not even research at all. In this paper we describe four different ...
Replicate and Bundle (RnB) -- A Mechanism for Relieving Bottlenecks in Data Centers
IPDPS '13: Proceedings of the 2013 IEEE 27th International Symposium on Parallel and Distributed Processing

This work addresses the scalability and efficiency of RAM-based storage systems wherein multiple objects must be retrieved per user request. Here, much of the CPU work is per server transaction, not per requested item. Adding servers and spreading the ...
Improving Data Grids Performance by Using Popular File Replicate First Algorithm
BWCCA '11: Proceedings of the 2011 International Conference on Broadband and Wireless Computing, Communication and Applications

In this paper, we propose an adaptive data replication algorithm, called the Popular File Replicate First algorithm (PFRF for short), which is developed on a star-topology data grid with limited storage space based on aggregated information on previous ...

Comments

Information & Contributors

Information

Published In

cover image GetMobile: Mobile Computing and Communications

GetMobile: Mobile Computing and Communications Volume 19, Issue 4

October 2015

34 pages

ISSN:2375-0529

EISSN:2375-0537

DOI:10.1145/2904337

Editor:
Eyal de Lara
University of Toronto

Issue’s Table of Contents

Copyright © 2016 Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 March 2016

Published in SIGMOBILE-GETMOBILE Volume 19, Issue 4

Check for updates

Qualifiers

Column

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
158
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)3

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hidaka TSei YNishida NYamanaka SShizuki B(2023)Advanced Investigation of Steering Performance with Error-Accepting DelaysInternational Journal of Human–Computer Interaction10.1080/10447318.2023.219258640:14(3622-3635)Online publication date: 3-Apr-2023
https://doi.org/10.1080/10447318.2023.2192586
Galster MWeyns D(2023)Empirical research in software architecture — Perceptions of the communityJournal of Systems and Software10.1016/j.jss.2023.111684202(111684)Online publication date: Aug-2023
https://doi.org/10.1016/j.jss.2023.111684
Komninos A(2022)The Value of Open Data in HCI: A Case Report from Mobile Text Entry ResearchMultimodal Technologies and Interaction10.3390/mti60900716:9(71)Online publication date: 23-Aug-2022
https://doi.org/10.3390/mti6090071
Heitmayer M(2021)“It's Like Being Gone For A Second”Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447526.3472026(1-17)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3447526.3472026
Heitmayer MLahlou S(2020)Why Are Smartphones Disruptive? An Empirical Study of Smartphone Use in Real-Life ContextsComputers in Human Behavior10.1016/j.chb.2020.106637(106637)Online publication date: Nov-2020
https://doi.org/10.1016/j.chb.2020.106637
Morrison AXiong XHiggs MBell MChalmers MMandryk RHancock MPerry MCox A(2018)A Large-Scale Study of iPhone App Launch BehaviourProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173918(1-13)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3173918
Cockburn AGutwin CDix AMandryk RHancock MPerry MCox A(2018)HARK No MoreProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173715(1-12)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3173715

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents