How well do people report time spent on Facebook? An evaluation of established survey questions with recommendations

SK Ernala, M Burke, A Leavitt, NB Ellison - Proceedings of the 2020 CHI …, 2020 - dl.acm.org
Proceedings of the 2020 CHI conference on human factors in computing systems, 2020dl.acm.org
Many studies examining social media use rely on self-report survey questions about how
much time participants spend on social media platforms. Because they are challenging to
answer accurately and susceptible to various biases, these self-reported measures are
known to contain error--although the specific contours of this error are not well understood.
This paper compares data from ten self-reported Facebook use survey measures deployed
in 15 countries (N= 49,934) against data from Facebook's server logs to describe factors …
Many studies examining social media use rely on self-report survey questions about how much time participants spend on social media platforms. Because they are challenging to answer accurately and susceptible to various biases, these self-reported measures are known to contain error -- although the specific contours of this error are not well understood. This paper compares data from ten self-reported Facebook use survey measures deployed in 15 countries (N = 49,934) against data from Facebook's server logs to describe factors associated with error in commonly used survey items from the literature. Self-reports were moderately correlated with actual Facebook use (r = 0.42 for the best-performing question), though participants significantly overestimated how much time they spent on Facebook and underestimated the number of times they visited. People who spent a lot of time on the platform were more likely to misreport their time, as were teens and younger adults, which is notable because of the high reliance on college-aged samples in many fields. We conclude with recommendations on the most accurate ways to collect time-spent data via surveys.
ACM Digital Library