Confidence Levels for Empirical Research Using Twitter Data

Concerns of a "reproducibility crisis" in scientific research have become increasingly prevalent. The field of meta science - the scientific study of science itself - is thriving and has examined the existence and prevalence of threats to reproducible and robust research in designed experiments or surveys. Nonetheless, largely missing are replication efforts devoted to examining empirical studies with "organic data" - e.g., data organically generated by ubiquitous sensors or mobile applications, twitter feeds, click streams, etc. Given the growing popularity of using Twitter as the source of research data in psychology, we must take proper care of the data handling process if Twitter as a data source is to be a robust, reliable, and reproducible endeavor into the future. Our research studies scholarly publications in psychology to establish the confidence (or the lack thereof) in their handling practices of Twitter data.