Overview
- Discusses the principles and limitations of statistical significance tests
- Provides hands-on examples of t-tests, ANOVA, and multiple comparison procedures with Excel and R
- Introduces tools for designing effective experiments by leveraging topic set size design and for power analysis
Part of the book series: The Information Retrieval Series (INRE, volume 40)
Access this book
Tax calculation will be finalised at checkout
Other ways to access
About this book
Chapters 1–5 review parametric significance tests for comparing system means, namely, t-tests and ANOVAs, and show how easily they can be conducted using Microsoft Excel or R. These chapters also discuss a few multiple comparison procedures for researcherswho are interested in comparing every system pair, including a randomised version of Tukey's Honestly Significant Difference test. The chapters then deal with known limitations of classical significance testing and provide practical guidelines for reporting research results regarding comparison of means.
Chapters 6 and 7 discuss statistical power. Chapter 6 introduces topic set size design to enable test collection builders to determine an appropriate number of topics to create. Readers can easily use the author’s Excel tools for topic set size design based on the paired and two-sample t-tests, one-way ANOVA, and confidence intervals. Chapter 7 describes power-analysis-based methods for determining an appropriate sample size for a new experiment based on a similar experiment done in the past, detailing how to utilize the author’s R tools for power analysis and how to interpret the results. Case studies from IR for both Excel-based topic set size design and R-basedpower analysis are also provided.
Similar content being viewed by others
Keywords
Table of contents (8 chapters)
Authors and Affiliations
About the author
Bibliographic Information
Book Title: Laboratory Experiments in Information Retrieval
Book Subtitle: Sample Sizes, Effect Sizes, and Statistical Power
Authors: Tetsuya Sakai
Series Title: The Information Retrieval Series
DOI: https://doi.org/10.1007/978-981-13-1199-4
Publisher: Springer Singapore
eBook Packages: Computer Science, Computer Science (R0)
Copyright Information: Springer Nature Singapore Pte Ltd. 2018
Hardcover ISBN: 978-981-13-1198-7Published: 04 October 2018
Softcover ISBN: 978-981-13-4581-4Published: 29 December 2018
eBook ISBN: 978-981-13-1199-4Published: 22 September 2018
Series ISSN: 1871-7500
Series E-ISSN: 2730-6836
Edition Number: 1
Number of Pages: IX, 150
Number of Illustrations: 10 b/w illustrations, 43 illustrations in colour
Topics: Information Storage and Retrieval, Statistics for Engineering, Physics, Computer Science, Chemistry and Earth Sciences