Abstract
We propose a new application of the Friedman statistical test of significance to compare multiple retrieval methods. After measuring the average precision at the eleven standard levels of recall, our application of the Friedman test provides a global comparison of the methods. In some experiments this test provides additional and useful information to decide if methods are different.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Hull, D.: Using statistical testing in the evaluation of retrieval experiments. In: Proc. of ACM SIGIR ’93, Pittsburgh, Pennsylvania, United States, pp. 329–338. ACM Press, New York (1993), doi:10.1145/160688.160758
Sanderson, M., Zobel, J.: Information retrieval system evaluation: effort, sensitivity, and reliability. In: Proc. of ACM SIGIR ’05, Salvador, Brazil, pp. 162–169. ACM Press, New York (2005), doi:10.1145/1076034.1076064
Sheskin, D.J.: Handbook of parametric and nonparametric statistical procedures, pp. 669–672. Chapman & Hall/CRC, Boca Raton (2000)
Hull, D.: Stemming algorithms: A case study for detailed evaluation. JASIS 47(1), 70–84 (1996)
Kekäläinen, J., Järvelin, K.: The impact of query structure and query expansion on retrieval performance. In: Proc. of ACM SIGIR ’98, Melbourne, Australia, pp. 130–137. ACM Press, New York (1998), doi:10.1145/290941.290978
Conover, W.J.: Practical Nonparametric Statistics, 2nd edn. John Wiley & Sons, Chichester (1980)
Davis, C.S.: Statistical Methods for Analysis of Repeated Measurements. Springer, Heidelberg (2002)
Zhai, C., Lafferty, J.: A study of smoothing methods for language models applied to information retrieval. ACM Trans. Inf. Syst. 22(2), 179–214 (2004), doi:10.1145/984321.984322
Zobel, J.: How reliable are the results of large-scale information retrieval experiments? In: Proc. of ACM SIGIR ’98, Melbourne, Australia, pp. 307–314. ACM Press, New York (1998), doi:10.1145/290941.291014
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Casanova, J.M., Presedo Quindimil, M.A., Barreiro, Á. (2007). Overall Comparison at the Standard Levels of Recall of Multiple Retrieval Methods with the Friedman Test. In: Amati, G., Carpineto, C., Romano, G. (eds) Advances in Information Retrieval. ECIR 2007. Lecture Notes in Computer Science, vol 4425. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71496-5_68
Download citation
DOI: https://doi.org/10.1007/978-3-540-71496-5_68
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71494-1
Online ISBN: 978-3-540-71496-5
eBook Packages: Computer ScienceComputer Science (R0)