poster

Generalising multiple capture-recapture to non-uniform sample sizes

Author:

SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

Pages 839 - 840

https://doi.org/10.1145/1390334.1390531

Published: 20 July 2008 Publication History

Get Access

Abstract

Algorithms in distributed information retrieval often rely on accurate knowledge of the size of a collection. The "multiple capture-recapture" method of Shokouhi et al. is one of the more reliable algorithms for determining collection size, but it relies on samples with a uniform number of documents. Such uniform samples are often hard to obtain in a working system.

A simple generalisation of multiple capture-recapture does not rely on uniform sample sizes. Simulations show it is as accurate as the original method even when sample sizes vary considerably, making it a useful technique in real tools.

References

[1]

K. Bharat and A. Broder. A technique for measuring the relative size and overlap of public web search engines. In Proc. WWW, 1998.

Digital Library

Google Scholar

[2]

J. Callan and M. Connell. Query-based sampling of text databases. ACM Trans. Info. Systems, 19(2), 2001.

Digital Library

Google Scholar

[3]

K.-L. Liu, A. Santoso, C. Yu, W. Meng, and C. Zhang. Discovering the representative of a search engine. In Proc. CIKM, 2001. Poster.

Digital Library

Google Scholar

[4]

M. Shokouhi, J. Zobel, F. Scholer, and S. M. M. Tahaghoghi. Capturing collection size for distributed non-cooperative retrieval. In Proc. ACM SIGIR, 2006.

Digital Library

Google Scholar

[5]

P. Thomas and D. Hawking. Evaluating sampling methods for uncooperative collections. In Proc. ACM SIGIR, 2007.

Digital Library

Google Scholar

Cited By

View all

Khelghati MHiemstra Dvan Keulen MPardede E(2012)Size estimation of non-cooperative data collectionsProceedings of the 14th International Conference on Information Integration and Web-based Applications & Services10.1145/2428736.2428774(239-246)Online publication date: 3-Dec-2012
https://dl.acm.org/doi/10.1145/2428736.2428774

Index Terms

Generalising multiple capture-recapture to non-uniform sample sizes
1. Information systems
  1. Information retrieval
    1. Search engine architectures and scalability
      1. Distributed retrieval
      2. Peer-to-peer retrieval
  2. Information storage systems
    1. Storage architectures
      1. Distributed storage

Recommendations

Assessing Software Designs Using Capture-Recapture Methods
Special issue on software reliability

The number of faults not discovered by the design review can be estimated by using capture-recapture methods. Since these methods were developed for wildlife population estimation, the assumptions used to derive them do not match design review ...
The effects of misclassification on estimates from capture - recapture studies (dual system estimation, em algorithm)
Capture-Recapture Sampling for Estimating Software Error Content

Mills capture-recapture sampling method allows the estimation of the number of errors in a program by randomly inserting known errors and then testing the program for both inserted and indigenous errors. This correspondence shows how correct confidence ...

Comments

Information & Contributors

Information

Published In

SIGIR '08: Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval

July 2008

934 pages

ISBN:9781605581644

DOI:10.1145/1390334

General Chairs:
Tat-Seng Chua
National University of Singapore
,
Mun-Kew Leong
National Library Board, Singapore
,
Program Chairs:
Syung Hyon Myaeng
Information and Communications University, Korea
,
Douglas W. Oard
University of Maryland, College Park, USA
,
Fabrizio Sebastiani
Consiglio Nazionale delle Ricerche, Italy

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

size estimation

Qualifiers

Poster

Conference

SIGIR '08

Sponsor:

SIGIR '08: The 31st Annual International ACM SIGIR Conference

July 20 - 24, 2008

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
252
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Khelghati MHiemstra Dvan Keulen MPardede E(2012)Size estimation of non-cooperative data collectionsProceedings of the 14th International Conference on Information Integration and Web-based Applications & Services10.1145/2428736.2428774(239-246)Online publication date: 3-Dec-2012
https://dl.acm.org/doi/10.1145/2428736.2428774

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Assessing Software Designs Using Capture-Recapture Methods

The effects of misclassification on estimates from capture - recapture studies (dual system estimation, em algorithm)

Capture-Recapture Sampling for Estimating Software Error Content