Article

Efficient search engine measurements

Authors:

Ziv Bar-Yossef,

Maxim GurevichAuthors Info & Claims

WWW '07: Proceedings of the 16th international conference on World Wide Web

Pages 401 - 410

https://doi.org/10.1145/1242572.1242627

Published: 08 May 2007 Publication History

Abstract

We address the problem of measuring global quality met-rics of search engines, like corpus size, index freshness, anddensity of duplicates in the corpus. The recently proposedestimators for such metrics [2, 6] suffer from significant biasand/or poor performance, due to inaccurate approximationof the so called .document degrees.We present two new estimators that are able to overcomethe bias introduced by approximate degrees. Our estimatorsare based on a careful implementation of an approximateimportance sampling procedure. Comprehensive theoreti-cal and empirical analysis of the estimators demonstratesthat they have essentially no bias even in situations wheredocument degrees are poorly approximated.Building on an idea from [6], we discuss Rao Blackwelliza-tion as a generic method for reducing variance in searchengine estimators. We show that Rao-Blackwellizing ourestimators results in significant performance improvements,while not compromising accuracy.

References

[1]

Z. Bar-Yossef, A. Berg, S. Chien, J. Fakcharoenphol, and D. Weitz. Approximating aggregate queries about Web pages via random walks. In Proc. 26th VLDB, pages 535--544, 2000.

Digital Library

[2]

Z. Bar-Yossef and M. Gurevich. Random sampling from a search engine's index. In Proc. 15th WWW, pages 367--376, 2006.

Digital Library

[3]

Z. Bar-Yossef and M. Gurevich. Efficient search engine measurements, 2007. Full version available at http://www.ee.technion.ac.il/people/zivby.

[4]

K. Bharat and A. Broder. A technique for measuring the relative size and overlap of public Web search engines. In Proc. 7th WWW, pages 379--388, 1998.

Digital Library

[5]

E. T. Bradlow and D. C. Schmittlein. The little engines that could: Modeling the performance of World Wide Web search engines. Marketing Science, 19:43--62, 2000.

Digital Library

[6]

A. Broder, M. Fontoura, V. Josifovski, R. Kumar, R. Motwani, S. Nabar, R. Panigrahy, A. Tomkins, and Y. Xu. Estimating corpus size via queries. Proc. 15th CIKM, 2006.

Digital Library

[7]

G. Casella and C. P. Robert. Rao-Blackwellisation of sampling schemes. Biometrika, 83(1):81--94, 1996.

[8]

M. Cheney and M. Perry. A comparison of the size of the Yahoo! and Google indices. Available at http://vburton.ncsa.uiuc.edu/indexsize.html, 2005.

[9]

dmoz. The open directory project. http://dmoz.org.

[10]

A. Dobra and S. E. Fienberg. How large is the World Wide Web? Web Dynamics, pages 23--44, 2004.

[11]

O. Goldreich. A sample of samplers -- a computational perspective on sampling (survey). ECCC, 4(20), 1997.

[12]

A. Gulli and A. Signorini. The indexable Web is more than 11.5 billion pages. In Proc. 14th WWW, pages 902--903, 2005.

Digital Library

[13]

W. K. Hastings. Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1):97--109, 1970.

[14]

M. R. Henzinger, A. Heydon, M. Mitzenmacher, and M. Najork. Measuring index quality using random walks on the Web. In Proc. 8th WWW, pages 213--225, 1999.

Digital Library

[15]

M. R. Henzinger, A. Heydon, M. Mitzenmacher, and M. Najork. On near-uniform URL sampling. In Proc. 9th WWW, pages 295--308, 2000.

Digital Library

[16]

T. C. Hesterberg. Advances in Importance Sampling. PhD thesis, Stanford University, 1988.

[17]

S. Lawrence and C. L. Giles. Searching the World Wide Web. Science, 5360(280):98, 1998.

[18]

S. Lawrence and C. L. Giles. Accessibility of information on the Web. Nature, 400:107--109, 1999.

[19]

S.-M. Lee and A. Chao. Estimating population size via sample coverage for closed capture-recapture models. Biometrics, 50(1):88--97, 1994.

[20]

J. S. Liu. Monte Carlo Strategies in Scientific Computing. Springer, 2001.

Digital Library

[21]

A. W. Marshal. The use of multi-stage sampling schemes in Monte Carlo computations. In Symposium on Monte Carlo Methods, pages 123--140, 1956.

[22]

N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller. Equations of state calculations by fast computing machines. J. of Chemical Physics, 21:1087--1091, 1953.

[23]

P. Rusmevichientong, D. Pennock, S. Lawrence, and C. L. Giles. Methods for sampling pages uniformly from the World Wide Web. In Proc. AAAI Symp. on Using Uncertainty within Computation, 2001.

[24]

J. von Neumann. Various techniques used in connection with random digits. In John von Neumann, Collected Works, volume V. Oxford, 1963.

Cited By

Zhou ZZhao PSheng VXu JLi ZWu JCui Z(2018)Efficient sampling methods for characterizing POIs on maps based on road networksFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-016-6146-612:3(582-592)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.1007/s11704-016-6146-6
Lee CDo XEun Y(2017)On the rao-blackwellization and its application for graph sampling via neighborhood explorationIEEE INFOCOM 2017 - IEEE Conference on Computer Communications10.1109/INFOCOM.2017.8057071(1-9)Online publication date: May-2017
https://doi.org/10.1109/INFOCOM.2017.8057071
Wang PHe WLiu X(2016)Efficiently Estimating Statistics of Points of Interests on MapsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2015.248039728:2(425-438)Online publication date: 1-Feb-2016
https://dl.acm.org/doi/10.1109/TKDE.2015.2480397
Show More Cited By

Index Terms

Efficient search engine measurements
1. Information systems
  1. Information retrieval

Recommendations

Efficient Search Engine Measurements

We address the problem of externally measuring aggregate functions over documents indexed by search engines, like corpus size, index freshness, and density of duplicates in the corpus. State of the art estimators for such quantities [Bar-Yossef and ...
What users see - Structures in search engine results pages

This paper investigates the composition of search engine results pages. We define what elements the most popular web search engines use on their results pages (e.g., organic results, advertisements, shortcuts) and to which degree they are used for ...
Brand and its effect on user perception of search engine performance

In this research we investigate the effect of search engine brand on the evaluation of searching performance. Our research is motivated by the large amount of search traffic directed to a handful of Web search engines, even though many have similar ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '07: Proceedings of the 16th international conference on World Wide Web

May 2007

1382 pages

ISBN:9781595936547

DOI:10.1145/1242572

General Chairs:
Carey Williamson
University of Calgary, Canada
,
Mary Ellen Zurko
IBM, USA
,
Program Chairs:
Peter Patel-Schneider
Bell Labs Research, USA
,
Prashant Shenoy
University of Massachusetts at Amherst, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

ACM: Association for Computing Machinery

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

WWW'07

Sponsor:

ACM

WWW'07: 16th International World Wide Web Conference

May 8 - 12, 2007

Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

31
Total Citations
View Citations
985
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou ZZhao PSheng VXu JLi ZWu JCui Z(2018)Efficient sampling methods for characterizing POIs on maps based on road networksFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-016-6146-612:3(582-592)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.1007/s11704-016-6146-6
Lee CDo XEun Y(2017)On the rao-blackwellization and its application for graph sampling via neighborhood explorationIEEE INFOCOM 2017 - IEEE Conference on Computer Communications10.1109/INFOCOM.2017.8057071(1-9)Online publication date: May-2017
https://doi.org/10.1109/INFOCOM.2017.8057071
Wang PHe WLiu X(2016)Efficiently Estimating Statistics of Points of Interests on MapsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2015.248039728:2(425-438)Online publication date: 1-Feb-2016
https://dl.acm.org/doi/10.1109/TKDE.2015.2480397
Ghansah BWu S(2015)Distributed Information Retrieval: Developments and StrategiesInternational Journal of Engineering Research in Africa10.4028/www.scientific.net/JERA.16.11016(110-144)Online publication date: Jun-2015
https://doi.org/10.4028/www.scientific.net/JERA.16.110
Khelghati MHiemstra Dvan Keulen MAnderst-Kotsis GIndrawan-Santiago M(2015)Towards complete coverage in focused web harvestingProceedings of the 17th International Conference on Information Integration and Web-based Applications & Services10.1145/2837185.2837208(1-9)Online publication date: 11-Dec-2015
https://dl.acm.org/doi/10.1145/2837185.2837208
Bian ZLi YYue TLei PZhao DXiao Y(2015)An Information Update Method Towards Internal Search EngineProceedings of the 2015 12th Web Information System and Application Conference (WISA)10.1109/WISA.2015.69(211-216)Online publication date: 11-Sep-2015
https://dl.acm.org/doi/10.1109/WISA.2015.69
Wang YLiang JLu JJia YRokne JWu XEster MXu G(2014)Estimating the size of hidden data sources by queriesProceedings of the 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining10.5555/3191835.3191975(712-719)Online publication date: 17-Aug-2014
https://dl.acm.org/doi/10.5555/3191835.3191975
Wang PHe WLiu X(2014)An efficient sampling method for characterizing points of interests on maps2014 IEEE 30th International Conference on Data Engineering10.1109/ICDE.2014.6816719(1012-1023)Online publication date: Mar-2014
https://doi.org/10.1109/ICDE.2014.6816719
Wang YLiang JLu J(2014)Estimating the size of hidden data sources by queries2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2014)10.1109/ASONAM.2014.6921664(712-719)Online publication date: Aug-2014
https://doi.org/10.1109/ASONAM.2014.6921664
Katzir LLiberty ESomekh OCosma I(2014)Estimating Sizes of Social Networks via Biased SamplingInternet Mathematics10.1080/15427951.2013.86288310:3-4(335-359)Online publication date: 15-Sep-2014
https://doi.org/10.1080/15427951.2013.862883
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents