article

Self-Improving Algorithms

Authors:

Bernard Chazelle,

Kenneth L. Clarkson,

Wolfgang Mulzer, and

C. SeshadhriAuthors Info & Claims

SIAM Journal on Computing, Volume 40, Issue 2

Pages 350 - 375

https://doi.org/10.1137/090766437

Published: 01 April 2011 Publication History

Abstract

We investigate ways in which an algorithm can improve its expected performance by fine-tuning itself automatically with respect to an unknown input distribution $\mathcal{D}$. We assume here that $\mathcal{D}$ is of product type. More precisely, suppose that we need to process a sequence $I_1,I_2,\ldots$ of inputs $I=(x_1,x_2,\ldots,x_n)$ of some fixed length $n$, where each $x_i$ is drawn independently from some arbitrary, unknown distribution $\mathcal{D}_i$. The goal is to design an algorithm for these inputs so that eventually the expected running time will be optimal for the input distribution $\mathcal{D}=\prod_i\mathcal{D}_i$. We give such self-improving algorithms for two problems: (i) sorting a sequence of numbers and (ii) computing the Delaunay triangulation of a planar point set. Both algorithms achieve optimal expected limiting complexity. The algorithms begin with a training phase during which they collect information about the input distribution, followed by a stationary regime in which the algorithms settle to their optimized incarnations.

References

[1]

P. Afshani, J. Barbay, and T. M. Chan, Instance-optimal geometric algorithms, in Proceedings of the 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2009, pp. 129-138.

[2]

P. Agarwal and J. Erickson, Geometric range searching and its relatives, in Advances in Discrete and Computational Geometry, Contemp. Math. 223, AMS, Providence, RI, 1998, pp. 1-56.

[3]

A. Aggarwal, L. J. Guibas, J. Saxe, and P. W. Shor, A linear-time algorithm for computing the Voronoi diagram of a convex polygon, Discrete Comput. Geom., 4 (1989), pp. 591-604.

[4]

S. Albers and M. Mitzenmacher, Average case analyses of list update algorithms, with applications to data compression, Algorithmica, 21 (1998), pp. 312-329.

[5]

S. Albers and J. Westbrook, Self-organizing data structures, in Online Algorithms (Schloss Dagstuhl, 1996), Lecture Notes in Comput. Sci. 1442, Springer-Verlag, Berlin, 1998, pp. 13-51.

[6]

B. Allen and I. Munro, Self-organizing binary search trees, J. ACM, 25 (1978), pp. 526-535.

[7]

N. Alon and J. H. Spencer, The Probabilistic Method, 2nd ed., Wiley-Interscience Series in Discrete Mathematics and Optimization, Wiley-Interscience, New York, 2000.

[8]

S. Ar, B. Chazelle, and A. Tal, Self-customized BSP trees for collision detection, Comput. Geom. Theory Appl., 15 (2000), pp. 91-102.

[9]

S. Arora and B. Barak, Computational Complexity: A Modern Approach, Cambridge University Press, Cambridge, UK, 2009.

[10]

S. Arya, T. Malamatos, D. M. Mount, and K. C. Wong, Optimal expected-case planar point location, SIAM J. Comput., 37 (2007), pp. 584-610.

[11]

J. L. Bentley and C. C. McGeoch, Amortized analyses of self-organizing sequential search heuristics, Comm. ACM, 28 (1985), pp. 404-411.

[12]

M. Bergde Berg, O. Cheong, M. van Kreveld, and M. Overmars, Computational Geometry: Algorithms and Applications, 3rd ed., Springer-Verlag, Berlin, 2008.

[13]

J. R. Bitner, Heuristics that dynamically organize data structures, SIAM J. Comput., 8 (1979), pp. 82-110.

[14]

J.-D. Boissonnat and M. Yvinec, Algorithmic Geometry, Cambridge University Press, Cambridge, UK, 1998.

[15]

A. Borodin and R. El-Yaniv, Online Computation and Competitive Analysis, Cambridge University Press, Cambridge, UK, 1998.

[16]

K. Buchin and W. Mulzer, Delaunay triangulations in ${O}(\text{sort}(n))$ time and more, in Proceedings of the 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2009, pp. 139-148.

[17]

T. M. Chan and M. Pa&caron;traşcu, Voronoi diagrams in $n2^{O(\sqrt{\lg \lg n})}$ time, in Proceedings of the 39th Annual ACM Symposium on Theory of Computing (STOC), 2007, pp. 31-39.

[18]

T. M. Chan and M. Pa&caron;traşcu, Transdichotomous results in computational geometry, I: Point location in sublogarithmic time, SIAM J. Comput., 39 (2009), pp. 703-729.

[19]

B. Chazelle, An optimal algorithm for intersecting three-dimensional convex polyhedra, SIAM J. Comput., 21 (1992), pp. 671-696.

[20]

B. Chazelle, The Discrepancy Method, Cambridge University Press, Cambridge, UK, 2000.

[21]

B. Chazelle, O. Devillers, F. Hurtado, M. Mora, V. Sacristán, and M. Teillaud, Splitting a Delaunay triangulation in linear time, Algorithmica, 34 (2002), pp. 39-46.

[22]

B. Chazelle and W. Mulzer, Computing hereditary convex structures, in Proceedings of the 25th Annual ACM Symposium on Computational Geometry (SoCG), 2009, pp. 61-70.

[23]

B. Chazelle and W. Mulzer, Markov incremental constructions, Discrete Comput. Geom., 42 (2009), pp. 399-420.

[24]

K. L. Clarkson, A randomized algorithm for closest-point queries, SIAM J. Comput., 17 (1988), pp. 830-847.

[25]

K. L. Clarkson and P. W. Shor, Applications of random sampling in computational geometry, II, Discrete Comput. Geom., 4 (1989), pp. 387-421.

[26]

K. L. Clarkson and K. Varadarajan, Improved approximation algorithms for geometric set cover, Discrete Comput. Geom., 37 (2007), pp. 43-58.

[27]

T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 3rd ed., MIT Press, Cambridge, MA, 2009.

[28]

T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed., Wiley-Interscience, New York, 2006.

[29]

H. Edelsbrunner and E. P. Mücke, Simulation of simplicity: A technique to cope with degenerate cases in geometric algorithms, ACM Trans. Graph., 9 (1990), pp. 66-104.

[30]

V. Estivill-Castro and D. Wood, A survey of adaptive sorting algorithms, ACM Comput. Surv., 24 (1992), pp. 441-476.

[31]

M. L. Fredman, How good is the information theory bound in sorting?, Theoret. Comput. Sci., 1 (1975/76), pp. 355-361.

[32]

G. H. Gonnet, J. I. Munro, and H. Suwanda, Exegesis of self-organizing linear search, SIAM J. Comput., 10 (1981), pp. 613-637.

[33]

Y. Han, Deterministic sorting in ${O}(n\log\log n)$ time and linear space, J. Algorithms, 50 (2004), pp. 96-105.

[34]

Y. Han and M. Thorup, Integer sorting in ${O}(n \sqrt {\log \log n})$ expected time and linear space, in Proceedings of the 43rd Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2002, pp. 135-144.

[35]

J. H. Hester and D. S. Hirschberg, Self-organizing linear search, ACM Comput. Surv., 17 (1985), pp. 295-311.

[36]

D. G. Kirkpatrick, Efficient computation of continuous skeletons, in Proceedings of the 20th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 1979, pp. 18-27.

[37]

D. T. Lee, On $k$-nearest neighbor Voronoi diagrams in the plane, IEEE Trans. Comput., 31 (1982), pp. 478-487.

[38]

J. Matoušek, Reporting points in halfspaces, Comput. Geom. Theory Appl., 2 (1992), pp. 169-186.

[39]

J. Matoušek, R. Seidel, and E. Welzl, How to net a lot with little: Small $\epsilon$-nets for disks and halfspaces, in Proceedings of the 6th Annual ACM Symposium on Computational Geometry (SoCG), 1990, pp. 16-22.

[40]

J. McCabe, On serial files with relocatable records, Operations Res., 13 (1965), pp. 609-618.

[41]

K. Mehlhorn, Data Structures and Algorithms 1: Sorting and Searching, EATCS Monographs on Theoretical Computer Science, Springer-Verlag, Berlin, 1984.

[42]

R. Motwani and P. Raghavan, Randomized Algorithms, Cambridge University Press, Cambridge, UK, 1995.

[43]

E. Pyrga and S. Ray, New existence proofs for $\epsilon$-nets, in Proceedings of the 24th Annual ACM Symposium on Computational Geometry (SoCG), 2008, pp. 199-207.

[44]

R. Rivest, On self-organizing sequential search heuristics, Comm. ACM, 19 (1976), pp. 63-67.

[45]

D. D. Sleator and R. E. Tarjan, Amortized efficiency of list update and paging rules, Comm. ACM, 28 (1985), pp. 202-208.

[46]

D. D. Sleator and R. E. Tarjan, Self-adjusting binary search trees, J. ACM, 32 (1985), pp. 652-686.

Cited By

Jin KCheng SChiu MWong M(2022)A Generalization of Self-Improving AlgorithmsACM Transactions on Algorithms10.1145/353122718:3(1-32)Online publication date: 11-Oct-2022
https://dl.acm.org/doi/10.1145/3531227
Lykouris TVassilvitskii S(2021)Competitive Caching with Machine Learned AdviceJournal of the ACM10.1145/344757968:4(1-25)Online publication date: 14-Jul-2021
https://dl.acm.org/doi/10.1145/3447579
Anand KGe RPanigrahi DDaumé HSingh A(2020)Customizing ML predictions for online algorithmsProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3524967(303-313)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3524967
Show More Cited By

Recommendations

Work-Time Optimal k-Merge Algorithms on the PRAM

For 2 k n, the k-merge problem is to merge a collection of k sorted sequences of total length n into a new sorted sequence. The k-merge problem is fundamental as it provides a common generalization of both merging and sorting. The main contribution of ...
Read More
A Generalization of Self-Improving Algorithms
Ailon et al. [SICOMP’11] proposed self-improving algorithms for sorting and Delaunay triangulation (DT) when the input instances x₁, ... , x_n follow some unknown product distribution. That is, x_i is drawn independently from a fixed unknown distribution 𝒟_...
Read More
Multi-point sampling in parallel algorithms for combinatorial search
Read More

Comments

Information & Contributors

Information

Published In

cover image SIAM Journal on Computing

SIAM Journal on Computing Volume 40, Issue 2

March 2011

372 pages

ISSN:0097-5397

Issue’s Table of Contents

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 April 2011

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Jin KCheng SChiu MWong M(2022)A Generalization of Self-Improving AlgorithmsACM Transactions on Algorithms10.1145/353122718:3(1-32)Online publication date: 11-Oct-2022
https://dl.acm.org/doi/10.1145/3531227
Lykouris TVassilvitskii S(2021)Competitive Caching with Machine Learned AdviceJournal of the ACM10.1145/344757968:4(1-25)Online publication date: 14-Jul-2021
https://dl.acm.org/doi/10.1145/3447579
Anand KGe RPanigrahi DDaumé HSingh A(2020)Customizing ML predictions for online algorithmsProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3524967(303-313)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3524967
Cheng SJin KYan L(2020)Extensions of Self-Improving SortersAlgorithmica10.1007/s00453-019-00604-682:1(88-106)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.1007/s00453-019-00604-6
Afshani PBarbay JChan T(2017)Instance-Optimal Geometric AlgorithmsJournal of the ACM10.1145/304667364:1(1-38)Online publication date: 17-Mar-2017
https://dl.acm.org/doi/10.1145/3046673
Yampolskiy R(2015)Analysis of Types of Self-Improving SoftwareProceedings of the 8th International Conference on Artificial General Intelligence - Volume 920510.1007/978-3-319-21365-1_39(384-393)Online publication date: 22-Jul-2015
https://dl.acm.org/doi/10.1007/978-3-319-21365-1_39
Ngo HNguyen DRe CRudra AHull RGrohe M(2014)Beyond worst-case analysis for joins with minesweeperProceedings of the 33rd ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems10.1145/2594538.2594547(234-245)Online publication date: 18-Jun-2014
https://dl.acm.org/doi/10.1145/2594538.2594547
Rudek R(2014)Exact and parallel metaheuristic algorithms for the single processor total weighted completion time scheduling problem with the sum-of-processing-time based modelsComputers and Operations Research10.1016/j.cor.2014.01.00846(91-101)Online publication date: 1-Jun-2014
https://dl.acm.org/doi/10.1016/j.cor.2014.01.008
Rudek R(2013)The computational complexity analysis of the two-processor flowshop problems with position dependent job processing timesApplied Mathematics and Computation10.5555/2745046.2745189221:C(819-832)Online publication date: 15-Sep-2013
https://dl.acm.org/doi/10.5555/2745046.2745189
Löffler MMulzer W(2013)Unions of onionsProceedings of the 13th international conference on Algorithms and Data Structures10.1007/978-3-642-40104-6_42(487-498)Online publication date: 12-Aug-2013
https://dl.acm.org/doi/10.1007/978-3-642-40104-6_42
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents