research-article

Twenty (simple) questions

Authors:

Shay MoranAuthors Info & Claims

STOC 2017: Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing

Pages 9 - 21

https://doi.org/10.1145/3055399.3055422

Published: 19 June 2017 Publication History

Abstract

A basic combinatorial interpretation of Shannon's entropy function is via the "20 questions" game. This cooperative game is played by two players, Alice and Bob: Alice picks a distribution Π over the numbers {1,…,n}, and announces it to Bob. She then chooses a number x according to Π, and Bob attempts to identify x using as few Yes/No queries as possible, on average.

An optimal strategy for the "20 questions" game is given by a Huffman code for Π: Bob's questions reveal the codeword for x bit by bit. This strategy finds x using fewer than H(Π)+1 questions on average. However, the questions asked by Bob could be arbitrary. In this paper, we investigate the following question: *Are there restricted sets of questions that match the performance of Huffman codes, either exactly or approximately?

Our first main result shows that for every distribution Π, Bob has a strategy that uses only questions of the form "x < c?" and "x = c?", and uncovers x using at most H(Π)+1 questions on average, matching the performance of Huffman codes in this sense. We also give a natural set of O(rn^1/r) questions that achieve a performance of at most H(Π)+r, and show that Ωrn^1/r) questions are required to achieve such a guarantee.

Our second main result gives a set Q of 1.25^n+o(n) questions such that for every distribution Π, Bob can implement an optimal strategy for Π using only questions from Q. We also show that 1.25^n-o(n) questions are needed, for infinitely many n. If we allow a small slack of r over the optimal strategy, then roughly (rn)^Θ(1/r) questions are necessary and sufficient.

Supplementary Material

MP4 File (d2_sa_t5.mp4)

Download
153.36 MB

References

[1]

Julia Abrahams. 1997.

[2]

Code and parse trees for lossless source encoding. In Compression and Complexity of Sequences. 145–171.

[3]

Micah Adler and Bruce M. Maggs. 2001. Protocols for Asymmetric Communication Channels. J. Comput. System Sci. 63, 4 (2001), 573–596.

Digital Library

[4]

Rudolf Ahlswede and Ingo Wegener. 1987.

[5]

Search problems. John Wiley & Sons, Inc., New York.

[6]

Nir Ailon and Bernard Chazelle. 2005.

[7]

Lower bounds for linear degeneracy testing. J. ACM 52, 2 (2005), 151–171.

Digital Library

[8]

Richard Anderson, Sampath Kannan, Howard Karloff, and Richard E. Ladner. 2002.

[9]

Thresholds and optimal binary comparison search trees. Journal of Algorithms 44 (2002), 338–358.

Digital Library

[10]

Javad A. Aslam and Aditi Dhagat. 1991. Searching in the presence of linearly bounded errors. In Proceedings of the twenty-third annual ACM symposium on Theory of computing (STOC ’91). 486–493.

Digital Library

[11]

Harout Aydinian, Ferdinando Cicalese, and Christian Deppe (Eds.). 2013.

[12]

Information Theory, Combinatorics, and Search Theory. Springer-Verlag Berlin Heidelberg.

[13]

Michael B. Baer. 2007. Twenty (or so) Questions: D-ary Length-Bounded Prefix Coding. In IEEE International Symposium on Information Theory (ISIT 2007). 896– 900.

[14]

Yosi Ben-Asher, Eitan Farchi, and Ilan Newman. 1999. Optimal search in trees. SIAM J. Comput. 28, 6 (1999), 2090–2102.

Digital Library

[15]

Michael Ben-Or. 1983. Lower bounds for algebraic computation trees. In Proceedings of the 15th ACM Symposium on Theory of Computing. 80–86.

Digital Library

[16]

Prosenjit Bose and Karim Douïeb. 2009. Efficient Construction of Near-Optimal Binary and Multiway Search Trees. In Algorithms and Data Structures (WADS 2009). 230–241.

Digital Library

[17]

Renato M. Capocelli, Raffaele Giancarlo, and Indeer Jeet Taneja. 1986. Bounds on the Redundancy of Huffman Codes. IEEE Transactions on Information Theory IT-32, 6 (1986), 854–857.

Digital Library

[18]

Jean Cardinal, John Iacono, and Aurélien Ooms. 2016. Solving k-SUM using few linear queries. In 24th Annual European Symposium on Algorithms (ESA 2016). LIPIcs, 25:1–25:17.

[19]

Marek Chrobak, Mordecai Golin, J. Ian Munro, and Neal E. Young. 2015. Optimal Search Trees with 2-Way Comparisons. In Algorithms and Computation (ISAAC 2015). 71–82.

[20]

David Cohn, Les Atlas, and Richard Ladner. 1994.

[21]

Improving Generalization with Active Learning. Machine Learning 15 (1994), 201–221.

Digital Library

[22]

C. J. Colbourn, J. H. Dinitz, and D. R. Stinson. 1993. Applications of combinatorial designs to communications, cryptography, and networking. In Surveys in Combinatorics, 1993, K. Walker (Ed.). London Mathematical Society Lecture Note Series, Vol. 187. Cambridge University Press.

[23]

Thomas M. Cover and Joy A. Thomas. 2006.

[24]

Elements of information theory (2. ed.). Wiley.

[25]

Roberto De Prisco and Alfredo De Santis. 1993. On Binary Search Trees. Inform. Process. Lett. 45, 5 (1993), 249–253.

Digital Library

[26]

Aditi Dhagat, Peter Gács, and Peter Winkler. 1992. On Playing “Twenty Questions” with a Liar. In Proceedings of 3rd Symposium on Discrete Algorithms (SODA’92). 16–22.

[27]

Robert Dorfman. 1943. The Detection of Defective Members of Large Populations. The Annals of Mathematical Statistics 14, 4 (1943), 436–440.

[28]

Ding-Zhu Du and Frank K. Hwang. 1999.

[29]

Combinatorial Group Testing and Its Applications (2nd ed.). Series on Applied Mathematics, Vol. 12. World Scientific.

[30]

Dwight Duffus, Bill Sands, and Peter Winkler. 1990.

[31]

Maximal chains and antichains in Boolean lattices. SIAM Journal on Discrete Mathematics 3, 2 (1990), 197–205. Twenty (Simple) Questions STOC’17, June 2017, Montreal, Canada

[32]

Jeff Erickson. 1999.

[33]

Lower bounds for linear satisfiability problems. Chicago Journal of Theoretical Computer Science 8 (1999).

[34]

William Evans and David Kirkpatrick. 2004. Restructuring ordered binary trees. Journal of Algorithms 50, 2 (2004), 168–193.

Digital Library

[35]

Ester Ezra and Micha Sharir. 2016. The Decision Tree Complexity for k-SUM is at most Nearly Quadratic. Manuscript. (2016).

[36]

Newton Faller. 1973. An adaptive system for data compression. In Record of the 7th Asilomar Conference on Circuits, Systems, and Computers. 593–597.

[37]

Michael L. Fredman. 1976. How good is the information theory bound in sorting? Theoretical Computer Science 1, 4 (1976), 355–361.

[38]

Ari Freund. 2015. Improved Subquadratic 3SUM. Algorithmica (2015), 1–19.

Digital Library

[39]

Robert G. Gallager. 1978. Variations on a Theme by Huffman. IEEE Transactions on Information Theory IT-24, 6 (1978), 668–674.

Digital Library

[40]

Michael R. Garey. 1974. Optimal Binary Search Trees with Restricted Maximal Depth. SIAM J. Comput. 3, 2 (1974), 101–110.

Digital Library

[41]

Michael R. Garey and Ronald L. Graham. 1974.

[42]

Performance bounds on the splitting algorithm for binary testing. Acta Informatica 3 (1974), 347–355.

Digital Library

[43]

Adriano M. Garsia and Michelle L. Wachs. 1977. A new algorithm for minimum cost binary trees. SIAM J. Comput. 6, 4 (1977), 622–642.

Digital Library

[44]

Edgar N. Gilbert. 1971.

[45]

Codes based on inaccurate source probabilities. IEEE Transactions on Information Theory 17, 3 (1971), 304–314.

Digital Library

[46]

E. N. Gilbert and E. F. Moore. 1959.

[47]

Variable-Length Binary Encodings. Bell System Technical Journal 38 (1959), 933–967.

[48]

Dennis Grinberg, Sivaramakrishnan Rajagopalan, Ramarathnam Venkatesan, and Vicor K. Wei. 1995. Splay Trees for Data Compression. In Proceedings of the sixth annual ACM–SIAM symposium on Discrete algorithms (SODA’95). 522–530.

Digital Library

[49]

Allan Grønlund and Seth Pettie. 2014. Threesomes, Degenerates, and Love Triangles. In 55th Annual Symposium on Foundations of Computer Science (FOCS’14). 621–630.

Digital Library

[50]

L. H. Harper, T. H. Payne, J. E. Savage, and E. Straus. 1975. Sorting X+Y. Commun. ACM 18, 6 (1975), 347–349.

Digital Library

[51]

James H. Hester, Daniel S. Hirschberg, Shou-Hsuan Stephen Huang, and C. K. Wong. 1986.

[52]

Faster Construction of Optimal Binary Split Trees. Journal of Algorithms 7, 3 (1986), 412–424.

[53]

Yasuichi Horibe. 1977. An improved bound for weight-balanced tree. Information and Control 34, 2 (1977), 148–151.

[54]

T. C. Hu and K. C. Tan. 1972. Path Length of Binary Search Trees. SIAM J. Appl. Math. 22, 2 (1972), 225–234.

Digital Library

[55]

T. C. Hu and Alan Curtiss Tucker. 1971.

[56]

Optimal computer search trees and variable length alphabetic codes. Journal of Applied Mathematics 21 (1971), 514–532.

[57]

Shou-Hsuan Stephen Huang and C. K. Wong. 1984.

[58]

Generalized binary split trees. Acta Informatica 21, 1 (1984), 113–123.

Digital Library

[59]

Shou-Hsuan Stephen Huang and C. K. Wong. 1984. Optimal binary split trees. Journal of Algorithms 5, 1 (1984), 69–79.

[60]

David A. Huffman. 1952.

[61]

A Method for the Construction of Minimum-Redundancy Codes. In Proceedings of the I.R.E. 1098–1103.

[62]

Laurent Hyafil and Ronald L. Rivest. 1976. Constructing optimal binary decision trees is NP-complete. Inform. Process. Lett. 5, 1 (1976), 15–17.

[63]

Ottar Johnsen. 1980.

[64]

On the Redundancy of Binary Huffman Codes. IEEE Transactions on Information Theory IT-26, 2 (1980), 220–222.

Digital Library

[65]

Jeff Kahn and Jeong Han Kim. 1995. Entropy and Sorting. J. Comput. System Sci. 51, 3 (1995), 390–399.

Digital Library

[66]

Jeff Kahn and Michael Saks. 1984. Balancing poset extensions. Order 1, 2 (1984), 113–126.

[67]

Richard M. Karp. 1961. Minimum-redundancy coding for the discrete noiseless channel. I.R.E. Transactions on Information Theory (1961), 27–38.

[68]

Gyula O. H. Katona. 1973.

[69]

Combinatorial Search Problems. In A Survery of Combinatorial Theory, J. N. Srivastava et al. (Ed.). North-Holland Publishing Company.

[70]

Daniel J. Kleitman and Michael E. Saks. 1981. Set orderings requiring costliest alphabetic binary trees. SIAM Journal on Algebraic Discrete Mathematics 2, 2 (1981), 142–146.

[71]

Donald E. Knuth. 1971.

[72]

Optimum binary search trees. Acta Informatica 1, 1 (1971), 14–25.

Digital Library

[73]

Donald E. Knuth. 1985. Dynamic Huffman coding. Journal of Algorithms 6 (1985), 163–180.

Digital Library

[74]

S. Rao Kosaraju, Teresa M. Przytycka, and Ryan Borgstrom. 1999.

[75]

On an Optimal Split Tree Problem. In Algorithms and Data Structures: 6th International Workshop, WADS’99 Vancouver, Canada, August 11–14, 1999 Proceedings, Frank Dehne, Jörg-Rüdiger Sack, Arvind Gupta, and Roberto Tamassia (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 157–168.

Digital Library

[76]

T. W. Lam and F. L. Yue. 2001.

[77]

Optimal edge ranking of trees in linear time. Algorithmica 30, 1 (2001), 12–33.

[78]

Jean-Luc Lambert. 1992.

[79]

Sorting the sums (x i + y j ) in O (n 2 ) comparisons. Theoretical Computer Science 103, 1 (1992), 137–141.

Digital Library

[80]

Lawrence L. Larmore. 1987.

[81]

Height-restricted optimal binary trees. SIAM J. Comput. 16 (1987), 1115–1123.

Digital Library

[82]

Nati Linial and Michael Saks. 1985. Every poset has a central element. Journal of Combinatorial Theory 40 (1985), 195–210.

[83]

Nati Linial and Michael Saks. 1985.

[84]

Searching order structures. Journal of Algorithms 6 (1985), 86–103.

[85]

Zbigniew Lonc and Ivan Rival. 1987. Chains, Antichains, and Fibres. Journal of Combinatorial Theory, Series A 44 (1987), 207–228.

Digital Library

[86]

Dietrich Manstetten. 1992. Tight Bounds on the Redundancy of Huffman Codes. IEEE Transactions on Information Theory IT-38, 1 (1992), 144–151.

Digital Library

[87]

S. Meiser. 1993. Point location in arrangements of hyperplanes. Information and Computation 106, 2 (1993), 286–303.

Digital Library

[88]

Friedhelm Meyer auf der Heide. 1984. A polynomial linear search algorithm for the n-dimensional knapsack problem. J. ACM 31 (1984), 668–676.

Digital Library

[89]

Soheil Mohajer, Payam Pakzad, and Ali Kakhbod. 2006.

[90]

Tight Bounds on the Redundancy of Huffman Codes. In Information Theory Workshop (ITW ’06). 131– 135.

[91]

Bruce L. Montgomery and Julia Abrahams. 1987. On the Redundancy of Optimal Binary Prefix-Condition Codes for Finite and Infinite Sources. IEEE Transactions on Information Theory IT-33, 1 (1987), 156–160.

Digital Library

[92]

Shay Moran and Amir Yehudayoff. 2016. A note on average-case sorting. Order 33, 1 (2016), 23–28.

[93]

Shay Mozes, Krzysztof Onak, and Oren Weimann. 2008. Finding an optimal tree searching strategy in linear time. In Proceedings of 19th Symposium on Discrete Algorithms (SODA’08). 1096–1105.

Digital Library

[94]

S. V. Nagaraj. 1997. Optimal binary search trees. Theoretical Computer Science 188, 1–2 (1997), 1–44.

Digital Library

[95]

Narao Nakatsu. 1991. Bounds on the Redundancy of Binary Alphabetical Codes. IEEE Transactions on Information Theory IT-37, 4 (1991), 1225–1229.

Digital Library

[96]

Krzysztof Onak and Paweł Parys. 2006. Generalization of Binary Search: Searching in Trees and Forest-Like Partial Orders. In 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2006), 21-24 October 2006, Berkeley, California, USA, Proceedings. 379–388.

Digital Library

[97]

Yehoshua Perl. 1984.

[98]

Optimum split trees. Journal of Algorithms 5, 3 (1984), 367–374.

Digital Library

[99]

J. Ross Quinlan. 1986. Induction of Decision Trees. Machine Learning 1 (1986), 81–106.

[100]

Jorma Rissanen. 1973. Bounds for weight balanced trees. IBM Journal of Research and Development 17 (1973), 101–105.

Digital Library

[101]

Ronald L. Rivest, Albert R. Meyer, Daniel J. Kleitman, Karl Winklmann, and Joel Spencer. 1980. Coping with errors in binary search procedures. J. Comput. System Sci. 20 (1980), 396–404.

[102]

Baruch Schieber. 1998. Computing a Minimum Weight k-Link Path in Graphs with the Concave Monge Property. Journal of Algorithms 29 (1998), 204–222.

Digital Library

[103]

Claude Elwood Shannon. 1948. A Mathematical Theory of Communication. Bell System Technical Journal 27 (1948), 379–423.

[104]

B. A. Sheil. 1978.

[105]

Median split trees: a fast lookup technique for frequently occuring keys. Commun. ACM 21, 11 (1978), 947–958.

Digital Library

[106]

Dafna Sheinwald. 1992.

[107]

On binary alphabetical codes. In Data Compression Conference (DCC’92). 112–121.

[108]

Cedric A. B. Smith. 1947.

[109]

The Counterfeit Coin Problem. The Mathematical Gazette 31, 293 (1947), 31–39.

[110]

Joel Spencer and Peter Winkler. 1992. Three thresholds for a liar. Combinatorics, Probability and Computing 1, 1 (1992), 81–93.

[111]

David Spuler. 1994. Optimal search trees using two-way key comparisons. Acta Informatica 31 (1994), 729–740.

Digital Library

[112]

David A. Spuler. 1994.

[113]

Optimal Binary Trees With Two-Way Key Comparisons. Ph.D. Dissertation. James Cook University.

[114]

Jan van Leeuwen. 1987.

[115]

On the construction of Huffman trees. In Third International Colloquium on Automata, Languages and Programming (ICALP ’87). 382–410.

[116]

Jeffrey Scott Vitter. 1987.

[117]

Design and analysis of dynamic Huffman codes. J. ACM 34, 4 (1987), 825–845.

Digital Library

[118]

W. A. Walker and C. C. Gottlieb. 1972.

[119]

A top-down algorithm for constructing nearly optimal lexicographical trees. Academic Press, 303–323.

[120]

John Watkinson, Micah Adler, and Faith E. Fich. 2001. New Protocols for Asymmetric Communication Channels. In 8th International Colloquium on Structural Information and Communication Complexity (SIROCCO). 337–350.

[121]

Raymond W. Yeung. 1991.

Cited By

Bruno RDe Prisco RDe Santis AVaccaro U(2024)Bounds and Algorithms for Alphabetic Codes and Binary Search TreesIEEE Transactions on Information Theory10.1109/TIT.2024.342869970:10(6974-6988)Online publication date: Oct-2024
https://doi.org/10.1109/TIT.2024.3428699
Muroň MDařena FProcházka DKern R(2024)User-Driven Toponym Disambiguation Using DialogueJournal of Map & Geography Libraries10.1080/15420353.2024.236321219:3(198-219)Online publication date: 17-Jun-2024
https://doi.org/10.1080/15420353.2024.2363212
Chrobak MYoung N(2023)Classification via Two-Way Comparisons (Extended Abstract)Algorithms and Data Structures10.1007/978-3-031-38906-1_19(275-290)Online publication date: 28-Jul-2023
https://doi.org/10.1007/978-3-031-38906-1_19
Show More Cited By

Index Terms

Twenty (simple) questions
1. Mathematics of computing
  1. Discrete mathematics
    1. Combinatorics
      1. Combinatoric problems
  2. Information theory

Recommendations

Twenty (Short) Questions
Abstract
A basic combinatorial interpretation of Shannon’s entropy function is via the “20 questions” game. This cooperative game is played by two players, Alice and Bob: Alice picks a distribution π over the numbers {1, …, n}, and announces it to Bob. She ...
Teleportation with an imperfect state

The standard teleportation protocol requires the availability of a maximally entangled state. Because such states are difficult to consistently generate experimentally, we study teleportation in which the entanglement used need not be maximal. The ...
Fast HEVC Inter-Prediction Algorithm Based on Matching Block Features

A fast inter-prediction algorithm based on matching block features is proposed in this article. The position of the matching block of the current CU in the previous frame is found by the motion vector estimated by the corresponding located CU in the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

STOC 2017: Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing

June 2017

1268 pages

ISBN:9781450345286

DOI:10.1145/3055399

General Chairs:
Hamed Hatami
McGill University
,
Pierre McKenzie
Université de Montréal
,
Program Chair:
Valerie King
University of Victoria

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

STOC '17

Sponsor:

SIGACT

STOC '17: Symposium on Theory of Computing

June 19 - 23, 2017

Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 1,469 of 4,586 submissions, 32%

Upcoming Conference

STOC '25

Sponsor:
sigact

57th Annual ACM Symposium on Theory of Computing (STOC 2025)

June 23 - 27, 2025

Prague , Czech Republic

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
155
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)2

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bruno RDe Prisco RDe Santis AVaccaro U(2024)Bounds and Algorithms for Alphabetic Codes and Binary Search TreesIEEE Transactions on Information Theory10.1109/TIT.2024.342869970:10(6974-6988)Online publication date: Oct-2024
https://doi.org/10.1109/TIT.2024.3428699
Muroň MDařena FProcházka DKern R(2024)User-Driven Toponym Disambiguation Using DialogueJournal of Map & Geography Libraries10.1080/15420353.2024.236321219:3(198-219)Online publication date: 17-Jun-2024
https://doi.org/10.1080/15420353.2024.2363212
Chrobak MYoung N(2023)Classification via Two-Way Comparisons (Extended Abstract)Algorithms and Data Structures10.1007/978-3-031-38906-1_19(275-290)Online publication date: 28-Jul-2023
https://doi.org/10.1007/978-3-031-38906-1_19
Chrobak MGolin MMunro JYoung N(2021)On The Cost of Unsuccessful Searches in Search Trees with Two-way ComparisonsInformation and Computation10.1016/j.ic.2021.104707(104707)Online publication date: Feb-2021
https://doi.org/10.1016/j.ic.2021.104707
Dereniowski DŁukasiewicz AUznański P(2021)An Efficient Noisy Binary Search in Graphs via Median ApproximationCombinatorial Algorithms10.1007/978-3-030-79987-8_19(265-281)Online publication date: 30-Jun-2021
https://doi.org/10.1007/978-3-030-79987-8_19

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents