research-article

A Brief on Short Descriptions

Authors:

Marius ZimandAuthors Info & Claims

ACM SIGACT News, Volume 47, Issue 1

Pages 42 - 67

https://doi.org/10.1145/2902945.2902957

Published: 10 March 2016 Publication History

Abstract

We discuss research developments on the complexity of shortest programs since the turn of the millennium. In particular, we will delve into the phenomenon of list approximation: while it's impossible to compute the shortest description for a given string, we can efficiently generate a short list of candidates which includes a (nearly) shortest description.

References

[1]

B. Bauwens and A. Shen. Complexity of complexity and maximal plain versus prefix-free Kolmogorov complexity. Journal of Symbolic Logic, 79(2):620--632, 2014.

[2]

Bruno Bauwens, Anton Makhlin, Nikolay K. Vereshchagin, and Marius Zimand. Short lists with short programs in short time. In Proceedings of the 28th Conference on Computational Complexity, CCC 2013, Palo Alto, California, USA, 5-7 June, 2013, pages 98--108, 2013.

[3]

Bruno Bauwens and Marius Zimand. Linear list-approximation for short programs (or the power of a few random bits). In IEEE 29th Conference on Computational Complexity, CCC 2014, Vancouver, BC, Canada, June 11-13, 2014, pages 241--247, 2014.

Digital Library

[4]

Richard Beigel, Harry Buhrman, Peter Fejer, Lance Fortnow, Piotr Grabowski, Luc Longpré, Andrej Muchnik, Frank Stephan, and Leen Torenvliet. Enumerations of the Kolmogorov function. The Journal of Symbolic Logic, 71(2):501--528, 2006.

[5]

Charles H. Bennett. Logical depth and physical complexity. In The universal Turing machine: a half-century survey, Oxford Science Publications, pages 227--257. Oxford Univ Press, New York, 1988.

Digital Library

[6]

Cristian S. Calude and G. J. Chaitin. What is::: a halting probability? Notices of the American Mathematical Society, 57(2):236--237, 2010.

[7]

Cristian S. Calude and André Nies. Chaitin numbers and strong reducibilities. In Proceedings of the First Japan-New Zealand Workshop on Logic in Computer Science (Auckland, 1997), volume 3, pages 1162--1166, 1997.

[8]

John Case. Homework assignment for students. Computer and Information Sciences Department, University of Delaware, 1990.

[9]

Gregory J. Chaitin. Information-theoretic limitations of formal systems. Journal of the Association for Computing Machinery, 21:403--424, 1974.

Digital Library

[10]

Daniyar Chumbalov and Andrei Romashchenko. Randomized polynomial time protocol for combinatorial Slepian-Wolf problem. In Giuseppe F. Italiano, Giovanni Pighizzini, and Donald T. Sannella, editors, Mathematical Foundations of Computer Science 2015, volume 9235 of Lecture Notes in Computer Science, pages 235--247. Springer Berlin Heidelberg, 2015.

[11]

Rudi Cilibrasi and Paul M. B. Vitányi. Clustering by compression. IEEE Transactions on Information Theory, 51(4):1523--1545, April 2005.

Digital Library

[12]

Thomas M. Cover and Joy A. Thomas. Elements of information theory. Wiley-Interscience {John Wiley & Sons}, Hoboken, NJ, second edition, 2006.

Digital Library

[13]

Imre Csiszár and János Körner. Information theory: Coding theorems for discrete memoryless systems. Cambridge University Press, Cambridge, second edition, 2011.

[14]

George Davie. http://cs.nyu.edu/pipermail/fom/2002-May/005535.html.

[15]

J. C. E. Dekker. Two notes on recursively enumerable sets. Proceedings of the American Mathematical Society, 4:495--501, 1953.

[16]

Rod G. Downey, Denis R. Hirschfeldt, André Nies, and Frank Stephan. Trivial reals. In Rod Downey, Ding Decheng, Tung Shih Ping, Qiu Yu Hui, and Mariko Yasugi, editors, Proceedings of the 7th and 8th Asian Logic Conferences (Hsi-Tou, 1999), pages 103--131. Singapore University Press and World Scientific, 2003.

[17]

Rodney G. Downey and Denis R. Hirschfeldt. Algorithmic randomness and complexity. Theory and Applications of Computability. Springer, New York, 2010.

Digital Library

[18]

Abbas El Gamal and Young-Han Kim. Network information theory. Cambridge University Press, Cambridge, 2011.

[19]

Stephen Fenner and Marcus Schaefer. Bounded immunity and btt-reductions. Mathematical Logic Quarterly, 45(1):3--21, 1999.

[20]

Harvey Fridman. http://cs.nyu.edu/pipermail/fom/2002-February/005289.html.

[21]

Richard M. Friedberg. Two recursively enumerable sets of incomparable degrees of unsolvability (solution of Post's problem, 1944). Proc. Nat. Acad. Sci. U.S.A., 43:236--238, 1957.

[22]

Richard M. Friedberg. Three theorems on recursive enumeration. I. Decomposition. II. Maximal set. III. Enumeration without duplication. The Journal of Symbolic Logic, 23:309--316, 1958.

[23]

Richard M. Friedberg and Hartley Rogers, Jr. Reducibility and completeness for sets of integers. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 5:117--125, 1959.

[24]

Peter Gács. On the symmetry of algorithmic information. Soviet Mathematics Doklady, 15:1477--1480, 1974.

[25]

Galileo Galilei. Dialogue Concerning the Two Chief World Systems, Ptolemaic and Copernican. University of California Press, August 1962. Translated by Stillman Drake with a forward by Albert Einstein.

[26]

Phil Gibbs and Sugihara Hiroshi. What is Occam's Razor? http://math.ucr.edu/home/baez/physics/General/occam.html.

[27]

Kurt Gödel. On formally undecidable propositions of Principia Mathematica and related systems. Dover Publications, Inc., New York, 1992. Translated from the German and with a preface by B. Meltzer, With an introduction by R. B. Braithwaite, Reprint of the 1963 translation.

[28]

Venkatesan Guruswami, Christopher Umans, and Salil Vadhan. Unbalanced expanders and randomness extractors from Parvaresh-Vardy codes. Journal of the ACM, 56(4):20:1--20:34, July 2009.

Digital Library

[29]

P. Hall. On representatives of subsets. Journal of the London Mathematical Society, s1- 10(1):26--30, 1935.

[30]

Valentina Harizanov, Martin Kummer, and Jim Owings. Frequency computations and the cardinality theorem. Journal of Symbolic Logic, 57(2):682--687, 1992.

[31]

Sanjay Jain. An infinite class of functions identi able using minimal programs in all Kolmogorov numberings. International Journal of Foundations of Computer Science, 06(01):89--94, 1995.

[32]

Sanjay Jain, Frank Stephan, and Jason Teutsch. Index sets and universal numberings. Journal of Computer and System Sciences, 77(4):760--773, 2011.

Digital Library

[33]

Sanjay Jain and Jason Teutsch. Enumerations including laconic enumerators. Manuscript.

[34]

Jefim Kinber. On btt-degrees of sets of minimal numbers in Gödel numberings. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 23(3):201--212, 1977.

[35]

Antonín Kučera. An alternative, priority-free, solution to Post's problem. In Mathematical foundations of computer science (MFCS 1986), volume 233 of Lecture Notes in Computer Science, pages 493--500. Springer, Berlin, 1986.

Digital Library

[36]

Martin Kummer. An easy priority-free proof of a theorem of Friedberg. Theoretical Computer Science, 74(2):249--251, 1990.

Digital Library

[37]

Martin Kummer. A proof of Beigel's cardinality conjecture. The Journal of Symbolic Logic, 57(2):677--681, 1992.

[38]

Martin Kummer. On the complexity of random strings (extended abstract). In Claude Puech and Rüdiger Reischuk, editors, 13th Annual Symposium on Theoretical Aspects of Computer Science (STACS), volume 1046 of Lecture Notes in Computer Science, pages 25--36. Springer, Berlin, 1996.

Digital Library

[39]

Martin Kummer. A note on the complexity of inclusion. Manuscript, 2008.

[40]

A. H. Lachlan. wtt-complete sets are not necessarily tt-complete. Proceedings of the American Mathematical Society, 48:429--434, 1975.

[41]

Ming Li and Paul Vitányi. An introduction to Kolmogorov complexity and its applications. Texts in Computer Science. Springer, New York, third edition, 2008.

Digital Library

[42]

Ming Li and Paul M.B. Vitányi. Inductive reasoning and Kolmogorov complexity. Journal of Computer and System Sciences, 44(2):343--384, 1992.

Digital Library

[43]

Albert R. Meyer. Program size in restricted programming languages. Information and Control, 21:382--394, 1972.

[44]

Robin A. Moser and Gábor Tardos. A constructive proof of the general Lovász local lemma. Journal of the ACM, 57(2):11:1--11:15, February 2010.

Digital Library

[45]

Andrej A. Muchnik. Conditional complexity and codes. Theoretical Computer Science, 271(1- 2):97--109, 2002.

Digital Library

[46]

A. A. Mučnik. On the unsolvability of the problem of reducibility in the theory of algorithms. Doklady Akademii Nauk SSSR, 108:194--197, 1956.

[47]

Daniil Musatov, Andrei Romashchenko, and Alexander Shen. Variations on muchnik's conditional complexity theorem. Theory of Computing Systems, 49(2):227--245, 2011.

Digital Library

[48]

John Myhill. Creative sets. Zeitschrift für Mathematische Logik und Grundlagen der Mathematik, 1:97--108, 1955.

[49]

Isaac Newton. The Principia: mathematical principles of natural philosophy. University of California Press, Berkeley, CA, 1999. A new translation by I. Bernard Cohen and Anne Whitman, assisted by Julia Budenz, Preceded by "A guide to Newton's Principia" by Cohen.

[50]

Piergiorgio Odifreddi. Classical recursion theory, volume 125 of Studies in Logic and the Foundations of Mathematics. North-Holland Publishing Co., Amsterdam, 1989. The theory of functions and sets of natural numbers.

[51]

James C. Owings, Jr. A cardinality version of Beigel's nonspeedup theorem. The Journal of Symbolic Logic, 54(3):761--767, 1989.

[52]

Emil L. Post. Recursively enumerable sets of positive integers and their decision problems. Bulletin of the American Mathematical Society, 50:284--316, 1944.

[53]

A. Romashchenko. Complexity interpretation for the fork network coding. Information Processes, 5(1):20--28, 2005. In Russian.

[54]

James S. Royer and John Case. Subrecursive programming systems: Complexity & succinctness. Progress in Theoretical Computer Science. Birkh auser Boston, Inc., Boston, MA, 1994.

Digital Library

[55]

Boris Ryabko, Zhanna Reznikova, Alexey Druzyaka, and Sofia Panteleeva. Using ideas of Kolmogorov complexity for studying biological texts. Theory of Computing Systems, 52(1):133--147, 2013.

Digital Library

[56]

Marcus Schaefer. A guided tour of minimal indices and shortest descriptions. Archive for Mathematical Logic, 37(8):521--548, 1998.

[57]

Alexander Shen, Vladimir Uspensky, and Nikolay Vereshchagin. Kolmgorov Complexity and Algorithmic Randomness. Original Russian published by MCCME Publishers, Moscow in 2012. Draft of English translation available at http://www.lirmm.fr/~ashen/kolmbook-eng.pdf.

[58]

David Slepian and Jack K. Wolf. Noiseless coding of correlated information sources. Institute of Electrical and Electronics Engineers. Transactions on Information Theory, IT-19:471--480, 1973.

Digital Library

[59]

Robert I. Soare. Recursively enumerable sets and degrees. Perspectives in Mathematical Logic. Springer-Verlag, Berlin, 1987. A study of computable functions and computably generated sets.

Digital Library

[60]

Frank Stephan. On the structures inside truth-table degrees. Journal of Symbolic Logic, 66(2):731--770, 2001.

[61]

Frank Stephan. The complexity of the set of nonrandom numbers. In Christian S. Calude, editor, Randomness and Complexity, From Leibniz to Chaitin, chapter 12, pages 217--230. World Scientific, 2007.

[62]

Frank Stephan and Jason Teutsch. Immunity and hyperimmunity for sets of minimal indices. Notre Dame Journal of Formal Logic, 49(2):107--125, 2008.

[63]

Frank Stephan and Jason Teutsch. An incomplete set of shortest descriptions. The Journal of Symbolic Logic, 77(1):291--307, March 2012.

[64]

Amnon Ta-Shma, Christopher Umans, and David Zuckerman. Lossless condensers, unbalanced expanders, and extractors. Combinatorica, 27:213--240, 2007.

Digital Library

[65]

Jason Teutsch. On the Turing degrees of minimal index sets_Annals of Pure and Applied Logic, 148:63--80, 2007.

[66]

Jason Teutsch. Short lists for shortest descriptions in short time. Computational Complexity, 23(4):565--583, 2014.

Digital Library

[67]

Jason Teutsch and Marius Zimand. On approximate decidability of minimal programs. ACM Transactions on Computational Theory, 7(4):17:1--17:16, August 2015.

Digital Library

[68]

W. M. Thorburn. The myth of Occam's Razor. Mind, XXVII(3):345--353, 1918.

[69]

Nikolay Vereshchagin. Short lists with short programs for functions. CoRR, abs/1409.5906, 2014.

[70]

Paul Vitányi. Analysis of sorting algorithms by Kolmogorov complexity (a survey). In Imre Csiszár, Gyula O.H. Katona, Gábor Tardos, and Gábor Wiener, editors, Entropy, Search, Complexity, volume 16 of Bolyai Society Mathematical Studies, pages 209--232. Springer Berlin Heidelberg, 2007.

[71]

Alfred North Whitehead and Bertrand Russell. Principia mathematica to *56. Cambridge Mathematical Library. Cambridge University Press, Cambridge, 1997. Reprint of the second (1927) edition.

[72]

Marius Zimand. Short lists with short programs in short time|a short proof. In Arnold Beckmann, Erzsébet Csuhaj-Varjú, and Klaus Meer, editors, Proceedings 10-th CiE, Budapest, Hungary, Language, Life, Limits, volume 8493 of Lecture Notes in Computer Science, pages 403--408. Springer International Publishing, 2014.

[73]

Marius Zimand. Kolmogorov complexity version of Slepian-Wolf coding. CoRR, abs/1511.03602, 2015.

[74]

A. K. Zvonkin and L. A. Levin. The complexity of finite objects and the development of the concepts of information and randomness by means of the theory of algorithms. Russian Mathematical Surveys, 25(6):83--124, 1970.

Cited By

Vitányi P(2020)How Incomputable Is Kolmogorov Complexity?Entropy10.3390/e2204040822:4(408)Online publication date: 3-Apr-2020
https://doi.org/10.3390/e22040408
Calude CJain SMerkle WStephan F(2019)Searching for shortest and least programsTheoretical Computer Science10.1016/j.tcs.2019.10.011Online publication date: Oct-2019
https://doi.org/10.1016/j.tcs.2019.10.011
Zimand MHatami HMcKenzie PKing V(2017)Kolmogorov complexity version of Slepian-Wolf codingProceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing10.1145/3055399.3055421(22-32)Online publication date: 19-Jun-2017
https://dl.acm.org/doi/10.1145/3055399.3055421
Show More Cited By

Recommendations

Short lists for shortest descriptions in short time

Is it possible to find a shortest description for a binary string? The well-known answer is "no, Kolmogorov complexity is not computable." Faced with this barrier, one might instead seek a short list of candidates which includes a laconic description. ...
Brief Announcement: A Stabilizing Algorithm for Finding Two Disjoint Paths in Arbitrary Networks
SSS '09: Proceedings of the 11th International Symposium on Stabilization, Safety, and Security of Distributed Systems

The problem of finding disjoint paths in a network is a fundamental problem with numerous applications. Two paths in a network are said to be (node) disjoint if they do not share any nodes except for the endpoints. The two node disjoint paths problem is ...
Short permutation strings

A permutation string is a string of symbols over the numerals 1, 2, ..., n such that all permutations of the string 1 2 ... n are subsequences. The search for short permutation strings arose out of studies into the complexity of shortest path algorithms ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGACT News

ACM SIGACT News Volume 47, Issue 1

March 2016

107 pages

ISSN:0163-5700

DOI:10.1145/2902945

Editor:
Stephen Fenner
University of South Carolina

Issue’s Table of Contents

Copyright © 2016 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 March 2016

Published in SIGACT Volume 47, Issue 1

Check for updates

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
129
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Vitányi P(2020)How Incomputable Is Kolmogorov Complexity?Entropy10.3390/e2204040822:4(408)Online publication date: 3-Apr-2020
https://doi.org/10.3390/e22040408
Calude CJain SMerkle WStephan F(2019)Searching for shortest and least programsTheoretical Computer Science10.1016/j.tcs.2019.10.011Online publication date: Oct-2019
https://doi.org/10.1016/j.tcs.2019.10.011
Zimand MHatami HMcKenzie PKing V(2017)Kolmogorov complexity version of Slepian-Wolf codingProceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing10.1145/3055399.3055421(22-32)Online publication date: 19-Jun-2017
https://dl.acm.org/doi/10.1145/3055399.3055421
Jain STeutsch J(2017)Enumerations including laconic enumeratorsTheoretical Computer Science10.1016/j.tcs.2017.08.001700:C(89-95)Online publication date: 14-Nov-2017
https://dl.acm.org/doi/10.1016/j.tcs.2017.08.001

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents