## Information-theoretic computational complexity (1974)

Venue: | IEEE Transactions on Information Theory |

Citations: | 35 - 10 self |

### BibTeX

@ARTICLE{Chaitin74information-theoreticcomputational,

author = {Gregory J. Chaitin},

title = {Information-theoretic computational complexity},

journal = {IEEE Transactions on Information Theory},

year = {1974},

volume = {20},

pages = {10--15}

}

### Years of Citing Articles

### OpenURL

### Abstract

This paper attempts to describe, in nontechnical language, some of the concepts and methods of one school of thought regarding computational complexity. It applies the viewpoint of information theory to computers. This will first lead us to a definition of the degree of randomness of individual binary strings, and then to an information-theoretic version of Gödel's theorem on the limitations of the axiomatic method. Finally, we will examine in the light of these ideas the scientific method and von Neumann's views on the basic conceptual problems of biology. This field's fundamental concept is the complexity of a binary string, that is, a string of bits, of zeros and ones. The complexity of a binary string is the minimum quantity of information needed to define the string. For example, the string of length n consisting entirely of ones is of complexity approximately log 2 n, because only log 2 n bits of information are required to specify n in binary notation. However, this is rather vague. Exactly what is meant by the definition of a string? To make this idea precise a computer is used. One says that a string defines another when the first string gives instructions for constructing the second string. In other words, one string defines another when it is a

### Citations

1050 |
Theory of games and economic behavior
- Neumann, Morgenstem
- 1947
(Show Context)
Citation Context ...na. His first effort in this direction was his work Theory of Games and Economic Behavior, in which he analyzes what is a rational way to behave in situations in which there are conflicting interests =-=[3]-=-. The Computer and the Brain, his notes for a lecture series, was published shortly after his death [5]. This book discusses the differences and similarities between the computer and the brain, as a f... |

838 |
Theory of Recursive Functions and Effective Computability
- ROGERS
- 1967
(Show Context)
Citation Context ...tes and the function is thus undefined for some values of its arguments, then the function is called partial recursive. (Full treatments of these concepts can be found in standard texts, e.g., Rogers =-=[16]-=-.) In what follows we are concerned with computations involving binary strings. The binary strings are considered to be ordered in the following manner: Λ, 0, 1, 00, 01, 10, 11, 000, 001, 010, ... The... |

406 |
A formal theory of inductive inference
- Solomonoff
- 1964
(Show Context)
Citation Context ...a computer mathematically. What remains to be analyzed is which definition should be adopted, inasmuch as some computers are easier to program than others. A decade ago Solomonoff solved this problem =-=[7]-=-. He constructed a definition of a computer whose programs are not much longer than those of any other computer. More exactly, Solomonoff's machine simulates running a program on another computer, whe... |

224 |
Selforganization of matter and the evolution of biological macromolecules
- Eigen
(Show Context)
Citation Context ...n of life in this abstract setting remains to be solved; and to express mathematically the evolution of the complexity of organisms, one must first define complexity precisely. (In an important paper =-=[14]-=-, Eigen studies these questions from the point of view of thermodynamics and biochemistry.) We submit that ``organism'' must also be defined, and have tried elsewhere to suggest how this might perhaps... |

91 |
Logical Basis for Information Theory and Probability Theory
- Kolmogorov
- 1968
(Show Context)
Citation Context ...ead or tail, it is highly probable that the complexity of this string will be very close to n. In 1965 Kolmogorov proposed calling random those strings of length n whose complexity is approximately n =-=[8]-=-. We made the same proposal independently [9]. It can be shown that a string that is random in this sense has the statistical properties that one would expect. For example, zeros and ones appear in su... |

50 |
From Frege to Gödel. A Source Book
- Heijenoort
- 1967
(Show Context)
Citation Context ...culating ``the first string that can be proven to be of complexity greater than 1 000 000 000?'' The answer depends on the concept of a formal axiom system, whose importance was emphasized by Hilbert =-=[1]-=-. Hilbert proposed that mathematics be made as exact and precise as possible. In order to avoid arguments between mathematicians about the validity of proofs, he set down explicitly the methods of rea... |

20 | On the Difficulty of Computations
- Chaitin
- 1970
(Show Context)
Citation Context ...omplexity of this string will be very close to n. In 1965 Kolmogorov proposed calling random those strings of length n whose complexity is approximately n [8]. We made the same proposal independently =-=[9]-=-. It can be shown that a string that is random in this sense has the statistical properties that one would expect. For example, zeros and ones appear in such strings with relative frequencies that ten... |

2 |
The Undecidable---Basic Papers on Undecidable Propositions, Unsolvable Problems and Computable Functions
- Davis
- 1965
(Show Context)
Citation Context ...ity Pagina 2 di 10 to define an idealized computer with unlimited memory. This was done in a very intuitive way by Turing and also by Post, and there are elegant definitions based on other principles =-=[2]-=-. The theory of recursive functions (or computability theory) has grown up around the questions of what is computable and what is not. Thus it is not difficult to define a computer mathematically. Wha... |

1 |
a mathematical definition of `life
- ``To
- 1970
(Show Context)
Citation Context ...tudies these questions from the point of view of thermodynamics and biochemistry.) We submit that ``organism'' must also be defined, and have tried elsewhere to suggest how this might perhaps be done =-=[10]-=-. We believe that the concept of complexity that has been presented here may be the tool that von Neumann felt is needed. It is by no means accidental that biological phenomena are considered to be ex... |

1 | Statistical Independence in Probability, Analysis - Kac - 1959 |

1 |
Artificial intelligence---a case for agnosticism
- Chandrasekaran, Reeker
- 1972
(Show Context)
Citation Context ... he sees, or uses natural languages to communicate. We cannot carry out these tasks by computer because they are as yet too complex for us---the programs would be too long. (Chandrasekaran and Reeker =-=[15]-=- discuss the relevance of complexity to artificial intelligence.) Appendix In this Appendix we try to give a more detailed idea of how the results concerning formal axiom systems that were stated are ... |