## The miraculous universal distribution (1997)

Venue: | Mathematical Intelligencer |

Citations: | 20 - 3 self |

### BibTeX

@ARTICLE{Kirchherr97themiraculous,

author = {Walter Kirchherr and Ming Li and Paul Vitányi},

title = {The miraculous universal distribution},

journal = {Mathematical Intelligencer},

year = {1997},

volume = {19},

pages = {7--15}

}

### Years of Citing Articles

### OpenURL

### Abstract

scientific hypothesis formulated? How does one choose one hypothesis over another? It may be surprising that questions such as these are still discussed. Even more surprising, perhaps, is the fact that the discussion is still moving forward, that new ideas are still being added to the debate. Certainly most surprising of all is the fact that, over the last thirty years or so, the normally concrete field of computer science has provided fundamental new insights. Scientists engage in what is usually called inductive reasoning. Inductive reasoning entails making predictions about future behavior based on past observations. But defining the proper method of formulating such predictions has occupied philosophers throughout the ages. In fact, the British philosopher David Hume (1711 – 1776) has argued convincingly that in some sense proper induction is impossible, [3]. It is impossible because we can only reach conclusions by using known data and methods. Therefore, the conclusion is logically already contained in the start configuration. Consequently, the only form of induction possible is deduction. Philosophers have tried to find

### Citations

1682 | An Introduction to Kolmogorov Complexity and its Applications
- Li, Vitányi
- 1997
(Show Context)
Citation Context ... notions, showing that these two questions are fundamentally related, and made major strides toward answering the age-old questions described above. (An extensive history of the field can be found in =-=[7]-=-.) For those of us living in the computer age, the notion of `computable' is pretty much intuitive. Many of us have written programs; most of us have run computers. We know what a computer program is.... |

1157 |
M.: On computable numbers, with an application to the Entscheidungsproblem
- Turing
- 1936
(Show Context)
Citation Context ...d that the Turing machine is a mathematical precise version of what our intuition tells us is "computable," and therefore the universal Turing machine can compute all intuitively computable =-=functions [11]. Since th-=-e latter statement is not a mathematical one, it cannot be proved: it is known as Turing's Thesis; in related form known as "Church's Thesis." Just like the Kolmogorov complexity is minimal ... |

448 |
A Treatise of Human Nature
- Hume
(Show Context)
Citation Context ...g such predictions has occupied philosophers throughout the ages. In fact, the British philosopher David Hume (1711 -- 1776) has argued convincingly that in some sense proper induction is impossible, =-=[3]-=-. It is impossible because we can only reach conclusions by using known data and methods. Therefore, the conclusion is logically already contained in the start configuration. Consequently, the only fo... |

380 |
An Essay towards solving a Problem in the Doctrine of Chances
- Bayes
- 1763
(Show Context)
Citation Context ...lity that a hypothesis is true is proportional to the prior probability of the hypothesis multiplied by the probability that the observed data would have occurred assuming that the hypothesis is true.=-=[2]-=- Suppose we have a priori a distribution of the probabilities P (H) of the various possible hypotheses. We want the list of hypotheses to be exhaustive and mutually exclusive so that P P (H) = 1, the ... |

117 |
Philosophiae naturalis principia mathematica (Jussu Societas Regiae ac Typis Josephi Streater
- Newton
- 1686
(Show Context)
Citation Context ...re wading not alone but together with the greatest scientist of all time, "Fortunate Newton, happy childhood of science!" in Einstein's phrase. Isaac Newton (1642 -- 1727) formulated in his =-=Principia [8]-=-: Newton's Rule #1 for doing natural philosophy: We are to admit no more causes of natural things than such as are both true and sufficient to explain the appearances. To this purpose the philosophers... |

50 |
A Device for Quantizing, Grouping and Coding Amplitude Modulated Pulses,” master’s thesis
- Kraft
- 1949
(Show Context)
Citation Context ...nywhere else), the programs written in our language form a prefix-free set.) The reason this approach saved the day is due to a key theorem proved in 1949 by L.G. Kraft (in his master's thesis at MIT =-=[4]-=-). For our purposes, it says in essence that if we restrict our attention to prefix-free sets, then the resulting a priori probabilities will sum to no more than one. From now on we'll use this slight... |

10 |
A philosophical essay on probabilities, 1819. English translation
- Laplace
- 1951
(Show Context)
Citation Context ...Bayes's Rule as given is not due to Bayes. Pierre-Simon, Marquis de Laplace, whom we will meet again later in this narrative, stated Bayes's Rule in its proper form and attached Bayes's name to it in =-=[5]. Whe-=-re Does This Leave Alice? So now that Alice knows the thoughts of the ancients, what should she do. Should she take the bet? Her basic question is, "What process (that is, what kind of coin) caus... |

6 |
The Life of Samuel
- Boswell
- 1792
(Show Context)
Citation Context ...as clearly right; yet the seeing of two extremes, each of which is in some degree more conspicuous than the rest, could not but strike one in a stronger manner than the sight of any other two numbers.=-=[1] Many-=- of us (including Alice) would agree with Boswell. (Most of us are not Samuel Johnson, of whom it was also said, "There's no arguing with Johnson; for when his pistol misses fire, he knocks you d... |

1 |
Worst case somplexity is equal to average case complexity under the universal distribution, Information Processing Letters
- Li, Vit'anyi
- 1992
(Show Context)
Citation Context ...f any distribution. It is somehow natural to assume that in many cases the average is better (in this case lower) than the worst case. It came as a surprise when it was shown by two of us (ML and PV) =-=[6]-=- that if we assume the universal distribution, that is, if we assume that inputs with low complexity (ones with `short descriptions') are more likely than inputs with high complexity, then the running... |

1 |
Lucretius Carus, The Nature of the Universe (Ronald Latham, translator
- Titus
- 1965
(Show Context)
Citation Context ...ted. For you would not be able to establish conclusively that he died by the sword or of cold or of illness or perhaps by poison, but we know that there is something of this kind that happened to him.=-=[9] This-=- multiple explanations approach is sometimes called the principle of indifference. The British philosopher Bertrand Russell (1872 -- 1970) summarizes it as follows: "When there are several possib... |

1 |
A History of Western Philosophy, Simon and
- Russel
- 1945
(Show Context)
Citation Context ... The British philosopher Bertrand Russell (1872 -- 1970) summarizes it as follows: "When there are several possible naturalistic explanations . . . there is no point in trying to decide between t=-=hem."[10]-=- In other words: Principle of Indifference: Keep all hypotheses that are consistent with the facts. (To be fair, it should be pointed out that Epicurean philosophy is not concerned with scientific pro... |

1 |
Collected Works, Volume V
- Neumann
- 1963
(Show Context)
Citation Context ...on (of a model) is solely and precisely that it is expected to work. . . . Furthermore, it must satisfy certain aesthetic criteria -- that is, in relation to how much it describes, it must be simple. =-=[12]-=- Of course there are problems with this. Why should a scientist be governed by `aesthetic' criteria? What is meant by `simple'? Isn't such a concept hopelessly subjective? We're wading in deep waters ... |