Results 1  10
of
17
Applying MDL to Learning Best Model Granularity
, 1994
"... The Minimum Description Length (MDL) principle is solidly based on a provably ideal method of inference using Kolmogorov complexity. We test how the theory behaves in practice on a general problem in model selection: that of learning the best model granularity. The performance of a model depends ..."
Abstract

Cited by 20 (8 self)
 Add to MetaCart
The Minimum Description Length (MDL) principle is solidly based on a provably ideal method of inference using Kolmogorov complexity. We test how the theory behaves in practice on a general problem in model selection: that of learning the best model granularity. The performance of a model depends critically on the granularity, for example the choice of precision of the parameters. Too high precision generally involves modeling of accidental noise and too low precision may lead to confusion of models that should be distinguished. This precision is often determined ad hoc. In MDL the best model is the one that most compresses a twopart code of the data set: this embodies "Occam's Razor." In two quite different experimental settings the theoretical value determined using MDL coincides with the best value found experimentally. In the first experiment the task is to recognize isolated handwritten characters in one subject's handwriting, irrespective of size and orientation. Base...
Algorithmic Complexity
 M B
, 1993
"... The theory of algorithmic complexity (commonly known as Kolmogorov complexity) or algorithmic information theory is a novel mathematical approach combining the theory of computation with information theory. It is the theory that finally formalizes the elusive notion of the amount of information in i ..."
Abstract

Cited by 15 (10 self)
 Add to MetaCart
The theory of algorithmic complexity (commonly known as Kolmogorov complexity) or algorithmic information theory is a novel mathematical approach combining the theory of computation with information theory. It is the theory that finally formalizes the elusive notion of the amount of information in individual objects, in contrast to entropy that is a statistical notion of average code word length to transmit a message form a random source. This powerful new theory has successfully resolved ancient questions about the nature of randomness of individual objects, inductive reasoning and prediction, and has applications in mathematics, computer science, physics, biology, and other sciences, including social and behavioral sciences.
On the KolmogorovChaitin Complexity for short sequences
, 2007
"... A drawback to KolmogorovChaitin complexity (K) is that it is uncomputable in general, and that limits its range of applicability. Another critique concerns the dependence of K on a particular universal Turing machine U for which predictions for short sequencesshorter for example than typical compi ..."
Abstract

Cited by 7 (7 self)
 Add to MetaCart
A drawback to KolmogorovChaitin complexity (K) is that it is uncomputable in general, and that limits its range of applicability. Another critique concerns the dependence of K on a particular universal Turing machine U for which predictions for short sequencesshorter for example than typical compiler lengths can be arbitrary. In practice one can approximate it by computable compression methods. However, such compression methods do not provide a good approximation for short sequences. Herein is suggested an empirical approach to overcome the problem that compression approximations do not work well for short sequences. Additionally, our results demonstrate that there is a strong correlation in terms of sequence frequencies across the output of several systems including such abstract systems as cellular automata and Turing machines, as well as repositories containing a sample of realworld information such as images and human DNA fragments. Our results suggest
No Free Lunch, Program Induction and Combinatorial Problems
 In Genetic Programming, Proceedings of EuroGP 2003
, 2003
"... This paper has three aims. Firstly, to clarify the poorly understood No Free Lunch Theorem (NFL) which states all search algorithms perform equally. Secondly, search algorithms are often applied to program induction and it is suggested that NFL does not hold due to the universal nature of the ma ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
This paper has three aims. Firstly, to clarify the poorly understood No Free Lunch Theorem (NFL) which states all search algorithms perform equally. Secondly, search algorithms are often applied to program induction and it is suggested that NFL does not hold due to the universal nature of the mapping between program space and functionality space. Finally, NFL and combinatorial problems are examined.
Information theory, evolutionary computation, and Dembski’s “complex specified information”’, Synthese 128(2): 237–270
, 2011
"... Intelligent design advocate William Dembski has introduced a measure of information called “complex specified information”, or CSI. He claims that CSI is a reliable marker of design by intelligent agents. He puts forth a “Law of Conservation of Information” which states that chance and natural laws ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
Intelligent design advocate William Dembski has introduced a measure of information called “complex specified information”, or CSI. He claims that CSI is a reliable marker of design by intelligent agents. He puts forth a “Law of Conservation of Information” which states that chance and natural laws are incapable of generating CSI. In particular, CSI cannot be generated by evolutionary computation. Dembski asserts that CSI is present in intelligent causes and in the flagellum of Escherichia coli, and concludes that neither have natural explanations. In this paper we examine Dembski’s claims, point out significant errors in his reasoning, and conclude that there is no reason to accept his assertions. 1
What is a Random Sequence
 The Mathematical Association of America, Monthly
, 2002
"... there laws of randomness? These old and deep philosophical questions still stir controversy today. Some scholars have suggested that our difficulty in dealing with notions of randomness could be gauged by the comparatively late development of probability theory, which had a ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
there laws of randomness? These old and deep philosophical questions still stir controversy today. Some scholars have suggested that our difficulty in dealing with notions of randomness could be gauged by the comparatively late development of probability theory, which had a
On the Algorithmic Nature of the World
"... We propose a test based on the theory of algorithmic complexity and an experimental evaluation of Levin’s universal distribution to identify evidence in support of or in contravention of the claim that the world is algorithmic in nature. To this end we have undertaken a statistical comparison of the ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
We propose a test based on the theory of algorithmic complexity and an experimental evaluation of Levin’s universal distribution to identify evidence in support of or in contravention of the claim that the world is algorithmic in nature. To this end we have undertaken a statistical comparison of the frequency distributions of data from physical sources on the one hand– repositories of information such as images, data stored in a hard drive, computer programs and DNA sequences–and the frequency distributions computing devices such as Turing machines, cellular automata and Post Tag systems. Statistical correlations were found and their significance measured. 1.2
Modelling modelled
 S.E.E.D. Journal
"... A model is one of the most fundamental concepts: it is a formal and generalized explanation of a phenomenon. Only with models we can bridge the particulars and predict the unknown. Virtually all our intellectual work turns around finding models, evaluating models, using models. Because models are so ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
A model is one of the most fundamental concepts: it is a formal and generalized explanation of a phenomenon. Only with models we can bridge the particulars and predict the unknown. Virtually all our intellectual work turns around finding models, evaluating models, using models. Because models are so pervasive, it makes sense to take a look at modelling itself. We will approach this problem, of course, by
An Application of Information Theory to Intrusion Detection
 Proceedings of the Fourth IEEE International Workshop on Information Assurance (IWIA’06
, 2006
"... Zeroday attacks, new (anomalous) attacks exploiting previously unknown system vulnerabilities, are a serious threat. Defending against them is no easy task, however. Having identified “degree of system knowledge” as one difference between legitimate and illegitimate users, theorists have drawn on i ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Zeroday attacks, new (anomalous) attacks exploiting previously unknown system vulnerabilities, are a serious threat. Defending against them is no easy task, however. Having identified “degree of system knowledge” as one difference between legitimate and illegitimate users, theorists have drawn on information theory as a basis for intrusion detection. In particular, Kolmogorov complexity (K) has been used successfully. In this work, we consider information distance (Observed K − Expected K) as a method of detecting system scans. Observed K is computed directly, Expected K is taken from compression tests shared herein. Results are encouraging. Observed scan traffic has an information distance at least an order of magnitude greater than the threshold value we determined for normal Internet traffic. With 320 KB packet blocks, separation between distributions appears to exceed 4σ. 1.