Results 1 
5 of
5
A Theory of Program Size Formally Identical to Information Theory
, 1975
"... A new definition of programsize complexity is made. H(A;B=C;D) is defined to be the size in bits of the shortest selfdelimiting program for calculating strings A and B if one is given a minimalsize selfdelimiting program for calculating strings C and D. This differs from previous definitions: (1) ..."
Abstract

Cited by 333 (16 self)
 Add to MetaCart
A new definition of programsize complexity is made. H(A;B=C;D) is defined to be the size in bits of the shortest selfdelimiting program for calculating strings A and B if one is given a minimalsize selfdelimiting program for calculating strings C and D. This differs from previous definitions: (1) programs are required to be selfdelimiting, i.e. no program is a prefix of another, and (2) instead of being given C and D directly, one is given a program for calculating them that is minimal in size. Unlike previous definitions, this one has precisely the formal 2 G. J. Chaitin properties of the entropy concept of information theory. For example, H(A;B) = H(A) + H(B=A) + O(1). Also, if a program of length k is assigned measure 2 \Gammak , then H(A) = \Gamma log 2 (the probability that the standard universal computer will calculate A) +O(1). Key Words and Phrases: computational complexity, entropy, information theory, instantaneous code, Kraft inequality, minimal program, probab...
A Paradigm for Class Identification Problems
 IEEE Transactions on Information Theory
, 1994
"... AbsfracThe following problem arises in many applications involving classification, identification, and inference. There is a set of objects X, and a particular x E X is chosen (unknown to us). Based on information obtained about x in a sequential manner, we wish to decide whether x belongs to one c ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
AbsfracThe following problem arises in many applications involving classification, identification, and inference. There is a set of objects X, and a particular x E X is chosen (unknown to us). Based on information obtained about x in a sequential manner, we wish to decide whether x belongs to one class of objects A, or a different class of objects A,. We study a general paradigm applicable to a broad range of problems of this type, which we refer to as problems of class identification or discernibility. We consider various types of information sequences, and various success criteria including discernibility in the limit, discernibility with a stopping criterion, uniform discernibility, and discernibility in the Cesaro sense. We consider decision rules both with and without memory. Necessary and sufficient conditions for discernibility are provided for each case in terms of separability conditions on the sets A, and A,. We then show that for any sets A, and A,, various types of separability can be achieved by allowing failure on appropriate sets of small measure. Applications to problems in language identification, system identification, and discrete geometry are discussed. Index TemClassification, discernibility, identification, learning, Cesaro sense, in the limit, stopping rule, uniform,
Machine Learning (Lecture notes 10)
, 1994
"... that is shattered by C. After m examples A knows nothing about at least half the points in Y , and therefore it cannot predict the labels on those points better than a random coin toss. In fact, for finite VCdimension equation (10.1) is nearly tight, and we will show that for VCdimension d, achie ..."
Abstract
 Add to MetaCart
that is shattered by C. After m examples A knows nothing about at least half the points in Y , and therefore it cannot predict the labels on those points better than a random coin toss. In fact, for finite VCdimension equation (10.1) is nearly tight, and we will show that for VCdimension d, achieving error rate ffl requires c 1 d=ffl samples for some fixed constant c 1 . 101 102 Lecture 10: October 17, 1994 Let X = fx 1 ; : : : ; x d g and let C = 2 X . Obviously, VCdim(C) = d. We will show that PAClearning C requires\Omega\Gamma d=ffl) examples. Theorem 1 Let X = fx 1<F24
In Learning in the Limit and NonUniform (eps, delta)Learning
 in Proceedings of the Sixth ACM Workshop on Computational Learning Theory
, 2001
"... We compare the two most common types of success criteria for learning processes. The (ffl; ffi ) criterion employed in the PAC model and its many variants and extensions, and the identification in the limit criterion that is used in inductive inference models. By applying common techniques from t ..."
Abstract
 Add to MetaCart
We compare the two most common types of success criteria for learning processes. The (ffl; ffi ) criterion employed in the PAC model and its many variants and extensions, and the identification in the limit criterion that is used in inductive inference models. By applying common techniques from the theory of Probability, and a stochastic variant of Rissanen's MDL principle, we demonstrate close connections between these two types of learnability notions. We also show that, once computability issues are set aside, stochastic identification in the limit is intimately related to the countability of a concept class. 1