Results 1  10
of
16
2007b)How Simplicity Helps You Find the Truth Without Pointing at it
 Philosophy of Mathematics and Induction
"... It seems that a fixed bias toward simplicity should help one find the truth, since scientific theorizing is guided by such a bias. But it also seems that a fixed bias toward simplicity cannot indicate or point at the truth, since an indicator has to be sensitive to what it indicates. I argue that bo ..."
Abstract

Cited by 10 (9 self)
 Add to MetaCart
It seems that a fixed bias toward simplicity should help one find the truth, since scientific theorizing is guided by such a bias. But it also seems that a fixed bias toward simplicity cannot indicate or point at the truth, since an indicator has to be sensitive to what it indicates. I argue that both views are correct. It is demonstrated, for a broad range of cases, that the Ockham strategy of favoring the simplest hypothesis, together with the strategy of never dropping the simplest hypothesis until it is no longer simplest, uniquely minimizes reversals of opinion and the times at which the reversals occur prior to convergence to the truth. Thus, simplicity guides one down the straightest path to the truth, even though that path may involve twists and turns along the way. The proof does not appeal to prior probabilities biased toward simplicity. Instead, it is based upon minimization of worstcase cost bounds over complexity classes of possibilities. 0.1 The Simplicity Puzzle There are infinitely many alternative hypotheses consistent with any finite amount of experience, so how is one entitled to choose among them? Scientists boldly respond with appeals to “Ockham’s razor”, which selects the “simplest ” hypothesis among them,
Ockham’s Razor, Empirical Complexity, and Truthfinding Efficiency
 THEORETICAL COMPUTER SCIENCE
, 2007
"... The nature of empirical simplicity and its relationship to scientific truth are longstanding puzzles. In this paper, empirical simplicity is explicated in terms of empirical effects, which are defined in terms of the structure of the inference problem addressed. Problem instances are classified acc ..."
Abstract

Cited by 6 (6 self)
 Add to MetaCart
The nature of empirical simplicity and its relationship to scientific truth are longstanding puzzles. In this paper, empirical simplicity is explicated in terms of empirical effects, which are defined in terms of the structure of the inference problem addressed. Problem instances are classified according to the number of empirical effects they present. Simple answers are satisfied by simple worlds. An efficient solution achieves optimum worstcase cost over each complexity class with respect to such costs such as the number of retractions or errors prior to convergence and elapsed time to convergence. It is shown that always choosing the simplest theory compatible with experience and hanging onto it while it remains simplest is both necessary and sufficient for efficiency.
Mind change efficient learning
 Info. & Comp
, 2005
"... Abstract. This paper studies efficient learning with respect to mind changes. Our starting point is the idea that a learner that is efficient with respect to mind changes minimizes mind changes not only globally in the entire learning problem, but also locally in subproblems after receiving some evi ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
Abstract. This paper studies efficient learning with respect to mind changes. Our starting point is the idea that a learner that is efficient with respect to mind changes minimizes mind changes not only globally in the entire learning problem, but also locally in subproblems after receiving some evidence. Formalizing this idea leads to the notion of uniform mind change optimality. We characterize the structure of language classes that can be identified with at most α mind changes by some learner (not necessarily effective): A language class L is identifiable with α mind changes iff the accumulation order of L is at most α. Accumulation order is a classic concept from pointset topology. To aid the construction of learning algorithms, we show that the characteristic property of uniformly mind change optimal learners is that they output conjectures (languages) with maximal accumulation order. We illustrate the theory by describing mind change optimal learners for various problems such as identifying linear subspaces and onevariable patterns. 1
Ockham’s Razor, Truth, and Information
, 2007
"... In science, one faces the problem of selecting the true theory from a range of alternative theories. The typical response is to select the simplest theory compatible with available evidence, on the authority of “Ockham’s Razor”. But how can a fixed bias toward simplicity help one find possibly compl ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
In science, one faces the problem of selecting the true theory from a range of alternative theories. The typical response is to select the simplest theory compatible with available evidence, on the authority of “Ockham’s Razor”. But how can a fixed bias toward simplicity help one find possibly complex truths? A short survey of standard answers to this question reveals them to be either wishful, circular, or irrelevant. A new explanation is presented, based on minimizing the reversals of opinion prior to convergence to the truth. According to this alternative approach, Ockham’s razor does not inform one which theory is true but is, nonetheless, the uniquely most efficient strategy for arriving at the true theory, where efficiency is a matter of minimizing reversals of opinion prior to finding the true theory. 1
An Abstract
 Machine Simulator, Third International Conference on Computer Assisted Learning
, 1990
"... Efficient transmission methods for fading radio channels often require an iterative decoder. This is for example the case for systems using turbo codes. Receiver decoder iterations could potentially lead to a latency problem which impacts the performance of the medium access control protocol. In thi ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Efficient transmission methods for fading radio channels often require an iterative decoder. This is for example the case for systems using turbo codes. Receiver decoder iterations could potentially lead to a latency problem which impacts the performance of the medium access control protocol. In this paper, we present modifications based on the carrier sense multiple access with collision avoidance (CSMA/CA) medium access control (MAC) protocol to accommodate the increased latency in the iterative processing. One area of applications is wireless local area networks (WLANs) with high data rate. The simulation results performed in the IEEE 802.11a WLAN environment by replacing the 802.11a’s convolutional coding with turbo coding demonstrate that the proposed algorithm provides the throughput gain over the conventional method. I.
Simplicity, Truth, and the Unending Game of Science
, 2005
"... This paper presents a new explanation of how preferring the simplest theory compatible with experience assists one in finding the true answer to a scientific question when the answers are theories or models. Science is portrayed as an infinite game between science and nature. Simplicity is a structu ..."
Abstract
 Add to MetaCart
This paper presents a new explanation of how preferring the simplest theory compatible with experience assists one in finding the true answer to a scientific question when the answers are theories or models. Science is portrayed as an infinite game between science and nature. Simplicity is a structural invariant reflecting sequences of theory choices nature could force the scientist to produce. It is demonstrated that among the methods that converge to the truth in an empirical problem, the ones that do so with a minimum number of reversals of opinion prior to convergence are exactly the ones that prefer simple theories. The idea explains not only simplicity tastes in model selection, but aspects of theory testing and the unwillingness of natural science to break symmetries without a reason. In natural science, one typically faces a situation in which several (or even infinitely many) available theories are compatible with experience. Standard practice is to choose the simplest theory among them and to cite “Ockham’s razor ” as the excuse (figure
Learning, Simplicity, Truth, and Misinformation
"... Both in learning and in natural science, one faces the problem of selecting among a range of theories, all of which are compatible with the available evidence. The traditional response to this problem has been to select the simplest such theory on the basis of “Ockham’s Razor”. But how can a fixed b ..."
Abstract
 Add to MetaCart
Both in learning and in natural science, one faces the problem of selecting among a range of theories, all of which are compatible with the available evidence. The traditional response to this problem has been to select the simplest such theory on the basis of “Ockham’s Razor”. But how can a fixed bias toward simplicity help us find possibly complex truths? I survey the current, textbook answers to this question and find them all to be wishful, circular, or irrelevant. Then I present a new approach based on minimizing the number of reversals of opinion prior to convergence to the truth. According to this alternative approach, Ockham’s razor is a good idea when it seems to be (e.g., in selecting among parametrized models) and is not a good idea when it feels dubious (e.g., in the inference of arbitrary computable functions). Hence, the proposed vindication of Ockham’s razor can be used to separate vindicated applications In science and learning, one must eventually face up to the problem of choosing among several or even infinitely many theories compatible with all available information. How ought one to choose? The traditional answer is to choose the “simplest ” and to invoke
Mind Change Optimal Learning: . . .
, 2007
"... Learning theories play a significant role to machine learning as computability and complexity theories to software engineering. Gold’s language learning paradigm is one cornerstone of modern learning theories. The aim of this thesis is to establish an inductive principle in Gold’s language learning ..."
Abstract
 Add to MetaCart
Learning theories play a significant role to machine learning as computability and complexity theories to software engineering. Gold’s language learning paradigm is one cornerstone of modern learning theories. The aim of this thesis is to establish an inductive principle in Gold’s language learning paradigm to guide the design of machine learning algorithms. We follow the common practice of using the number of mind changes to measure complexity of Gold’s language learning problems, and study efficient learning with respect to mind changes. Our starting point is the idea that a learner that is efficient with respect to mind changes minimizes mind changes not only globally in the entire learning problem, but also locally in subproblems after receiving some evidence. Formalizing this idea leads to the notion of mind change optimality. We characterize mind change complexity of language collections with Cantor’s classic concept of accumulation order. We show that the characteristic property of mind change optimal learners is that they output conjectures (languages) with maximal accumulation order. Therefore, we obtain an inductive principle in Gold’s language learning paradigm based on the simple topological concept accumulation order. The new
Ockham Efficiency Theorem for Stochastic Empirical Methods
, 2010
"... Ockham’s razor is the principle that, all other things being equal, scientists ought to prefer simpler theories. In recent years, philosophers have argued that simpler theories make better predictions, possess theoretical virtues like explanatory power, and have other pragmatic virtues like computa ..."
Abstract
 Add to MetaCart
Ockham’s razor is the principle that, all other things being equal, scientists ought to prefer simpler theories. In recent years, philosophers have argued that simpler theories make better predictions, possess theoretical virtues like explanatory power, and have other pragmatic virtues like computational tractability. However, such arguments fail to explain how and why a preference for simplicity can help one find true theories in scientific inquiry, unless one already assumes that the truth is simple. One new solution to that problem is the Ockham efficiency theorem (Kelly 2002, 2004, 2007ad, Kelly and Glymour 2004), which states that scientists who heed Ockham’s razor retract their opinions less often and sooner than do their nonOckham competitors. The theorem neglects, however, to consider competitors following random (“mixed”) strategies and in many applications random strategies are known to achieve better worstcase loss than deterministic strategies. In this paper, we describe two ways to extend the result to a very general class of random, empirical strategies. The first extension concerns expected retractions, retraction times, and errors and the second extension concerns retractions in chance, times of retractions in chance, and chances of errors.
Noname manuscript No. (will be inserted by the editor) Ockham Efficiency Theorem for Stochastic Empirical Methods
"... the date of receipt and acceptance should be inserted later Abstract Ockham’s razor is the principle that, all other things being equal, scientists ought to prefer simpler theories. In recent years, philosophers have argued that simpler theories make better predictions, possess theoretical virtues l ..."
Abstract
 Add to MetaCart
the date of receipt and acceptance should be inserted later Abstract Ockham’s razor is the principle that, all other things being equal, scientists ought to prefer simpler theories. In recent years, philosophers have argued that simpler theories make better predictions, possess theoretical virtues like explanatory power, and have other pragmatic virtues like computational tractability. However, such arguments fail to explain how and why a preference for simplicity can help one find true theories in scientific inquiry, unless one already assumes that the truth is simple. One new solution to that problem is the Ockham efficiency theorem (Kelly 2002, 2004, 2007ad, Kelly and Glymour 2004), which states that scientists who heed Ockham’s razor retract their opinions less often and sooner than do their nonOckham competitors. The theorem neglects, however, to consider competitors following random (“mixed”) strategies and in many applications random strategies are known to achieve better worstcase loss than deterministic strategies. In this paper, we describe two ways to extend the result to a very general class of random, empirical strategies. The first extension concerns expected retractions, retraction times, and errors and the second extension concerns retractions in chance, times of retractions in chance, and chances of errors.