## A Survey of Inductive Inference with an Emphasis on Queries (1997)

Venue: | Complexity, Logic, and Recursion Theory, number 187 in Lecture notes in Pure and Applied Mathematics Series |

Citations: | 4 - 0 self |

### BibTeX

@INPROCEEDINGS{Gasarch97asurvey,

author = {William Gasarch and Carl H. Smith},

title = {A Survey of Inductive Inference with an Emphasis on Queries},

booktitle = {Complexity, Logic, and Recursion Theory, number 187 in Lecture notes in Pure and Applied Mathematics Series},

year = {1997},

publisher = {M. Dekker}

}

### OpenURL

### Abstract

this paper M 0 ; M 1 ; : : : is a standard list of all Turing machines, M

### Citations

893 |
Language identification in the limit
- Gold
- 1967
(Show Context)
Citation Context ... has not entered the set 8 yet, you do not know if it ever will; by contrast, when learning (say) a 0-1 valued recursive function f , you eventually know f(17). Language learning was first defined in =-=[Gol67]-=-, but was later developed more in [BB75, CL82, OW82]. Language learning with a team is discussed in [JS90, JS93c, JS95a]. Restrictions on strategies for language learning are discussed in [Ful90a, JS9... |

306 |
Inductive inference: theory and methods
- Angluin, Smith
- 1983
(Show Context)
Citation Context ...ches are are sketchier than others. There have been other surveys of some of this material. A survey of inductive inference that included this material and models that were not recursion theoretic is =-=[AS83]-=-. A survey concentrating on team inference is [Smi94b]. A survey that looks at the features of learning that are most similar to human learning is [GS95]. 2 Standard Notation We use standard notation ... |

259 | Toward a mathematical theory of inductive inference - Blum, Blum - 1975 |

222 | Computational complexity of probabilistic Turing machines - Gill - 1977 |

163 |
Comparison of identification criteria for machine inductive inference, Theoretical Computer Science
- Case, Smith
- 1983
(Show Context)
Citation Context ...variations on inference that have received considerable attention in the literature. Definition 3.6 Let M be an IIM, f be a recursive function, c; d; n 2 N with c; ds1, and a 2 N [ fg. 1. Mindchanges =-=[CS83]-=-. If M infers f and only at most n times outputs a conjecture that is different from the most recent previous conjecture, then M infers the function f withsn mindchanges. S 2 EX n if there is an IIM M... |

140 |
On a decision method in restricted secondorder arithmetic
- Büchi
- 1962
(Show Context)
Citation Context ...y we will consider inferring sets instead of functions. This will not affect any of the results because if (say) RECSET = 2 QEX[Succ;!] 2 then REC = 2 QEX[Succ;!] 2 . Definition 6.3 A Buchi Automaton =-=[Buc62]-=- is a nondeterministic finite automatonsA = hQ; \Sigma; \Delta; s; F i where Q is the set of states, \Sigma is the alphabet, \Delta maps Q \Theta \Sigma to 2 Q , s 2 Q (s is the start state), and F ` ... |

92 | Machine inductive inference and language identification - Case, Lynes - 1982 |

59 | Prudence and other conditions on formal language learning - Fulk - 1990 |

52 |
The decision problem for exponential diophantine equations
- Davis, Putnam, et al.
- 1961
(Show Context)
Citation Context ...s the QIM is allowed to make. Theorem 6.1 ([GS92]) REC 2 Q 1 EX[+; \Theta]. REC 2 Q 2 EX 0 [+; \Theta]. Proof sketch: Both proofs use the following theorem (proved in [Mat70], but builds on work from =-=[DPR61]-=-): If A is any r.e. set, then there exists a polynomial p(x; x 1 ; : : : ; x k ) (with integer coefficients) such that A = fx : (9x 1 ; : : : ; x k )[p(x; x 1 ; : : : ; x k ) = 0]g: This theorem was a... |

49 |
Periodicity in generations of automata
- Case
- 1974
(Show Context)
Citation Context ... know the function is in S 4 . From the information gathered so far, and a few more questions, we can determine a program for f . To show that S = 2 [1; 3]BC a , we use the Operator Recursion Theorem =-=[Cas74]-=- and diagonalization. For an easier use of the Operator Recursion Theorem in inductive inference, see [CS83]. 28 Open Problem 6.17 We conjecture that for Ls[Succ; !] 2 the following hold. 1. QEX 1 [L]... |

43 | Inductive Inference and Unsolvability - Adleman, Blum - 1991 |

35 |
Theories of automata on !-tapes: a simplified approach
- Choueka
- 1974
(Show Context)
Citation Context ...uantifiers are handled easily, since Buchi automata are nondeterministic. The only real hard step is showing that !-regular sets are closed under complementation. This was first shown in [Buc62]. See =-=[Cho74]-=- for a good exposition and see [Saf88] for a more efficient proof. Lemma 6.5 If q(X) is a query in [Succ; !] 2 then the set fA : q(A)g is !- regular. Furthermore, there is a recursive procedure to tra... |

35 | Learning via queries
- Gasarch, Smith
- 1992
(Show Context)
Citation Context ...[Succ; !] 2 can be interpreted as Buchi automata (to be defined later), hence we have the mathematical tools to prove sophisticated theorems, and (2) other languages can be `reduced' to this one (see =-=[GH95]-=- or Definition 6.11), hence results about this language imply results about other languages. For this study we will consider inferring sets instead of functions. This will not affect any of the result... |

34 |
On the role of procrastination in machine learning
- Freivalds, Smith
- 1993
(Show Context)
Citation Context ...s to an auxiliary oracle [FJG + 94, GP89, JS93b, KS96, Ste95]? 3. What if your inductive inference machine has limited memory [FKS95]? 4. What if the number of mindchanges is allowed to be an ordinal =-=[FS93]-=-? 5. What if you want to infer functions with domain N but range the rationals [CM95] or the reals [AFS95]? Variations on the output 1. What if the number of errors allowed is an ordinal [FS93]? 2. Wh... |

30 |
On the complexity of inductive inference
- Daley, Smith
- 1986
(Show Context)
Citation Context ... 2 [FKS96, JS94a, JS95b]? 4. How do you choose inductive inference machines to form a team [AFS]? 5. Is there a way to measure the complexity of inference (similar to Blum Complexity for computation) =-=[DS86]-=-? 4 Results in Inductive Inference We first look at how much power is needed before REC can be inferred. Theorem 4.1 Let a 2 N and ms1. 1. REC = 2 BC [Bar74, CS83]. 2. REC = 2 BC a [CS83]. 3. REC = 2 ... |

29 | Terse, superterse, and verbose sets
- Beigel, Gasarch, et al.
- 1993
(Show Context)
Citation Context ...how that S 2 [2; 4]EX 0 and hence it is in [1; 2]EX 0 [A]. From this one can show that K T A. The proof is unusual in that it is not a direct algorithm. It uses the following notions. Definition 7.18 =-=[BGGO93]-=- Let X and Y be sets and a; bs1. C X a is the function C X a (x 1 ; : : : ; x a ) = X(x 1 ) \Delta \Delta \Delta X(x a ) (2 f0; 1g a ). A function f is benumerable if there is a Turing machine M that ... |

26 | Two theorems on the limiting synthesis of functions - Barzdins - 1974 |

25 | On the intrinsic complexity of learning - Freivalds, Kinber, et al. - 1994 |

24 | Weak second order arithmetic and finite automata. Zeitscrift fur mathematische Logic und Grundlagen der Mathematik - Büchi - 1960 |

23 |
Robust separations in inductive inference
- Fulk
- 1990
(Show Context)
Citation Context ...the only two types of inference used. Barzdins [Bar80] conjectured that these might be the only types of inference there are (formalizing this conjecture is non-trivial). The conjecture was disproved =-=[Ful90b]-=-. An exact characterization of EX (and BC which will be described in Definition 3.6) is in [Wie78b, Wie78a, FKW95]. Definition 3.5 Let a 2 N. If f and g are functions, then f = a g means that f and g ... |

20 | Tradeoffs in inductive inference of nearly minimal sized programs - Chen - 1982 |

19 | Learning via queries to an oracle - Gasarch, Pleszkoch - 1989 |

18 |
Relations between probabilistic and team one-shot learners
- Daley, Pitt, et al.
- 1991
(Show Context)
Citation Context ...nd theorem we state is about the case when the probability iss1=2. In this case far less is known. Theorem 4.7 Let k; n; r; ss1 (rss) and 1 2 ! ps1. 1. If 0 ! r+1 s+2 ! ps1 then EX 0 hpi ` [r; s]EX 0 =-=[DPVW91]-=-. 2. If n+1 2n+1 ! psn 2n\Gamma1 then (1) EX 0 hpi = EX 0 h n 2n\Gamma1 i [Fre79] and (2) [n; 2n \Gamma 1]EX 0 ae [n + 1; 2n + 1]EX 0 [DPVW91]. (Hence the EX 0 hpi inference types change at the breakp... |

16 | Manguelle. Refinements of inductive inference by Popperian and reliable machines - Case, Jain, et al. - 1994 |

16 |
Breaking the probability 1 2 barrier in FIN-type learning
- Daley, Kalyanasundaram, et al.
- 1995
(Show Context)
Citation Context ...that these bounds are not necessarily best possible. 4. Let p; c; d be such that 24 49 ! p = c d ! 1 2 . Then (1) EXhpi = [2; 4]EX 0 , (2) [c; d]EX 0 = [2; 4]EX 0 , and (3) [2; 4]EX 0 ae [24; 49]EX 0 =-=[DKV95]-=-. Proof sketch: One feels a moral obligation to comment on the numbers 24 and 49. Efim Kinber commented (jokingly) that they are `nonrecursive numbers' meaning only that they seem unnatural. The autho... |

10 | On the impact of forgetting on learning machines
- Freivalds, Kinber, et al.
- 1993
(Show Context)
Citation Context ...GS92, Ste95]? 2. What if your inductive inference machine could ask queries to an auxiliary oracle [FJG + 94, GP89, JS93b, KS96, Ste95]? 3. What if your inductive inference machine has limited memory =-=[FKS95]-=-? 4. What if the number of mindchanges is allowed to be an ordinal [FS93]? 5. What if you want to infer functions with domain N but range the rationals [CM95] or the reals [AFS95]? Variations on the o... |

10 | Asking questions versus verifiability - Gasarch, Velauthapillai - 1992 |

9 | Learning in the presence of inaccurate information - Fulk, Jain - 1989 |

8 | Learning with Higher Order Additional Information - Baliga, Case - 1994 |

8 |
Tradeoffs in Machine Inductive Inference
- Chen
- 1981
(Show Context)
Citation Context ...ing worse and worse. (The programs must get worse and worse. That is, it is known that if REC 2 BC via M then there will be recursive functions f 11 for which the number of errors grows without bound =-=[Che81]-=-.) Here is the inference scheme. Given oe = f(0)f(1) \Delta \Delta \Delta f(s), output a program that does the following. On input x, run each of ' 0 ; : : : ; ' s on f0; : : : ; sg for x steps. Let e... |

8 | How Inductive Inference Strategies Discover Their Errors - Freivalds, Kinber, et al. - 1995 |

7 |
Co-learning of total recursive functions
- Freivalds, Karpinski, et al.
- 1994
(Show Context)
Citation Context ...o be given some additional information [BC94, JS93a]? 2. What if the information presented to the learner is not quite accurate [FJ89, Jai94]? 3. Can you learn something by the process of elimination =-=[FKS94]-=-? Misc. 1. If I and J are two inference types, and I 6` J , then is there a way to measure the strength of that non-inclusion [Kum94]? 2. Given a class of functions S, let's look at the class of all i... |

6 | Program synthesis in the presence of infinite number of inaccuracies - Jain - 1994 |

5 | Classification using information - Gasarch, Pleszkoch, et al. - 1994 |

5 | Recursion theoretic models of learning: some results and intuitions
- Gasarch, Smith
- 1995
(Show Context)
Citation Context ...nd models that were not recursion theoretic is [AS83]. A survey concentrating on team inference is [Smi94b]. A survey that looks at the features of learning that are most similar to human learning is =-=[GS95]-=-. 2 Standard Notation We use standard notation from recursion theory. For an elementary introduction to recursion theory see [Smi94a]. For a more advanced treatment see [Soa87]. Notation 2.1 The natur... |

3 | Kolmogorov numberings and minimal identification, in - Freivalds, Jain |

2 |
On the inductive inference of real valued functions
- itis, Freivalds, et al.
- 1995
(Show Context)
Citation Context ...e has limited memory [FKS95]? 4. What if the number of mindchanges is allowed to be an ordinal [FS93]? 5. What if you want to infer functions with domain N but range the rationals [CM95] or the reals =-=[AFS95]-=-? Variations on the output 1. What if the number of errors allowed is an ordinal [FS93]? 2. What if you allow the number of errors to be infinite [GSSV92, Roy86, SV90]? 3. What if you want to learn a ... |

2 |
On the duality between mechanistic learners and what it is they learn
- Freivalds, Smith
- 1997
(Show Context)
Citation Context ...pes, and I 6` J , then is there a way to measure the strength of that non-inclusion [Kum94]? 2. Given a class of functions S, let's look at the class of all inductive inference machines that learn it =-=[FS97]-=-. 3. Is there a way to say that S 1 is harder to infer than S 2 [FKS96, JS94a, JS95b]? 4. How do you choose inductive inference machines to form a team [AFS]? 5. Is there a way to measure the complexi... |

1 |
On duality in learning and the selection of learning team. Information and Computation. To appear. Earlier version appeared in STOC94 under the title Choosing a learning team: a topological approach
- Apsitis, Freivalds, et al.
(Show Context)
Citation Context ...nductive inference machines that learn it [FS97]. 3. Is there a way to say that S 1 is harder to infer than S 2 [FKS96, JS94a, JS95b]? 4. How do you choose inductive inference machines to form a team =-=[AFS]-=-? 5. Is there a way to measure the complexity of inference (similar to Blum Complexity for computation) [DS86]? 4 Results in Inductive Inference We first look at how much power is needed before REC ca... |

1 |
Probabilistic PFIN-type learning: General properties
- Ambainis
- 1996
(Show Context)
Citation Context ...oints for EX 0 hpi less than 24 49 are known, but each one requires a new proof and no pattern is known. If only total functions can be output then the following stunning result is known. Theorem 4.9 =-=[Amb96]-=- 1. The set of breakpoints for PEX 0 has order type ffl 0 . (ffl 0 is the ordinal that is the limit of the sequence x 0 = !, x i+1 = ! x i .) 2. The problem of, given p; q 2 [0; 1], determining if PEX... |

1 |
Inductive inference of functions on the rationals
- Cenzer, Moser
- 1995
(Show Context)
Citation Context ...ive inference machine has limited memory [FKS95]? 4. What if the number of mindchanges is allowed to be an ordinal [FS93]? 5. What if you want to infer functions with domain N but range the rationals =-=[CM95]-=- or the reals [AFS95]? Variations on the output 1. What if the number of errors allowed is an ordinal [FS93]? 2. What if you allow the number of errors to be infinite [GSSV92, Roy86, SV90]? 3. What if... |

1 |
Mahendra Velauthapillai. Trade-off among parameters affecting inductive inference
- Freivalds, Smith
- 1989
(Show Context)
Citation Context ...t what the program does ). 7 8. Combinations. The parameters above can be combined to yield inference types like [c; d]EX a n . Comparisons between these inference types have been studied extensively =-=[FSV89]-=-. We will deal with combinations like PEX n , PEX a n , EX 0 , and [1; d]BC a . For [1; d]BC a we will not allowsa = , since the class of all recursive functions is in BC [CS83]. (We prove this in The... |

1 | Mahendra Velauthapillai. Learning programs with an easy to calculate set of errors. Fundamentica Informaticae - Gasarch, Sitaraman, et al. - 1992 |