## Inductive Inference, DFAs and Computational Complexity (1989)

Venue: | 2nd Int. Workshop on Analogical and Inductive Inference (AII |

Citations: | 77 - 1 self |

### BibTeX

@INPROCEEDINGS{Pitt89inductiveinference,,

author = {Leonard Pitt},

title = {Inductive Inference, DFAs and Computational Complexity},

booktitle = {2nd Int. Workshop on Analogical and Inductive Inference (AII},

year = {1989},

pages = {18--44}

}

### Years of Citing Articles

### OpenURL

### Abstract

This paper surveys recent results concerning the inference of deterministic finite automata (DFAs). The results discussed determine the extent to which DFAs can be feasibly inferred, and highlight a number of interesting approaches in computational learning theory. 1

### Citations

10959 |
Computers and Intractability: A Guide to the Theory of NP-Completeness
- Garey, Johnson
- 1979
(Show Context)
Citation Context ...ty theoretic assumption that RP ~ NP is widely used; if RP = NP, there would be "efficient" (randomized) algorithms for solving problems for which there is significant evidence of intractability. See =-=[23, 24, 59]-=- for further justification and background. An alphabet E is a finite set of symbols. The set E* consists of all finite length sequences formed by concatenating zero or more elements of ]~. Elements of... |

3839 |
Introduction to Automata Theory, Languages, and Computation
- Hopcroft, Ullman
- 1979
(Show Context)
Citation Context ...metric difference of L1 and L2, defined by LltL2 = (L1 - L~) U (L~ - L1). 2.1 Deterministic Finite Automata All of the background material regarding DFAs that is assumed in this paper may be found in =-=[32]-=-. Here we briery review the formal definitions and notation that will be used. A deterministic finite automaton (DFA), also called a finite state machine, is a 5-tuple M = (Q, ~, 6, q0, F), where Q is... |

1697 | A theory of the learnable
- Valiant
- 1984
(Show Context)
Citation Context ...4, 7] are summarized, showing that there is no polynomial time algorithm for exactly identifying the class of DFAs from examples alone. In Section 4 the "distribution-free" inference model of Valiant =-=[58]-=-, and generalizations of this model are considered. The results of Pitt and *Supported in part by NSF grant IRI-8809570, and by the Department of Computer Science, University of l]linois at Urbana-Cha... |

893 |
Language identification in the limit
- Gold
- 1967
(Show Context)
Citation Context ...inferred, and highlight a number of interesting approaches in computational learning theory. 1 Introduction Beginning with Gold's seminal paper on the identification of formal languages from examples =-=[26]-=-, there has been extensive research into the problem of inferring DFAs (deterministic finite automata) from examples supplied to an inference algorithm. The problem of identifying an unknown DFA from ... |

674 | Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm
- Littlestone
- 1988
(Show Context)
Citation Context ...iving the i-th ezample and outputting the i-th conjectured DFA is at most q(n, ml +... + mi), where mj is the length of the j-th ezampIe. A model of prediction in the limit has also been investigated =-=[12, 13, 14, 16, 37, 44, 45]-=-. This model (as applied to DFAs) dictates that the inference algorithm receives an unlabeled string w, and must predict whether or not w is in the language of the target DFA M. After the prediction, ... |

625 |
Learnability and the VapnikChervonenkis dimension
- Blumer, Ehrenfeucht, et al.
- 1989
(Show Context)
Citation Context ... a distribution-independent model of randomized inference that has been called PAC-identification. Subsequently, the model has been refined and extended in a number of ways; the reader should consult =-=[17, 29, 43]-=- for interesting variations. Here we consider the version(s) most applicable to the class of DFAs. "PAC" abbreviates "probably approximately correct". The goal of a PAC-identification algorithm is to ... |

510 |
Learning regular sets from queries and counterexamples
- Angluin
- 1987
(Show Context)
Citation Context ...h, or have the ability to obtain, additional information about the DFA to be inferred, as well as the problem of inferring certain restricted classes of DFAs. We describe a number of results (Angluin =-=[6, 9]-=-, R~vest and Schapire [50, 51, 52], rbarra and Jiang [33], Helmbotd, Sloan, and Warmuth [30, 31], and Abe [2]) showing that the inference problem is efficiently solvable in these more favorable settin... |

375 | Learning decision lists - Rivest - 1987 |

305 | Cryptographic limitations on learning Boolean formulae and finite automata - Kearns, Valiant - 1994 |

289 |
Digitalized Signatures and Public Key Functions as Intractable as Factoring
- Rabin
- 1979
(Show Context)
Citation Context ... a well known public key cryptosystem that has received much analysis. There is currently no known polynomial-time algorithm for inverting the I~SA function. Similarly, the encryption scheme of Rabin =-=[47]-=- is thought to be hard to invert. In Rabin's encryption scheme, the problem of decryption can be shown to be equivalent (with respect to polynomial-time probabilistic computation) to the problem of fa... |

259 |
Toward a mathematical theory of inductive inference
- Blum, Blum
- 1975
(Show Context)
Citation Context ...iving the i-th ezample and outputting the i-th conjectured DFA is at most q(n, ml +... + mi), where mj is the length of the j-th ezampIe. A model of prediction in the limit has also been investigated =-=[12, 13, 14, 16, 37, 44, 45]-=-. This model (as applied to DFAs) dictates that the inference algorithm receives an unlabeled string w, and must predict whether or not w is in the language of the target DFA M. After the prediction, ... |

221 |
Computational complexity of probabilistic Turing machines
- Gill
- 1977
(Show Context)
Citation Context ...olynomial time by nondeterministic Turing machines, and RP is the class of languages accepted by probabilistic polynomial time 'Ihring machines with one-sided error bounded away from 0, as defined in =-=[24]-=-. It is known that P C_ RP C_ NP, and it is not known if any of the inclusions are proper. Most of the inference problems considered in this paper would be triviM if RP = NP. The complexity theoretic ... |

220 |
Complexity of Automaton Identification from Given Data
- Gold
- 1978
(Show Context)
Citation Context ...point, the smallest DFA consistent with the set of examples seen up to that point. This general approach has been taken in a number of practical and theoretical settings [2, 12~ 36, 48, 49, 56]. Gold =-=[25]-=- showed that an algorithm for the minimum consistent DFA problem could be used to achieve identification in the limit~ and that such a technique would be optimally data efficient. However, Gold also s... |

192 |
A general lower bound on the number of examples needed for learning
- Ehrenfeucht, Haussler, et al.
- 1988
(Show Context)
Citation Context ...tion needed to infer DFAs according to the models presented? Such bounds might be obtained by computing the VC-dimension [17] of the classes considered, and applying techniques used for other domains =-=[22]-=-. 4. Are (not necessarily zero-reversible) commutative DFAs (or NFAs) PAC-identifiable? 5. Is the class of Boolean formulas polynomiaUy approximately predictable if membership queries are also allowed... |

192 |
An Introduction to the Theory of Numbers
- Niven, Montgomery
- 1991
(Show Context)
Citation Context ...-time solution is known: Let Z~ be the multiplicative group modulo N (Z,~ contains only elements of Ziv that are relatively prime to N) and let J(z, N) denote the Jacobi symbol of z with respect to N =-=[39]-=-. Then the quadratic residue problem is the following. Given a number N that is the product of two (unknown) primes p and q, each of length l, and given a number z E Z~ such that J(~, N) = 1, determin... |

191 | Computational limitations on learning from examples
- Pitt, Valiant
- 1988
(Show Context)
Citation Context ... constant.) is not PAC-identifiable unless RP = NP, but there is a PAC-algorithm for this class in terms of the class k-CNF of conjunctive normal form expressions where each clause has size at most k =-=[41]-=-. Finally, there is a notion of polynomial time predictability that corresponds to the criterion of PAC-identification. We say that DFAs are (polynomially) approzimately predictable iff there exists a... |

168 | Inference of finite automata using homing sequences
- Rivest, Schapire
- 1993
(Show Context)
Citation Context ...tain, additional information about the DFA to be inferred, as well as the problem of inferring certain restricted classes of DFAs. We describe a number of results (Angluin [6, 9], R~vest and Schapire =-=[50, 51, 52]-=-, rbarra and Jiang [33], Helmbotd, Sloan, and Warmuth [30, 31], and Abe [2]) showing that the inference problem is efficiently solvable in these more favorable settings. Finally~ Section 6 provides a ... |

163 |
Comparison of identification criteria for machine inductive inference, Theoretical Computer Science
- Case, Smith
- 1983
(Show Context)
Citation Context ...ntifiable in the limit Lff there exists a single inference algorithm A such that for all DFAs M, A identifies M in the limit. This criterion of successful inference is also known as EX-identification =-=[20]-=-. We also consider a presentation of only the positive examples of the DFA M. A positive presentation of M is the same as a presentation of M, except that the sequence contains all and only the string... |

162 | Inference of Reversible Languages - Angluin - 1982 |

148 |
Gedanken-experiments on sequential machines
- Moore
- 1956
(Show Context)
Citation Context ...en Theorem 2 applies. In the absence of any additional information, it is known that DFAs cannot be identified by an algorithm in finite time, i.e., by one that halts after making a single conjecture =-=[38]-=-. However, if an upper bound n on the number of states of M is provided, then a canonical DFA may be found by querying sufficiently many strings so that at most one canonical acceptor of size less tha... |

135 |
RSA and Rabin functions: Certain parts are as hard as the whole
- Alexi, Chor, et al.
- 1988
(Show Context)
Citation Context ...cult for z chosen randomly from Z~r according to a uniform distribution. It has been shown that finding the least significant bit of z given RSA(N, e, z) is as hard as determining the entire string z =-=[3]-=-. This suggests the following inference problem R based on the P~SA iunction. The class of representations R consists of triples (p, q, e) with the required special properties. A positive example of (... |

97 |
Negative results for equivalence queries
- Angluin
- 1990
(Show Context)
Citation Context ...the limit due to Gold [26], and discusses the problem of augmenting the definition so as to incorporate a notion of computational efficiency. Various definitions are presented, and results of Angluin =-=[4, 7]-=- are summarized, showing that there is no polynomial time algorithm for exactly identifying the class of DFAs from examples alone. In Section 4 the "distribution-free" inference model of Valiant [58],... |

92 | On the complexity of minimum inference of regular sets - Angluin - 1978 |

90 |
Equivalence of models for polynomial learnability
- HAUSSLER, KEARNS, et al.
- 1988
(Show Context)
Citation Context ... a distribution-independent model of randomized inference that has been called PAC-identification. Subsequently, the model has been refined and extended in a number of ways; the reader should consult =-=[17, 29, 43]-=- for interesting variations. Here we consider the version(s) most applicable to the class of DFAs. "PAC" abbreviates "probably approximately correct". The goal of a PAC-identification algorithm is to ... |

82 | The minimum consistent DFA problem cannot be approximated within any polynomial
- Pitt, Warmuth
- 1993
(Show Context)
Citation Context ...les alone remains intractable. The related optimization problem of finding an approximately small DFA that is consistent with given data is also considered, and strongly negative results are reviewed =-=[34, 42]-=-. Section 5 considers inference algorithms that are provided with, or have the ability to obtain, additional information about the DFA to be inferred, as well as the problem of inferring certain restr... |

77 |
Learning conjunctive concepts in structural domains
- Haussler
- 1989
(Show Context)
Citation Context ...y negative results for the PAC-identification criterion must make the standard complexity-theoretic assumption that RP ~ NP. There are negative results based only on this assumption (for example, see =-=[28, 41]-=-). These negative results are typically obtained by showing that the consistency problem for the class of representations is NP-hard [17, 28, 41]. (The consistency problem is that of producing any ele... |

74 | Diversity-based Inference of Finite Automata
- Rivest, Schapire
- 1994
(Show Context)
Citation Context ...tain, additional information about the DFA to be inferred, as well as the problem of inferring certain restricted classes of DFAs. We describe a number of results (Angluin [6, 9], R~vest and Schapire =-=[50, 51, 52]-=-, rbarra and Jiang [33], Helmbotd, Sloan, and Warmuth [30, 31], and Abe [2]) showing that the inference problem is efficiently solvable in these more favorable settings. Finally~ Section 6 provides a ... |

58 | Inductive reasoning and Kolmogorov complexity - Li, Vitányi - 1992 |

48 |
A survey of inductive inference: Theory and methods, Computing Surveys
- Angluin, Smith
- 1983
(Show Context)
Citation Context ... of the results discussed here with heuristic approaches that have been presented in earlier work in more practical settings. Many of these approaches are discussed in the survey by Angluin and Smith =-=[12]-=-. Hopefully, we have convinced the reader that the general area of DFA inference is rich with interesting problems and solutions; we close with a list of open problems for fttrther consideration. 1. C... |

36 |
Prediction preserving reducibility
- Pitt, Warmuth
- 1990
(Show Context)
Citation Context ...s of this model are considered. The results of Pitt and *Supported in part by NSF grant IRI-8809570, and by the Department of Computer Science, University of l]linois at Urbana-Charnpaign.19 Warmuth =-=[43]-=- and Kearns and Valiant f34] are reviewed, which show, based on various complexitytheoretic assumptions, that despite these less demanding inference criteria, the problem of inferring DFAs from exampl... |

36 |
Learning automata from ordered examples
- Porat, Feldman
- 1991
(Show Context)
Citation Context ...identifying the class of DFAs accepting finite languages is no easier than identifying the full class of DFAs. These general reductions play an important role in proving Theorem 29. Porat and Feldman =-=[46]-=- also consider the problem of identifying DFAs from ordered examples. They present an algorithm that identifies the class of DFAs in the limit from a lexicographic presentation. While their algorithm ... |

30 |
On the complexity of inductive inference
- Daley, Smith
- 1986
(Show Context)
Citation Context ...e small. Bounding the number of "mind changes" before the inference algorithm has converged to a correct hypothesis has also been considered as a measure of complexity of inference (see, for example, =-=[12, 20, 21]-=-). We might require that the number of changes of hypothesis be at most p(n) for some polynomial p, in addition to requiring polynomial update time. This criterion of success is satisfied by the foUow... |

30 | A formal theory of inductive inference. part i - Solomonoff - 1964 |

26 |
System identification via state characterization
- Gold
- 1972
(Show Context)
Citation Context ...hich is slightly better than the bound stated here) is shown to be optimal to within a constant factor [9]. We sketch the ideas in the proof of Theorem 23, which was motivated by the approach of Gold =-=[27]-=-. We first briefly review the state minimization algorithm for DFAs [32]. Given any DFA M = (Q, Z, ~, q0, F), a pair of states qz, q2 of Q is said to be distinguishable if there exists a string w such... |

16 |
Learning One-Counter Languages in Polynomial Time
- Berman, Roos
- 1987
(Show Context)
Citation Context ...e polynomially approximately predictable if membership queries were available in addition to randomly generated examples. Extensions to Theorem 24 have been obtained in other domains. Berman and Roos =-=[15]-=- show that deterministic one-counter languages can be identified in polynomial time using equivalence and membership queries. Sakakibara shows that context-free grammars may be identified in polynomia... |

9 |
Learning nested differences of intersection closed concept classes
- Helmbold, Sloan, et al.
- 1989
(Show Context)
Citation Context ... as the problem of inferring certain restricted classes of DFAs. We describe a number of results (Angluin [6, 9], R~vest and Schapire [50, 51, 52], rbarra and Jiang [33], Helmbotd, Sloan, and Warmuth =-=[30, 31]-=-, and Abe [2]) showing that the inference problem is efficiently solvable in these more favorable settings. Finally~ Section 6 provides a brief sumraary, and suggests some problems for further researc... |

8 |
Prognostication of automata and functions
- Barzdin
(Show Context)
Citation Context ...iving the i-th ezample and outputting the i-th conjectured DFA is at most q(n, ml +... + mi), where mj is the length of the j-th ezampIe. A model of prediction in the limit has also been investigated =-=[12, 13, 14, 16, 37, 44, 45]-=-. This model (as applied to DFAs) dictates that the inference algorithm receives an unlabeled string w, and must predict whether or not w is in the language of the target DFA M. After the prediction, ... |

7 |
On the learnabillty of finite automata
- Li, Vazlraul
- 1988
(Show Context)
Citation Context ...ximation guarantee. More succinctly, for DFAs, PAC-identification is equivalent to data compression. Consequently, this approach to inferring DFAs has recently received more scrutiny. Li and Vazirani =-=[35]-=- provided the first nonapproximabillty result extending the NP-completeness result of [25], by showing that it is NP-hard to produce a consistent DFA that is at most ] times larger than the smallest c... |

6 |
Equivalence queries and approximate fingerprints
- Angluin
- 1989
(Show Context)
Citation Context ...the limit due to Gold [26], and discusses the problem of augmenting the definition so as to incorporate a notion of computational efficiency. Various definitions are presented, and results of Angluin =-=[4, 7]-=- are summarized, showing that there is no polynomial time algorithm for exactly identifying the class of DFAs from examples alone. In Section 4 the "distribution-free" inference model of Valiant [58],... |

5 |
Polynomial Learnability of Semilinear Sets
- Abe
- 1989
(Show Context)
Citation Context ... inferring certain restricted classes of DFAs. We describe a number of results (Angluin [6, 9], R~vest and Schapire [50, 51, 52], rbarra and Jiang [33], Helmbotd, Sloan, and Warmuth [30, 31], and Abe =-=[2]-=-) showing that the inference problem is efficiently solvable in these more favorable settings. Finally~ Section 6 provides a brief sumraary, and suggests some problems for further research. 2 Definiti... |

5 |
Comparing various concepts of function prediction, Part I
- Podnieks
- 1974
(Show Context)
Citation Context |

2 |
A note on diversity
- Angluin
- 1987
(Show Context)
Citation Context ... measure of size are presented, and theorems analogous to Theorem 27 and Theorem 28 are proven. These results subsume the results in [52] on the inference of "visible simple assignment automata". See =-=[8, 50]-=- for further discussion of the diversity measure.39 5.3 Equivalence Queries with Ordered Counterexamples Ibarra and Jiang [33] present an interesting theorem that contrasts nicely with the negative r... |

2 |
On the prediction of general recursive functions. Soviet Mathematics Doklady
- Barzdin, Freivald
- 1972
(Show Context)
Citation Context |

2 |
Learning Submodules and Reversible Commutative Regular Languages
- Helmbold, Sloan, et al.
- 1989
(Show Context)
Citation Context ... as the problem of inferring certain restricted classes of DFAs. We describe a number of results (Angluin [6, 9], R~vest and Schapire [50, 51, 52], rbarra and Jiang [33], Helmbotd, Sloan, and Warmuth =-=[30, 31]-=-, and Abe [2]) showing that the inference problem is efficiently solvable in these more favorable settings. Finally~ Section 6 provides a brief sumraary, and suggests some problems for further researc... |

1 | Polynomial learnability as a formal model of natural language acquisition - Abe - 1989 |

1 |
A note on the number of queries needed to identify regular languages
- Angluln
- 1981
(Show Context)
Citation Context ...h, or have the ability to obtain, additional information about the DFA to be inferred, as well as the problem of inferring certain restricted classes of DFAs. We describe a number of results (Angluin =-=[6, 9]-=-, R~vest and Schapire [50, 51, 52], rbarra and Jiang [33], Helmbotd, Sloan, and Warmuth [30, 31], and Abe [2]) showing that the inference problem is efficiently solvable in these more favorable settin... |

1 |
Queries and concept learning
- Anglnin
- 1987
(Show Context)
Citation Context ...e where the inference algorithm is required to output a correct DFA (in the limit). The definition of polynomial time identification in the limit is also closely related to the definitions of Angluin =-=[7, 11]-=- considered in the next section, and the development of Definition 3 nicely motivates her model of polynomial time identification using equivalence queries. 3.3 Polynomial-time Identification using Eq... |

1 |
Sufficent conditions for the necessity of Occam algorithms
- Board, Pitt
- 1989
(Show Context)
Citation Context ... recently prove that ifa hypothesis class H is "closed under exceptions" then the converse of Theorem 11 holds. They observe that DFAs have this property, and obtain the following theorem. Theorem 14 =-=[19]-=- If DFAs are PA C-identifiable then there ezists a randomized polynomial.time Occam algorithm ]or DFAs. Together with Corollary 13, Theorem t4 shows that the PAC-identifiabillty of DFAs is equivalent ... |

1 |
Learning reguJar languages from counterexamples
- l'barra, Jiang
- 1988
(Show Context)
Citation Context ...out the DFA to be inferred, as well as the problem of inferring certain restricted classes of DFAs. We describe a number of results (Angluin [6, 9], R~vest and Schapire [50, 51, 52], rbarra and Jiang =-=[33]-=-, Helmbotd, Sloan, and Warmuth [30, 31], and Abe [2]) showing that the inference problem is efficiently solvable in these more favorable settings. Finally~ Section 6 provides a brief sumraary, and sug... |

1 |
of Occam. "Entities should not be multiplied unnecessarily", c. 1320. Comment regarding superfluous elaboration
- William
(Show Context)
Citation Context ... already received significant attention. This is not surprising, in that independent of any formal justification from the standpoint of computational inference models, the principle of Occain's Razor =-=[40]-=-, is appealing both intuitively and philosophically as an inference technique. In the context of inference, Oecam's Razor is taken to mean that among competing hypotheses, the simplest is preferable. ... |