## Probabilistic Pattern Matching and the Evolution of Stochastic Regular Expressions (1999)

### Cached

### Download Links

- [www.cosc.brocku.ca]
- [www.cosc.brocku.ca]
- [sandcastle.cosc.brocku.ca]
- [www.cs.bham.ac.uk]
- DBLP

### Other Repositories/Bibliography

Venue: | International Journal of Applied Intelligence |

Citations: | 9 - 5 self |

### BibTeX

@ARTICLE{Ross99probabilisticpattern,

author = {Brian J. Ross and Brian J. Ross},

title = {Probabilistic Pattern Matching and the Evolution of Stochastic Regular Expressions},

journal = {International Journal of Applied Intelligence},

year = {1999},

volume = {13},

pages = {285--300}

}

### Years of Citing Articles

### OpenURL

### Abstract

The use of genetic programming for probabilistic pattern matching is investigated. A stochastic regular expression language is used. The language features a statistically sound semantics, as well as a syntax that promotes efficient manipulation by genetic programming operators. An algorithm for efficient string recognition based on approaches in conventional regular language recognition is used. When attempting to recognize a particular test string, the recognition algorithm computes the probabilities of generating that string and all its prefixes with the given stochastic regular expression. To promote efficiency, intermediate computed probabilities that exceed a given cut-off value will pre-empt particular interpretation paths, and hence prune unconstructive interpretation. A few experiments in recognizing stochastic regular languages are discussed. Application of the technology in bioinformatics is in progress.

### Citations

4490 | A tutorial on hidden Markov models and selected applications in speech recognition
- Rabiner
- 1989
(Show Context)
Citation Context ...lf is maximally connected. When particular transitions are not required, the probabilities associated with those nodes are set to zero. HMMs have found extensive use in language and speech processing =-=[8, 15]-=-. Strangely enough, stochastic regular expressions have not been extensively studied; one example paper is [16]. Language inference has been successfully done using genetic algorithms (GA) and genetic... |

3944 |
Introduction to Automata theory, languages and computation
- Hopcroft, Ullman
- 1979
(Show Context)
Citation Context ...ely weaker than stochastic contextfree languages, have also found their practical niche in applications. Regular languages are definable by finite automata, regular grammars, and regular expressions (=-=Hopcroft and Ullman 1979-=-). Similarly, stochastic regular languages are defined by stochastic versions of these three representations. Examples of work in stochastic grammar inference is in (Maryanski and Booth 1977, van der ... |

871 | An introduction to hidden Markov models
- Rabiner, Juang
- 1986
(Show Context)
Citation Context ...nce is in (Maryanski and Booth 1977, van der Mude and Walker 1978, Carrasco and Forcada 1996, Carrasco and Oncina 1998). Stochastic finite automata are defined in terms of Hidden Markov Models (HMM) (=-=Rabiner and Juang 1986-=-). An HMM is a finite automaton with probabilities marking the transition links between nodes. Each node is connected to all other nodes, and so the network itself is maximally connected. When particu... |

574 |
Statistical Language Learning
- Charniak
(Show Context)
Citation Context ...nguages enjoy both expressitivity and tractable properties, for example, the existence of useful inference algorithms (Lari and Young 1990). They have also found practical use in language processing (=-=Charniak 1993-=-). Stochastic regular languages, albeit descriptively weaker than stochastic contextfree languages, have also found their practical niche in applications. Regular languages are definable by finite aut... |

550 |
Introduction to the Theory of Computation
- Sipser
- 1997
(Show Context)
Citation Context ...+Closure: Similar to (c) and (d) above. 2 3.2 Implementation of an SRE Processor Given a regular expression, determining whether particular strings are members of its language is a tractable problem (=-=Sipser 1996-=-, Hopcroft and Ullman 1979). There are different 11 ways in which this may be performed. One technique is to convert the regular expression into an equivalent nondeterministic finite automaton, which ... |

385 |
A Tutorial on
- Rabiner
- 1989
(Show Context)
Citation Context ...f is maximally connected. When particular transitions are not required, the probabilities associated with those nodes are set to zero. HMMs have found extensive use in language and speech processing (=-=Rabiner 1989-=-, Charniak 1993). Strangely enough, stochastic regular expressions have not been extensively studied; one example paper is (Garg et al. 1996). 4 Language inference has been successfully done using gen... |

380 |
The estimation of stochastic context-free grammars using the inside-outside algorithm. Computer Speech and Language
- Lari, Young
- 1990
(Show Context)
Citation Context ...more discriminating than in a non--stochastic setting. Stochastic context free languages enjoy both expressitivity and tractable properties, for example, the existence of useful inference algorithms (=-=Lari and Young 1990-=-). They have also found practical use in language processing (Charniak 1993). Stochastic regular languages, albeit descriptively weaker than stochastic contextfree languages, have also found their pra... |

210 |
Syntactic Pattern recognition and Applications
- Fu
- 1982
(Show Context)
Citation Context ...ammar in that each grammar rule is marked with a probability associated with its use, and the set of probabilities for a grammar encode a probability distribution for the resulting derived language. (=-=Fu 1982-=-) has an extensive treatment of stochastic grammars, their derivation, and their application in pattern recognition. Stochastic grammars are also more complex than their non--stochastic kin, as the di... |

170 |
The PROSITE database, its status in
- Hofmann, Bucher, et al.
- 1999
(Show Context)
Citation Context ..., are used for this purpose. SRE is a natural vehicle for this problem area, since its regular expression basis conforms to the pattern languages commonly used (eg. that used in the PROSITE database (=-=Hofmann et al. 1999-=-)), while its stochastic features conveniently model the probabilistic characteristics of DNA sequences themselves. Acknowledgement: Thanks to Tom Jenkyns for helpful discussions about probability. Th... |

138 | Approaches to the automatic discovery of patterns in biosequences
- Brazma, Jonassen, et al.
- 1998
(Show Context)
Citation Context ..., the applicability of SRE in bioinformatics problems is being investigated. A fundamental problem in DNA and protein sequencing is to determine a common pattern shared amongst a family of sequences (=-=Brazma et al. 1995-=-), which can be used for both search and analytical purposes. A number of techniques, such as HMM's and regular pattern languages, are used for this purpose. SRE is a natural vehicle for this problem ... |

79 |
Computational learning theory: survey and selected bibliography
- Angluin
- 1992
(Show Context)
Citation Context ...nd future directions conclude the paper in section 6. 2 RELATED WORK Formal language induction has a long history as a fundamental problem in machine learning (Fu and Booth 1975a, Fu and Booth 1975b, =-=Angluin 1992-=-, Sakakibara 1997). The specialized topic of stochastic languages has also been studied for some time (Fu and Huang 1972). A stochastic grammar differs from a conventional grammar in that each grammar... |

74 |
The semantics of programming languages: an elementary introduction using structural operational semantics
- Hennessy
- 1990
(Show Context)
Citation Context ...lta \Delta (ff k ;p k ) \Gamma! E j E (s;p k ) =) E (s = ff 1 :::ff k ) Figure 1 contains transitional rules for the relations, which define the structural operational semantics of the SRE operators (=-=Hennessy 1990-=-). These inference rules define an abstract interpreter for SRE expressions, and are the basis of an SRE recognizer. In fact, with languages such as Prolog, these rules can be compiled into Prolog sta... |

72 |
Logic Grammars
- Abramson, Dahl
- 1989
(Show Context)
Citation Context ...ned so that evolution is given a helpful push towards program structures that are more sensible for the problem at hand. The grammar used by DCTG-GP is a definite clause translation grammar, or DCTG (=-=Abramson and Dahl 1989-=-). A DCTG is a logical version of a contextfree attribute grammar. Each DCTG production has a syntactic component, which defines a context--free syntax rule. In addition, each production can have incl... |

61 | Grammatically-based genetic programming
- Whigham
- 1995
(Show Context)
Citation Context ...-GP system (Ross 1999). DCTG-GP performs grammar--based genetic programming, in which the target language of the evolved program population is defined in terms of a context--free grammar (Lucas 1994, =-=Whigham 1995-=-, Wong and Leung 1995, Geyer-Shulz 1997). A major advantage of grammatical GP systems is that the search space is syntactically constrained so that evolution is given a helpful push towards program st... |

54 |
Recent advances of grammatical inference
- Sakakibara
- 1997
(Show Context)
Citation Context ...ctions conclude the paper in section 6. 2 RELATED WORK Formal language induction has a long history as a fundamental problem in machine learning (Fu and Booth 1975a, Fu and Booth 1975b, Angluin 1992, =-=Sakakibara 1997-=-). The specialized topic of stochastic languages has also been studied for some time (Fu and Huang 1972). A stochastic grammar differs from a conventional grammar in that each grammar rule is marked w... |

45 | Learning deterministic regular grammars from stochastic samples in polynomial time
- Carrasco, Oncina
- 1999
(Show Context)
Citation Context ...fined by stochastic versions of these three representations. Examples of work in stochastic grammar inference is in (Maryanski and Booth 1977, van der Mude and Walker 1978, Carrasco and Forcada 1996, =-=Carrasco and Oncina 1998-=-). Stochastic finite automata are defined in terms of Hidden Markov Models (HMM) (Rabiner and Juang 1986). An HMM is a finite automaton with probabilities marking the transition links between nodes. E... |

42 |
Grammatical Inference: Introduction and Survey Parts I and II
- Fu, Booth
- 1975
(Show Context)
Citation Context ...e discussed in section 5. A discussion and future directions conclude the paper in section 6. 2 RELATED WORK Formal language induction has a long history as a fundamental problem in machine learning (=-=Fu and Booth 1975-=-a, Fu and Booth 1975b, Angluin 1992, Sakakibara 1997). The specialized topic of stochastic languages has also been studied for some time (Fu and Huang 1972). A stochastic grammar differs from a conven... |

38 | Grammatical Inference from Positive and Negative Samples by Genetic Search: the GIG Method
- Dupont, “Regular
- 1994
(Show Context)
Citation Context ...t negative examples) and automata size. (Dunay et al. 1994)'s approach is similar to (Zhou and Grefenstette 1986), except that finite automata are denoted in GP--style nested S--expression notation. (=-=Dupont 1994-=-) uses an automata--theoretic partition representation for regular languages. This has the advantage of preserving language properties of chromosomes during GA reproduction, unlike the more fragile FA... |

36 |
Denotational semantics
- Stoy
- 1977
(Show Context)
Citation Context ...ay to consider SRE expressions is that every expression defines a specific probability function over strings in \Sigma : E : \Sigma ! p (0sps1) Using a denotational semantics style of representation (=-=Stoy 1977-=-), the probability function for SRE expression E is denoted by [[E]], and its application to a particular string s is denoted [[E]]s, which denotes the probability associated with string s in the lang... |

29 | Evolving deterministic finite automata using cellular encoding
- Brave
- 1996
(Show Context)
Citation Context ...ette 1986). His fitness function scores both language performance and automata size. He successfully evolved a large set of regular languages, including the benchmark Tomita languages (Tomita 1982). (=-=Brave 1997) uses an -=-abstract "cellular encoding" representation for deterministic FA's, which builds the network structure of a FA during interpretation. The intention of this denotation is to preserve structur... |

22 | Logic-based genetic programming with definite clause translation grammars
- Ross
- 1999
(Show Context)
Citation Context ...ngs 2 E (ff;p) \Gamma! E 0 E 0 (s;q) =) E 00 E (ffs;pq) =) E 00 Figure 1: Transitional semantics of SRE 13 implementation uses a logical grammar definition of SRE, which is part of the DCTGGP system (=-=Ross 1999-=-) (see Section 4). Prolog's backtracking is advantageously used to investigate different paths of an expression's derivation. In addition, string recognition is performed by pattern matching on an arg... |

21 | Stochastic Context-Free Grammar Induction with a Genetic Algorithm Using Local Search,” in
- Kammeyer, Belew
- 1996
(Show Context)
Citation Context ...sure of the grammar's ability to accept prefixes of the target grammar. A few experiments were performed, and their GA performance compares well with standard regular--language inference algorithms. (=-=Kammeyer and Belew 1997-=-) uses a GA to evolve stochastic context--free grammars. They use a liberal representation for grammars in which correct grammars are parsed from the genome when evaluated; this permits intron or junk... |

20 |
Context free grammar induction using genetic algorithms
- Wyard
- 1993
(Show Context)
Citation Context ...pressions are directly encoded as program trees, and fitness is based on correct example classification. He successfully evolved the Tomita languages. Context--free languages have also been studied. (=-=Wyard 1991-=-) uses a GA to evolve context--free grammars. Chromosomes takes the form of lists of production rules, which guarantees correctness at all times. The fitness function scores example classification per... |

19 |
Introduction to the Theory
- Sipser
- 2006
(Show Context)
Citation Context ...+Closure: Similar to (c) and (d) above. ✷ 3.2. Implementation of an SRE Processor Given a regular expression, determining whether particular strings are members of its language is a tractable proble=-=m [9, 34]-=-. There are different ways in which this may be performed. One technique is to convert the regular expression into an equivalent nondeterministic finite automaton, which can be done in polynomial time... |

18 |
Dynamic construction of finite automata from examples using hill-climbing
- Tomita
- 1982
(Show Context)
Citation Context ...ou and Grefenstette 1986). His fitness function scores both language performance and automata size. He successfully evolved a large set of regular languages, including the benchmark Tomita languages (=-=Tomita 1982). (Brave -=-1997) uses an abstract "cellular encoding" representation for deterministic FA's, which builds the network structure of a FA during interpretation. The intention of this denotation is to pre... |

15 | Structuring Chromosomes for Context-Free Grammar Evolution
- Lucas
- 1994
(Show Context)
Citation Context ... not included in the example set. These additional evaluation considerations give the GA more informa5 tion with which to drive evolution. He applied the GA to a number of CFG and regular languages. (=-=Lucas 1994-=-) uses a binary--encoded normal form for CFG productions, which preserves language properties during reproduction, and promotes convergence. His fitness strategy scores example classification and gram... |

11 |
W.P.: Regular Language Induction with Genetic Programming
- Dunay, Petry, et al.
- 1994
(Show Context)
Citation Context ...destructive effects during crossover and mutation. Their unspecified fitness function scores language performance (ability to accept positive strings and reject negative examples) and automata size. (=-=Dunay et al. 1994-=-)'s approach is similar to (Zhou and Grefenstette 1986), except that finite automata are denoted in GP--style nested S--expression notation. (Dupont 1994) uses an automata--theoretic partition represe... |

11 |
Induction of finite automata by genetic algorithms
- Zhou, Grefenstette
- 1986
(Show Context)
Citation Context ... GP uses a variable--sized parse tree. Some of the following use encodings with characteristics of both approaches. With respect to regular languages, an early work in evolving finite automata is in (=-=Zhou and Grefenstette 1986-=-). They used a GA with a binary encoding of the automata as a set of state transitions, capped at a size of 8 states. A weakness of this encoding is that the represented automata are susceptible to de... |

10 | Inference of stochastic regular grammars by massively parallel genetic algorithms
- Schwehm, Ost
- 1995
(Show Context)
Citation Context ...t--sensitive languages, they used a compositional approach, in which the GA had access to TM building blocks evolved in earlier runs. Finally, the evolution of stochastic languages has been studied. (=-=Schwehm and Ost 1995-=-) uses a GA for evolving stochastic regular languages. Two different encodings are studied -- production rules with probabilities, and quotient automata. The fitness function uses grammar complexity (... |

9 |
Grammatical inference: Introduction and survey---part II
- Fu, Booth
- 1975
(Show Context)
Citation Context ...odel of SRE is now given. Let s = α1,..αn ∈� ∗ . • Atomic actions: [[α]]β = 1 ifα = β [[α]]β = 0 ifα �= β • Choice (including guarded choice): �� �� � Ei(ni) s = � ��=-=� nk � j n � · [[ E k ]]s j i k (1) (2) Since-=- every term might recognize s, the overall probability for a choice expression is the sum of all the term probabilities with respect to s. • Concatenation: [[ E 1 : E 2]]s = n� ([[ E 1]]α1.. αi ... |

8 | On the inference of stochastic regular grammars - Mude, Walker - 1978 |

7 |
Grammatical inference with a genetic algorithm
- Lankhorst
- 1994
(Show Context)
Citation Context ...es takes the form of lists of production rules, which guarantees correctness at all times. The fitness function scores example classification performance. Two simple CFG's were successfully evolved. (=-=Lankhorst 1994-=-) uses a vector encoding to represent grammar productions. His fitness function is more involved than most others, as it scores example classificaton performance, the length of substrings of examples ... |

7 |
Inference of Finite-State Probabilistic Grammars
- Maryanski, Booth
- 1977
(Show Context)
Citation Context ...ressions (Hopcroft and Ullman 1979). Similarly, stochastic regular languages are defined by stochastic versions of these three representations. Examples of work in stochastic grammar inference is in (=-=Maryanski and Booth 1977-=-, van der Mude and Walker 1978, Carrasco and Forcada 1996, Carrasco and Oncina 1998). Stochastic finite automata are defined in terms of Hidden Markov Models (HMM) (Rabiner and Juang 1986). An HMM is ... |

6 |
Stochastic grammars and languages
- Fu, Huang
- 1972
(Show Context)
Citation Context ...s a fundamental problem in machine learning (Fu and Booth 1975a, Fu and Booth 1975b, Angluin 1992, Sakakibara 1997). The specialized topic of stochastic languages has also been studied for some time (=-=Fu and Huang 1972-=-). A stochastic grammar differs from a conventional grammar in that each grammar rule is marked with a probability associated with its use, and the set of probabilities for a grammar encode a probabil... |

6 |
Evolutionary learning of large grammars
- Longshaw
- 1997
(Show Context)
Citation Context ...ors which permit automata composition. The fitness function tallied the number of correctly classified sentences. All but one of the Tomita languages were successfully inferred using this technique. (=-=Longshaw 1997-=-) uses a straight--forward state--transition representation for automata. However, his GA uses a population seeded with correct but overly general automata. Specialized reproduction operators manipula... |

5 | Inferring stochastic regular grammars with recurrent neural networks
- Carrasco, Forcada, et al.
- 1996
(Show Context)
Citation Context ...ic regular languages are defined by stochastic versions of these three representations. Examples of work in stochastic grammar inference is in (Maryanski and Booth 1977, van der Mude and Walker 1978, =-=Carrasco and Forcada 1996-=-, Carrasco and Oncina 1998). Stochastic finite automata are defined in terms of Hidden Markov Models (HMM) (Rabiner and Juang 1986). An HMM is a finite automaton with probabilities marking the transit... |

4 |
Solving complex problems with genetic algorithms
- Dunay, Petry
- 1995
(Show Context)
Citation Context ...p' an empty stack. (Lankhorst 1995) extends this idea towards nondeterministic pushdown automata. His fitness additionally considers prefix sizes and the stack size after a string has been consumed. (=-=Dunay and Petry 1995-=-) use a Turing machine representation in their GA experiments. Although this powerful notation can denote the entire set of languages in the Chomsky, it does not necessary mean that search will be eas... |

4 |
A genetic algorithms for the induction of pushdown automata
- Lankhorst
- 1995
(Show Context)
Citation Context ... pushdown automata, which is an alternative to the grammar representation for CFG's. Fitness scores example recognition performance, and whether the PDA attempts to erroneously 'pop' an empty stack. (=-=Lankhorst 1995-=-) extends this idea towards nondeterministic pushdown automata. His fitness additionally considers prefix sizes and the stack size after a string has been consumed. (Dunay and Petry 1995) use a Turing... |

4 |
Learning to construct pushdown automata for accepting deterministic context-free languages
- Sen, Janakiraman
- 1992
(Show Context)
Citation Context ...nary--encoded normal form for CFG productions, which preserves language properties during reproduction, and promotes convergence. His fitness strategy scores example classification and grammar size. (=-=Sen and Janakiraman 1992-=-) applies a GA towards inferring deterministic pushdown automata, which is an alternative to the grammar representation for CFG's. Fitness scores example recognition performance, and whether the PDA a... |

3 |
Probabilistic language framework for stochastic discrete event systems
- Garg, Kumar, et al.
- 1996
(Show Context)
Citation Context ...MMs have found extensive use in language and speech processing (Rabiner 1989, Charniak 1993). Strangely enough, stochastic regular expressions have not been extensively studied; one example paper is (=-=Garg et al. 1996-=-). 4 Language inference has been successfully done using genetic algorithms (GA) and genetic programming (GP). The distinguishing difference between GA and GP approaches is one of denotation: a pure G... |

3 |
The next 700 programming languages for genetic programming
- Geyer-Shulz
- 1997
(Show Context)
Citation Context ...forms grammar--based genetic programming, in which the target language of the evolved program population is defined in terms of a context--free grammar (Lucas 1994, Whigham 1995, Wong and Leung 1995, =-=Geyer-Shulz 1997-=-). A major advantage of grammatical GP systems is that the search space is syntactically constrained so that evolution is given a helpful push towards program structures that are more sensible for the... |

2 |
Learning regular languages using genetic programming
- Svingen
- 1998
(Show Context)
Citation Context ...ates. The overall intention is to refine the general automata into a more specific one for the language in question. His fitness function scores example classification performance and automata size. (=-=Svingen 1998-=-) uses a GP on regular expressions. Regular expressions are directly encoded as program trees, and fitness is based on correct example classification. He successfully evolved the Tomita languages. Con... |

2 |
Learning programs in different paradigms using genetic programming
- Wong, Leung
- 1995
(Show Context)
Citation Context ...ss 1999). DCTG-GP performs grammar--based genetic programming, in which the target language of the evolved program population is defined in terms of a context--free grammar (Lucas 1994, Whigham 1995, =-=Wong and Leung 1995-=-, Geyer-Shulz 1997). A major advantage of grammatical GP systems is that the search space is syntactically constrained so that evolution is given a helpful push towards program structures that are mor... |

1 | Genetic Programming 1997 (John R. Koza et al, Ed - Proc |

1 |
A Primer in Probability
- Subrahmaniam
- 1979
(Show Context)
Citation Context ...eturning non--zero probabilities for particular strings: s 2 L(E) iff [[E]]s ? 0 s 62 L(E) iff [[E]]s = 0 Definition 3.1 All probability functions pf must adhere to the following two characteristics (=-=Subrahmaniam 1979-=-): (i) for all x i in the sample space of the experiment: X i pf(x i ) = 1 (6) (ii) For every event x i : 0spf(x i )s1 (7) 2 Consequently, if SRE expressions are to define well--formed probability fun... |

1 |
The Semantics of Programming Languages—An Elementary Introduction Using Structural Operational Semantics
- Hennessy
- 1990
(Show Context)
Citation Context ...strings: E (α1,p1) −→ ··· (αk,pk) −→ E ≡ E (s, pk ) �⇒ E (s = α1,...,αk) Figure 1 contains transitional rules for the relations, which define the structural operational semantics =-=of the SRE operators [35]-=-. These inference rules define an abstract interpreter for SRE expressions, and are the basis of an SRE recognizer. In fact, with languages such as Prolog, these rules can be compiled into Prolog stat... |