Results 1 
4 of
4
A Spectral Learning Algorithm for Finite State
"... Abstract. FiniteState Transducers (FSTs) are a popular tool for modeling paired inputoutput sequences, and have numerous applications in realworld problems. Most training algorithms for learning FSTs rely on gradientbased or EM optimizations which can be computationally expensive and suffer from ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
Abstract. FiniteState Transducers (FSTs) are a popular tool for modeling paired inputoutput sequences, and have numerous applications in realworld problems. Most training algorithms for learning FSTs rely on gradientbased or EM optimizations which can be computationally expensive and suffer from local optima issues. Recently, Hsu et al. [13] proposed a spectral method for learning Hidden Markov Models (HMMs) which is based on an Observable Operator Model (OOM) view of HMMs. Following this line of work we present a spectral algorithm to learn FSTs with strong PACstyle guarantees. To the best of our knowledge, ours is the first result of this type for FST learning. At its core, the algorithm is simple, and scalable to large data sets. We present experiments that validate the effectiveness of the algorithm on synthetic and real data. 1
A spectral approach for probabilistic grammatical inference on trees
 In: Conference on Algorithmic Learning Theory (ALT
, 2010
"... Abstract. We focus on the estimation of a probability distribution over a set of trees. We consider here the class of distributions computed by weighted automata a strict generalization of probabilistic tree automata. This class of distributions (called rational distributions, or rational stochasti ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Abstract. We focus on the estimation of a probability distribution over a set of trees. We consider here the class of distributions computed by weighted automata a strict generalization of probabilistic tree automata. This class of distributions (called rational distributions, or rational stochastic tree languages RSTL) has an algebraic characterization: All the residuals (conditional) of such distributions lie in a finitedimensional vector subspace. We propose a methodology based on Principal Components Analysis to identify this vector subspace. We provide an algorithm that computes an estimate of the target residuals vector subspace and builds a model which computes an estimate of the target distribution. 1
Machine Learning manuscript No. (will be inserted by the editor) PAutomaC: a PFA/HMM Learning Competition
, 2013
"... Abstract Approximating distributions over strings is a hard learning problem. Typical techniques involve using finite state machines as models and attempting to learn these; these machines can either be hand built and then have their weights estimated, or built by grammatical inference techniques: t ..."
Abstract
 Add to MetaCart
Abstract Approximating distributions over strings is a hard learning problem. Typical techniques involve using finite state machines as models and attempting to learn these; these machines can either be hand built and then have their weights estimated, or built by grammatical inference techniques: the structure and the weights are then learned simultaneously. The Probabilistic Automata learning Competition (PAutomaC), run in 2012, was the first grammatical inference challenge that allowed the comparison between these methods and algorithms. Its main goal was to provide an overview of the stateoftheart techniques for this hard learning problem. Both artificial data and real data were presented and contestants were to try to estimate the probabilities of strings. The purpose of this paper is to describe some of the technical and intrinsic challenges such a competition has to face, to give a broad state of the art concerning both the problems dealing with learning grammars and finite state machines and the relevant literature. This paper also provides the results of the competition and a brief description and analysis of the different approaches the main participants used.
Learning FiniteState Machines Statistical and Algorithmic Aspects
"... The present thesis addresses several machine learning problems on generative and predictive models on sequential data. All the models considered have in common that they can be defined in terms of finitestate machines. On one line of work we study algorithms for learning the probabilistic analog of ..."
Abstract
 Add to MetaCart
The present thesis addresses several machine learning problems on generative and predictive models on sequential data. All the models considered have in common that they can be defined in terms of finitestate machines. On one line of work we study algorithms for learning the probabilistic analog of Deterministic Finite Automata (DFA). This provides a fairly expressive generative model for sequences with very interesting algorithmic properties. Statemerging algorithms for learning these models can be interpreted as a divisive clustering scheme where the “dependency graph ” between clusters is not necessarily a tree. We characterize these algorithms in terms of statistical queries and a use this characterization for proving a lower bound with an explicit dependency on the distinguishability of the target machine. In a more realistic setting, we give an adaptive statemerging algorithm satisfying the stringent algorithmic constraints of the data streams computing paradigm. Our algorithms come with strict PAC learning guarantees. At the heart of statemerging algorithms lies a statistical test for distribution similarity. In the streaming version this is replaced with a bootstrapbased test which yields faster convergence in many situations. We also studied a wider class of models for which the statemerging paradigm also yield PAC learning algorithms. Applications of this method are given to continuoustime Markovian models and stochastic transducers on pairs of aligned sequences. The main tools used for