Results 1  10
of
64
Dynamic Bayesian Networks: Representation, Inference and Learning
, 2002
"... Modelling sequential data is important in many areas of science and engineering. Hidden Markov models (HMMs) and Kalman filter models (KFMs) are popular for this because they are simple and flexible. For example, HMMs have been used for speech recognition and biosequence analysis, and KFMs have bee ..."
Abstract

Cited by 564 (3 self)
 Add to MetaCart
Modelling sequential data is important in many areas of science and engineering. Hidden Markov models (HMMs) and Kalman filter models (KFMs) are popular for this because they are simple and flexible. For example, HMMs have been used for speech recognition and biosequence analysis, and KFMs have been used for problems ranging from tracking planes and missiles to predicting the economy. However, HMMs
and KFMs are limited in their “expressive power”. Dynamic Bayesian Networks (DBNs) generalize HMMs by allowing the state space to be represented in factored form, instead of as a single discrete random variable. DBNs generalize KFMs by allowing arbitrary probability distributions, not just (unimodal) linearGaussian. In this thesis, I will discuss how to represent many different kinds of models as DBNs, how to perform exact and approximate inference in DBNs, and how to learn DBN models from sequential data.
In particular, the main novel technical contributions of this thesis are as follows: a way of representing
Hierarchical HMMs as DBNs, which enables inference to be done in O(T) time instead of O(T 3), where T is the length of the sequence; an exact smoothing algorithm that takes O(log T) space instead of O(T); a simple way of using the junction tree algorithm for online inference in DBNs; new complexity bounds on exact online inference in DBNs; a new deterministic approximate inference algorithm called factored frontier; an analysis of the relationship between the BK algorithm and loopy belief propagation; a way of
applying RaoBlackwellised particle filtering to DBNs in general, and the SLAM (simultaneous localization
and mapping) problem in particular; a way of extending the structural EM algorithm to DBNs; and a variety of different applications of DBNs. However, perhaps the main value of the thesis is its catholic presentation of the field of sequential data modelling.
Multiresolution markov models for signal and image processing
 Proceedings of the IEEE
, 2002
"... This paper reviews a significant component of the rich field of statistical multiresolution (MR) modeling and processing. These MR methods have found application and permeated the literature of a widely scattered set of disciplines, and one of our principal objectives is to present a single, coheren ..."
Abstract

Cited by 122 (18 self)
 Add to MetaCart
This paper reviews a significant component of the rich field of statistical multiresolution (MR) modeling and processing. These MR methods have found application and permeated the literature of a widely scattered set of disciplines, and one of our principal objectives is to present a single, coherent picture of this framework. A second goal is to describe how this topic fits into the even larger field of MR methods and concepts–in particular making ties to topics such as wavelets and multigrid methods. A third is to provide several alternate viewpoints for this body of work, as the methods and concepts we describe intersect with a number of other fields. The principle focus of our presentation is the class of MR Markov processes defined on pyramidally organized trees. The attractiveness of these models stems from both the very efficient algorithms they admit and their expressive power and broad applicability. We show how a variety of methods and models relate to this framework including models for selfsimilar and 1/f processes. We also illustrate how these methods have been used in practice. We discuss the construction of MR models on trees and show how questions that arise in this context make contact with wavelets, state space modeling of time series, system and parameter identification, and hidden
An Introduction to Factor Graphs
 IEEE SIGNAL PROCESSING MAG., JAN. 2004
, 2004
"... A large variety of algorithms in coding, signal processing, and artificial intelligence may be viewed as instances of the summaryproduct algorithm (or belief/probability ..."
Abstract

Cited by 121 (34 self)
 Add to MetaCart
A large variety of algorithms in coding, signal processing, and artificial intelligence may be viewed as instances of the summaryproduct algorithm (or belief/probability
Phylogenetic motif detection by expectationmaximization on evolutionary mixtures
 Pac. Symp. Biocomput
, 2004
"... The preferential conservation of transcription factor binding sites implies that noncoding sequence data from related species will prove a powerful asset to motif discovery. We present a unified probabilistic framework for motif discovery that incorporates of evolutionary information. We treat alig ..."
Abstract

Cited by 41 (1 self)
 Add to MetaCart
The preferential conservation of transcription factor binding sites implies that noncoding sequence data from related species will prove a powerful asset to motif discovery. We present a unified probabilistic framework for motif discovery that incorporates of evolutionary information. We treat aligned DNA sequence as a mixture of evolutionary models, for motif and background, and, following the example of the MEME program, provide an algorithm to estimate the parameters by ExpectationMaximization. We examine a variety of evolutionary models and show that our approach can take advantage of phylogenic information to avoid false positives and discover motifs upstream of groups of characterized target genes. We compare our method to traditional motif finding on only conserved regions. An implementation will be made available
Keyboard acoustic emanations revisited
 In Proceedings of the 12th ACM CCS
, 2005
"... We examine the problem of keyboard acoustic emanations. We present a novel attack taking as input a 10minute sound recording of a user typing English text using a keyboard and recovering up to 96 % of typed characters. There is no need for training recordings labeled with the corresponding clear te ..."
Abstract

Cited by 40 (1 self)
 Add to MetaCart
We examine the problem of keyboard acoustic emanations. We present a novel attack taking as input a 10minute sound recording of a user typing English text using a keyboard and recovering up to 96 % of typed characters. There is no need for training recordings labeled with the corresponding clear text. A recognizer bootstrapped from a 10minute sound recording can even recognize random text such as passwords: In our experiments, 90 % of 5character random passwords using only letters can be generated in fewer than 20 attempts by an adversary; 80 % of 10character passwords can be generated in fewer than 75 attempts by an adversary. In the attack, we use the statistical constraints of the underlying content, English language, to reconstruct text from sound recordings without knowing the corresponding clear text. The attack incorporates a combination of standard machine learning and speech recognition techniques, including cepstrum features, Hidden Markov Models, linear classification, and feedbackbased incremental learning.
Bundle Methods for Regularized Risk Minimization
"... A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Gaussian Processes, Logistic Regression, Conditional ..."
Abstract

Cited by 36 (2 self)
 Add to MetaCart
A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Gaussian Processes, Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for datalocality, and can deal with regularizers such as L1 and L2 penalties. In addition to the unified framework we present tight convergence bounds, which show that our algorithm converges in O(1/ɛ) steps to ɛ precision for general convex problems and in O(log(1/ɛ)) steps for continuously differentiable problems. We demonstrate the performance of our general purpose solver on a variety of publicly available datasets.
Sketched symbol recognition using zernike moments
 International Conference on Pattern Recognition
, 2004
"... In this paper, we present an online recognition method for handsketched symbols. The method is independent of strokeorder,number, anddirection, as well as invariant to rotation, scaling, and translation of symbols. Zernike moment descriptors are used to represent symbols and three different cla ..."
Abstract

Cited by 32 (0 self)
 Add to MetaCart
In this paper, we present an online recognition method for handsketched symbols. The method is independent of strokeorder,number, anddirection, as well as invariant to rotation, scaling, and translation of symbols. Zernike moment descriptors are used to represent symbols and three different classification techniques are compared: Support Vector Machines (SVM), Minimum Mean Distance (MMD), and Nearest Neighbor (NN). We have obtained 97 % accuracy rate on a dataset consisting of 7,410 sketched symbols using Zernike moment features and a SVM classifier. This method has been implemented in a software recognition package, HHreco [7]. 1.
Optimal Dynamic Treatment Regimes
 JOURNAL OF THE ROYAL STATISTICAL SOCIETY, SERIES B (WITH
, 2002
"... ... this paper is to use experimental or observational data to estimate decision regimes that result in a maximal mean response. To explicate our objective and state assumptions, we use the potential outcomes model. The proposed method makes smooth, parametric assumptions only on quantities directly ..."
Abstract

Cited by 32 (10 self)
 Add to MetaCart
... this paper is to use experimental or observational data to estimate decision regimes that result in a maximal mean response. To explicate our objective and state assumptions, we use the potential outcomes model. The proposed method makes smooth, parametric assumptions only on quantities directly relevant to the goal of estimating the optimal rules. We illustrate the proposed methodology via a small simulation.
Graphical models and point pattern matching
 IEEE Trans. PAMI
, 2006
"... Abstract—This paper describes a novel solution to the rigid point pattern matching problem in Euclidean spaces of any dimension. Although we assume rigid motion, jitter is allowed. We present a noniterative, polynomial time algorithm that is guaranteed to find an optimal solution for the noiseless c ..."
Abstract

Cited by 31 (6 self)
 Add to MetaCart
Abstract—This paper describes a novel solution to the rigid point pattern matching problem in Euclidean spaces of any dimension. Although we assume rigid motion, jitter is allowed. We present a noniterative, polynomial time algorithm that is guaranteed to find an optimal solution for the noiseless case. First, we model point pattern matching as a weighted graph matching problem, where weights correspond to Euclidean distances between nodes. We then formulate graph matching as a problem of finding a maximum probability configuration in a graphical model. By using graph rigidity arguments, we prove that a sparse graphical model yields equivalent results to the fully connected model in the noiseless case. This allows us to obtain an algorithm that runs in polynomial time and is provably optimal for exact matching between noiseless point sets. For inexact matching, we can still apply the same algorithm to find approximately optimal solutions. Experimental results obtained by our approach show improvements in accuracy over current methods, particularly when matching patterns of different sizes. Index Terms—Point pattern matching, graph matching, graphical models, Markov random fields, junction tree algorithm. 1
Probabilistic data management for pervasive computing: The data furnace project
 IEEE Data Eng. Bull
, 2006
"... The wide deployment of wireless sensor and RFID (Radio Frequency IDentification) devices is one of the key enablers for nextgeneration pervasive computing applications, including largescale environmental monitoring and control, contextaware computing, and “smart digital homes”. Sensory readings a ..."
Abstract

Cited by 15 (0 self)
 Add to MetaCart
The wide deployment of wireless sensor and RFID (Radio Frequency IDentification) devices is one of the key enablers for nextgeneration pervasive computing applications, including largescale environmental monitoring and control, contextaware computing, and “smart digital homes”. Sensory readings are inherently unreliable and typically exhibit strong temporal and spatial correlations (within and across different sensing devices); effective reasoning over such unreliable streams introduces a host of new data management challenges. The Data Furnace project at Intel Research and UCBerkeley aims to build a probabilistic data management infrastructure for pervasive computing environments that handles the uncertain nature of such data as a firstclass citizen through a principled framework grounded in probabilistic models and inference techniques. 1