Results 1  10
of
101
A MonteCarlo AIXI Approximation
, 2009
"... This paper describes a computationally feasible approximation to the AIXI agent, a universal reinforcement learning agent for arbitrary environments. AIXI is scaled down in two key ways: First, the class of environment models is restricted to all prediction suffix trees of a fixed maximum depth. Thi ..."
Abstract

Cited by 28 (9 self)
 Add to MetaCart
This paper describes a computationally feasible approximation to the AIXI agent, a universal reinforcement learning agent for arbitrary environments. AIXI is scaled down in two key ways: First, the class of environment models is restricted to all prediction suffix trees of a fixed maximum depth. This allows a Bayesian mixture of environment models to be computed in time proportional to the logarithm of the size of the model class. Secondly, the finitehorizon expectimax search is approximated by an asymptotically convergent Monte Carlo Tree Search technique. This scaled down AIXI agent is empirically shown to be effective on a wide class of toy problem domains, ranging from simple fully observable games to small POMDPs. We explore the limits of this approximate agent and propose a general heuristic framework for scaling this technique to much larger problems.
A Monte Carlo AIXI Approximation
 J. Artif. Intell. Res
"... This paper describes a computationally feasible approximation to the AIXI agent, a universal reinforcement learning agent for arbitrary environments. AIXI is scaled down in two key ways: First, the class of environment models is restricted to all prediction suffix trees of a fixed maximum depth. Thi ..."
Abstract

Cited by 21 (11 self)
 Add to MetaCart
(Show Context)
This paper describes a computationally feasible approximation to the AIXI agent, a universal reinforcement learning agent for arbitrary environments. AIXI is scaled down in two key ways: First, the class of environment models is restricted to all prediction suffix trees of a fixed maximum depth. This allows a Bayesian mixture of environment models to be computed in time proportional to the logarithm of the size of the model class. Secondly, the finitehorizon expectimax search is approximated by an asymptotically convergent Monte Carlo Tree Search technique. This scaled down AIXI agent is empirically shown to be effective on a wide class of toy problem domains, ranging from simple fully observable games to small POMDPs. We explore the limits of this approximate agent and propose a general heuristic framework for scaling this technique to much larger problems.
1 Web Query Recommendation via Sequential Query Prediction
"... Abstract — Web query recommendation has long been considered a key feature of search engines. Building a good Web query recommendation system, however, is very difficult due to the fundamental challenge of predicting users ’ search intent, especially given the limited user context information. In th ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
(Show Context)
Abstract — Web query recommendation has long been considered a key feature of search engines. Building a good Web query recommendation system, however, is very difficult due to the fundamental challenge of predicting users ’ search intent, especially given the limited user context information. In this paper, we propose a novel “sequential query prediction ” approach that tries to grasp a user’s search intent based on his/her past query sequence and its resemblance to historical query sequence models mined from massive search engine logs. Different query sequence models were examined, including the naive variablelength Ngram model, Variable Memory Markov (VMM) model, and our proposed Mixture Variable Memory Markov (MVMM) model. Extensive experiments were conducted to benchmark our sequence prediction algorithms against two conventional pairwise approaches on largescale search logs extracted from a commercial search engine. Results show that the sequencewise approaches significantly outperform the conventional pairwise ones in terms of prediction accuracy. In particular, our MVMM approach, consistently leads the pack, making it an effective and practical approach towards Web query recommendation.
Annotating proteins by mining protein interaction networks
 Bioinformatics
, 2006
"... doi:10.1093/bioinformatics/btl221 ..."
Automatic Bass Line Transcription from Streaming Polyphonic Audio
 Proceedings of the 2007 IEEE International Conference on Acoustics, Speech, and Signal Processing
, 2007
"... This paper proposes a method for the automatic transcription of the bass line in polyphonic music. The method uses a multipleF0 estimator as a frontend and this is followed by acoustic and musicological models. The acoustic modeling consists of separate models for bass notes and rests. The musicol ..."
Abstract

Cited by 12 (4 self)
 Add to MetaCart
(Show Context)
This paper proposes a method for the automatic transcription of the bass line in polyphonic music. The method uses a multipleF0 estimator as a frontend and this is followed by acoustic and musicological models. The acoustic modeling consists of separate models for bass notes and rests. The musicological model estimates the key and determines probabilities for the transitions between notes using a conventional bigram or a variableorder Markov model. The transcription is obtained with Viterbi decoding through the note and rest models. In addition, a causal algorithm is presented which allows transcription of streaming audio. The method was evaluated using 87 minutes of music from the RWC Popular Music Database. Recall and precision rates of 64 % and 60%, respectively, were achieved for discrete note events.
Automatic Generation of String Signatures for Malware Detection
"... Abstract. Scanning files for signatures is a proven technology, but exponential growth in unique malware programs has caused an explosion in signature database sizes. One solution to this problem is to use string signatures, each of which is a contiguous byte sequence that potentially can match many ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
(Show Context)
Abstract. Scanning files for signatures is a proven technology, but exponential growth in unique malware programs has caused an explosion in signature database sizes. One solution to this problem is to use string signatures, each of which is a contiguous byte sequence that potentially can match many variants of a malware family. However, it is not clear how to automatically generate these string signatures with a sufficiently low false positive rate. Hancock is the first string signature generation system that takes on this challenge on a large scale. To minimize the false positive rate, Hancock features a scalable model that estimates the occurrence probability of arbitrary byte sequences in goodware programs, a set of library code identification techniques, and diversitybased heuristics that ensure the contexts in which a signature is embedded in containing malware files are similar to one another. With these techniques combined, Hancock is able to automatically generate string signatures with a false positive rate below 0.1%.
A feedback informationtheoretic approach to the design of brain–computer interfaces,” Intl
 Journal of Human–Computer Interaction
, 2010
"... Feedback InformationTheoretic Approach to BCI 2 This paper presents and experimentally validates a new approach to designing braincomputer interfaces (BCIs) that explicitly takes into account both the inherent uncertainty in measurement and interpretation of neural signals and the important role o ..."
Abstract

Cited by 9 (7 self)
 Add to MetaCart
(Show Context)
Feedback InformationTheoretic Approach to BCI 2 This paper presents and experimentally validates a new approach to designing braincomputer interfaces (BCIs) that explicitly takes into account both the inherent uncertainty in measurement and interpretation of neural signals and the important role of sensory feedback in mitigating the effects of this uncertainty. This approach views a BCI as the means by which a user may communicate their intent to a prosthetic device. Although “intent ” may take different forms, in many applications of practical interest (e.g., text entry) it can be modeled as a string in an ordered symbolic language. This abstraction allows the problem of interface design to be reformulated as the problem of deriving an optimal communication protocol using tools from feedback information theory. For a wide class of BCIs, such a protocol is provided by the combination of arithmetic coding as a classical method of lossless data compression with posterior matching as a capacityachieving channel code that uses feedback to avoid the necessity of forward error correction. The remarkable thing about this protocol is that it is not only provably
A Machine Learning Approach for Statistical Software Testing
 in Proceedings, International Conference on Artificial Intelligence
"... Some Statistical Software Testing approaches rely on sampling the feasible paths in the control ow graph of the program; the difculty comes from the tiny ratio of feasible paths. This paper presents an adaptive sampling mechanism called EXIST for Exploration/eXploitation Inference for Software Test ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
Some Statistical Software Testing approaches rely on sampling the feasible paths in the control ow graph of the program; the difculty comes from the tiny ratio of feasible paths. This paper presents an adaptive sampling mechanism called EXIST for Exploration/eXploitation Inference for Software Testing, able to retrieve distinct feasible paths with high probability. EXIST proceeds by alternatively exploiting and updating a distribution on the set of program paths. An original representation of paths, accommodating longrange dependencies and data sparsity and based on extended Parikh maps, is proposed. Experimental validation on realworld and articial problems demonstrates dramatic improvements compared to the state of the art. 1
An Algorithm for Universal Lossless Compression With Side Information
"... Abstract—This paper proposes a new algorithm based on the ContextTree Weighting (CTW) method for universal compression of a finitealphabet sequence � � I with side information � � I available to both the encoder and decoder. We prove that with probability one the compression ratio converges to t ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
(Show Context)
Abstract—This paper proposes a new algorithm based on the ContextTree Weighting (CTW) method for universal compression of a finitealphabet sequence � � I with side information � � I available to both the encoder and decoder. We prove that with probability one the compression ratio converges to the conditional entropy rate for jointly stationary ergodic sources. Experimental results with Markov chains and English texts show the effectiveness of the algorithm. Index Terms—Arithmetic coding, conditional entropy, context tree weighting method, hidden Markov process, source coding, universal lossless data compression. I.
Universal divergence estimation for finitealphabet sources
 IEEE Trans. Inf. Theory
, 2006
"... Abstract—This paper studies universal estimation of divergence from the realizations of two unknown finitealphabet sources. Two algorithms that borrow techniques from data compression are presented. The first divergence estimator applies the Burrows–Wheeler block sorting transform to the concatenat ..."
Abstract

Cited by 9 (3 self)
 Add to MetaCart
(Show Context)
Abstract—This paper studies universal estimation of divergence from the realizations of two unknown finitealphabet sources. Two algorithms that borrow techniques from data compression are presented. The first divergence estimator applies the Burrows–Wheeler block sorting transform to the concatenation of the two realizations; consistency of this estimator is shown for all finitememory sources. The second divergence estimator is based on the Context Tree Weighting method; consistency is shown for all sources whose memory length does not exceed a known bound. Experimental results show that both algorithms perform similarly and outperform stringmatching and plugin methods. Index Terms—Block sorting, Burrows–Wheeler transform, context tree weighting method, divergence estimation, information divergence,