Results 1 
9 of
9
Direct Densityratio Estimation with Dimensionality Reduction via Leastsquares Heterodistributional Subspace Search
 NEURAL NETWORKS, VOL.24, NO.2, PP.183–198
, 2011
"... Methods for directly estimating the ratio of two probability density functions have been actively explored recently since they can be used for various data processing tasks such as nonstationarity adaptation, outlier detection, and feature selection. In this paper, we develop a new method which inc ..."
Abstract

Cited by 22 (13 self)
 Add to MetaCart
(Show Context)
Methods for directly estimating the ratio of two probability density functions have been actively explored recently since they can be used for various data processing tasks such as nonstationarity adaptation, outlier detection, and feature selection. In this paper, we develop a new method which incorporates dimensionality reduction into a direct densityratio estimation procedure. Our key idea is to find a lowdimensional subspace in which densities are significantly different and perform density ratio estimation only in this subspace. The proposed method, D³LHSS (Direct Densityratio estimation with Dimensionality reduction via Leastsquares Heterodistributional Subspace Search), is shown to overcome the limitation of baseline methods.
Computational Complexity of KernelBased DensityRatio Estimation: A Condition Number Analysis
 MACHINE LEARNING, VOL.90, NO.3, PP.431–460
, 2013
"... In this study, the computational properties of a kernelbased leastsquares densityratio estimator are investigated from the viewpoint of condition numbers. The condition number of the Hessian matrix of the loss function is closely related to the convergence rate of optimization and the numerical st ..."
Abstract

Cited by 14 (11 self)
 Add to MetaCart
In this study, the computational properties of a kernelbased leastsquares densityratio estimator are investigated from the viewpoint of condition numbers. The condition number of the Hessian matrix of the loss function is closely related to the convergence rate of optimization and the numerical stability. We use smoothed analysis techniques and theoretically demonstrate that the kernel leastsquares method has a smaller condition number than other Mestimators. This implies that the kernel leastsquares method has desirable computational properties. In addition, an alternate formulation of the kernel leastsquares estimator that possesses an even smaller condition number is presented. The validity of the theoretical analysis is verified through numerical experiments.
Learning under Nonstationarity: Covariate Shift Adaptation by Importance Weighting
 IN J. E. GENTLE , W. HÄRDLE , Y. MORI (EDS), HANDBOOK OF COMPUTATIONAL STATISTICS: CONCEPTS AND METHODS, 2ND EDITION. CHAPTER 31, PP.927–952, SPRINGER, BERLIN
, 2012
"... The goal of supervised learning is to estimate an underlying inputoutput function from its inputoutput training samples so that output values for unseen test input points can be predicted. A common assumption in supervised learning is that the training input points follow the same probability dist ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
The goal of supervised learning is to estimate an underlying inputoutput function from its inputoutput training samples so that output values for unseen test input points can be predicted. A common assumption in supervised learning is that the training input points follow the same probability distribution as the test input points. However, this assumption is not satisfied, for example, when outside of the training region is extrapolated. The situation where the training and test input points follow different distributions while the conditional distribution of output values given input points is unchanged is called covariate shift. Since almost all existing learning methods assume that the training and test samples are drawn from the same distribution, their fundamental theoretical properties such as consistency or efficiency no longer hold under covariate shift. In this chapter, we review recently proposed techniques for covariate shift adaptation. 1
Direct Density Ratio Estimation with Dimensionality Reduction
"... Methods for directly estimating the ratio of two probability density functions without going through density estimation have been actively explored recently since they can be used for various data processing tasks such as nonstationarity adaptation, outlier detection, conditional density estimation ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Methods for directly estimating the ratio of two probability density functions without going through density estimation have been actively explored recently since they can be used for various data processing tasks such as nonstationarity adaptation, outlier detection, conditional density estimation, feature selection, and independent component analysis. However, even the stateoftheart density ratio estimation methods still perform rather poorly in highdimensional problems. In this paper, we propose a new density ratio estimation method which incorporates dimensionality reduction into a density ratio estimation procedure. Our key idea is to identify a lowdimensional subspace in which the two densities corresponding to the denominator and the numerator in the density ratio are significantly different. Then the density ratio is estimated only within this lowdimensional subspace. Through numerical examples, we illustrate the effectiveness of the proposed method. 1
Density Ratio Estimation: A Comprehensive Review
, 2010
"... Density ratio estimation has attracted a great deal of attention in the statistics and machine learning communities since it can be used for solving various statistical data processing tasks such as nonstationarity adaptation, twosample test, outlier detection, independence test, feature selection ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Density ratio estimation has attracted a great deal of attention in the statistics and machine learning communities since it can be used for solving various statistical data processing tasks such as nonstationarity adaptation, twosample test, outlier detection, independence test, feature selection/extraction, independent component analysis, causal inference, and conditional probability estimation. When estimating the density ratio, it is preferable to avoid estimating densities since density estimation is known to be a hard problem. In this paper, we give a comprehensive review of density ratio estimation methods based on moment matching, probabilistic classification, and ratio matching.
Statistical Analysis of KernelBased LeastSquares DensityRatio Estimation
 MACHINE LEARNING, VOL.86, NO.3, PP.335–367
, 2012
"... The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihoodratio test), feature selection (mutual information), and conditional probability estimation. Several methods of direct ..."
Abstract
 Add to MetaCart
(Show Context)
The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihoodratio test), feature selection (mutual information), and conditional probability estimation. Several methods of directly estimating the density ratio have recently been developed, e.g., moment matching estimation, maximumlikelihood densityratio estimation, and leastsquares densityratio fitting. In this paper, we propose a kernelized variant of the leastsquares method for densityratio estimation, which is called kernel unconstrained leastsquares importance fitting (KuLSIF). We investigate its fundamental statistical properties including a nonparametric convergence rate, an analyticform solution, and a leaveoneout crossvalidation score. We further study its relation to other kernelbased densityratio estimators. In experiments, we numerically compare various kernelbased densityratio estimation methods, and show that KuLSIF compares favorably with other approaches.
IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, vol.E93A, no.4, pp.787–798, 2010. 1 Theoretical Analysis of Density Ratio Estimation
"... Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estima ..."
Abstract
 Add to MetaCart
(Show Context)
Density ratio estimation has gathered a great deal of attention recently since it can be used for various data processing tasks. In this paper, we consider three methods of density ratio estimation: (A) the numerator and denominator densities are separately estimated and then the ratio of the estimated densities is computed, (B) a logistic regression classifier discriminating denominator samples from numerator samples is learned and then the ratio of the posterior probabilities is computed, and (C) the density ratio function is directly modeled and learned by minimizing the empirical KullbackLeibler divergence. We first prove that when the numerator and denominator densities are known to be members of the exponential family, (A) is better than (B) and (B) is better than (C). Then we show that once the model assumption is violated, (C) is better than (A) and (B). Thus in practical situations where no exact model is available, (C) would be the most promising approach to density ratio estimation.
Twosample Homogeneity Tests Based on Divergence Measures
, 2014
"... ch ni ca lR ep or ..."
(Show Context)
A New Approach to Machine Learning Based on Density Ratios
"... This paper reviews a new framework for statistical machine learning that we introduced recently. A distinctive feature of this framework is that various machine learning problems are formulated as a problem of estimating the ratio of probability densities in a unied way. Then the density ratio is ..."
Abstract
 Add to MetaCart
(Show Context)
This paper reviews a new framework for statistical machine learning that we introduced recently. A distinctive feature of this framework is that various machine learning problems are formulated as a problem of estimating the ratio of probability densities in a unied way. Then the density ratio is estimated without going through the hard task of density estimation, which results in accurate estimation. This density ratio framework includes various machine learning tasks such as nonstationarity adaptation, outlier detection, dimensionality reduction, independent component analysis, and conditional density estimation. Thus, density ratio estimation is a highly versatile tool for machine learning.