Results 1  10
of
14
Clustering: Science or art
 NIPS 2009 Workshop on Clustering Theory
, 2009
"... This paper deals with the question whether the quality of different clustering algorithms can be compared by a general, scientifically sound procedure which is independent of particular clustering algorithms. In our opinion, the major obstacle is the difficulty to evaluate a clustering algorithm wit ..."
Abstract

Cited by 27 (1 self)
 Add to MetaCart
This paper deals with the question whether the quality of different clustering algorithms can be compared by a general, scientifically sound procedure which is independent of particular clustering algorithms. In our opinion, the major obstacle is the difficulty to evaluate a clustering algorithm without taking into account the context: why does the user cluster his data in the first place, and what does he want to do with the clustering afterwards? We suggest that clustering should not be treated as an applicationindependent mathematical problem, but should always be studied in the context of its enduse. Different techniques to evaluate clustering algorithms have to be developed for different uses of clustering. To simplify this procedure it will be useful to build a “taxonomy of clustering problems ” to identify clustering applications which can be treated in a unified way. Preamble Every year, dozens of papers on clustering algorithms get published. Researchers continuously invent new clustering algorithms and work on improving existing ones.
Pacbayesian inequalities for martingales
 IEEE Transactions on Information Theory
, 2012
"... Abstract—We present a set of highprobability inequalities that control the concentration of weighted averages of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. Our results extend the PACBayesian (probably approximately correct) analysis in learning the ..."
Abstract

Cited by 14 (4 self)
 Add to MetaCart
(Show Context)
Abstract—We present a set of highprobability inequalities that control the concentration of weighted averages of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. Our results extend the PACBayesian (probably approximately correct) analysis in learning theory from the i.i.d. setting to martingales opening the way for its application to importance weighted sampling, reinforcement learning, and other interactive learning domains, as well as many other domains in probability theory and statistics, where martingales are encountered. We also present a comparison inequality that bounds the expectation of a convex function of a martingaledifferencesequenceshiftedto the interval by the expectation of the same function of independent Bernoulli random variables. This inequality is applied to derive a tighter analog of Hoeffding–Azuma’s inequality. Index Terms—Bernstein’s inequality, Hoeffding–Azuma’s inequality, martingales, PACBayesian bounds.
PACBayesian Analysis of Contextual Bandits
"... We derive an instantaneous (perround) datadependent regret bound for stochastic multiarmed bandits with side information (also known as contextual bandits). pThe scaling of our regret bound with the number of states (contexts) N goes as NI⇢t (S; A), where I⇢t (S; A) is the mutual information betwe ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
We derive an instantaneous (perround) datadependent regret bound for stochastic multiarmed bandits with side information (also known as contextual bandits). pThe scaling of our regret bound with the number of states (contexts) N goes as NI⇢t (S; A), where I⇢t (S; A) is the mutual information between states and actions (the side information) used by the algorithm at round t. If the algorithm uses all the side information, the regret bound scales as p N ln K, where K is the number of actions (arms). However, if the side information I⇢t (S; A) is not fully used, the regret bound is significantly tighter. In the extreme case, when I⇢t (S; A) =0, the dependence on the number of states reduces from linear to logarithmic. Our analysis allows to provide the algorithm large amount of side information, let the algorithm to decide which side information is relevant for the task, and penalize the algorithm only for the side information that it is using de facto. We also present an algorithm for multiarmed bandits with side information with O(K) computational complexity per game round. 1
Balancing Safety and Exploitability in Opponent Modeling
"... Opponent modeling is a critical mechanism in repeated games. It allows a player to adapt its strategy in order to better respond to the presumed preferences of his opponents. We introduce a new modeling technique that adaptively balances exploitability and risk reduction. An opponent’s strategy is m ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
Opponent modeling is a critical mechanism in repeated games. It allows a player to adapt its strategy in order to better respond to the presumed preferences of his opponents. We introduce a new modeling technique that adaptively balances exploitability and risk reduction. An opponent’s strategy is modeled with a set of possible strategies that contain the actual strategy with a high probability. The algorithm is safe as the expected payoff is above the minimax payoff with a high probability, and can exploit the opponents ’ preferences when sufficient observations have been obtained. We apply them to normalform games and stochastic games with a finite number of stages. The performance of the proposed approach is first demonstrated on repeated rockpaperscissors games. Subsequently, the approach is evaluated in a humanrobot tabletennis setting where the robot player learns to prepare to return a served ball. By modeling the human players, the robot chooses a forehand, backhand or middle preparation pose before they serve. The learned strategies can exploit the opponent’s preferences, leading to a higher rate of successful returns.
Contextual Collaborative Filtering via Hierarchical Matrix Factorization
"... Matrix factorization (MF) has been demonstrated to be one of the most competitive techniques for collaborative filtering. However, stateoftheart MFs do not consider contextual information, where ratings can be generated under different environments. For example, users select items under various s ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
Matrix factorization (MF) has been demonstrated to be one of the most competitive techniques for collaborative filtering. However, stateoftheart MFs do not consider contextual information, where ratings can be generated under different environments. For example, users select items under various situations, such as happy mood vs. sad, mobile vs. stationary, movies vs. book, etc. Under different contexts, the preference of users are inherently different. The problem is that MF methods uniformly decompose the rating matrix, and thus they are unable to factorize for different contexts. To amend this problem and improve recommendation accuracy, we introducea“hierarchical” factorization model by considering the local context when performing matrix factorization. The intuition is that: as ratings are being generated from heterogeneous environments, certain userand item pairs tend to be more similar to each other than others, and hence they ought to receive more collaborative information from each other. To take the contextual information into consideration, the proposed “contextual collaborative filtering ” approach splits the rating matrix hierarchically by grouping similar users and items together, and factorizes each submatrix locally under different contexts. By building an ensemble model, the approach further avoids overfitting with less parameter tuning. We analyze and demonstrate that the proposed method is a modelaveraging gradient boosting model, and its error rate can be bounded. Experimental results show that it outperforms three stateoftheart algorithms on a number of realworld datasets (MovieLens, Netflix, etc). The source code and datasets are available for download 1. 1
PACBayesian analysis of martingales and multiarmed bandits. http://arxiv.org/abs/1105.2416
, 2011
"... Abstract We present two alternative ways to apply PACBayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Ber ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
(Show Context)
Abstract We present two alternative ways to apply PACBayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This lemma provides an alternative tool to HoeffdingAzuma inequality to bound concentration of martingale values. Our second approach is based on integration of HoeffdingAzuma inequality with PACBayesian analysis. We also introduce a way to apply PACBayesian analysis in situation of limited feedback. We combine the new tools to derive PACBayesian generalization and regret bounds for the multiarmed bandit problem. Although our regret bound is not yet as tight as stateoftheart regret bounds based on other wellestablished techniques, our results significantly expand the range of potential applications of PACBayesian analysis and introduce a new analysis tool to reinforcement learning and many other fields, where martingales and limited feedback are encountered.
PACBayesBernstein inequality for martingales and its application to multiarmed bandits
 JMLR Workshop and Conference Proceedings
"... We develop a new tool for datadependent analysis of the explorationexploitation tradeoff in learning under limited feedback. Our tool is based on two main ingredients. The first ingredient is a new concentration inequality that makes it possible to control the concentration of weighted averages o ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
We develop a new tool for datadependent analysis of the explorationexploitation tradeoff in learning under limited feedback. Our tool is based on two main ingredients. The first ingredient is a new concentration inequality that makes it possible to control the concentration of weighted averages of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. 1 The second ingredient is an application of this inequality to the explorationexploitation tradeoff via importance weighted sampling. We apply the new tool to the stochastic multiarmed bandit problem, however, the main importance of this paper is the development and understanding of the new tool rather than improvement of existing algorithms for stochastic multiarmed bandits. In the followup work we demonstrate that the new tool can improve over stateoftheart in structurally richer problems, such as stochastic multiarmed bandits with side information (Seldin et al., 2011a).
PACBayesian analysis of the explorationexploitation tradeoff
 In Online Trading of Exploration and Exploitation 2, ICML2011 workshop
"... We develop a coherent framework for integrative simultaneous analysis of the explorationexploitation and model order selection tradeoffs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PACBayesian analysis with Bernsteintype inequality for martingales. ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
We develop a coherent framework for integrative simultaneous analysis of the explorationexploitation and model order selection tradeoffs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PACBayesian analysis with Bernsteintype inequality for martingales. Such a combination is also of independent interest for studies of multiple simultaneously evolving martingales. 1.
Computing Centre Russian Academy of Sciences
"... We present a PACBayesEmpiricalBernstein inequality. The inequality is based on a combination of the PACBayesian bounding technique with an Empirical Bernstein bound. We show that when the empirical variance is significantly smaller than the empirical loss the PACBayesEmpiricalBernstein inequa ..."
Abstract
 Add to MetaCart
(Show Context)
We present a PACBayesEmpiricalBernstein inequality. The inequality is based on a combination of the PACBayesian bounding technique with an Empirical Bernstein bound. We show that when the empirical variance is significantly smaller than the empirical loss the PACBayesEmpiricalBernstein inequality is significantly tighter than the PACBayeskl inequality of Seeger (2002) and otherwise it is comparable. Our theoretical analysis is confirmed empirically on a synthetic example and several UCI datasets. The PACBayesEmpiricalBernstein inequality is an interesting example of an application of the PACBayesian bounding technique to selfbounding functions. 1
Machine Learning manuscript No. (will be inserted by the editor) Clustering: Science or Art? ⋆
"... Abstract We examine whether the quality of different clustering algorithms can be compared by a general, scientifically sound procedure which is independent of particular clustering algorithms. We argue that the major obstacle is the difficulty in evaluating a clustering algorithm without taking int ..."
Abstract
 Add to MetaCart
Abstract We examine whether the quality of different clustering algorithms can be compared by a general, scientifically sound procedure which is independent of particular clustering algorithms. We argue that the major obstacle is the difficulty in evaluating a clustering algorithm without taking into account the context: why does the user cluster his data in the first place, and what does he want to do with the clustering afterwards? We argue that clustering should not be treated as an applicationindependent mathematical problem, but should always be studied in the context of its enduse. Different techniques to evaluate clustering algorithms have to be developed for different uses of clustering. To simplify this procedure we argue that it will be useful to build a “taxonomy of clustering problems ” to identify clustering applications which can be treated in a unified way and that such an effort will be more fruitful than attempting the impossible — developing “optimal” domainindependent clustering algorithms or even classifying clustering algorithms in terms of how they work.