Results 1  10
of
41
New Support Vector Algorithms
, 2000
"... this article with the regression case. To explain this, we will introduce a suitable definition of a margin that is maximized in both cases ..."
Abstract

Cited by 351 (42 self)
 Add to MetaCart
this article with the regression case. To explain this, we will introduce a suitable definition of a margin that is maximized in both cases
The analysis of decomposition methods for support vector machines
 IEEE Transactions on Neural Networks
, 1999
"... Abstract. The decomposition method is currently one of the major methods for solving support vector machines. An important issue of this method is the selection of working sets. In this paper through the design of decomposition methods for boundconstrained SVM formulations we demonstrate that the w ..."
Abstract

Cited by 116 (19 self)
 Add to MetaCart
Abstract. The decomposition method is currently one of the major methods for solving support vector machines. An important issue of this method is the selection of working sets. In this paper through the design of decomposition methods for boundconstrained SVM formulations we demonstrate that the working set selection is not a trivial task. Then from the experimental analysis we propose a simple selection of the working set which leads to faster convergences for difficult cases. Numerical experiments on different types of problems are conducted to demonstrate the viability of the proposed method.
TreeBased Reparameterization Framework for Analysis of Belief Propagation and Related Algorithms
, 2001
"... We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and ext ..."
Abstract

Cited by 101 (21 self)
 Add to MetaCart
We present a treebased reparameterization framework that provides a new conceptual view of a large class of algorithms for computing approximate marginals in graphs with cycles. This class includes the belief propagation or sumproduct algorithm [39, 36], as well as a rich set of variations and extensions of belief propagation. Algorithms in this class can be formulated as a sequence of reparameterization updates, each of which entails refactorizing a portion of the distribution corresponding to an acyclic subgraph (i.e., a tree). The ultimate goal is to obtain an alternative but equivalent factorization using functions that represent (exact or approximate) marginal distributions on cliques of the graph. Our framework highlights an important property of BP and the entire class of reparameterization algorithms: the distribution on the full graph is not changed. The perspective of treebased updates gives rise to a simple and intuitive characterization of the fixed points in terms of tree consistency. We develop interpretations of these results in terms of information geometry. The invariance of the distribution, in conjunction with the fixed point characterization, enables us to derive an exact relation between the exact marginals on an arbitrary graph with cycles, and the approximations provided by belief propagation, and more broadly, any algorithm that minimizes the Bethe free energy. We also develop bounds on this approximation error, which illuminate the conditions that govern their accuracy. Finally, we show how the reparameterization perspective extends naturally to more structured approximations (e.g., Kikuchi and variants [52, 37]) that operate over higher order cliques.
Content based retrieval of VRML objects  an iterative and interactive approach
, 2001
"... We examine the problem of searching a database of threedimensional objects (given in VRML) for objects similar to a given object. We introduce an algorithm which is both iterative and interactive. Rather than base the search solely on geometric feature similarity, we propose letting the user influe ..."
Abstract

Cited by 99 (6 self)
 Add to MetaCart
(Show Context)
We examine the problem of searching a database of threedimensional objects (given in VRML) for objects similar to a given object. We introduce an algorithm which is both iterative and interactive. Rather than base the search solely on geometric feature similarity, we propose letting the user influence future search results by marking some of the results of the current search as `relevant' or `irrelevant', thus indicating personal preferences. A novel approach, based on SVM, is used for the adaptation of the distance measure consistently with these markings, which brings the `relevant' objects closer and pushes the `irrelevant' objects farther. We show that in practice very few iterations are needed for the system to converge well on what the user "had in mind".
Random Cascades on Wavelet Trees and Their Use in Analyzing and Modeling Natural Images
 Applied and Computational Harmonic Analysis
, 2001
"... in signal and image processing, including image denoising, coding, and superresolution. # 2001 Academic Press 1. INTRODUCTION Stochastic models of natural images underlie a variety of applications in image processing and lowlevel computer vision, including image coding, denoising and 1 MW supp ..."
Abstract

Cited by 89 (15 self)
 Add to MetaCart
in signal and image processing, including image denoising, coding, and superresolution. # 2001 Academic Press 1. INTRODUCTION Stochastic models of natural images underlie a variety of applications in image processing and lowlevel computer vision, including image coding, denoising and 1 MW supported by NSERC 1967 fellowship; AW and MW by AFOSR Grant F496209810349 and ONR Grant N0001491J1004. Address correspondence to MW. 2 ES supported by NSF Career Grant MIP9796040 and an Alfred P. Sloan fellowship. 89 10635203/01 $35.00 Copyright # 2001 by Academic Press All rights of reproduction in any form reserved. 90 WAINWRIGHT, SIMONCELLI, AND WILLSKY restoration, interpolation and synthesis. Accordingly, the past decade has witnessed an increasing amount of research devoted to developing stochastic models of images (e.g., [19, 38, 45, 48, 55]). Simultaneously, wavel
Treereweighted belief propagation algorithms and approximate ML estimation by pseudomoment matching
 In AISTATS
, 2003
"... In previous work [10], we presented a class of upper bounds on the log partition function of an arbitrary undirected graphical model based on solving a convex variational problem. Here we develop a class of local messagepassing algorithms, which we call treereweighted belief propagation, for ..."
Abstract

Cited by 55 (4 self)
 Add to MetaCart
(Show Context)
In previous work [10], we presented a class of upper bounds on the log partition function of an arbitrary undirected graphical model based on solving a convex variational problem. Here we develop a class of local messagepassing algorithms, which we call treereweighted belief propagation, for ef ciently computing the value of these upper bounds, as well as the associated pseudomarginals.
Learning and Value Function Approximation in Complex Decision Processes
, 1998
"... In principle, a wide variety of sequential decision problems  ranging from dynamic resource allocation in telecommunication networks to financial risk management  can be formulated in terms of stochastic control and solved by the algorithms of dynamic programming. Such algorithms compute and sto ..."
Abstract

Cited by 38 (4 self)
 Add to MetaCart
In principle, a wide variety of sequential decision problems  ranging from dynamic resource allocation in telecommunication networks to financial risk management  can be formulated in terms of stochastic control and solved by the algorithms of dynamic programming. Such algorithms compute and store a value function, which evaluates expected future reward as a function of current state. Unfortunately, exact computation of the value function typically requires time and storage that grow proportionately with the number of states, and consequently, the enormous state spaces that arise in practical applications render the algorithms intractable. In this thesis, we study tractable methods that approximate the value function. Our work builds on research in an area of artificial intelligence known as reinforcement learning. A point of focus of this thesis is temporaldifference learning  a stochastic algorithm inspired to some extent by phenomena observed in animal behavior. Given a selection of...
A theoretical and numerical comparison of some semismooth algorithms for complementarity problems
 Comput. Optim. Appl
"... Abstract: In this paper we introduce a general line search scheme which easily allows us to dene and analyze known and new semismooth algorithms for the solution of nonlinear complementarity problems. We enucleate the basic assumptions that a search direction to be used in the general scheme has to ..."
Abstract

Cited by 23 (3 self)
 Add to MetaCart
(Show Context)
Abstract: In this paper we introduce a general line search scheme which easily allows us to dene and analyze known and new semismooth algorithms for the solution of nonlinear complementarity problems. We enucleate the basic assumptions that a search direction to be used in the general scheme has to enjoy in order to guarantee global convergence, local superlinear/quadratic convergence or nite convergence. We examine in detail several dierent semismooth algorithms and compare their theoretical features and their practical behavior on a set of largescale problems.
Sparse Regression Ensembles in Infinite and Finite Hypothesis Spaces
, 2000
"... We examine methods for constructing regression ensembles based on a linear program (LP). The ensemble regression function consists of linear combina tions of base hypotheses generated by some boostingtype base learning algorithm. Unlike the classification case, for regression the set of possible h ..."
Abstract

Cited by 20 (9 self)
 Add to MetaCart
We examine methods for constructing regression ensembles based on a linear program (LP). The ensemble regression function consists of linear combina tions of base hypotheses generated by some boostingtype base learning algorithm. Unlike the classification case, for regression the set of possible hypotheses producible by the base learning algorithm may be infinite. We explicitly tackle the issue of how to define and solve ensemble regression when the hypothesis space is infinite. Our approach is based on a semiinfinite linear program that has an infinite number of constraints and a finite number of variables. We show that the regression problem is well posed for infinite hypothesis spaces in both the primal and dual spaces. Most importantly, we prove there exists an optimal solution to the infinite hypothesisspace problem consisting of a finite number of hypothesis. We propose two algorithms for solving the infinite and finite hypothesis problems. One uses a column generation simplextype algorithm and the other adopts an exponential barrier approach. Furthermore, we give sufficient conditions for the base learning algorithm and the hypothesis set to be used for infinite regression ensembles. Computational resultsshow that these methods are extremely promising.
Holonic Manufacturing Scheduling: Architecture, . . .
 COMPUTERS IN INDUSTRY
, 1998
"... A Holonic Manufacturing System HMS is a manufacturing system where key elements, such as machines, cells, factories, parts, products, operators, teams, etc., are modeled as `holons' having autonomous and cooperatie properties. The decentralized information structure, the distributed decision ..."
Abstract

Cited by 19 (1 self)
 Add to MetaCart
A Holonic Manufacturing System HMS is a manufacturing system where key elements, such as machines, cells, factories, parts, products, operators, teams, etc., are modeled as `holons' having autonomous and cooperatie properties. The decentralized information structure, the distributed decisionmaking authority, the integration of physical and informational aspects, and the cooperative relationship among holons, make the HMS a new paradigm, with great potential for meeting today's agile manufacturing challenges. Critical issues to be investigated include how to define holons for a given problem context, what should be the appropriate system architecture, and how to design effective cooperation mechanisms for good system performance. In this paper, holonic scheduling is developed for a factory consisting of multiple cells. Relevant holons are identified, and their relationships are delineated through a novel modeling of the interactions among parts, machines, and cells. The cooperation mechanisms among holons are established based on the pricing concept of market economy following `Lagrangian relaxation' of mathematical optimization, and cooperation across cells is performed without accessing individual cells' local information nor intruding on their decision authority. The system also possesses structural recursivity and extendibility. Numerical testing shows that the method can generate nearoptimal schedules with quantifiable quality in a timely fashion, and has comparable computational requirements and performance as compared to the centralized method following singlelevel Lagrangian relaxation. The method thus provides a theoretical foundation for guiding the cooperation among holons, leading to globally nearoptimal performance.