Results 1  10
of
24
On the Learnability and Design of Output Codes for Multiclass Problems
 In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory
, 2000
"... . Output coding is a general framework for solving multiclass categorization problems. Previous research on output codes has focused on building multiclass machines given predefined output codes. In this paper we discuss for the first time the problem of designing output codes for multiclass problem ..."
Abstract

Cited by 176 (5 self)
 Add to MetaCart
. Output coding is a general framework for solving multiclass categorization problems. Previous research on output codes has focused on building multiclass machines given predefined output codes. In this paper we discuss for the first time the problem of designing output codes for multiclass problems. For the design problem of discrete codes, which have been used extensively in previous works, we present mostly negative results. We then introduce the notion of continuous codes and cast the design problem of continuous codes as a constrained optimization problem. We describe three optimization problems corresponding to three different norms of the code matrix. Interestingly, for the l 2 norm our formalism results in a quadratic program whose dual does not depend on the length of the code. A special case of our formalism provides a multiclass scheme for building support vector machines which can be solved efficiently. We give a time and space efficient algorithm for solving the quadratic program. We describe preliminary experiments with synthetic data show that our algorithm is often two orders of magnitude faster than standard quadratic programming packages. We conclude with the generalization properties of the algorithm. Keywords: Multiclass categorization,output coding, SVM 1.
Machine learning for sequential data: A review
 Structural, Syntactic, and Statistical Pattern Recognition
, 2002
"... Abstract. Statistical learning problems in many fields involve sequential data. This paper formalizes the principal learning tasks and describes the methods that have been developed within the machine learning research community for addressing these problems. These methods include sliding window met ..."
Abstract

Cited by 90 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Statistical learning problems in many fields involve sequential data. This paper formalizes the principal learning tasks and describes the methods that have been developed within the machine learning research community for addressing these problems. These methods include sliding window methods, recurrent sliding windows, hidden Markov models, conditional random fields, and graph transformer networks. The paper also discusses some open research issues. 1
ErrorCorrecting Output Coding for Text Classification
, 1999
"... This paper applies errorcorrecting output coding (ECOC) to the task of document categorization. ECOC, of recent vintage in the AI literature, is a method for decomposing a multiway classification problem into many binary classification tasks, and then combining the results of the subtasks int ..."
Abstract

Cited by 58 (0 self)
 Add to MetaCart
This paper applies errorcorrecting output coding (ECOC) to the task of document categorization. ECOC, of recent vintage in the AI literature, is a method for decomposing a multiway classification problem into many binary classification tasks, and then combining the results of the subtasks into a hypothesized solution to the original problem. There has been much recent interest in the machine learning community about algorithms which integrate "advice" from many subordinate predictors into a single classifier, and errorcorrecting output coding is one such technique. We provide experimental results on several realworld datasets, extracted from the Internet, which demonstrate that ECOC can o#er significant improvements in accuracy over conventional classification algorithms. 1
Bidirectional Conversion Between Graphemes and Phonemes Using a Joint Ngram Model
, 2001
"... We present in this paper a statistical model for languageindependent bidirectional conversion between spelling and pronunciation, based on joint grapheme/phoneme units 1 extracted from automatically aligned data. The model is evaluated on spellingtopronunciation and pronunciationtospelling conv ..."
Abstract

Cited by 23 (2 self)
 Add to MetaCart
We present in this paper a statistical model for languageindependent bidirectional conversion between spelling and pronunciation, based on joint grapheme/phoneme units 1 extracted from automatically aligned data. The model is evaluated on spellingtopronunciation and pronunciationtospelling conversion on the NetTalk database and the CMU dictionary. We also study the effect of including lexical stress in the pronunciation. Although a direct comparison is difficult to make, our model's performance appears to be as good or better than that of other datadriven approaches that have been applied to the same tasks. 1.
Improved output coding for classification using continuous relaxation
 In Advances in Neural Information Processing Systes 13 (NIPS*00
, 2001
"... Output coding is a general method for solving multiclass problems by reducing them to multiple binary classification problems. Previous research on output coding has employed, almost solely, predefined discrete codes. We describe an algorithm that improves the performance of output codes by relaxing ..."
Abstract

Cited by 10 (0 self)
 Add to MetaCart
(Show Context)
Output coding is a general method for solving multiclass problems by reducing them to multiple binary classification problems. Previous research on output coding has employed, almost solely, predefined discrete codes. We describe an algorithm that improves the performance of output codes by relaxing them to continuous codes. The relaxation procedure is cast as an optimization problem and is reminiscent of the quadratic program for support vector machines. We describe experiments with the proposed algorithm, comparing it to standard discrete output codes. The experimental results indicate that continuous relaxations of output codes often improve the generalization performance, especially for short codes. 1
Machine Learning and Natural Language Processing
, 2000
"... In this report, some collaborative work between the fields of Machine Learning (ML) and Natural Language Processing (NLP) is presented. The document is structured in two parts. The first part includes a superficial but comprehensive survey covering the stateoftheart of machine learning techniq ..."
Abstract

Cited by 6 (0 self)
 Add to MetaCart
In this report, some collaborative work between the fields of Machine Learning (ML) and Natural Language Processing (NLP) is presented. The document is structured in two parts. The first part includes a superficial but comprehensive survey covering the stateoftheart of machine learning techniques applied to natural language learning tasks. In the second part, a particular problem, namely Word Sense Disambiguation (WSD), is studied in more detail. In doing so, four algorithms for supervised learning, which belong to different families, are compared in a benchmark corpus for the WSD task. Both qualitative and quantitative conclusions are drawn. This document stands for the complementary documentation for the conference "Aprendizaje autom 'atico aplicado al procesamiento del lenguaje natural", given by the author within the course: "Curso de Industrias de la Lengua: La Ingenier'ia Lingu'istica en la Sociedad de la Informaci'on", Fundaci'on Duques de Soria. Soria. July 2000. 1 Con...
Upper bounds on the training error of ECOC SVM ensembles
, 2001
"... Error Correcting Output Coding Support Vector Machines (ECOCSVM) are ensemble of SVM based on ECOC decomposition methods. They exploit the accuracy of SVM dichotomizers and the error recovering capabilities of ECOC ensembles. ECOC decomposition is an Output Coding (OC) decomposition method for multi ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
Error Correcting Output Coding Support Vector Machines (ECOCSVM) are ensemble of SVM based on ECOC decomposition methods. They exploit the accuracy of SVM dichotomizers and the error recovering capabilities of ECOC ensembles. ECOC decomposition is an Output Coding (OC) decomposition method for multiclass classification: It generates ensembles of learning machines decomposing a polychotomic problem in a set of dichotomic subproblems that can be solved separately by different dichotomic learning machines. The tradeoff between error recovering capabilities and complexity of the dichotomies induced by the ECOC decomposition is an open problem in OC methods. Upper bounds on the training error of ECOCSVM ensembles, using margin based loss function derived from the optimization problem associated with the SVM learning algorithm, are presented. They highlight the relations between the error recovering power of ECOC methods and the complexity of the induced ECOC decomposition, separ...
The DivideandConquer Manifesto
 Proceedings of the Eleventh International Conference on Algorithmic Learning Theory
, 2000
"... . Existing machine learning theory and algorithms have focused on learning an unknown function from training examples, where the unknown function maps from a feature vector to one of a small number of classes. Emerging applications in science and industry require learning much more complex funct ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
. Existing machine learning theory and algorithms have focused on learning an unknown function from training examples, where the unknown function maps from a feature vector to one of a small number of classes. Emerging applications in science and industry require learning much more complex functions that map from complex input spaces (e.g., 2dimensional maps, time series, and strings) to complex output spaces (e.g., other 2dimensional maps, time series, and strings). Despite the lack of theory covering such cases, many practical systems have been built that work well in particular applications. These systems all employ some form of divideandconquer, where the inputs and outputs are divided into smaller pieces (e.g., "windows"), classified, and then the results are merged to produce an overall solution. This paper defines the problem of divideandconquer learning and identifies the key research questions that need to be studied in order to develop practical, generalp...