A sensible use of classifiers must be based on the estimated reliability of their predictions. A cautious classifier would delegate the difficult or uncertain predictions to other, possibly more specialised, classifiers. In this paper we analyse and develop this idea of delegating classifiers in a systematic way. First, we design a two-step scenario where a first classifier chooses which examples to classify and delegates the difficult examples to train a second classifier. Secondly, we present an iterated scenario involving an arbitrary number of chained classifiers. We compare these scenarios to classical ensemble methods, such as bagging and boosting. We show experimentally that our approach is not far behind these methods in terms of accuracy, but with several advantages: (i) improved efficiency, since each classifier learns from fewer examples than the previous one; (ii) improved comprehensibility, since each classification derives from a single classifier; and (iii) the possibility to simplify the overall multiclassifier by removing the parts that lead to delegation.

Abstract — Committees of classification and approximation models are used to improve accuracy and decrease the variance of individual models. Each model has an equal right to vote (democratic procedure), despite obvious differences in model competence in different regions of the feature space. Adding competence factors to different models before calculation of the committee decision (undemocratic procedure) improves the quality of the committee. A method for creation of a committee of competent models is described and several real-life empirical tests performed. Significant improvement of results is observed. I.

Committees of models are frequently employed to improve accuracy and decrease the variance of individual models. Each model has an equal right to vote (democratic procedure), despite obvious differences in model competence in different regions of the feature space. Adding competence factors to different models before calculation of the committee decision (undemocratic procedure) improves the quality of the committee. A method for creation of a committee of competent models is described and empirical tests presented.

In the past decades Machine Learning tools have been successfully used in 11 several medical diagnostic problems. While they often significantly out- 12 perform expert physicians (in terms of diagnostic accuracy, sensitivity, and 13 specificity), they are mostly not being used in practice. One reason for this 14 is that it is difficult to obtain an unbiased estimation of diagnose's reliabil- 15 ity. We discuss how reliability of diagnoses is assessed in medical decision 16 making and propose a general framework for reliability estimation in Ma- 17 chine Learning, based on transductive inference. We compare our approach 18 with a usual (Machine Learning) probabilistic approach as well as with clas- 19 sical stepwise diagnostic process where reliability of diagnose is presented 20 as its post-test probability. The proposed transductive approach is evaluated 21 on several medical datasets from the UCI (University of California, Irvine) 22 repository as well as on a practical problem of clinical diagnosis of the coro- 23 nary artery disease. In all cases significant improvements over existing tech- 24 niques are achieved. 25 26 27 Keywords: transduction, machine learning, medical diagnosis, reliability 28 estimation, coronary artery disease. 29 1

Abstract – The paper scope is multi-agent technology and software tool for the joint engineering, implementation, deployment and, possibly, use of applied multi-agent distributed data mining and distributed decision making systems. The core problem of distributed data mining and decision making technology does not concern s particular data mining techniques, because the respective library of classes can be extended when necessary. Instead of this, its core problem is development of an infrastructure and protocols supporting coherent collaborative work of distributed software components (agents) responsible for data mining and decision making. The paper is focused on architecture of multi-agent distributed data mining and decision making system, on its design technology, software tool and on the protocols of software tool agents ' interaction, mainly, in distributed data mining and decision making processes. The presented software tool is implemented and validated on the basis of several case studies from data fusion scope. 1.

Recently, mining from data streams has become an important and challenging task for many real-world applications such as credit card fraud protection and sensor networking. One popular solution is to separate stream data into chunks, learn a base classifier from each chunk, and then integrate all base classifiers for effective classification. In this paper, we propose a new dynamic classifier selection (DCS) mechanism to integrate base classifiers for effective mining from data streams. The proposed algorithm dynamically selects a single “best” classifier to classify each test instance at run time. Our scheme uses statistical information from attribute values, and uses each attribute to partition the evaluation set into disjoint subsets, followed by a procedure that evaluates the classification accuracy of each base classifier on these subsets. Given a test instance, its attribute values determine the subsets that the similar instances in the evaluation set have constructed, and the classifier with the highest classification accuracy on those subsets is selected to classify the test instance. Experimental results and comparative studies demonstrate the efficiency and efficacy of our method. Such a DCS scheme appears to be promising in mining data streams with dramatic concept drifting or with a significant amount of noise, where the base classifiers are likely conflictive or have low confidence. 1.

The experience proved that integration of multi-agent and data mining technologies provides mutual enrichment of them both and leads to emergence of new design technologies and systems with better properties. However, this integration put forward novel problems of different kinds. The paper considers agent-based distributed design technology of distributed classification systems and presents basic components of the software infrastructure intended for support of the aforementioned design technology. The paper also analyzes associated problems and outlines some basic solutions proposed. 1.

Abstract. We present three new voting schemes for multi-classifier biometric authentication using a reliability model to influence the importance of each base classifier’s vote. The reliability model is a metaclassifier computing the probability of a correct decision for the base classifiers. It uses two features which do not depend directly on the underlying physical signal properties, verification score and difference between user-specific and user-independent decision threshold. It is shown on two signature databases and two speaker databases that this reliability classification can systematically reduce the number of errors compared to the base classifier. Fusion experiments on the signature databases show that all three voting methods (rigged majority voting, weighted rigged majority voting, and selective rigged majority voting) perform significantly better than majority voting, and that given sufficient training data, they also perform significantly better than the best classifier in the ensemble. 1

The classification performance of an associative classifier is strongly dependent on the statistic measure or metric that is used to quantify the strength of the association between features and classes (i.e., confidence, correlation etc.). Previous studies have shown that classifiers produced by different metrics may provide conflicting predictions, and that the best metric to use is data-dependent and rarely known while designing the classifier. This uncertainty concerning the optimal match between metrics and problems is a dilemma, and prevents associative classifiers to achieve their maximal performance. This dilemma is the focus of this paper. A possible solution to this dilemma is to learn the competence, expertise, or assertiveness of metrics. The basic idea is that each metric has a specific sub-domain for

The classification performance of an associative classifier is strongly dependent on the statistic measure or metric that is used to quantify the strength of the association between features and classes (i.e., confidence, correlation etc.). Previous studies have shown that classifiers produced by different metrics may provide conflicting predictions, and that the best metric to use is data-dependent and rarely known while designing the classifier. This uncertainty concerning the optimal match between metrics and problems is a dilemma, and prevents associative classifiers to achieve their maximal performance. This dilemma is the focus of this paper. A possible solution to this dilemma is to learn the competence, expertise, or assertiveness of metrics. The basic idea is that each metric has a specific sub-domain for