In this paper we introduce part of a formal framework for Learning Classifier Systems (LCS) which, as a whole, aims at incorporating all components of LCS: function approximation, reinforcement learning and classifier replacement. The part introduced here concerns function approximation, and provides a formal problem definition, a formalisation of the LCS function approximation architecture, and a definition of the approximation aim. Additionally, we provide definitions of optimality and what conditions need to be fulfilled for a classifier to be optimal. Furthermore, as a demonstration of the usefulness of the framework, we derive commonly used algorithmic approaches that aim at reaching optimality from first principles, and introduce a new Kalman filter-based method that outperforms all currently implemented methods. How to mix classifiers to reach an overall approximation is simplified when compared to current LCS, and is justified by the Maximum Likelihood Estimate of a combination of all classifiers.

This paper is focused on studying the initialization stage of learning classifier systems (LCS) applying the Pittsburgh approach. It has a theoretical part where the covering probability of a random rule set is modelled and a practical part. The practical part has the objective of developing general initialization policies that have competent performance on a broad range of datasets. Two kinds of policies are tested: (1) ways of tuning the initialization probability of the system and (2) smart initialization operators that create rules that are generalized versions of randomly sampled training instances. The results identify a subset of settings that are robust enough to be considered candidates to be the default initialization policy. These settings have competent performance compared to several alternative machine learning systems. Beside identifying the good policies, the experimentation made is also useful to give hints about what kind of initial solutions is the system able to process successfully to create well generalized solutions

Abstract not found

Learning concept descriptions from data is a complex multiobjective task. The model induced by the learner should be accurate so that it can represent precisely the data instances, complete, which means it can be generalizable to new instances, and minimum, or easily readable. Learning Classifier Systems (LCSs) are a family or learners whose primary search mechanism is a genetic algorithm. Along the intense history of the field, the efforts of the community have been centered on the design of LCSs that solved these goals efficiently, resulting in the proposal of multiple systems. This paper revises the main LCS approaches and focuses on the analysis of the different mechanisms designed to fulfill the learning goals. Some of these mechanisms include implicit multiobjective learning mechanisms, while others use explicit multiobjective evolutionary algorithms. The paper analyses the advantages of using multiobjective evolutionary algorithms, especially in Pittsburgh LCSs, such as controlling the so-called bloat effect, and offering the human expert a set of concept description alternatives. 1 A Multiobjective Motivation Classification is a central task in data mining and machine learning applications. It consists

Michigan-style learning classifier systems (LCSs), such as the accuracy-based XCS system, evolve distributed problem solutions represented by a population of rules. Recently, it was shown that decomposable problems may require effective processing of subsets of problem attributes, which cannot be generally assured with standard crossover operators. A number of competent crossover operators capable of effective identification and processing of arbitrary subsets of variables or string positions were proposed for genetic and evolutionary algorithms. This paper effectively introduces two competent crossover operators to XCS by incorporating techniques from competent genetic algorithms (GAs): the extended compact GA (ECGA) and the Bayesian optimization algorithm (BOA). Instead of applying standard crossover operators, here a probabilistic model of the global population is built and sampled to generate offspring classifiers locally. Various offspring generation methods are introduced and evaluated. Results indicate that the performance of the proposed learning classifier systems XCS/ECGA and XCS/BOA is similar to that of XCS with informed crossover operators that is given all information about problem structure on input and exploits this knowledge using problemspecific crossover operators.

UCS is a supervised learning classifier system that was introduced in 2003 for classification in data mining tasks. The representation of a rule in UCS as a univariate classification rule is straightforward for a human to understand. However, the system may require a large number of rules to cover the input space. Artificial neural networks, on the other hand, normally provide a more compact representation. However, it is not a straightforward task to understand the network. In this paper, we propose a novel way to incorporate neural networks into UCS. The approach offers a good compromise between compactness, expressiveness, and accuracy. By using a simple artificial neural network as the classifier’s action, we obtain a more compact population size, better generalization, and the same or better accuracy, while maintaining a reasonable level of expressiveness. We also apply negative correlation learning (NCL) during the training of the resultant neural network ensemble. NCL is shown to improve the generalization of the ensemble. I.

In this paper we empirically evaluate several local search (LS) mechanisms that heuristically edit classification rules and rule sets to improve their performance. Two kinds of operators are studied, (1) rule-wise operators, that edit individual rules, and (2) a rule set-wise operator, which takes the rules from N parents (N ≥ 2) to generate a new offspring, selecting the minimum subset of candidate rules that obtains maximum training accuracy. Moreover, various ways of integrating these operators within the evolutionary cycle of Learning Classifier Systems are studied. The combinations of LS operators and policies are integrated in a Pittsburgh approach framework that we call MPLCS for Memetic Pittsburgh Learning Classifier System. MPLCS is systematically evaluated using various metrics. Several datasets were employed with the objective of identifying which combination of operators and policies scale well, are robust to noise, generate compact solutions and use the least amount of computational resources to solve the problems.

Abstract — In this paper the recently introduced evolving fuzzy classifier method called eClass is studied in respect to its architecture and evolution of the fuzzy rule-base. The proposed classifier has an open/evolving structure and can start ‘from scratch’, learning and adapting to the new data samples. Alternatively, if an initial fuzzy rule-based classifier, generated beforehand in off-line mode or provided by the operator, exists then eClass can evolve this initial classifier in on-line mode. In other words, the fuzzy rule base will evolve incorporating new rules, modifying and/or, possibly, removing some of the previously existing ones. Additionally, the parameters of both, the antecedent and the consequent parts are adapted. Note that eClass can start with an empty rule-base, which is a unique feature of this approach. The proposed approach is free from user-specified parameters and the mechanism of forming new rules is very robust. In this paper, four different modelling architectures are described and compared. The architectures are based on i) unsupervised cluster partitions, eClassC; ii) Sugeno fuzzy models with singleton consequents, eClassA; iii) Takagi-Sugeno fuzzy models with linear consequent functions, eClassB; and iv) a multi-model classification architecture, where separate TS regression models are combined to form an overall classification output of the system, eClassM. A thorough comparison of the results when applying each of these architectures and the results using previously existing classifiers has been made using an online interactive self-adaptive image classification framework.

This report gives an overview of the XCSFJava 1.1 code, available from the web. The document specifies where to get the code and how to compile and run the code. Moreover, the document specifies the features of the code. In short, XCSFJava 1.1 is an XCSF classifier system implementation that can be used at least for the following purposes: (1) testing and evaluating XCSF on various function approximation problems, including binary and realvalued problems, (2) visualizing the evolutionary process in XCSF in 2D or 3D, (3) enhancing XCSF’s capabilities, such as adding new types of classifier conditions or predictions.

©2010 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any