. On-line learning in domains where the target concept depends on some hidden context poses serious problems. A changing context can induce changes in the target concepts, producing what is known as concept drift. We describe a family of learning algorithms that flexibly react to concept drift and can take advantage of situations where contexts reappear. The general approach underlying all these algorithms consists of (1) keeping only a window of currently trusted examples and hypotheses; (2) storing concept descriptions and re-using them when a previous context reappears; and (3) controlling both of these functions by a heuristic that constantly monitors the system's behavior. The paper reports on experiments that test the systems' performance under various conditions such as different levels of noise and different extent and rate of concept drift. Keywords: Incremental concept learning, on-line learning, context dependence, concept drift, forgetting 1. Introduction The work presen...

The paper presents the system FRANN, which exploits the idea of radial-basis functions for the needs of learning in numeric domains under concept drift. The classification accuracy of the program compares favourably to that of older algorithms that are based on symbol manipulation. The system tolerates noise and is able to learn symbolic, numeric, and mixed concepts with nonlinear boundaries in environments with abrupt as well as gradual concept drift. Research area. Inductive learning Key words. concept drift, radial-basis functions Demo request. No Address for Correspondence: Miroslav Kubat, Institute for Systems Sciences, Johannes Kepler University, A-4040 Linz, Austria, e-mail: mirek@cast.uni-linz.ac.at 1 Introduction Recently, the problem of on-line learning in time-varying domains has received attention in the machine learning community. The essence is to make the learner recognize gradual or abrupt changes in the target concept and adjust accordingly the internal representa...

In this paper, we review five heuristic strategies for handling context-sensitive features in supervised machine learning from examples. We discuss two methods for recovering lost (implicit) contextual information. We mention some evidence that hybrid strategies can have a synergetic effect. We then show how the work of several machine learning researchers fits into this framework. While we do not claim that these strategies exhaust the possibilities, it appears that the framework includes all of the techniques that can be found in the published literature on context-sensitive learning.

Daily experience shows that in the real world, the meaning of many concepts heavily depends on some implicit context, and changes in that context can cause more or less radical changes in the concepts. Incremental concept learning in such domains requires the ability to recognize and adapt to such changes. This paper presents a solution for incremental learning tasks where the domain provides explicit clues as to the current context (e.g., attributes with characteristic values). We present a general two-level learning model, and its realization in a system named MetaL(B), that can learn to detect certain types of contextual clues, and can react accordingly when a context change is suspected. The model consists of a base level learner that performs the regular on-line learning and classification task, and a meta-learner that identifies potential contextual clues. Context learning and detection occur during regular on-line learning, without separate training phases for context recogniti...

The paper deals with incremental concept learning from classified examples. In many real-world applications, the target concepts of interest may change over time, and incremental learners should be able to track such changes and adapt to them. The problem is known in the literature as concept drift. The paper presents a new method for learning in such changing environments. In particular, it addresses the problem of learning drifting concepts from noisy data. We present an algorithm that is both robust against noise and quick at recognizing and adapting to changes in the target concepts. The method has been implemented in a system named FLORA4, the latest member of a whole family of learning algorithms. Experiments demonstrate significant improvement over previous results, both in noise-free and noisy situations. 1 Introduction In many real-world domains, the context on which some concepts of interest depend may change, resulting in more or less abrupt and radical changes in the defin...

In this paper we present a multiple window incremental learning algorithm that distinguishes between virtual concept drift and real concept drift. The algorithm is unsupervised and uses a novel approach to tracking concept drift that involves the use of competing windows to interpret the data. Unlike previous methods which use a single window to determine the drift in the data, our algorithm uses three windows of different sizes to estimate the change in the data. The advantage of this approach is that it allows the system to progressively adapt and predict the change thus enabling it to deal more effectively with different types of drift. We give a detailed description of the algorithm and present the results obtained from its application to two real world problems: computing the background image and sound recognition. We also compare its performance with FLORA, an existing concept drift tracking algorithm. 1

Abstract. The paper focuses on a difficult problem when formalizing knowledge: What about the possible concepts that didn’t make it into the formalization? We call such concepts the unconsidered context of the formalized knowledge and argue that erroneous and inadequate behavior of systems based on formalized knowledge can be attributed to different states of the unconsidered context; either while formalizing or during application of the formalization. We then propose an automatic strategy to identify different states of unconsidered context inside a given formalization and to classify which parts of the formalization to use in a given application situation. The goal of this work is to uncover unconsidered context by observing sucess and failure of a given system in use. The paper closes with the evaluation of the proposed procedures in an error diagnosis scenario featuring a plan based user interface.

de l'information

In data mining, concept drift refers to the phenomenon that the underlying model (or concept) is changing over time. The aim of this paper is twofold. First, we propose a fundamental characterization and quantification of different types of concept drift. The proposed theory enables a rigorous investigation of learning system performance on streamed data. In particular, we investigate the impact of different amounts and types of concept drift on evolutionary classification systems focusing on the learning classifier system approach. We compare performance of one Pittsburgh-type system, GAssist, which learns in batch mode using windowing techniques, with a Michigan-type system, XCS, which is a natural online learner. The results show that both systems are able to handle the various concept drifts well. Behavioral differences are discussed revealing task dependencies, representation dependencies as well as dynamics dependencies. Discussions and conclusions outline the path towards more detailed measures for problem dynamics in the data mining realm. 1

A large body of research in machine learning is concerned with supervised learning from examples. The examples are typically represented as vectors in a multi-dimensional feature space (also known as attribute-value descriptions). A teacher partitions a set of training examples into a finite number of classes. The task of the learning algorithm is to induce a concept from the training examples. In this paper, we formally distinguish three types of features: primary, contextual, and irrelevant features. We also formally define what it means for one feature to be context-sensitive to another feature. Context-sensitive features complicate the task of the learner and potentially impair the learner's performance. Our formal definitions make it possible for a learner to automatically identify context-sensitive features. After context-sensitive features have been identified, there are several strategies that the learner can employ for managing the features; however, a disc...