Color histograms are used to compare images in many applications. Their advantages are efficiency, and insensitivity to small changes in camera viewpoint. However, color histograms lack spatial information, so images with very di#erent appearances can have similar histograms. For example, a picture of fall foliage might contain a large number of scattered red pixels

Designing computer programs to automatically categorize images using low-level features is a challenging research topic in computer vision. In this paper, we present a new learning technique, which extends Multiple-Instance Learning (MIL), and its application to the problem of region-based image categorization. Images are viewed as bags, each of which contains a number of instances corresponding to regions obtained from image segmentation. The standard MIL problem assumes that a bag is labeled positive if at least one of its instances is positive; otherwise, the bag is negative.

This paper describes results from an ongoing project concerned with recognizing objects in complex scene domains, and especially in the domain that includes the natural outdoor world. Traditional machine recognition paradigms assume either (1) that all objects of interest are definable by a relatively small number of explicit shape models, or (2) that all objects of interest have characteristic, locally measurable features. The failure of both assumptions in a complex domain such as the natural outdoor world has a dramatic impact on the form of an acceptable architecture for an object recognition system. In our work, we make the use of contextual information a central issue, and explicitly design a system to identify and use context as an integral part of recognition. In so doing, we provide a new paradigm for visual recognition that eliminates the traditional dependence on stored geometric models and universal image partitioning algorithms. This paradigm combines the results of many s...

AbstractÐA method for deformable shape detection and recognition is described. Deformable shape templates are used to partition the image into a globally consistent interpretation, determined in part by the minimum description length principle. Statistical shape models enforce the prior probabilities on global, parametric deformations for each object class. Once trained, the system autonomously segments deformed shapes from the background, while not merging them with adjacent objects or shadows. The formulation can be used to group image regions obtained via any region segmentation algorithm, e.g., texture, color, or motion. The recovered shape models can be used directly in object recognition. Experiments with color imagery are reported. Index TermsÐImage segmentation, region merging, object detection and recognition, deformable templates, nonrigid shape models, statistical shape models. 1

The knowledge-directed approach to image interpretation, popular in the 1980's, sought to identify objects in unconstrained two-dimensional images and to determine the threedimensional relationships between these objects and the camera by applying large amounts of object- and domain-specific knowledge to the interpretation problem. Among the primary issues faced by these systems were variations among instances of an object class and differences in how object classes were defined in terms of shape, color, function, texture, size, and/or substructures. This paper

In this paper we discuss our use of multi-modal input to improve human computer interaction. Specifically we look at the methods used in the Intelligent Classroom to combine multiple input modes, and examine in particular the visual input modes. The Classroom provides context that improves the functioning of the visual input modes. It also determines which visual input modes are needed when. We examine a number of visual input modes to see how they fit into the general scheme, and look at how the Classroom controls their operation.

DATA REPROCESSING IN SIGNAL UNDERSTANDING SYSTEMS SEPTEMBER 1996 FRANK I. KLASSNER, III B.S., UNIVERSITY OF SCRANTON M.S., UNIVERSITY OF MASSACHUSETTS AMHERST Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Victor R. Lesser Signal understanding systems have the difficult task of interpreting environmental signals: decomposing them and explaining their components in terms of an arbitrary number of instances of perceptual object categories whose properties can interact with one another. This dissertation addresses the problem of designing blackboard-based perceptual systems for interpreting signals from complex environments. A "complex environment" is one that can (1) produce signal-to-noise ratios that vary unpredictably over time, and (2) can contain perceptual objects that mutually interfere with each others' signal signature, or have arbitrary time-dependent behaviors. The traditional design paradigm for perceptual systems assumes that some particular set of ...

Contextual information can facilitate automatic extraction of objects from digital imagery. This paper addresses the use of context for the automatic extraction of roads from aerial imagery. Context is restricted to knowledge about relations between roads and other objects and is hierarchically structured. More specific, context is used to guide road extraction on a global and on a local level. On a global level it is used to emphasize characteristic parts of the road model (context regions). On a local level it initiates contextual reasoning (context sketches). 1 Introduction The automatic extraction of objects from digital imagery is a very complex task. It is widely accepted that the complexity can be reduced by integrating context information into the extraction process, e.g. (Strat 1992). Generally speaking, context means that there exists knowledge not only about the object of interest but also about other relevant facts and their relations to the object of interest. Apart from ...

In this paper we describe a new multi resolution approach to automatic road extraction from aerial images. We make use of the fact that different characteristics of objects such as roads can be best detected in different scales. Two different resolutions of the same image are used, a coarse one with 2 m per pixel, and a fine one with 0.25 m per pixel. In the coarse resolution roads are modeled as bright lines and are extracted by a combination of local and global thresholding. In the fine resolution roads are assumed to have two parallel edges, be bright, and have a homogeneous texture. A multi step procedure has been designed to find the roads according to these criteria. Subsequently both outputs are merged using a rule based system. The developed method has been tested on real imagery, and some preliminary results are reported. Based on the existing experience the multi resolution approach is claimed to be superior to a road extraction in one resolution only. 1. INTRODUCTION One ...

Contextual information is often essential for visual recognition, but the design of image-understanding systems that effectively use context has remained elusive. We describe some of our experiences in attempting to employ contextual information in computer vision systems. By making explicit the built-in assumptions inherent in all computer vision algorithms, an architecture can be designed in which context can influence the recognition process. This paper describes such an architecture for context-based vision (CBV). 1 Introduction It is generally accepted that the surroundings of an object may have a profound influence on, and in some cases, may be necessary for, visual recognition of the object. What is not so well established is how to design computer vision systems that can exploit such contextual information. When a human observes a scene, or even studies a photograph, he normally has at his disposal a wealth of information that is not captured by the image alone. For example, i...