This paper describes an efficient control mechanism for incorporating picture-specific context in the task of image interpretation. Although other knowledge-based vision systems use general domain context in reducing the computational burden of image interpretation, to our knowledge, this is the first effort in exploring picture-specific collateral information. We assume that constraints on the picture are generated from a natural language understanding module which processes descriptive text accompanying the pictures. We have developed a unified framework for exploiting these constraints both in the object location and identification (labeling) stage. In particular, we describe a technique for incorporating constrained search in context-based vision. Finally, we demonstrate the effectiveness of this approach in PICTION, a system that uses captions to label human faces in newspaper photographs. 1 Introduction To solve the inherently under-constrained task of image ...

In this paper, we propose a probabilistic region classification scheme for natural scene images. In conventional generative methods, a generative model is learnt for each class using all the available training data belonging to that class. However, if an input image has been generated from only a subset of the model support, use of the full model to assign generative probabilities can produce serious artifacts in the probability assignments. This problem arises mainly when the different classes have multimodal distributions with considerable overlap in the feature space. We propose an approach to constrain the class generative probability of a set of newly observed data by exploiting the distribution of the new data itself and using linear weighted mixing. A Kullback -- Leibler Divergence-based fast model selection procedure is also proposed for learning mixture models in a low dimensional feature space. The preliminary results on the natural scene images support the effectiveness of the proposed approach.

This paper presents a generic cognitive vision platform for the automatic recognition of natural complex objects. The recognition consists of three steps : image processing for numerical object description, mapping of numerical data into symbolic data and semantic interpretation for object recognition. The focus of this paper is the distributed platform architecture composed of three highly specialized Knowledge Based Systems (KBS). The first KBS is dedicated to semantic interpretation. The second one has to deal with the anchoring of symbolic data into image data. The last KBS is dedicated to intelligent image processing. After a brief overview of the natural object recognition problem, this paper describes the three subcomponents of the platform. Keywords : Cognitive Vision, Natural Object Recognition, Knowledge Based System 1.

This thesis is available for Library use on the understanding that it is copyright material and that no quotation from this thesis may be published without proper acknowledgement. I certify that all material in this thesis which is not my own work has been identified and that no material has previously been submitted and approved for the award of this degree by this or any other university.

Our research explores the interaction of textual and photographic information in document understanding. Specifically, we have been working on a computational model whereby textual captions are used as collateral information in the interpretation of the corresponding photographs. The final understanding of the picture and caption reflects a consolidation of the information obtained from each of the two sources and can thus be used in intelligent information retrieval tasks. The problem of performing general-purpose vision without a priori knowledge is very difficult at best. The concept of using collateral information in scene understanding has been explored in systems that use general scene context in the task of object identification. The work described here extends this notion by incorporating picture specific information. A multi-stage system PICTION which uses captions to identify humans in an accompanying photograph is described. 1 Introduction Our research has focused on devel...

Scene analysis is an important area of research with the aim of identifying objects and their relationships in natural scenes. MINERVA benchmark has been recently introduced in this area for testing different image processing and classification schemes. In this paper we present results on the classification of eight natural objects in the complete set of 448 natural images using neural networks. An exhaustive set of experiments with this benchmark has been conducted using four different segmentation methods and five texture-based feature extraction methods. The results in this paper show the performance of a neural network classifier on a ten fold cross-validation task. On the basis of the results produced, we are able to rank how well different image segmentation algorithms are suited to the task of region of interest identification in these images, and we also see how well texture extraction algorithms rank on the basis of classification results.

iii Preface Needless to say, this work would not have been possible without the continuing support of Robert Hummel and Benjamin Goldberg. To them goes my deepest gratitude. iv Table of Contents Acknowledgements............................................................................. iii

In various application domains such as entertainment, biomedicine, commerce, education, and crime prevention, the volume of digital data archives is growing rapidly. The very large repository of digital information raises challenging problems in retrieval and various other information manipulation tasks. Content-based image retrieval (CBIR) is aimed at efficient retrieval of relevant images from large image databases based on automatically derived imagery features. However, images with high feature similarities to the query image may be very different from the query in terms of semantics. This discrepancy between low-level content features (such as color, texture, and shape) and high-level semantic concepts (such as sunset, flowers, outdoor scene, etc.) is known as “semantic gap, ” which is an open challenging problem in current CBIR systems. With the ultimate goal of narrowing the semantic gap, this thesis makes three contributions to the field of CBIR. The first contribution is a novel region-based im-age similarity measure. An image is represented by a set of segmented regions each of which is characterized by a fuzzy feature (fuzzy set) reflecting color, texture, and

Current learning approaches to computer vision have mainly focussed on low-level image processing and object recognition, while tending to ignore higher level processing for understanding. We propose an approach to scene analysis that facilitates the transition from recognition to understanding. It begins by segmenting the image into regions using standard approaches, which are then classified using a discovered fuzzy Cartesian granule feature classifier. Understanding is made possible through the transparent and succinct nature of the discovered models. The recognition of roads in images is taken as an illustrative problem. The discovered fuzzy models while providing high levels of accuracy (97%), also provide understanding of the problem domain through the transparency of the learnt models. The learning step in the proposed approach is compared with other techniques such as decision trees, nave Bayes and neural networks using a variety of performance criteria such as accuracy, unders...

Das Werk einschließlich aller seiner Teile ist urheberrechtlich geschützt. Jede Verwertung außerhalb der engen Grenzen des Urheberrechtsgesetzes ist ohne Zustimmung des Verlages unzulässig und strafbar. Dies gilt insbesondere für Vervielfältigungen, Übersetzungen, Mikroverfilmungen und elektronische Speicherformen sowie die Einspeicherung und Verarbeitung in elektronischen Systemen.