1 Language in the World How does language relate to the non-linguistic world? If an agent is able to communicate linguistically and is also able to directly perceive and/or act on the world, how do perception, action, and language interact with and influence each other? Such questions are surely amongst the most important in Cognitive Science and Artificial Intelligence (AI). Language, after all, is a central aspect of the human mind – indeed it may be what distinguishes us from other species. There is sometimes a tendency in the academic world to study language in isolation, as a formal system with rules for well-constructed sentences; or to focus on how language relates to formal notations such as symbolic logic. But language did not evolve as an isolated system or as a way of communicating symbolic logic; it presumably evolved as a mechanism for exchanging information about the world, ultimately providing the medium for cultural transmission across generations. Motivated by these observations, the goal of this special issue is to bring together research in AI that focuses on relating language to the physical world. Language is of course also used to communicate about non-physical referents, but the ubiquity of physical metaphor in language [21] suggests that grounding in the physical world provides the foundations of semantics.

Within a computer vision context color naming is the action of assigning linguistic color labels to image pixels. In general, research on color naming applies the following paradigm: a collection of color chips is labelled with color names within a well-defined experimental setup by multiple test subjects. The collected data set is subsequently used to label RGB values in real-world images with a color name. Apart from the fact that this collection process is time consuming, it is unclear to what extent color naming within a controlled setup is representative for color naming in realworld images. Therefore we propose to learn color names from real-world images. Furthermore, we avoid test subjects by using Google Image to collect a data set. Due to limitations of Google Image this data set contains a substantial quantity of wrongly labelled data. The color names are learned using a PLSA model adapted to this task. Experimental results show that color names learned from realworld images significantly outperform color names learned from labelled color chips on retrieval and classification. 1.

The accumulation of large collections of digital images has created the need for efficient and intelligent schemes for content-based image retrieval. Our goal is to organize the contents semantically, according to meaningful categories. We present a new approach for semantic classification that utilizes a recently proposed color-texture segmentation algorithm (by Chen et al.), which combines knowledge of human perception and signal characteristics to segment natural scenes into perceptually uniform regions. The color and texture features of these regions are used as medium level descriptors, based on which we extract semantic labels, first at the segment and then at the scene level. The segment features consist of spatial texture orientation information and color composition in terms of a limited number of locally adapted dominant colors. The focus of this paper is on region classification. We use a hierarchical vocabulary of segment labels that is consistent with those used in the NIST TRECVID 2003 development set. We test the approach on a database of 9000 segments obtained from 2500 photographs of natural scenes. For training and classification we use the Linear Discriminant Analysis (LDA) technique. We examine the performance of the algorithm (precision and recall rates) when different sets of features (e.g., one or two most dominant colors versus four quantized dominant colors) are used. Our results indicate that the proposed approach offers significant performance improvements over existing approaches.

Color names are required in real-world applications such as image retrieval and image annotation. Traditionally, they are learned from a collection of labelled color chips. These color chips are labelled with color names within a well-defined experimental setup by human test subjects. However naming colors in real-world images differs significantly from this experimental setting. In this paper, we investigate how color names learned from color chips compare to color names learned from real-world images. To avoid hand labelling real-world images with color names we use Google Image to collect a data set. Due to limitations of Google Image this data set contains a substantial quantity of wrongly labelled data. We propose several variants of the PLSA model to learn color names from this noisy data. Experimental results show that color names learned from real-world images significantly outperform color names learned from labelled color chips for both image retrieval and image annotation. I.

Recent developments in sensor technology have made it feasible to use mobile robots in several fields, but robots still lack the ability to accurately sense the environment. A major challenge to the widespread deployment of mobile robots is the ability to function autonomously, learning useful models of environmental features, recognizing environmental changes, and adapting the learned models in response to such changes. This article focuses on such learning and adaptation in the context of color segmentation on mobile robots in the presence of illumination changes. The main contribution of this article is a survey of vision algorithms that are potentially applicable to color-based mobile robot vision. We therefore look at algorithms for color segmentation, color learning and illumination invariance on mobile robot platforms, including approaches that tackle just the underlying vision problems. Furthermore, we investigate how the interdependencies between these modules and high-level action planning can be exploited to achieve autonomous learning and adaptation. The goal is to determine the suitability of the state-of-the-art vision algorithms for mobile robot domains, and to identify the challenges that still need to be addressed to enable mobile robots to learn and adapt models for color, so as to operate autonomously in natural conditions.

Dedicated to the children around the world. If only they could retain their inquisitiveness forever and continue to dream fearlessly... Acknowledgments I would like to thank Dr. Peter Stone and Dr. Benjamin Kuipers for guiding me throughout my thesis work. Peter was a great source of advice and encouragement, and gave me a lot of freedom in choosing my research focus. My discussions with Ben helped me put things in perspective and take a critical look at my research. Thanks are also due to my other committee members, Dr. Joydeep Ghosh, Dr. Alan Bovik and Dr. Gregory Dudek for their help and advice. Like any other doctoral dissertation, there were periods when the going was not smooth. I would like to take this opportunity to thank my cousin Suresh Venkat, his wife Karen and their son Aditya for being a constant source of support and cheer. Talking to them and visiting them helped me keep myself sane while still being focused on my research. This was all the more important considering my introversive nature and extended work hours. I was fortunate to work with Dr. Daniel Lee for a semester at the University of Pennsylvania. In addition to enjoying my stay there, I gained very valuable experience working with robots that operate outdoors. I would like to thank him for providing me with such an opportunity. Over the years I have interacted with several professors at conferences and on other occasions. Some of them have provided a lot of encouragement and helped me put my problems in better perspective. I would like to especially thank the following:

We present a study on how people use size modifiers when referring to visible objects. We find strong evidence that the selection of modifiers like tall, thin, and big is brought about by several interacting factors, including how a target object’s physical dimensions differ from another object of the same type, and the relationship between the target object’s individual dimensions. Findings from this study are used to inform the design of a referring expression generation algorithm capable of referring to objects naturally, providing a further link between visual cues and corresponding linguistic forms.

Within a computer vision context color naming is the action of assigning linguistic color labels to pixels, regions or objects in images. Humans use color names routinely and seemingly without effort to describe the world around us. They have been primarily studied in the fields of visual psychology, anthropology and linguistics [17]. Color names are for example used in the context of image retrieval. A user might query an image search engine for "red cars". The system recognizes the color name "red", and orders the retrieved results on "car " based on their resemblance to the human usage of "red’. Furthermore, knowledge of visual attributes can be used to assist object recognition methods. For example, for an image annotated with the text "Orange stapler on table", knowledge of the color name orange would greatly simplify the task of discovering where (or what) the stapler is. Color names are further applicable in automatic content labelling of images, colorblind assistance, and linguistic humancomputer interaction [41].

We use images that have been collected using an Internet search engine to train color name models for color naming and recognition tasks. Considering color histogram bands as being words of an image and the color names as classes, we use the supervised latent Dirichlet allocation to train our model. To pre-process the training data, we use state-of-the art salient object detection and a Kullback–Leibler divergence based outlier detection. In summary, we achieve state-of-the-art performance on the eBay data set and improve the similarity between labels assigned by our model and human observers by approximately 14%. 1