Abstract—Although it has been studied for years by the computer vision and machine learning communities, image annotation is still far from practical. In this paper, we propose a novel attempt at model-free image annotation, which is a data-driven approach that annotates images by mining their search results. Some 2.4 million images with their surrounding text are collected from a few photo forums to support this approach. The entire process is formulated in a divide-and-conquer framework where a query keyword is provided along with the uncaptioned image to improve both the effectiveness and efficiency. This is helpful when the collected data set is not dense everywhere. In this sense, our approach contains three steps: 1) the search process to discover visually and semantically similar search results, 2) the mining process to identify salient terms from textual descriptions of the search results, and 3) the annotation rejection process to filter out noisy terms yielded by Step 2. To ensure real-time annotation, two key techniques are leveraged—one is to map the high-dimensional image visual features into hash codes, the other is to implement it as a distributed system, of which the search and mining processes are provided as Web services. As a typical result, the entire process finishes in less than 1 second. Since no training data set is required, our approach enables annotating with unlimited vocabulary and is highly scalable and robust to outliers. Experimental results on both real Web images and a benchmark image data set show the effectiveness and efficiency of the proposed algorithm. It is also worth noting that, although the entire approach is illustrated within the divide-andconquer framework, a query keyword is not crucial to our current implementation. We provide experimental results to prove this.

Browsing and retrieving images from large image collections are becoming common and important activities. Recent semantic image analysis techniques, which automatically detect high level semantic contents of images for annotation, are promising solutions toward this problem. However, few efforts have been made to convey the annotation results to users in an intuitive manner to enable effective image browsing and retrieval. There also lack methods to monitor and evaluate the automatic image analysis algorithms due to the high dimensional nature of image data, features, and contents. In this paper, we propose a novel, scalable semantic image browser by applying existing information visualization techniques to semantic image analysis. This browser not only allows users to effectively browse and search in large image databases according to semantic content of images, but also allows analysts to evaluate their annotation process through interactive visual exploration. The major visualization components of this browser are Multi-Dimensional Scaling (MDS) based image layout, the Value and Relation (VaR) display that allows effective high dimensional visualization without dimension reduction, and a rich set of interaction tools such as search by sample images and content relationship detection. Our preliminary user study showed that the browser was easy to use and understand, and effective in supporting image browsing and retrieval tasks.

In this paper, a hierarchical classification framework has been proposed for bridging the semantic gap effectively and achieving multi-level image annotation automatically. First, the semantic gap between the low-level computable visual features and the users ’ real information needs is partitioned into four smaller gaps, and multiple approaches are proposed to bridge these smaller gaps more effectively. To learn more reliable contextual relationships between the atomic image concepts and the co-appearances of salient objects, a multimodal boosting algorithm is proposed. To enable hierarchical image classification and avoid inter-level error transmission, a hierarchical boosting algorithm is proposed by incorporating concept ontology and multi-task learning to achieve hierarchical image classifier training with automatic error recovery. To bridge the gap between the computable image concepts and the users ’ real information needs, a novel hyperbolic visualization framework is seamlessly incorporated to enable intuitive query specification and evaluation by acquainting the users with a good global view of largescale image collections. Our experiments on large-scale image databases have also obtained very positive results.

the input data is a collection of images that are annotated with a given keyword, such as “car”. The problem is to attribute the annotation to specific parts of the images. There exists plenty of suitable input data readily

Abstract. Traditional content based image retrieval attempts to retrieve images using syntactic features for a query image. Annotated image banks and Google allow the use of text to retrieve images. In this paper, we studied the task of using the content of an image to retrieve information in general. We describe the significance of object identification in an information retrieval paradigm that uses image set as intermediate means in indexing and matching. We also describe a unique Singapore Tourist Object Identification Collection with associated queries and relevance judgments for evaluating the new task and the need for efficient image matching using simple image features. We present comprehensive experimental evaluation on the effects of feature dimensions, context, spatial weightings, coverage of image indexes, and query devices on task performance. Lastly we describe the current system developed to support mobile image-based tourist information retrieval. 1

The paper presents a contribution to image indexing consisting in a weighting model for visible objects – or image objects – in home photographs. To improve its effectiveness this weighting model has been designed according to human perception criteria about what is estimated as important in photographs. Four basic hypotheses related to human perception are presented, and their validity is estimated as compared to actual observations from a user study. Finally a formal definition of this weighting model is presented and its consistence with the user study is evaluated.

Our goal is to determine if artificially imagined or synthesized images can be beneficial to interactive visual search. We present a novel approach for using artificially imagined images in relevance feedback. Since the search engine constructs the synthetic images itself, any feedback given by the user on these images allows it to obtain a better understanding of what the user is looking for than it would from feedback on database images alone. We evaluated and compared our image synthesis approach with a normal Rocchio-based system on a well-known texture database with real users. 1.

Digital images databases open the way for content-based searching. Content Based Image Retrieval occupies a well ranked position among the research areas as it provides the practical solution for narrowing the semantic gap between the image retrieval process and the human perception. The main objective of this paper is to propose a framework for region content based image retrieval based on a distributed clustered image dataset. The proposed framework introduces a new perspective to measure the similarity between the image query and the clustered dataset images. Moreover, a development by adopting three relevance feedback techniques is used to refine the results of the retrieval system which are the well known Query Point Movement and Query Expansion, besides to the proposed third technique which is Query Modified Re-Weighting technique.