We demonstrate that it is possible to automatically find representative example images of a specified object category. These canonical examples are perhaps the kind of images that one would show a child to teach them what, for example a horse is – images with a large object clearly separated from the background. Given a large collection of images returned by a web

Abstract. The paper stresses the importance of automatically analyzing and semantically annotating creative forms of human expression, among which are textual sources. A number of challenging and emerging research directions are enumerated and illustrated with results obtained by the research group of the author.

We develop and demonstrate automatic image description methods using a large captioned photo collection. One contribution is our technique for the automatic collection of this new dataset – performing a huge number of Flickr queries and then filtering the noisy results down to 1 million images with associated visually relevant captions. Such a collection allows us to approach the extremely challenging problem of description generation using relatively simple non-parametric methods and produces surprisingly effective results. We also develop methods incorporating many state of the art, but fairly noisy, estimates of image content to produce even more pleasing results. Finally we introduce a new objective performance measure for image captioning. 1