DMCA
Learning Models for Object Recognition From Natural Language Descriptions (2009)
| Citations: | 35 - 1 self |
BibTeX
@MISC{Wang09learningmodels,
author = {Josiah Wang and Katja Markert and Mark Everingham},
title = {Learning Models for Object Recognition From Natural Language Descriptions},
year = {2009}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We investigate the task of learning models for visual object recognition from natural language descriptions alone. The approach contributes to the recognition of fine-grain object categories, such as animal and plant species, where it may be difficult to collect many images for training, but where textual descriptions of visual attributes are readily available. As an example we tackle recognition of butterfly species, learning models from descriptions in an online nature guide. We propose natural language processing methods for extracting salient visual attributes from these descriptions to use as ‘templates ’ for the object categories, and apply vision methods to extract corresponding attributes from test images. A generative model is used to connect textual terms in the learnt templates to visual attributes. We report experiments comparing the performance of humans and the proposed method on a dataset of ten butterfly categories. 1
Keyphrases
object recognition natural language description visual attribute online nature guide fine-grain object category salient visual attribute butterfly specie generative model many image plant specie visual object recognition vision method textual term natural language description ten butterfly category object category natural language processing method test image textual description
