Abstract

The bottleneck in interactive visual classification is the exchange of information between human and machine. We introduce the concept of the visible model, which is an abstraction of an object superimposed on its picture. For a narrow domain, like flowers or faces, it may be as simple as an outline of the entire object to be classified or a set of characteristic points. For every new object to be classified, the machine proposes a model subject to constraints (learned from a training set) that help avoid implausible abstractions. The visible model is not by itself sufficient for classification, because it contains no intensity, color or texture information. However, using features extracted from the picture based on the model, the classes can be rank ordered according to the similarity of the unknown picture (in a hidden high-dimensional feature space) to the labeled reference pictures. If the rank ordering appears unsatisfactory, the operator may modify the model, resulting in the extraction of new features, and a new rank ordering. The interaction continues until the operator confirms a satisfactory match. The new object, with its model and label, is added to the reference database. Comprehensive experiments show that interactive recognition of flowers and faces is much more accurate than automated classification, much faster than unaided human classification, and that both machine and human performance improve with use.

Citations