The Problem: We use sketches as a medium for expressing ideas and saving thoughts. Sketching is especially common in early design as a means of communication, documentation and as a tool for stimulating thought. Despite the increasing availability of pen based PDAs and PCs, we still can’t interact with our devices via sketching as we do with people. As a group, we are building a generic multi-domain sketch recognition architecture to make computers sketch literate. This sketch recognition system will differ from existing architectures in many aspects, including a language for describing shapes, mechanisms for learning new shapes, and a blackboard based recognition architecture with top-down and bottom-up recognizers. Here we describe a part of this system that generates efficient bottom-up recognizers by compiling object descriptions. Motivation: As described in [2], current sketch recognition systems require users to hand-code individual recognizers as well as data structures for each object to be recognized. Hand-coding individual recognizers has a number of drawbacks: (i) writing recognizers and data structures is labor intensive and error prone, (ii) extending or modifying existing recognizers requires knowing how they work, (iii) because recognizers may be written by different programmers and may have different recognition algorithms, they lack a unified approach to recognition, (iv) users usually sketch parts of objects in a certain order and style, but current systems don’t have a systemic way of exploiting this information to improve recognition accuracy and speed.

In this paper, we describe a new framework for multi-domain sketch recognition which is being developed by the Design Rationale Group at the MIT AI laboratory. The framework uses a blackboard architecture for recognition in which the knowledge sources are a combination of domain-independent and domain-specific recognizers. Domain-specific recognizers are automatically generated from the domain description which is written using the domain description language syntax. Domain descriptions can be automatically generated by a system that learns shape descriptions from a drawn example.

We use sketches as a medium for expressing ideas and saving thoughts. Sketching is especially common in early design as a means of communication, documentation and as a tool for stimulating thought. Despite the increasing availability of pen based PDAs and PCs, we still can’t interact with our devices via sketching as we do with people. As a group, we are building a generic multi-domain sketch recognition architecture to make computers sketch literate. This sketch recognition system will differ from existing architectures in many aspects, including a language for describing shapes, mechanisms for learning new shapes, and a blackboard based recognition architecture with top-down and bottom-up recognizers. Here we describe a part of this system that generates efficient bottom-up recognizers by compiling object descriptions. 2.