We present SketchREAD, a multi-domain sketch recognition engine capable of recognizing freely hand-drawn diagrammatic sketches. Current computer sketch recognition systems are difficult to construct, and either are fragile or accomplish robustness by severely limiting the designer’s drawing freedom. Our system can be applied to a variety of domains by providing structural descriptions of the shapes in that domain; no training data or programming is necessary. Robustness to the ambiguity and uncertainty inherent in complex, freely-drawn sketches is achieved through the use of context. The system uses context to guide the search for possible interpretations and uses a novel form of dynamically constructed Bayesian networks to evaluate these interpretations. This process allows the system to recover from low-level recognition errors (e.g., a line misclassified as an arc) that would otherwise result in domain level recognition errors. We evaluated SketchREAD on real sketches in two domains— family trees and circuit diagrams—and found that in both domains the use of context to reclassify low-level shapes significantly reduced recognition error over a baseline system that did not reinterpret low-level classifications. We also discuss the system’s potential role in sketch-based user interfaces.

The effective use of visual languages requires a precise understanding of their meaning. Moreover, it is impossible to prove properties of visual languages like soundness of transformation rules or correctness results without having a formal language definition. Although this sounds obvious, it is surprising that only little work has been done about the semantics of visual languages, and even worse, there is no general framework available for the semantics specification of different visual languages. We present such a framework that is based on a rather general notion of abstract visual syntax. This framework allows a logical as well as a denotational approach to visual semantics, and it facilitates the formal reasoning about visual languages syntax and semantics for the visual languages VEX, Show and Tell, and Euler Circles. We demonstrate the semantics in action by proving a rule for visual reasoning with Euler Circles and by showing the correctness of a Show and Tell program.

In a graphical user interface, physical layout and abstract structure are two important aspects of a graph. This article proposes a new graph grammar formalism which integrates both the spatial and structural specification mechanisms in a single framework. This formalism is equipped with a parser that performs in polynomial time with an improved parsing complexity over its nonspatial predecessor, that is, the Reserved Graph Grammar. With the extended expressive power, the formalism is suitable for many user interface applications. The article presents its application in adaptive Web design and presentation.

Documents are composed of text and graphics. There is substantial work on automated text summarization but almost none on the automated summarization of graphics. Four examples

Abstract not found

For many problem domains domain-specific languages (DSLs) offer users more appropriate notations and abstractions in which to model systems when compared with general purpose programming languages. These benefits can often be amplified if a visual notation is used instead of textual notations. In many problem domains visual notations are preferred by practitioners as they often are the most intuitive representation of a problem. However, the lack of supporting infrastructure for constructing, implementing, and maintaining visual languages in general and domain-specific visual languages (DSVLs) in particular has been an impediment to gaining wider acceptance. This paper describes techniques used in the Moses tool-suite for defining the syntax and semantics of DSVLs, which are very general, yet are built on a few very simple concepts and are therefore easy to apply.

Compiler techniques are effective and efficient in processing textual programming languages. These techniques can be adapted to recognition and processing of two-dimensional languages (diagrams). Already, grammars and parsers have been used in a variety of diagramrecognition and diagram-processing tasks. Here we explore the use of two other compiler techniques in pattern recognition systems. The first is compiler-style use of trees and tree transformation. The second is a multi-pass control structure, with a clear separation between layout, lexical, syntactic, and semantic analysis. Our proposal is illustrated on a case study involving recognition of hand- drawn mathematics notation.

Document image analysis is the study of converting documents from paper form to an electronic form that captures the information content of the document. Necessary processing includes recognition of document layout (to determine reading order, and to distinguish text from diagrams), recognition of text (called Optical Character Recognition, OCR), and processing of diagrams and photographs. The processing of diagrams has been an active research area for several decades. A selection of existing diagram recognition techniques are presented in this paper. Challenging problems in diagram recognition include (1) the great diversity of diagram types, (2) the difficulty of adequately describing the syntax and semantics of diagram notations, and (3) the need to handle imaging noise. Recognition techniques that are discussed include blackboard systems, stochastic grammars, Hidden Markov Models, and graph grammars. 1.

Abstract. In this paper we describe a language-driven approach to the development of videogames. In our approach the development process starts with the design of a suitable domain-specific language for building games, along with an abstract syntax for the language and its operational semantics. Next an engine supporting the language is built. Finally games are built using the customized language and they are executed using the engine. This approach is exemplified with the <e-Game> project, which delivers the design of a language and the construction of an engine for the documental development of graphical adventure videogames with educational purposes. 1.

To take advantage of the ever-increasing volume of diagrams in electronic form, it is crucial that we have methods for parsing diagrams. Once a structured, content-based description is built for a diagram, it can be indexed for search, retrieval, and use. Whenever broadcoverage grammars are built to parse a wide range of objects, whether natural language or diagrams, the grammars will overgenerate, giving multiple parses. This is the ambiguity problem. This paper discusses the types of ambiguities that can arise in diagram parsing, as well as techniques to avoid or resolve them. One class of ambiguity is attachment, e.g., the determination of what graphic object is labeled by a text item. Two classes of ambiguities are unique to diagrams: segmentation and occlusion. Examples of segmentation ambiguities include the use of a portion of a single line as an entity itself. Occlusion ambiguities can be difficult to analyze if occlusion is deliberately used to create a novel object from its components. The paper uses our context-based constraint grammars to describe the origin and resolution of ambiguities. It assumes that diagrams are available as vector graphics, not bitmaps.