This paper describes an approach to the automatic presentation of multimedia documents based on parsing and syntax-directed translation using Relational Grammars. This translation is followed by a constraint solving mechanism to create the final layout. Grammatical rules provide the mechanism for mapping from a representation of the content of a presentation to forms that specify the media objects to be realized. These realization forms include sets of spatial and temporal constraints between elements of the presentation. Individual grammars encapsulate the "look and feel" of a presentation and can be used as generators of that style. By making the grammars sensitive to the requirements of the output medium, parsing can introduce flexibility into the information realization process. Keywords: Automatic design, grammar-directed design, visual languages, relational grammars, parsing, constraints 2: Introduction A fully functioning multimedia system requires a wide range of stages to achi...

The effective use of visual languages requires a precise understanding of their meaning. Moreover, it is impossible to prove properties of visual languages like soundness of transformation rules or correctness results without having a formal language definition. Although this sounds obvious, it is surprising that only little work has been done about the semantics of visual languages, and even worse, there is no general framework available for the semantics specification of different visual languages. We present such a framework that is based on a rather general notion of abstract visual syntax. This framework allows a logical as well as a denotational approach to visual semantics, and it facilitates the formal reasoning about visual languages syntax and semantics for the visual languages VEX, Show and Tell, and Euler Circles. We demonstrate the semantics in action by proving a rule for visual reasoning with Euler Circles and by showing the correctness of a Show and Tell program.

In order to implement graphical editors which allow both for structured and free editing, a parsing algorithm is needed which can analyze a diagram according to a graphical syntax, and derive the structure depicted. We propose to split this analysis in two phases. The first phase reads the picture objects as they were drawn, determines the spatial relations between them, and stores the whole in a graph. The second phase of the analysis searches this graph for patterns which form constructs of the language under consideration, and generates an abstract syntax graph of the depicted structure. We investigate whether graph grammars are a suitable formalism to define both graphs and to specify the translations between the two. The description of the translation would define the graphical syntax of L; graph rewriting according to this description would implement the corresponding graphical parser which performs the second phase of the analysis. 1 INTRODUCTION AND MOTIVATION Workstations wi...

This paper addresses issues in visual language theory with the help of logic formalisms that were developed for reasoning tasks by the artificial intelligence and spatial databases community, especially for spatial and diagrammatical reasoning. We describe an approach based on three formal components. Topology is used to define basic geometric objects. Theory about spatial relations from the domain of spatial databases is employed to define possible relationships between visual language elements. Description logic theory from the AI community is used to combine topology and spatial relations. The resulting theory has been successfully applied to formally specifying semantics of visual languages. The theory’s application is illustrated with a specification of entity-relationship diagrams. 1

We describe the object-oriented editor GenEd supporting the design of specifications for visual notations. Prominent features of GenEd are (1) it is generic, i.e. domain-specific syntax and semantics are specified by users; (2) built-in parser for actual drawings, driven by formal specifications; (3) powerful reasoning capabilities about diagrams and their specification. GenEd’s specification language is based on a fully formalized theory for describing visual notations. Three examples, place-transition petri nets, entity-relationship diagrams, and a small GIS application are presented.

For many problem domains domain-specific languages (DSLs) offer users more appropriate notations and abstractions in which to model systems when compared with general purpose programming languages. These benefits can often be amplified if a visual notation is used instead of textual notations. In many problem domains visual notations are preferred by practitioners as they often are the most intuitive representation of a problem. However, the lack of supporting infrastructure for constructing, implementing, and maintaining visual languages in general and domain-specific visual languages (DSVLs) in particular has been an impediment to gaining wider acceptance. This paper describes techniques used in the Moses tool-suite for defining the syntax and semantics of DSVLs, which are very general, yet are built on a few very simple concepts and are therefore easy to apply.

Formalisms for visual language specification have been investigated for more than two decades now. However, there has been little attempt to develop a systematic and comprehensive hierarchy of visual languages based on formal properties. Given the importance of the Chomsky hierarchy in the theory of textual languages and the di#culty of comparing the di#erent visual language formalisms, it is clear that there is a need for such a hierarchy. We develop a hierarchy for VLs and investigate the expressiveness and cost of parsing for classes defined therein. Although the hierarchy is based on the constraint multiset grammar formalism, we sketch how other visual language specification formalisms can be mapped into constraint multiset grammars. One consequence of our work is that a large class of "naturally occurring" visual languages are inherently context-sensitive, so that the core of such a hierarchy has to be built around di#erent forms of context-sensitivity. 1 Introduction Many di#er...

We discuss the generation of structured visual editors for formally specified languages. Our concept of structured visual editor is a structure-- oriented editor which supports the interactive construction of programs as well as provide language specific behavior. This paper describes an approach to the specification of the visual syntax of a language and the generation of a structured visual editor from the formal specification of that language. A simple visual expression editor is presented in order to demonstrate our approach. Our prototype implementation for the structured visual editor and visual definition language is presented at the end of the paper. 1 Introduction In recent years, a great variety of visual languages have evolved. One of the concerns related to visual languages is supporting environments for the construction of programs. Visual programming environments should provide a set of tools that deal with the software development process for visual languages. There are...

We propose Conditional Set Rewriting as a general mechanism for describing the syntax of multidimensional languages. We compare the approach with other existing methods, and give a number of examples that illustrate its strengths. 1 Introduction This paper introduces the notion of a Conditional Set Rewrite System (CSRS). Conditional Set Rewrite Systems are a generalization of Conditional Term Rewrite Systems [10]. Conditional Term Rewrite Systems deal with rewriting a single term, whereas Conditional Set Rewrite systems deal with rewriting a set of terms. Each conditional set rewrite rule specifies how to replace a subset of the set of terms by another set. We suggest two uses for CSRS: they can be used to describe the set of all legal phrases (pictures) of a multidimensional (visual) language, and they can be used to translate a phrase from one language (e.g. a visual one) into another language (e.g. a textual one). The ability of CSRS to translate visual programs into textual ones ...

In the visual languages community there has been a growing consensus that visual languages will be most successful in the case of special purpose languages. Furthermore, their success will largely depend on the programming environment which is provided for them. Programming environment generators, generate programming environments for formally specified languages. This paper discusses specification of visual languages and the generation of visual environments. We focus on a picture definition language, Vodl, which serves as the basis for defining the syntax of visual languages. We present the language definition and an example showing how Vodl is used in defining language syntax and thereafter generating visual editors. Finally, we discuss how to extend this approach in creating a visual specification formalism and a supporting environment for specifying the syntax and semantics of visual languages. 1 Introduction Although there is considerable optimism among some circles, there are...