Semiotics is "the mathematics of the humanities" in the sense that it provides an abstract language covering a diversity of special sign-usages (language, pictures, movies, theatre, etc.). In this capacity, semiotics is helpful for bringing insights from older media to the task of interface design, and for defining the special characteristics of the computer medium. However, semiotics is not limited to interface design but may also contribute to the proper design of program texts and yield predictions about the interaction between computer systems and their context of use. Keywords Computer based signs, aesthetics, context of use. 0. The mathematics of the humanities. In my experience, semiotics can be useful for the HCI-field, but the purely analytic character of traditional semiotics has to be supplemented by a constructive one. In addition, the semiotic community has to acquire a solid understanding of the technical possibilities and limitations of computer systems in order to b...

The purpose of this paper is two-fold. We begin by exploring the emerging trend to view multimedia information in terms of low-level and high-level components; the former being feature-based and the latter the \semantics" intrinsic to what is portrayed by the media object. Traditionally, this has been viewed by employing analogies with generative linguistics (e.g. compositional semantics). Recently, a new perspective based on the semiotic tradition has been alluded to in several papers. We believe this to be a more appropriate approach. From this, we propose an approach for tackling this problem which uses an associative data structure expressing authored information together with intelligent agents acting autonomously over this structure. We then show how neural networks can be used to implement such agents. The agents act as \vehicles" for bridging the gap between multimedia semantics and concrete expressions of high-level knowledge, but we suggest that traditional neural network tec...

Although metaphor is a commonly used device in the design of user-interfaces, it is not rigorously understood, and most guidance stops at the recommendation of its use. In this paper we seek to provide a systematic taxonomy of userinterface metaphors, based on and extending the framework of Lakoff and Johnson. We then suggest that some usability heuristics emerge directly from analysis of the taxonomy. We conclude that the taxonomy and heuristics may provide appreciable benefits in user-interface design and evaluation, and address some of the criticisms of metaphor use that have been made.

The ultimate goal of all computer science is the program. The performance of programs was once the noblest function of computer science, and computer science was indispensable to great programs. Today, programming and computer science exist in complacent isolation, and can only be rescued by the conscious co-operation and collaboration of all programmers. The universities were unable to produce this unity; and how indeed, should they have done so, since creativity cannot be taught? Designers, programmers and engineers must once again come to know and comprehend the composite character of a program, both as an entity and in terms of its various parts. Then their work will be filled with that true software spirit which, as “theory of computing”, it has lost. Universities must return to programming. The worlds of the formal methods and algorithm analysis, consisting only of logic and mathematics, must become once again a world in which things are built. If the young person who rejoices in creative activity now begins his career as in the older days by learning to program, then the unproductive “scientist ” will no longer be condemned to inadequate science, for their skills will be preserved for the programming in which they can achieve great things. Designers, programmers, engineers, we must all return to programming! There is no essential difference between the computer scientist and the programmer. The computer scientist is an exalted programmer. By the grace of Heaven and in rare moments of inspiration which transcend the will, computer science may unconsciously blossom from the labour of the hand, but a base in programming is essential to every computer scientist. It is there that the original source of creativity lies. Let us therefore create a new guild of programmers without the class-distinctions that raise an arrogant barrier between programmers and computer scientists! Let us desire, conceive, and create the new program of the future together. It will combine design, user-interfaces, and programming in a single form, and will one day rise towards the heavens from the hands of a million workers as the crystalline symbol of a new and coming faith. 1 1

Psychological and social systems provide us with a natural domain for the study of anticipations because these systems are based on and operate in terms of intentionality. Psychological systems can be expected to contain a model of themselves and their environments; social systems can be strongly anticipatory and therefore co-construct their environments, for example, in techno-economic (co-)evolutions. Using Dubois’s hyper-incursive and incursive formulations of the logistic equation, these two types of systems and their couplings can be simulated. In addition to their structural coupling, psychological and social systems are also coupled by providing meaning reflexively to each other’s meaning-processing. Luhmann’s distinctions among (1) interactions between intentions at the micro-level, (2) organization at the meso-level, and (3) self-organization of the fluxes of meaningful communication at the global level can be modeled and simulated using three hyper-incursive equations. The global level of self-organizing interactions among fluxes of communication is retained at the meso-level of organization. In a knowledge-based economy, these two levels of anticipatory structuration can be expected to propel each other at the supra-individual level.

– programming of social agents by children

Social order does not exist as a stable phenomenon, but can be considered as “an order of reproduced expectations. ” When anticipations operate upon one another, they can generate a non-linear dynamics which processes meaning. Although specific meanings can be stabilized, for example in social institutions, all meaning arises from a global horizon of possible meanings. Using Luhmann’s (1984) social systems theory and Rosen’s (1985) theory of anticipatory systems, I submit algorithms for modeling the non-linear dynamics of meaning in social systems. First, a self-referential system can use a model of itself for the anticipation. Under the condition of functional differentiation, the social system can be expected to entertain a set of models; each model can also contain a model of the other models. Two anticipatory mechanisms are then possible: a transversal one between the models, and a longitudinal one providing the system with a variety of meanings. A system containing two anticipatory mechanisms can become hyper-incursive. Without making decisions, however, a hyper-incursive system would be overloaded with uncertainty. Under this pressure, informed decisions tend to replace the “natural preferences ” of agents and a knowledge-based order can increasingly be shaped.

We present an approach to children’s programming inspired by the semiotics of comics. The idea is to build computer programs in a direct and concrete way by using a class of signs that we call contextual signs. There are two aspects that distinguish contextual signs from other sign systems used for programming. The first is that the signs are displayed in the immediate visual context of the object that they refer to. The second is that the signs are used to illustrate actions and properties in a way that is directly perceivable by the user. We argue that these two properties make contextual signs a promising high-level approach for building systems that are rich in dynamic properties, such as the ones that children often like to build.

Multichannel applications deliver the same content and a “similar interactive experience” using different devices and different technologies (e.g. web sites, palm held devices, car navigators, or interactive TVs). Various channels imply a number of differences, including screen (size), keyboard (size), pointing devices, output devices, performances, and the context of use (standing, sitting, walking, etc.). In most cases, today, applications for different channels are designed and implemented almost “independently”, with ineffectiveness for the developers (high costs) and ineffectiveness for the users (loss of consistency across the different channels and the perception that they are “different applications”). This paper presents IDM (Interactive Dialogue Model), a novel design model specifically tailored for multi-channel applications. The background research, moving from linguistic theories and practices, has led us to the development of a “channel-independent” design model (based on dialogue primitives). Design can start in a “conceptual”, channel-independent fashion, and then proceed into a further “logical” design oriented toward specific channels of communication. Designing an interactive application in two steps (channel-independent first, and channel-dependent later) allows a number of advantages without making more cumbersome the overall design process. Beside

This article has its starting point in a large number of empirical findings regarding computer-mediated work. These empirical findings have challenged our understanding of the role of mediation in such work; on the one hand as an aspect of communication and cooperation at work and on the other hand as an aspect of human engagement with instruments of work. On the basis of previous work in activity-theoretical and semiotic human–computer interaction, we propose a model to encompass both of these aspects. In a dialogue with our empirical findings we move on to propose a number of types of mediation that have helped to enrich our understanding of mediated work and the design of computer mediation for such work. 1.