This paper reports about an experiment addressing different modes of natural language instructions in spatial human-robot interaction. The experimental setting involves a mobile robot equipped with an elementary dialogue system and human users unfamiliar with it who are required to achieve joint spatial reference with the robot either in spoken or in written mode. In addition, the robot's output is varied between an initial scene description (indicating the robot's conceptual and linguistic knowledge), and no initial output. Our robot uses a computational model of spatial reference to interpret the linguistic instructions that is based on psycholinguistic evidence and on previous experiments. Results show that the model is successful with regard to the correct interpretation of the intended kinds of instructions, that scene descriptions can encourage users to refer directly to the goal object, and that there are lasting negative effects on communication if potentially successful spoken instructions are not recognized by the speech component.

To my parents, Paul and B.J., for encouraging both wonder and accomplishment. To my wife, Sarah, for her unflagging love, support, and understanding. To all my friends, who have helped in innumerable ways.

This paper deals with emotional speech characteristics in human-computer- and human-robot interaction. The focus is on the users ’ involuntary expression of emotion in reaction to system malfunction, which may cause severe problems for automatic speech recognition and processing. Investigating different user groups is shown to be a useful method for determining what makes speakers respond emotionally and for understanding the interpersonal differences that can be observed in reaction to system malfunction. Which aspects may be involved is illustrated by discussing the example of the personal relationship between user and system as evident from the different forms of address that can be found in the corpora. We shall draw on corpora of humancomputer and human-robot communication involving children and adults from both sexes. However, it will be demonstrated that the major factor that determines the users ’ expressive behaviour is their conceptualisation of the artificial agent and the situation. Knowledge about this is then used to develop means for influencing the speakers ’ attitude towards the system, which can be shown to change their expressive speech characteristics in situations of system malfunction much more effectively than by assessing the linguistic behaviour directly. Thus, interestingly, what turns out to be most suitable for guiding the user into a linguistic behaviour that is understandable for an automatic speech processing system is employing aspects of expressive speech as well. 1

Abstract not found