Abstract

This paper reports on user evaluation for a spoken dialogue system and the subsequent redesign of the dialogue strategy. An automatic telephone directory system developed at Philips Research, PADIS, was evaluated using both objective metrics and subjective evaluation techniques. Duration and the occurrence of misunderstanding leading to `dead-ends', were found to be the key factors that kept users from using the system. Therefore, the dialogue strategy was redesigned to avoid `dead-end' misunderstandings. Prompts were redesigned for speed. Reliability measures provided by the speech-recognition component are used to limit the number and length of verification questions. Tasks with a higher risk or cost need to be verified more accurately. 1. Introduction Spoken dialogue systems are becoming more and more used. The speech recognition is increasingly reliable. Yet, complete recognition will remain impossible; so occasionally the user will have to correct the system. In order to facilit...

Citations