The aim of the research reported on here is to develop a system for automatic assessment of foreign speakers’ pronunciation of Dutch. In this paper, special attention is paid to expert ratings of pronunciation, because they are used as a reference to validate the pronunciation scores obtained automatically. It is shown that the ratings can differ between raters and rater groups and it is concluded that these differences should be taken into consideration before going on to develop an automatic system for pronunciation grading. 1.

This paper describes two experiments aimed at exploring the relationship between objective properties of speech and perceived pronunciation quality in read and spontaneous speech, with a view to determining whether such quantitative measures can be used to develop objective pronunciation tests. Read and spontaneous speech of two groups of 60 learners of Dutch as a second language was scored for pronunciation quality by human raters and was analyzed by means of a continuous speech recognizer to calculate six quantitative measures of speech quality related to speech timing. The results show that quantitative, temporal measures of speech are strongly related to pronunciation quality, in both read and spontaneous speech, although not all variables suitable for measuring pronunciation quality in read speech are as effective in spontaneous speech. In particular, measures that express the rate at which sounds are produced without taking the frequency and distribution of pauses into account appear to be unsuitable for measuring pronunciation quality in spontaneous speech. 1.

Prosody—suprasegmental characteristics of speech such as pitch, rhythm, and loudness— is a rich source of information in spoken language and can tell a listener much about the internal state of a speaker. This thesis explores the role of prosody in conveying three very different types of speaker state: paralinguistic state, in particular emotion; pragmatic state, in particular questioning; and the state of spoken language proficiency of non-native English speakers. Paralinguistics. Intonational features describing pitch contour shape were found to dis-criminate emotion in terms of positive and negative affect. A procedure is described for clustering groups of listeners according to perceptual emotion ratings that foster further understanding of the relationship between acoustic-prosodic cues and emotion perception. Pragmatics. Student questions in a corpus of one-on-one tutorial dialogs were found to be signaled primarily by phrase-final rising intonation, an important cue used in conjunc-tion with lexico-pragmatic cues to differentiate the high rate of observed declarative questions from proper declaratives. The automatic classification of question form and

Experiments were carried out to determine whether log-likelihood ratios (LRs) can be employed to improve automatic assessment of Dutch pronunciation. Read speech of natives and non-natives was judged by three groups of expert raters and was then analyzed by means of a continuous speech recognizer. Three automatic measures were calculated, two LRs and rate of speech (ros), and then compared with the expert ratings. It appears that expert ratings of pronunciation quality can accurately be predicted on the basis of ros alone and that LRs do not contribute to better prediction. However, LRs can be useful to automatic pronunciation assessment because they can help detect fast speakers who produce totally wrong sentences. 1.

This paper investigates the feasibility of automated scoring of spoken English proficiency of non-native speakers. Unlike existing automated assessments of spoken English, our data consists of spontaneous spoken responses to complex test items. We first compute a set of features relevant for measuring communicative competence based on speech recognition output. We then perform both a quantitative and a qualitative analysis of these features using two different machine learning approaches. (1) We use support vector machines to produce a score and evaluate it with respect to a mode baseline and to human rater agreement. We find that scoring based on support vector machines yields accuracies approaching inter-rater agreement in some cases. (2) We use classification and regression trees to understand the role of different features and feature classes in the characterization of speaking proficiency by human scorers. Our analysis shows that across all the test items most or all the feature classes are used in the nodes of the trees suggesting that the scores are, appropriately, a combination of multiple components of speaking proficiency. Future research will concentrate on extending the set of features and introducing new feature classes to arrive at a scoring model that comprises additional relevant aspects of speaking proficiency.

The aim of our current research is to investigate the possibility of using likelihood ratios to perform utterance verification within the context of automatic oral proficiency assessment. The likelihood ratios under investigation have the appealing feature that they may be computed simply by using an off-theshelf automatic speech recognition system in two different recognition modes (forced and free phone) instead of using a system with specifically trained anti-models. We achieved 93% correct classification for 10 phonetically rich sentences uttered by 60 non-native language students. 1. INTRODUCTION The long-term goal of our research is to employ ASR technology in an automatic pronunciation test for Dutch as a second language. As a consequence of this aim we are not concerned with learners of Dutch with a specific mother tongue, but rather with a group of speakers who are highly varied in this respect. In this sense our situation is different from that of many studies on the use of...

Prosody, or intonation, is a critically important component of spoken communication. The automatic extraction of prosodic information is necessary for machines to process speech with human levels of proficiency. In this thesis we describe work on the automatic detection and classification of prosodic events – specifically, pitch accents and prosodic phrase boundaries. We present novel techniques, feature representations and state of the art performance in each of these tasks. We also present three proof-of-concept applications – speech summarization, story segmentation and non-native speech assessment – showing that access to hypothesized prosodic event information can be used to improve the performance of downstream spoken language processing tasks. We believe the contributions of this thesis advance the understanding of prosodic events and the use of prosody in spoken language processing towards the goal of human-like processing of speech by machines.

This paper describes a system aimed at automatically scoring two task types of high and medium-high linguistic entropy from a spoken English test with a total of six widely differing task types. We describe the speech recognizer used for this system and its acoustic model and language model adaptation; the speech features computed based on the recognition output; and finally the scoring models based on multiple regression and classification trees. For both tasks, agreement measures between machine and human scores (correlation, kappa) are close to or reach inter-human agreements. 1

Abstract not found