Hybrid Reinforcement/Supervised Learning for Dialogue Policies from COMMUNICATOR data

DMCA

Hybrid Reinforcement/Supervised Learning for Dialogue Policies from COMMUNICATOR data (2005)

Cached

Download Links

by James Henderson , Oliver Lemon , Kallirroi Georgila

Venue:	In IJCAI workshop on Knowledge and Reasoning in Practical Dialogue Systems
Citations:	99 - 26 self

BibTeX

@INPROCEEDINGS{Henderson05hybridreinforcement/supervised,
    author = {James Henderson and Oliver Lemon and Kallirroi Georgila},
    title = {Hybrid Reinforcement/Supervised Learning for Dialogue Policies from COMMUNICATOR data},
    booktitle = {In IJCAI workshop on Knowledge and Reasoning in Practical Dialogue Systems},
    year = {2005},
    pages = {68--75}
}

OpenURL

Abstract

We propose a method for learning dialogue management policies from a fixed dataset. The method is designed for use with “Information State Update” (ISU)-based dialogue systems, which represent the state of a dialogue as a large set of features, resulting in a very large state space and a very large policy space. To address the problem that any fixed dataset will only provide information about small portions of these state and policy spaces, we propose a hybrid model which combines reinforcement learning (RL) with supervised learning. The reinforcement learning is used to optimise a measure of dialogue reward, while the supervised learning is used to restrict the learnt policy to the portion of the space for which we have data. Linear function approximation is used to handle the large state space efficiently. We trained this model on a subset of the COMMUNICATOR corpus, to which we have added annotations for user actions and Information States. When tested with a user simulation trained on the same data, our model outperforms all the systems in the COMMUNICATOR data (it scores 37 % higher than the best COMMUNICA-TOR system). All of these advances will improve techniques for bootstrapping and automatic optimisation of dialogue management policies from limited initial datasets. 1

Keyphrases

communicator data hybrid reinforcement dialogue policy reinforcement learning fixed dataset dialogue management policy large state space information state user simulation linear function approximation information state update communicator corpus communica-tor system large policy space limited initial datasets hybrid model user action large set policy space dialogue reward learnt policy automatic optimisation dialogue system small portion supervised learning

DMCA