Abstract—In this paper, we present a framework which harvests grassroots-generated data from the Web (e.g., reviews, blogs), extracts latent information from these data, and provides a multimodal interface for review browsing and inquiring. A prescription-drug domain system is implemented under this framework. Patient-provided drug reviews were collected from various health-related forums, from which significant side effects correlated to each drug type were identified with association algorithms. A multimodal web-based spoken dialogue system was implemented to allow users to inquire about drugs and correlated side effects as well as browsing the reviews obtained from the Web. We report evaluation results on speech recognition, parse coverage and system response. I.

In this paper we present a framework for spoken recommendation systems. To provide reliable recommendations to users, we incorporate a review summarization technique which extracts informative opinion summaries from grass-roots users ‘ reviews. The dialogue system then utilizes these review summaries to support both quality-based opinion inquiry and feature-specific entity search. We propose a probabilistic language generation approach to automatically creating recommendations in spoken natural language from the text-based opinion summaries. A user study in the restaurant domain shows that the proposed approaches can effectively generate reliable and helpful recommendations in human-computer conversations. 1