Magic Music Desk: A Tangible and Ubiquitous Multimodal Music and Speech System
BibTeX
@MISC{Zhou_magicmusic,
author = {Zhiying Zhou and Adrian David Cheok and Wei Liu and Chen Xiangdong and Farzam Farbiz},
title = {Magic Music Desk: A Tangible and Ubiquitous Multimodal Music and Speech System},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Abstract. This paper describes a novel multi-modal multi-user audio-visual interface – the Magic Music Desk (MMD) which employs the principles of embodied interaction, and emphasizes social interaction between users. This research is aimed for use in wearable and mobile sound and music systems. We combine multiple modalities for the interfaces including speech recognition, hand gesture recognition, sound and visual mixed reality technologies. A new type of interaction which is called as “What You Say is What You See” (WYSIWYS) is demonstrated in our system. This interaction enables all users to visualize each others spoken words as 3D objects which could be seen by multi-users (this also allows multi-cultural social interaction). 1
