With the increasing amount of multimedia content on the web added as user generated content in Web 2.0 websites, conventional multimedia information retrieval is presented with new challenges. It is no longer possible to rely only on meta-data based retrieval but to consider also content based techniques combined with the collective knowledge generated by users’ contributions and geo-referenced meta-data. Tagging is a modest way to annotate such documents and fails to capture a full semantic description of the document content. This report concerns ongoing research to investigate a means to identify, model and utilise semantic descriptions of the user-generated content in Web 2.0 documents using a hybrid approach. The approach consists of three main components, natural language processing, image analysis and a shared knowledge base. In this paper we describe the complete model but, as the image analysis component is in its early stages, the results focus on the natural language processing and the knowledge base. We show that the additional use of these components can improve retrieval and analysis performance over that based only on Web 2.0 tags.

knowledge management, visualization, social web, tag semantics, topic modeling