Abstract. Most approaches to classifying media content assume a fixed, closed vocabulary of labels. In contrast, we advocate machine learning approaches which take advantage of the millions of free-form tags obtainable via online crowd-sourcing platforms and social tagging websites. The use of such open vocabularies presents learning challenges due to typographical errors, synonymy, and a potentially unbounded set of tag labels. In this work, we present a new approach that organizes these noisy tags into well-behaved semantic classes using topic modeling, and learn to predict tags accurately using a mixture of topic classes. This method can utilize an arbitrary open vocabulary of tags, reduces training time by 94% compared to learning from these tags directly, and achieves comparable performance for classification and superior performance for retrieval. We also demonstrate that on open vocabulary tasks, human evaluations are essential for measuring the true performance of tag classifiers, which traditional evaluation methods will consistently underestimate. We focus on the domain of tagging music clips, and demonstrate our results using data collected with a human computation game called TagATune.

In our submission we use a set of block-level features for three different tasks, namely genre classification, tag classification presents the feature set that is used and some specific details of the three submitted algorithms. 1.

We present and compare several models for automatic identification of instrument classes in polyphonic and poly-instrument audio. The goal is to be able to identify which categories of instrument (Strings, Woodwind, Guitar, Piano, etc.) are present in a given audio example. We use a machine learning approach to solve this task. We constructed a system to generate a large database of musically relevant poly-instrument audio. Our database is generated from hundreds of instruments classified in 7 categories. Musical audio examples are generated by mixing multi-track MIDI files with thousands of instrument combinations. We compare three different classifiers: a Support Vector Machine (SVM), a Multilayer Perceptron (MLP) and a Deep Belief Network (DBN). We show that the DBN tends to outperform both the SVM and the MLP in most cases. tion, we generated our own database of audio. Our goal was to have enough variability in the set of instruments so as to allow us to generalize to instruments not used in the training set. An overview of our system is illustrated in Figure 1. 1.

● What are we working on? Automatic tagging of music ● What is our problem? Finding hard-to-classify examples ● What is our solution? Metropolis-Hastings sampling algorithm ● Is it working? ● What else would work better? Automatic tagging of music Tag cloud from The Beatles on www.last.fm Can we recreate this tag cloud, from audio features, using machine learning? (can help with the cold-start problem) distribution of words for Give it Away by Red Hot Chili Peppers (taken from Turnbull et al. 2008) Automatic tagging of music Genre classification genre 1 genre 2 genre k GC special case of AT Automatic tagging tag 1 tag 2 tag k for 1 tag: would it be applied? Y/N more than some other tag? Y/NAutomatic tagging of music: Tag Data ● We use tags applied to artists on lastfm.com ● crawl done during Spring 2007 ● This dataset is big and noisy different from data obtained by an online game, or by paying annotators 7M tags (122K unique), applied to 280K artists

Music folksonomies have an inherent loose and open semantics, which hampers their use in structured browsing and recommendation. In this paper, we present a method for automatically obtaining a set of semantic facets underlying a folksonomy of music tags. The semantic facets are anchored upon the structure of the dynamic repository of universal knowledge Wikipedia. We illustrate the relevance of the obtained facets for the description of tags. Categories and Subject Descriptors

Music perception is highly intertwined with both emotions and context. Not surprisingly, many of the users ’ information seeking actions aim at retrieving music songs based on these perceptual dimensions – moods and themes, expressing how people feel about music or which situations they associate it with. In order to successfully support music retrieval along these dimensions, powerful methods are needed. Still, most existing approaches aiming at inferring some of the songs ’ latent characteristics focus on identifying musical genres. In this paper we aim at bridging this gap between users ’ information needs and indexed music features by developing algorithms for classifying music songs by moods and themes. We extend existing approaches by also considering the songs ’ thematic dimensions and by using social data from the Last.fm music portal, as support for the classification tasks. Our methods exploit both audio features and collaborative user annotations, fusing them to improve overall performance. Evaluation performed against the AllMusic.com ground truth shows that both kinds of information are complementary and should be merged for enhanced classification accuracy. 1.

the degree of Doctor of Philosophy. Human computation is the study of systems where humans perform a major part of the computation or are an integral part of the overall computational process. The ESP Game, for example, is a human computation system that maps images to tags, by engaging humans to play a game in which they are rewarded each time they agree on a description for an image. It was shown that these so-called Games with a Purpose are a reliable way to quickly collect millions of accurate image descriptors, which can then used to index images and facilitate search. However, most existing human computation systems operate without any machine intervention. Likewise, very few supervised learning systems are taking advantage of these powerful new platforms to elicit help from human teachers. It is therefore largely unknown what more a human computation system can achieve with machines in the loop. This thesis is centered around the problem of attribute learning – using the joint effort of human game players and machine learning algorithms to determine that a piece of music is “soothing”, that the bird in an image “has a red beak”, or that Ernest Hemingway is an “Nobel Prize winning author”. In particular, our work focuses on two aspects of the problem – how to acquire attributes and attribute values from human computers using incentive-compatible game mechanisms, and what active learning strategies to employ for attribute and attribute value acquisition.

We have developed a novel hybrid representation for Music Information Retrieval. Our representation is built by incorporating audio content into the tag space in a tag-track matrix, and then learning hybrid concepts using latent semantic analysis. We apply this representation to the task of music recommendation, using similarity-based retrieval from a query music track. We also develop a new approach to evaluating music recommender systems, which is based upon the relationship of users liking tracks. We are interested in measuring the recommendation quality, and the rate at which cold-start tracks are recommended. Our hybrid representation is able to outperform a tag-only representation, in terms of both recommendation quality and the rate that cold-start tracks are included as recommendations. 1

To automatically annotate songs with descriptive keywords, a variety of content-based auto-tagging strategies have been proposed in recent years. Different approaches may capture different aspects of a song’s musical content, such as timbre, temporal dynamics, rhythmic qualities, etc. As a result, some auto-taggers may be better suited to model the acoustic characteristics commonly associated with one set of tags, while being less predictive for other tags. This paper proposes decision-fusion, a principled approach to combining the predictions of a diverse collection of content-based autotaggers that focus on various aspects of the musical signal. By modeling the correlations between tag predictions of different auto-taggers, decision-fusion leverages the benefits of each of the original auto-taggers, and achieves superior annotation and retrieval performance. 1.

We introduce the Million Song Dataset, a freely-available collection of audio features and metadata for a million contemporary popular music tracks. We describe its creation process, its content, and its possible uses. Attractive features of the Million Song Database include the range of existing resources to which it is linked, and the fact that it is the largest current research dataset in our field. As an illustration, we present year prediction as an example application, a task that has, until now, been difficult to study owing to the absence of a large set of suitable data. We show positive results on year prediction, and discuss more generally the future development of the dataset. 1.