Semi-supervised clustering algorithms aim to improve clustering results using limited supervision. The supervision is generally given as pairwise constraints; such constraints are natural for graphs, yet most semi-supervised clustering algorithms are designed for data represented as vectors. In this paper, we unify vector-based and graph-based approaches. We first show that a recently-proposed objective function for semi-supervised clustering based on Hidden Markov Random Fields, with squared Euclidean distance and a certain class of constraint penalty functions, can be expressed as a special case of the weighted kernel k-means objective (Dhillon et al., in Proceedings of the 10th International Conference on Knowledge Discovery and Data Mining, 2004a). A recent theoretical connection between weighted kernel k-means and several graph clustering objectives enables us to perform semi-supervised clustering of data given either as vectors or as a graph. For graph data, this result leads to algorithms for optimizing several new semi-supervised graph clustering objectives. For vector data, the kernel approach also enables us to find clusters with non-linear boundaries in the input data space. Furthermore, we show that recent work on spectral learning (Kamvar et al., in Proceedings of the 17th International Joint Conference on Artificial Intelligence, 2003) may be viewed as a special case of our formulation. We empirically show that our algorithm is able to outperform current state-of-the-art semi-supervised algorithms on both vector-based and graph-based data sets.

Naming every individual person appearing in broadcast news videos with names detected from the video transcript leads to better access of the news video content. In this paper, we approach this challenging problem with a statistical learning method. Two categories of information extracted from multiple video modalities have been explored, namely features, which help distinguish the true name of every person, as well as constraints, which reveal the relationships among the names of different persons. The personnaming problem is formulated into a learning framework which predicts the most likely name for each person based on the features, and refines the predictions using the constraints. Experiments conducted on ABC World New Tonight and CNN Headline News videos demonstrate that this approach outperforms a nonlearning alternative by a large amount.

People identification is an essential task for video content analysis in a surveillance system. A good classifier, however, requires a large amount of training data, which may not be obtained in some scenario. In this paper, we propose an approach to augment insufficient training data with pairwise constraints that can be offered from video images that have removed people's identities by masking faces. We show user study results that human subjects can perform reasonably well in labeling pairwise constraints from face masked images. We also present a new discriminative learning algorithm that can handle uncertainties in pairwise constraints. The effectiveness of the proposed approach is demonstrated using video captured from a nursing home environment. The new method provides a way to obtain high accuracy of people identification from limited labeled data with noisy pairwise constraints, and meanwhile minimize the risk of exposing people's identities. 1.

Abstract. In this work we investigate a weakly-supervised approach to learning facial attributes of humans in video. Given a small set of images labeled with attributes and a much larger unlabeled set of video tracks, we train a classifier to recognize these attributes in video data. We make two contributions. First, we show that training on video data improves classification performance over training on images alone. Second, and more significantly, we show that tracks in video provide a natural mechanism for generalizing training data – in this case to new poses, lighting conditions and expressions. The advantage of our method is demonstrated on the classification of gender and age attributes in the movie “Love, Actually”. We show that the semi-supervised approach adds a significant performance boost, for example for gender increasing average precision from 0.75 on static images alone to 0.85. 1

We address the character identification problem in movies and television videos: assigning names to faces on the screen. Most prior work on person recognition in video assumes some supervised data such as screenplay or handlabeled faces. In this paper, our only source of ‘supervision’ are the dialog cues: first, second and third person references (such as “I’m Jack”, “Hey, Jack! ” and “Jack left”). While this kind of supervision is sparse and indirect, we exploit multiple modalities and their interactions (appearance, dialog, mouth movement, synchrony, continuityediting cues) to effectively resolve identities through local temporal grouping followed by global weakly supervised recognition. We propose a novel temporal grouping model that partitions face tracks across multiple shots while respecting appearance, geometric and film-editing cues and constraints. In this model, states represent partitions of the k most recent face tracks, and transitions represent compatibility of consecutive partitions. We present dynamic programming inference and discriminative learning for the model. The individual face tracks are subsequently assigned a name by learning a classifier from partial label constraints. The weakly supervised classifier incorporates multiple-instance constraints from dialog cues as well as soft grouping constraints from our temporal grouping. We evaluate both the temporal grouping and final character naming on several hours of TV and movies. 1.

In this paper we consider the problem of classification in the presence of pairwise constraints, which consist of pairs of examples as well as a binary variable indicating whether they belong to the same class or not. We propose a method which can effectively utilize pairwise constraints to construct an estimator of the decision boundary, and we show that the resulting estimator is sign-insensitive consistent with respect to the optimal linear decision boundary. We also study the asymptotic variance of the estimator and extend the method to handle both labeled and pairwise examples in a natural way. Several experiments on simulated datasets and real world classification datasets are conducted. The results not only verify the theoretical properties of the proposed method but also demonstrate its practical value in applications. 1.

Abstract. CareMedia captures and analyzes a continuous audio and video record of behavior and activity in a skilled nursing facility. Through computer vision and machine learning we automatically identify individuals, classify activities, recognize behaviors, and extract relevant events. Two extensive field trials have been undertaken which produced meaningful but sometimes limited clinical results. Based on an analysis of this experience, combined with the development of new approaches and algorithms, we describe a radically improved audiovisual recording and computing infrastructure to be implemented, enabling longitudinal studies with comprehensive video and audio coverage provided with a mix of resolution, frame-rate, compression and storage requirements. Changes in the amount or rate of social interaction, eating, walking, gait, arm swing, and other attributes of motion from each patient’s baseline behavior will also be made easy to flag for professional review and diagnosis through the recording and analysis infrastructure, as such changes are key indicators to the benefits and possibly detrimental side effects of pharmacological interventions. 1

In this paper, we address the problem of learning when some cases are fully labeled while other cases are only partially labeled, in the form of partial labels. Partial labels are represented as a set of possible labels for each training example, one of which is the correct label. We introduce a discriminative learning approach that incorporates partial label information into the conventional margin-based learning framework. The partial label learning problem is formulated as a convex quadratic optimization minimizing the L2-norm regularized empirical risk using hinge loss. We also present an efficient algorithm for classification in the presence of partial labels. Experiments with different data sets show that partial label information improves the performance of classification when there is traditional fully-labeled data, and also yields reasonable performance in the absence of any fully labeled data.

Abstract. In this paper, we address the semi-supervised learning problem when there is a small amount of labeled data augmented with pairwise constraints indicating whether a pair of examples belongs to a same class or different classes. We introduce a discriminative learning approach that incorporates pairwise constraints into the conventional margin-based learning framework. We also present an efficient algorithm, PCSVM, to solve the pairwise constraint learning problem. Experiments with 15 data sets show that pairwise constraint information significantly increases the performance of classification. Key words: classification, pairwise constraints, margin-based learning 1

We describe the ideas and methodologies that we developed in addressing the KDD Cup 2008 on early breast cancer detection, and discuss how they contributed to our success. The most important components of our solution were 1) the identification of predictive information in the patient identifier, 2) a linear SVM on the 117 provided features, and 3) a heuristic post-processing approach to optimize the evaluation criteria. 1. TASK AND DATA DESCRIPTION The KDD Cup 2008 was organized by Siemens medical solutions and consisted of two prediction tasks in breast cancer detection from images. The organizers provided data from 1712 patients for training; of these 118 had cancer. Siemens uses proprietary software to identify in each image (two views for each breast) suspect locations (called candidates),