In this paper, we investigate a new machine learning framework called Online Transfer Learning (OTL) that aims to transfer knowledge from some source domain to an online learning task on a target domain. We do not assume the target data follows the same class or generative distribution as the source data, and our key motivation is to improve a supervised online learning task in a target domain by exploiting the knowledge that had been learned from large amount of training data in source domains. OTL is in general challenging since data in both domains not only can be different in their class distributions but can be also different in their feature representations. As a first attempt to this problem, we propose techniques to address two kinds of OTL tasks: one is to perform OTL in a homogeneous domain, and the other is to perform OTL across heterogeneous domains. We show the mistake bounds of the proposed OTL algorithms, and empirically examine their performance on several challenging OTL tasks. Encouraging results validate the efficacy of our techniques. 1.

We present a hierarchical approach for information sharing among different classification tasks. Our core approach is designed for the joint learning scenario and is later extended to the knowledge-transfer scenario. In the joint learning scenario we consider multi-task and multi-class settings. We engage a top-down iterative method, which begins by posing an optimization problem with an incentive for large scale sharing among all classes. This incentive to share is gradually decreased, until there is no sharing and all tasks are considered separately. The method therefore exploits different levels of sharing within a given group of related tasks, without having to make hard decisions about the grouping of tasks. In order to deal with large scale problems, with many tasks and many classes, we extend our batch approach to an online setting and provide regret analysis of the algorithm. Based on the structure of shared information discovered in the joint learning setttings we propose two different knowledge-transfer methods, for learning novel tasks, specificaly fitting large scale settings where the question of ’what to transfer? ’ is even more challanging. We tested our approach extensively on synthetic and real datasets, showing significant improvement over baseline and state-of-theart methods. 1.