My thesis research focuses on a sub-discipline of machine learning known as semi-supervised learning where large amounts of unannotated data is available in addition to the annotated training sets used in traditional supervised learning. For my thesis, my collaborators and I developed methods for inferring domain-specific prior distributions over the parameters of a machine classifier from this pool of unannotated data. The methods developed are similar to those of transductive methods which propagate predictions over graphs of instances. Such methods essentially induce a Gaussian prior with a non-diagonal covariance matrix over the predictions of a non-parametric classifier. In contrast, the methods we developed specify a Gaussian prior with nondiagonal covariance matrix over the parameters of a parametric classifier. Consequently, classifiers learned using parameter network priors can be applied to instances not available at training time, something which transductive learners typically cannot do. Furthermore, the covariance matrix of these priors is specified as a sparse parameter network and is very efficient to use when training a model. While most regularization methods assume independence between model parameters, parameter network priors allow us to generate a full covariance matrix over the model parameters but without hitting memory limitations that would be encountered were we to specify the covariance matrix directly. Using statistics mined from pools of unlabeled data to construct the parameter network, my collaborators and

Even though this thesis bears my name, a lot of people have contributed, directly or indirectly, over the course of my graduate studies. First and foremost is my academic advisor, Dr Lyle Ungar. His supervision, guidance and ideas, his support, encouragement and motivation over the years were a sufficient and, more importantly, a necessary condition for completing my thesis. I would not have made it without his help and I owe him my deepest gratitude. Second only to my adviser are my collaborators. The work we did together has become a big part of the thesis. Dr Shane Jensen, who is also a member of my dissertation committee, helped me in presenting and clarifying the information in many chapters and contributed in the work of chapter 4. Dr Phineas Upham helped me a lot in gathering the data and analyzing the clusters in chapter 2. Ted Sandler (who is also to join the league of doctors in a few months- good luck in your defense Ted!) helped in chapter 4 and in many other places too numerous to count. And of course the members of my thesis committee, Doctors Mitch Marcus, Ben Taskar, David Blei and Shane Jensen whose insightful comments and suggestions helped make this dissertation so much better. There are many more people who were not directly involved in the work presented in this thesis, but who were indirectly responsible for its completion. First, the faculty of the department of Computer and Information Science in the University of Pennsylvania. They accepted my application, taught me a lot about Computer Science and provided an environment with plenty of opportunities to improve, to the point that I could fulfill the requirements to become a Doctor of Philosophy. ii I am grateful that they were willing to bet on my success as a graduate student. Second, the Greek State Scholarship Foundation (IKY) for providing partial financial support during my studies. It was an honor for me to receive that scholarship.

First and foremost, I would like to thank my academic advisor, Lyle Ungar. Lyle