Abstract

Latent variables allow probabilistic graphical models to capture nuance and structure in im-portant domains such as network science, natural language processing, and computer vision. Naive approaches to learning such complex models can be prohibitively expensive—because they require repeated inferences to update beliefs about la-tent variables—so lifting this restriction for use-ful classes of models is an important problem. Hinge-loss Markov random fields (HL-MRFs) are graphical models that allow highly scalable inference and learning in structured domains, in part by representing structured problems with continuous variables. However, this representa-tion leads to challenges when learning with la-tent variables. We introduce paired-dual learn-ing, a framework that greatly speeds up training by using tractable entropy surrogates and avoid-ing repeated inferences. Paired-dual learning op-timizes an objective with a pair of dual inference problems. This allows fast, joint optimization of parameters and dual variables. We evaluate on social-group detection, trust prediction in social networks, and image reconstruction, finding that paired-dual learning trains models as accurate as those trained by traditional methods in much less time, often before traditional methods make even a single parameter update. 1.