Many NLP tasks rely on accurately estimating word dependency probabilities P(w 1 |w 2 ), where the words w 1 and w 2 have a particular relationship (such as verb-object). Because of the sparseness of counts of such dependencies, smoothing and the ability to use multiple sources of knowledge are important challenges. For example, if the probability P(N ) of noun N being the subject of verb V is high, and V takes similar objects to V # , and V # is synonymous to V ## , then we want to conclude that P(N ## ) should also be reasonably high---even when those words did not cooccur in the training data. To capture

Many NLP tasks rely on accurately estimating word dependency probabilities P(w 1 |w 2 ), where the words w 1 and w 2 have a particular relationship (such as verb-object). Because of the sparseness of counts of such dependencies, smoothing and the ability to use multiple sources of knowledge are important challenges. For example, if the probability P(N ) of noun N being the subject of verb V is high, and V takes similar objects to V # , and V # is synonymous to V ## , then we want to conclude that P(N ## ) should also be reasonably high---even when those words did not cooccur in the training data.