Texts frequently contain expression whose meaning is not strictly literal, such as idioms. Idiomatic and non-literal expressions pose a major challenge to natural language processing technology as they often exhibit lexical and syntactic idiosyncrasies. We propose a novel unsupervised method for distinguishing literal and non-literal usages of expressions. Our method determines how well a literal interpretation of the expression is linked to the overall cohesive structure of the discourse. If only weak cohesive links can be found, the expression is classified as idiomatic. We propose two methods to model the cohesive links in our task: the lexical-chain-based approach and the cohesion-graph-based approach. While the chain-based approach is effective at distinguishing literal and non-literal usage, it is sensitive to chaining algorithms, parameter settings and data setup. We further develop the chain-based approach into a graph-based approach in order to overcome these problems. This development makes our cohesion-based approach unsupervised while maintaining a high performance.