Abstract

Document representation is a crucial step in any Information Retrieval (IR) system. Since most of the traditional methods do not consider much of semantic or syntactic information, the representation becomes insuciently informative for an IR task. We describe a novel approach to incorporating Natural Language Processing (NLP) in document representation for addressing this problem. Use is made of additional information about the sentences, viz., (i) syntactic links among the words found by the Link Parser and (ii) heuristically determined semantic attributes of the words. After mapping this information to the document level using Self-Organizing Map (SOM), we use it for embellishing the document vectors constructed by the TFIDF method. The ecacy of the proposed method is established by showing that the document vectors (i) have higher mutual information content and (ii) achieve better class separation.

Citations