We describe experiments that show that the concepts of rhetorical analysis and nuclearity can be used effectively for determining the most important units in a text. We show how these concepts can be implemented and we discuss results that we obtained with a discourse-based summarization program. 1 Motivation The evaluation of automatic summarizers has always been a thorny problem: most papers on summarization describe the approach that they use and give some "convincing " samples of the output. In very few cases, the direct output of a summarization program is compared with a human-made summary or evaluated with the help of human subjects; usually, the results are modest. Unfortunately, evaluating the results of a particular implementation does not enable one to determine what part of the failure is due to the implementation itself and what part to its underlying assumptions. The position that we take in this paper is that, in order to build high-quality summarization programs, one ...

We derive the rhetorical structures of texts by means of two new, surface-form-based algorithms: one that identifies discourse usages of cue phrases and breaks sen- tences into clauses, and one that produces valid rhetorical structure trees for unre- stricted natural language texts. The algo- rithms use information that was derived from a corpus analysis of cue phrases.

We present a new, data-driven approach to text planning, which can be used not only to map full knowledge pools into natural language texts, but also to generate texts that satisfy multiple, high-level communicative goals. The approach explains how global coherence can be achieved by exploiting the local coherence constraints of rhetorical relations. The local constraints were derived from a corpus analysis. Motivation 1 All current flexible approaches to text planning that assume that the abstract structure of text is a tree-like structure are, esentially, top-down approaches. Some of them define plan operators and exploit hierarchical planning techniques (Hovy 1993; Moore and Paris 1993; Moore and Swartout 1991; Cawsey 1991; Maybury 1992) and partial-order planning techniques (Young and Moore 1994). Others assume that plans are hierarchically organized sets of frames that can be derived through a top-down expansion process (Nirenburg et al. 1989; Meteer 1992). And the recursive app...

This paper exploresthe extent to which well-formed rhetorical structures can be automatically derived by means of surface-form-based algorithms. These algorithms identify discourse usages of cue phrases and break sentences into clauses, hypothesize rhetorical relations that holdamong textual units, and produce valid rhetorical structure trees for unrestricted natural language texts. The algorithms are empirically grounded in a corpus analysis of cue phrases and rely on a #rst-order formalization of rhetorical structure trees

Abstract not found

A method is described by which a rhetoricalstructure tree can bc realized by a text structure made up of sections, paragraphs, sentences, vertical lists, and other textual patterns, with discourse connectives added (in the correct positions) to mark rhetorical relations. We show that text-structuring can be forInulated as a Constraint Satisfaction Prob- lem, so that all solutions respecting constraints on text-structure formation and structural comt)atibility can be efficiently generated. Of the nany solutions generated by this method, some are stylistically preferable to others; we show how further constraints can be applied in order to select the best versions. Finally, we discuss some extensions such as the generation of indented text structures.

The way in which discourse features express connections back to the previous dis- course has been described in the literature in terms of adjoining at the right frontier of discourse structure. But this does not allow for discourse features that express expectations about what is to come in the subsequent discourse. After characterizing these expectations and their distribution in text, we show how an approach that makes use of substitution as well as adjoining on a suitably defined right frontier, can be used to both process expectations and constrain discouse processing in general.

Abstract not found

RST-Tool is a graphical interface for marking up the rhetorical structure of a text. This

Information Retrieval (IR) is the discipline that deals with retrieval of unstructured data, especially textual documents, in response to a query or topic statement, which may itself be unstructured, e.g., a sentence or even another document, or which may be structured, e.g., a boolean expression. The need for effective methods of automated IR has grown in importance because of the tremendous explosion in the amount of unstructured data, both internal, corporate document collections, and the immense and growing number of document sources on the Internet. This report is a tutorial and survey of the state of the art, both research and commercial, in this dynamic field. The topics covered include: formulation of structured and unstructured queries and topic statements, indexing (including term weighting) of document collections, methods for computing the similarity of queries and documents, classification and routing of documents in an incoming stream to users on the basis of topic or nee...