Abstract

This paper presents TextTiling, a method for partitioning full-length text documents into coherent multiparagraph units. The layout of text tiles is meant to reflect the pattern of subtopics contained in an expository text. The approach uses lexical analyses based on tf.idf, an information retrieval measurement, to determine the extent of the tiles, incorporating thesaural information via a statistical disambiguation algorithm. The tiles have been found to correspond well to human judgements of the major subtopic boundaries of science magazine articles.

Citations

1302	WordNet: an online lexical database - Miller, Beckwith, et al. - 1990
1095	Automatic text processing: The transformation, analysis, and retrieval of information by computer - Salton
920	Attention, intentions, and the structure of discourse - Grosz, Sidner - 1986
352	Rhetorical Structure Theory: A Theory of Text Organization (No. ISI/RS-87–190). Marina del Rey - Mann, Thompson - 1987
269	Lexical cohesion computed by thesaural relations as an indicator of the structure of text - Morris, Hirst - 1991
169	Subtopic Structuring for Full-length Document Access - Hearst, Plaunt - 1993
63	The intonational structuring of discourse - Hirschberg, Pierrehumbert - 1986
52	Redundancy in collaborative dialogue - Walker - 1992
41	Lexical cohesion, the thesaurus, and the structure of text - Morris, Hirst - 1991
33	Generating Text without Trees - Sibun - 1992
31	Automatic indexing using selective NLP and first-order thesauri - Evans, Ginther-Webster, et al. - 1991
24	Talking Voices: Repetition, Dialogue, and Imagery in Conversational Discourse - Tannen - 1989
21	Parsimonious and Profligate Approaches to the Question of Discourse Structure Relations - Hovy - 1990
16	Scale space filtering: A new approach to multiscale description - Witkin - 1984
15	Adaptive method of automatic abstracting and indexing - Skorochod’ko - 1972
14	Discourse Analysis. Cambridge Textbooks in Linguistics - Brown, Yule - 1998
13	Clustering analysis of subjective partitions of text - Rotondo - 1984
11	The generation of high-level structure for extended explanations - Mooney, Carberry, et al. - 1990
3	Language and Writing: Applications of Linguistics to Rhetoric and Composition - Raskin - 1987
2	Effects of text organization on memory; test of a cognitive effort hypothesis with limited exposure time - Britton, Muth - 1986
2	What do paragraph markers do - Stark - 1988
1	A parser for real-time speech synthesis of conversational texts - Bachenko, Daugherty - 1992
1	Acoustic segmentation and classification - Glass - 1987
1	On the psychological reality of story structure - Mandler - 1987
1	Micro- and macroprocesses in text comprehension - Vipond - 1980