A Compression-based Algorithm for Chinese Word Segmentation

A Compression-based Algorithm for Chinese Word Segmentation (0)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by W. J. Teahan , Yingying Wen , Rodger Mcnab , Ian H. Witten

Venue:	Computational Linguistics
Citations:	48 - 7 self

Versions

Version History

Metadata Version 3

User correction supplied by SystemCorrections

Datum	Value	Source
TITLE	A Compression-based Algorithm for Chinese Word Segmentation	INFERENCE
AUTHOR NAME	W. J. Teahan	user correction - Legacy Corrections
AUTHOR AFFIL	The Robert Gordon University University of Waikato	user correction - Legacy Corrections
AUTHOR NAME	Yingying Wen	user correction - Legacy Corrections
AUTHOR AFFIL	The Robert Gordon University University of Waikato	user correction - Legacy Corrections
AUTHOR NAME	Rodger Mcnab	user correction - Legacy Corrections
AUTHOR AFFIL	University of Waikato University of Waikato	user correction - Legacy Corrections
AUTHOR NAME	Ian H. Witten	user correction - Legacy Corrections
AUTHOR AFFIL	University of Waikato University of Waikato	user correction - Legacy Corrections
ABSTRACT	This paper describes a general scheme for segmenting text by inferring the position of word boundaries, thus supplying a necessary preprocessing step for applications like those mentioned above. Unlike other approaches, which involve a dictionary of legal words and are therefore language-specific, it works by using a corpus of already segmented text for training and thus can easily be retargeted for any language for which a suitable corpus of segmented material is available. To infer word boundaries, a general adaptive text compression technique is used that predicts upcoming characters on the basis of their preceding context. Spaces are inserted into positions where their presence enables the text to be compressed more effectively. This approach means that we can capitalize on existing research in text compression to create good models for word segmentation. To build a segmenter for a new language, the only resource required is a corpus of segmented text to train the compression model...	user correction - Legacy Corrections
YEAR	0	user correction - Legacy Corrections
VENUE	Computational Linguistics	INFERENCE
VENUE TYPE	JOURNAL	INFERENCE
PAGES	375--393	INFERENCE
VOLUME	26	INFERENCE
CITATIONS	26 found	ParsCit 1.0