A Study of Chinese Word Segmentation Based on the Characteristics of Chinese

Lifeng Han, Derek F. Wong, Lidia S. Chao, Liangye He, Ling Zhu, Shuo Li

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

Abstract

This paper introduces the research on Chinese word segmentation (CWS). The word segmentation of Chinese expressions is difficult due to the fact that there is no word boundary in Chinese expressions and that there are some kinds of ambiguities that could result in different segmentations. To distinguish itself from the conventional research that usually emphasizes more on the algorithms employed and the workflow designed with less contribution to the discussion of the fundamental problems of CWS, this paper firstly makes effort on the analysis of the characteristics of Chinese and several categories of ambiguities in Chinese to explore potential solutions. The selected conditional random field models are trained with a quasi-Newton algorithm to perform the sequence labeling. To consider as much of the contextual information as possible, an augmented and optimized set of features is developed. The experiments show promising evaluation scores as compared to some related works.
Original languageEnglish
Title of host publicationLanguage Processing and Knowledge in the Web
Subtitle of host publication25th International Conference, GSCL 2013, Darmstadt, Germany, September 25-27, 2013, Proceedings
EditorsIryna Gurevych, Chris Biemann, Torsten Zesch
Place of PublicationHeidelberg
PublisherSpringer Berlin
Pages111-118
Number of pages8
ISBN (Electronic)9783642407222
ISBN (Print)9783642407215
DOIs
Publication statusPublished - 21 Aug 2013

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume8105
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Keywords

  • : Natural language processing
  • Chinese word segmentation
  • Characteristics of Chinese
  • Optimized features

Fingerprint

Dive into the research topics of 'A Study of Chinese Word Segmentation Based on the Characteristics of Chinese'. Together they form a unique fingerprint.

Cite this