Automatic Machine Translation Evaluation with Part-of-Speech Information

Lifeng Han, Derek F. Wong, Lidia S. Chao, Liangye He

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review


One problem of automatic translation is the evaluation of the result. The result should be as close to a human reference translation as possible, but varying word order or synonyms have to be taken into account for the evaluation of the similarity of both. In the conventional methods, researchers tend to employ many resources such as the synonyms vocabulary, paraphrasing, and text entailment data, etc. To make the evaluation model both accurate and concise, this paper explores the evaluation only using Part-of-Speech information of the words, which means the method is based only on the consilience of the POS strings of the hypothesis translation and reference. In this developed method, the POS also acts as the similar function with the synonyms in addition to its syntactic or morphological behaviour of the lexical item in question. Measures for the similarity between machine translation and human reference are dependent on the language pair since the word order or the number of synonyms may vary, for instance. This new measure solves this problem to a certain extent by introducing weights to different sources of information. The experiment results on English, German and French languages correlate on average better with the human reference than some existing measures, such as BLEU, AMBER and MP4IBM1.
Original languageEnglish
Title of host publicationText, Speech, and Dialogue
Subtitle of host publication16th International Conference, TSD 2013, Pilsen, Czech Republic, September 1-5, 2013, Proceedings
EditorsIvan Habernal, Václav Matoušek
Place of PublicationHeidelberg
PublisherSpringer Berlin
Number of pages8
ISBN (Electronic)9783642405853
ISBN (Print)9783642405846
Publication statusPublished - 22 Aug 2013

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


  • Natural language processing
  • Machine translation evaluation
  • Part-of-Speech
  • Reference translation


Dive into the research topics of 'Automatic Machine Translation Evaluation with Part-of-Speech Information'. Together they form a unique fingerprint.

Cite this