Morphological analysis of the corpus of spontaneous Japanese

Kiyotaka Uchimoto; Kazuma Takaoka; Chikashi Nobata; Atsushi Yamada; Satoshi Sekine; Hitoshi Isahara

doi:10.1109/TSA.2004.828700

Morphological analysis of the corpus of spontaneous Japanese

Kiyotaka Uchimoto, Kazuma Takaoka, Chikashi Nobata, Atsushi Yamada, Satoshi Sekine, Hitoshi Isahara

Research output: Contribution to journal › Article › peer-review

Abstract

This paper describes two methods for detecting word segments and their morphological information in a Japanese spontaneous speech corpus, and describes how to tag a large spontaneous speech corpus accurately by using the two methods. The first method is used to detect any type of word segments. The second method is used when there are several definitions for word segments and their POS categories, and when one type of word segments includes another type of word segments. In this paper, we show that by using semi-automatic analysis, we achieve a precision of better than 99% for detecting and tagging short-unit words and 97% for long-unit words; the two types of words that comprise the corpus. We also show that better accuracy is achieved by using both methods than by using only the first.

Original language	English
Pages (from-to)	382-390
Number of pages	8
Journal	IEEE Transactions on Speech and Audio Processing
Volume	12
Issue number	4
DOIs	https://doi.org/10.1109/TSA.2004.828700
Publication status	Published - Jul 2004

Keywords

Japanese spontaneous speech corpus
Maximum entropy models
Morphological analysis
Natural language processing
Unknown words

Access to Document

10.1109/TSA.2004.828700

Cite this

@article{8e91afad0c6643f68feaad43013a1411,

title = "Morphological analysis of the corpus of spontaneous Japanese",

abstract = "This paper describes two methods for detecting word segments and their morphological information in a Japanese spontaneous speech corpus, and describes how to tag a large spontaneous speech corpus accurately by using the two methods. The first method is used to detect any type of word segments. The second method is used when there are several definitions for word segments and their POS categories, and when one type of word segments includes another type of word segments. In this paper, we show that by using semi-automatic analysis, we achieve a precision of better than 99% for detecting and tagging short-unit words and 97% for long-unit words; the two types of words that comprise the corpus. We also show that better accuracy is achieved by using both methods than by using only the first.",

keywords = "Japanese spontaneous speech corpus, Maximum entropy models, Morphological analysis, Natural language processing, Unknown words",

author = "Kiyotaka Uchimoto and Kazuma Takaoka and Chikashi Nobata and Atsushi Yamada and Satoshi Sekine and Hitoshi Isahara",

year = "2004",

month = jul,

doi = "10.1109/TSA.2004.828700",

language = "English",

volume = "12",

pages = "382--390",

journal = "IEEE Transactions on Speech and Audio Processing",

issn = "1558-2353",

publisher = "IEEE",

number = "4",

}

TY - JOUR

T1 - Morphological analysis of the corpus of spontaneous Japanese

AU - Uchimoto, Kiyotaka

AU - Takaoka, Kazuma

AU - Nobata, Chikashi

AU - Yamada, Atsushi

AU - Sekine, Satoshi

AU - Isahara, Hitoshi

PY - 2004/7

Y1 - 2004/7

N2 - This paper describes two methods for detecting word segments and their morphological information in a Japanese spontaneous speech corpus, and describes how to tag a large spontaneous speech corpus accurately by using the two methods. The first method is used to detect any type of word segments. The second method is used when there are several definitions for word segments and their POS categories, and when one type of word segments includes another type of word segments. In this paper, we show that by using semi-automatic analysis, we achieve a precision of better than 99% for detecting and tagging short-unit words and 97% for long-unit words; the two types of words that comprise the corpus. We also show that better accuracy is achieved by using both methods than by using only the first.

AB - This paper describes two methods for detecting word segments and their morphological information in a Japanese spontaneous speech corpus, and describes how to tag a large spontaneous speech corpus accurately by using the two methods. The first method is used to detect any type of word segments. The second method is used when there are several definitions for word segments and their POS categories, and when one type of word segments includes another type of word segments. In this paper, we show that by using semi-automatic analysis, we achieve a precision of better than 99% for detecting and tagging short-unit words and 97% for long-unit words; the two types of words that comprise the corpus. We also show that better accuracy is achieved by using both methods than by using only the first.

KW - Japanese spontaneous speech corpus

KW - Maximum entropy models

KW - Morphological analysis

KW - Natural language processing

KW - Unknown words

U2 - 10.1109/TSA.2004.828700

DO - 10.1109/TSA.2004.828700

M3 - Article

SN - 1558-2353

VL - 12

SP - 382

EP - 390

JO - IEEE Transactions on Speech and Audio Processing

JF - IEEE Transactions on Speech and Audio Processing

IS - 4

ER -

Morphological analysis of the corpus of spontaneous Japanese

Abstract

Keywords

Access to Document

Fingerprint

Cite this