Correlation between human assessment of essays and ROUGE evaluation of essays' summaries

Seemab Latif; Mary McGee Wood; Goran Nenadic

doi:10.1109/SNLP.2009.5340933

Correlation between human assessment of essays and ROUGE evaluation of essays' summaries

Seemab Latif, Mary McGee Wood, Goran Nenadic

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In this paper we have addressed the qualitative (human evaluation) and quantitative (ROUGE) evaluation of computer generated summaries of the students' essays. The experimental results show that there is a positive high correlation between ROUGE scores and human assessment of the essays (human assigned marks). We have also found out that human evaluation of the automatic summaries positively correlates with the human assessment of the essays. These correlations can be used to classify students' essays into broad bands of quality. © 2009 IEEE.

Original language	English
Title of host publication	2009 8th International Symposium on Natural Language Processing, SNLP '09\|Int. Symp. Nat. Lang. Process., SNLP
Pages	122-127
Number of pages	5
DOIs	https://doi.org/10.1109/SNLP.2009.5340933
Publication status	Published - 2009
Event	2009 8th International Symposium on Natural Language Processing, SNLP '09 - Bangkok Duration: 1 Jul 2009 → …

Conference

Conference	2009 8th International Symposium on Natural Language Processing, SNLP '09
City	Bangkok
Period	1/07/09 → …

Access to Document

10.1109/SNLP.2009.5340933

Cite this

@inproceedings{a02d57d9e5ab4dd3b152b5d07d4c71e7,

title = "Correlation between human assessment of essays and ROUGE evaluation of essays' summaries",

abstract = "In this paper we have addressed the qualitative (human evaluation) and quantitative (ROUGE) evaluation of computer generated summaries of the students' essays. The experimental results show that there is a positive high correlation between ROUGE scores and human assessment of the essays (human assigned marks). We have also found out that human evaluation of the automatic summaries positively correlates with the human assessment of the essays. These correlations can be used to classify students' essays into broad bands of quality. {\textcopyright} 2009 IEEE.",

author = "Seemab Latif and Wood, {Mary McGee} and Goran Nenadic",

year = "2009",

doi = "10.1109/SNLP.2009.5340933",

language = "English",

isbn = "9781424441389",

pages = "122--127",

booktitle = "2009 8th International Symposium on Natural Language Processing, SNLP '09|Int. Symp. Nat. Lang. Process., SNLP",

note = "2009 8th International Symposium on Natural Language Processing, SNLP '09 ; Conference date: 01-07-2009",

}

Latif, S, Wood, MM & Nenadic, G 2009, Correlation between human assessment of essays and ROUGE evaluation of essays' summaries. in 2009 8th International Symposium on Natural Language Processing, SNLP '09|Int. Symp. Nat. Lang. Process., SNLP. pp. 122-127, 2009 8th International Symposium on Natural Language Processing, SNLP '09, Bangkok, 1/07/09. https://doi.org/10.1109/SNLP.2009.5340933

Correlation between human assessment of essays and ROUGE evaluation of essays' summaries. / Latif, Seemab; Wood, Mary McGee; Nenadic, Goran.
2009 8th International Symposium on Natural Language Processing, SNLP '09|Int. Symp. Nat. Lang. Process., SNLP. 2009. p. 122-127.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Correlation between human assessment of essays and ROUGE evaluation of essays' summaries

AU - Latif, Seemab

AU - Wood, Mary McGee

AU - Nenadic, Goran

PY - 2009

Y1 - 2009

N2 - In this paper we have addressed the qualitative (human evaluation) and quantitative (ROUGE) evaluation of computer generated summaries of the students' essays. The experimental results show that there is a positive high correlation between ROUGE scores and human assessment of the essays (human assigned marks). We have also found out that human evaluation of the automatic summaries positively correlates with the human assessment of the essays. These correlations can be used to classify students' essays into broad bands of quality. © 2009 IEEE.

AB - In this paper we have addressed the qualitative (human evaluation) and quantitative (ROUGE) evaluation of computer generated summaries of the students' essays. The experimental results show that there is a positive high correlation between ROUGE scores and human assessment of the essays (human assigned marks). We have also found out that human evaluation of the automatic summaries positively correlates with the human assessment of the essays. These correlations can be used to classify students' essays into broad bands of quality. © 2009 IEEE.

U2 - 10.1109/SNLP.2009.5340933

DO - 10.1109/SNLP.2009.5340933

M3 - Conference contribution

SN - 9781424441389

SP - 122

EP - 127

BT - 2009 8th International Symposium on Natural Language Processing, SNLP '09|Int. Symp. Nat. Lang. Process., SNLP

T2 - 2009 8th International Symposium on Natural Language Processing, SNLP '09

Y2 - 1 July 2009

ER -

Correlation between human assessment of essays and ROUGE evaluation of essays' summaries

Abstract

Conference

Access to Document

Fingerprint

Cite this