[PDF][PDF] Inter-annotator Agreement for a German Newspaper Corpus.

T Brants - LREC, 2000 - Citeseer
T Brants
LREC, 2000Citeseer
This paper presents the results of an investigation on inter-annotator agreement for the
NEGRA corpus, consisting of German newspaper texts. The corpus is syntactically
annotated with part-of-speech and structural information. Agreement for part-of-speech is
98.6%, the labeled F-score for structures is 92.4%. The two annotations are used to create a
common final version by discussing differences and by several iterations of cleaning. Initial
and final versions are compared. We identify categories causing large numbers of …
Abstract
This paper presents the results of an investigation on inter-annotator agreement for the NEGRA corpus, consisting of German newspaper texts. The corpus is syntactically annotated with part-of-speech and structural information. Agreement for part-of-speech is 98.6%, the labeled F-score for structures is 92.4%. The two annotations are used to create a common final version by discussing differences and by several iterations of cleaning. Initial and final versions are compared. We identify categories causing large numbers of differences and categories that are handled inconsistently.
Citeseer