Temporal Ordering Annotation on ‘the Balanced Corpus of Contemporary Written Japanese’

Abstract

Temporal information extraction can be divided into the following tasks: temporal expression extraction, time normalization and temporal ordering relation resolution. The first task is a subtask of a named entity and numeral expression extraction. The second task is often performed by rewriting systems. The third task consists of event anchoring. This paper proposed a Japanese temporal ordering annotation scheme and performed annotations by referring to ‘the Balanced Corpus of Contemporary Written Japanese’ (BCCWJ). We extracted verbal and adjective event expressions as 〈EVENT〉 in a subset of BCCWJ and annotated a temporal ordering relation 〈TLINK〉 on the pairs of the above event expressions and time expressions obtained from a previous study (Konishi et al. 2013). The recognition of temporal ordering by language recipients tends to disagree compared to the normalization of time expressions. We should not regard making unique gold annotation data as an objective in such a situation. If anything, we should evaluate the degree of inter-annotator discrepancy by subjects of experiments. Then, we analysed inter-annotator discrepancies by three annotators in temporal ordering annotation. The result showed that boundaries of time segments barely exhibit any agreement, whereas the annotation of temporal relative ordering tendency exhibits good agreement by the annotators.