As a first step of word sense disambiguation (WSD) errors analysis, generally we need investigate the causes of errors and classify them. For this purpose, seven analysts classified the error data for analysis from their unique standpoints. Next, we attempted to merge the results from the analyses. However, merging these results through discussions was difficult because the results differed significantly. Therefore, we used a clustering method for a certain level of automatic merger. Consequently, we classified WSD errors into nine types, and it turned out that the three main types of errors covers 90% of the total WSD errors. Moreover, we showed that the merged error types represented seven results and was standardized by defining the similarity between two classifications and comparing it with each analysis result.

View full abstract

The development and spread of social media services have made it possible for new approaches to be used to survey the public and society. One popular application is health surveillance, that is, predicting disease epidemics and symptoms from texts on social media services. In this paper, we address an application of natural language processing for detecting an episode of a disease/symptom (e.g., flu and cold) in social media texts. Following an error analysis of the state-of-the-art system, we identified two important and generic subtasks for improving the accuracy of the system: factuality analysis and subject identification. We address these subtasks and demonstrate their impact on detecting an episode of a disease/symptom.

View full abstract

Event factuality is information pertaining to whether events mentioned in the natural language correspond to either actual events that have occurred in the real world or events that are of uncertain interpretation. In factuality analysis, sufficient performance is yet to be achieved because of the complexity of issues such as functional expression and linguistic scope. This paper discusses the issues involved in factuality analysis by analyzing errors when applying a rule-based system to 3,734 events in 1,533 sentences. We annotate functional expression labels for all events. In the main events, the factuality analyzer, consisting of simple functional expression rules, achieves approximately 90% accuracy if correct functional expression labels are provided. In subordinate events, we found many errors specific to subordinate events, such as errors caused by predicates and linguistic scopes. We provide guidelines for factuality analysis through additional discussion regarding predicates and linguistic scope.

View full abstract

This paper provides a deep analysis of linguistic phenomena related to argument ellipsis, one of central issues for improving the accuracy of Japanese predicate-argument structure analysis. We specifically focus on cases where a target predicate and its ellipsed argument appear in the same sentence, and we categorize instances based on two criteria: a clue annotation by a human annotator and systematic categorization based on both syntactic and semantic structure. We then show both the distribution of instances among the categories and the accuracy for each category achieved by a state-of-the-art system. As a result, we show that 58% of the intra-sentencial zero anaphora are the case when an argument of a target predicate P is shared with another predicate O that is in a direct syntactic dependency relation with the argument. This fact implies that analyzing syntactic and semantic relations between O and P is important for Japanese predicate-argument structure analysis. We also show that the distribution of clue combinations is very broad. Finally, we discovered that not only are there cases where each clue independently increases the certainty, but we also discovered cases where clues became relevant when all of them composed a chain.

View full abstract