An Integrated Framework for Processing Grammatically Ill-Formed Sentences

Abstract

Interaction with computers using natural language has been a major goal of artificial intelligence. Though many natural language processing systems have been developed, most of them assume that input sentences are grammatically correct. However, when users communicate with the system, they often ues grammatically ill-formed sentences, especially in spoken dialogues. For example, the users omit some words, change the word order, or make some careless errors such as agreement errors, misspellings or adding of extra words. To use NLP systems in real applications, we need to construct an NLP system that can handle not only grammatically well-formed inputs but also grammatically ill-formed inputs. This paper describes an integrated method for processing grammatically ill-formed inputs. We use partial parses of the input sentence for recovering from parsing failure. In order to select partial parses appropriate for error recovery, cost and reward are assigned to them. The notion of cost and reward is introduced in order to select a partial parse appropriate for error recovery. Cost and reward represent the badness and goodness of a partial parse, respectively. The most appropriate partial parse is selected on the basis of cost and reward trade-off. Cost is calculated by cost-based unification proposed here. The system contains three modules. Module A handles local ill-formedness such as constraint violations. Module B handles non-local ill-formedness such as word order violations, and Module C handles non-local ill-formedness such as contextual ellipses. These three modules work in a uniform framework based on the notions of cost and reward.