Towards Natural Language Inference with CCG Parser and Automated Theorem Prover

Abstract

This paper proposes a natural language inference system based on theoretical linguistics. The framework integrates lightblue, a robust syntactic and semantic parser based on combinatory categorical grammar (CCG) and dependent type semantics (DTS) that performs syntactic parsing, semantic composition, and type checking/inference, with wani, an automated theorem prover for dependent type theory. Since each computational step in lightblue corresponds to a theoretical assumption, its evaluation can be seen as a hypothesis verification of the adopted theories. In that sense, this framework is an example of a rationalist approach to computational linguistics, as opposed to the empiricist approach underlying large language models (LLMs). This paper outlines the implementation of our inference system and describes the evaluation methodology using the Japanese semantic test suit (JSeM). It also presents a methodology for providing feedback to linguistic theories and improving the system through error analysis based on the evaluation results.