On Representing the Set of All Parse Trees with a Decision Diagram

Abstract

In this paper, we construct decision diagrams (DDs) representing the set of all parse trees of a context-free grammar (CFG) from a sequence data and analyze DD size. CFG is widely used in the field of natural language processing and bioinformatics to estimate the hidden structures of sequence data. A decision diagram is a data structure that represents a Boolean function in a concise form. By using DDs to represent the set of all parse trees, we can efficiently perform many useful operations over the parse trees, such as finding parse trees that satisfy additional constraints and finding the most probable parse tree. Since the time complexity of these operations strongly depends on DD size, selecting an appropriate DD variant is important. Experiments on the parse trees of a simple CFG show that the Zero-suppressed Sentential Decision Diagram (ZSDD) is better than other DDs; we also give theoretical upper bounds on ZSDD size of a simple CFG. Moreover, we propose an efficient method based on CYK (Cocke-Younger-Kasami) algorithm to construct ZSDDs that can represent the set of all parse trees. Experiments show that the method can construct ZSDDs much faster than the naive method based on compiling a Boolean function.