Analyzing Character-level representations for Multilingual DRS Semantic Parsing

Abstract

Even in the era of massive language models, it has been suggested that character-level representations improve the performance of neural models. The state-of-the-art neural semantic parser for Discourse Representation Structures (DRSs) uses character-level representations, improving performance in all four languages on the Parallel Meaning Bank dataset. However, how and why character-level information improves the parser's performance remains unclear. This study provides in-depth analyses of performance changes by order of character sequences. In the experiments, we compare F1-scores by shuffling the order and randomizing character sequences. Our results indicate that the neural DRS parser is not sensitive to correct character order in English, German, and Dutch. Although we observe overall improvements by incorporating character-level tokens in German, Dutch, and Italian, we find hundreds of cases in which character-level tokens decrease performance.