Abstract
This paper describes an association model for natural language generation based on motion symbol sequences. In particular, our method enables the model to not only recognize motions but infer the actor of those motions, because a motion sequence is often unique to a certain kind of people. For example, in a baseball game, a pitcher throws a ball much more times than any other players on the field. We constructed this model with Conditional Random Fields, while motion recognition is accomplished by Hidden Markov Models. In the learning phase, this model breaks training sentences into pieces, learn the transition of each piece corresponding to motion symbols, and reconstructs them when generating. This method successfully generates sentences including proper actors. We further discuss the possibility of inference with unknown motion sequences and motions.
