Visualization of Conjugate Distributions in Latent Dirichlet Allocation Method

Abstract

In Bayesian probability theory, if the posterior distributions p(θ|x) are in the same family as the prior probability distribution p(θ), the prior and posterior are then called conjugate distributions, and the prior is called a conjugate prior for the likelihood function. In the Latent Dirichlet Allocation model, the likelihood function is Multinomial and the prior function is Dirichlet. There the Dirichlet distribution is a conjugate prior and then the posterior function becomes also Dirichlet. The posterior function is a parameter mixture distribution where the parameter of the likelihood function is distributed according to the given Dirichlet function. Because the process is complicated, it is hard to understand the process. The paper visualizes the parameter mixture distribution. The visualization is helpful to understand the algorithm of the Latent Dirichlet Allocation method.