Grouped Statistical Modeling for Heterogeneous Data

Abstract

In Big Data era, large-scale data is being utilized in a variety of fields. On the other hand, as data becomes larger and larger, there are more and more situations where heterogeneous groups are mixed together, and simple statistical modeling based on the conventional “one-model-fits-the-whole-population approach” is not sufficient to perform appropriate statistical analysis. Although various methodologies have already been developed for this situation, there is still a lack of methodologies that can perform flexible statistical modeling at a realistic computational cost. In this paper, we describe methodologies that can simultaneously perform grouping of data (discovery of heterogeneous groups) and estimation of statistical models for each group (discovery of the unique structure of each group) in the context of clustered data and spatial data.