A Generalized Information Bottleneck Method: A Decision-Theoretic Perspective

抄録

The information bottleneck (IB) method seeks a compressed representation of data that preserves information relevant to a target variable for prediction while discarding irrelevant information from the original data. In its classical formulation, the IB method employs mutual information to evaluate the compression between the original and compressed data, as well as the utility of the representation for the target variable. In this study, we investigate a generalized IB problem, where the evaluation of utility is based on the H-mutual information that satisfies the concave (CV) and averaging (AVG) conditions. This class of information measures admits a statistical decision-theoretic interpretation via its equivalence to the expected value of sample information. Based on this interpretation, we derive a block coordinate descent algorithm to assess the tradeoff between compression and utility in the generalized IB problem.