On Fuzzy c-Means using Tsallis and Renyi Divergences and Their Variants

Abstract

Hard clustering algorithms, such as the K-means or hard c-means (HCM) algorithm, assign each data point to a single cluster. In contrast, fuzzy clustering algorithms allow data points to belong to multiple clusters with varying degrees of membership. Among these, the fuzzy c-means (FCM) algorithm proposed by Bezdek is the most well-known, achieving fuzzification by raising membership degrees to a power in the objective function. This algorithm is referred to as the Bezdek-type FCM (BFCM) to distinguish it from its variants. BFCM was later extended by introducing cluster weights, resulting in the modified BFCM (mBFCM). Another line of development involves regularizing the objective function with divergences, leading to algorithms such as KLFCM (using the Kullback–Leibler divergence), TFCM (using the Tsallis divergence), and RFCM (using the R ́enyi divergence). Previous studies have reported mixed results regarding the accuracy of TFCM and RFCM on real datasets. Motivated by the observation that neither TFCM nor RFCM consistently outperforms the other, this study proposes a new algorithm that combines their advantages. First, we point out the inadequacy of the original optimization formulation in RFCM and propose a revised version, referred to as revised RFCM (rRFCM). Then, we introduce two generalized algorithms, SMFCM and MFCM, which utilize the Sharma-Mittal and Masi divergences, respectively — both of which generalize the Tsallis and R ́enyi divergences.