This paper consider sexploratory methods for investigating the factor invariance or factor changes in multigroup, multiset, and three-mode data, orthogonal rotations achieving the congruence of loading matrices, canonical analysis of component scores, and simultaneous principal component analytical methods with various optimization criteria and constraints. Desirable properties which should be shared by exploratory factor analytical methods are discussed based on simple schematic models of differences of factor loadings and factor scores between two data matrices. The hierarchical principal component analysis method is derived as an extension of van de Geer's Maxbet method for multiset data. The method is equivalent to the slightly generalized Tuckals2 model(Kroonenberg and de Leeuw)for three-mode data, and can be modified to be applicable to multigroup data. It is demonstrated that these methods have the desirable properties as exploratory analysis.