Coupled Helmholtz machine for PCA

Abstract

A coupled Helmholtz machine for principal component analysis (PCA), where sub-machines are related through sharing some latent variables and associated weights, is presented. A wake-sleep PCA algorithm for training the coupled Helmholtz machine is then presented, showing that the algorithm iteratively determines principal eigenvectors of a data covariance matrix without any rotational ambiguity, in contrast to some existing methods that perform factor analysis or principal subspace analysis. The coupled Helmholtz machine provides a unified view of principal component analysis, including various existing algorithms as its special cases. The validity of the wake-sleep PCA algorithm is confirmed by numerical experiments.