Principal component analysis for topologists

GAUGE-TOPOLOGY-SYMPLECTIC

Speaker:

Jon Bloom - Broad Institute

Abstract:

Geometrically, PCA finds the k-dimensional subspace nearest a point cloud in m-dimensional Euclidean space. Numerically, PCA also specifies a basis for this subspace, the principal directions of the data. Deep learning-ly, PCA is the shallowest of autoencoder models, that of a linear autoencoder (LAE) which learns the subspace but not the basis. I’ll relate these views by considering the gradient dynamics of learning from the perspective of a simple F_2-perfect Morse function on the real Grassmannian. I’ll then prove that L_2-regularized LAEs are symmetric at all critical points and recover the principal directions as singular vectors of the decoder, with implications for eigenvector algorithms and perhaps learning in the brain. If asked, I’ll speculate on a role for Morse homology in deep learning. Based on “Loss Landscapes of Regularized Linear Autoencoders” (ICML 2019) with Daniel Kunin, Aleksandrina Goeva, and Cotton Seed at the Broad Institute.