Decomposition methods for unsupervised learning

Abstract

This thesis presents the application and development of decomposition methods for Unsupervised Learning. It covers topics from classical factor analysis based decomposition and its variants such as Independent Component Analysis, Non-negative Matrix Factorization and Sparse Coding to their generalizations to multi-way array, i.e. tensor decomposition, through models such as the CanDecomp/ PARAFAC and the Tucker model. Extensions for these types of decomposition models to incorporate shift, reverberation and general transformations are also described. Finally, a connection between decomposition methods and clustering problems is derived both in terms of classical point clustering but also in terms of community detection in complex networks. A guiding principle throughout this thesis is the principle of parsimony. Hence, the goal of Unsupervised Learning is here posed as striving for simplicity in the decompositions. Thus, it is demonstrated how a wide range of decomposition methods explicitly or implicitly strive to attain this goal. Applications of the derived decompositions are given ranging from multi-media analysis of image and sound data, analysis of biomedical data such as electroencephalography to the analysis of social network data