Towards complex features: Competitive receptive fields in unsupervised deep networks

In this paper we propose a simple unsupervised approach to learning higher order features. This model is based on the recent success of lightweight approaches such as SOMNet and PCANet to the challenging task of image classification. Contrary to the more complex deep learning models such as convolutional neural networks (CNNs), these methods use naive algorithms to model the input distribution. Our endeavour focuses on the self-organizing map (SOM) based method and extends it by incorporating a competitive connection layer between filter learning stages. This simple addition encourages the second filter learning stage to learn complex combinations of first layer filters and simultaneously decreases channel depth. This approach to learning complex representations offers a competitive alternative to common deep learning models whilst maintaining an efficient framework. We test our proposed approach on the popular MNIST and challenging CIFAR-10 datasets.