Large Margin Classification in Infinite Neural Networks
The authors introduce a new family of positive-definite kernels for large margin classification in Support Vector Machines (SVMs). These kernels mimic the computation in large neural networks with one layer of hidden units. They also show how to derive new kernels, by recursive composition, that may be viewed as mapping their inputs through a series of nonlinear feature spaces. These recursively derived kernels mimic the computation in deep networks with multiple hidden layers. They evaluate SVMs with these kernels on problems designed to illustrate the advantages of deep architectures. Comparing to previous benchmarks, they find that on some problems, these SVMs yield state-of-the-art results, beating not only other SVMs, but also deep belief nets.