An Asymptotic Analysis of Smooth Regularizers
Percy liang, Francis Bach, Guillaume Bouchard, Michael Jordan
Many types of regularization schemes have been employed in statistical learning, each motivated by some assumption about the problem domain. In this paper, we present a unified asymptotic analysis of smooth regularizers, which allows us to see how the validity of these assumptions impacts the success of a particular regularizer. In addition, our analysis motivates an algorithm for optimizing regularization parameters, which in turn can be analyzed within our framework. We apply our analysis to several examples, including hybrid generative-discriminative learning and multi-task learning.
NIPS 2009, (Neural Information Processing Systems Conference), Vancouver, B.C., Canada, 7-10 December, 2009. Full paper available on NIPS Website