Iteratively Reweighted Group Lasso Based on Log-Composite Regularization

Abstract

The paper considers supervised learning problems of labeled data with grouped input features. The groups are nonoverlapped such that the model coefficients corresponding to the input features form disjoint groups. The coefficients have group sparsity structure in the sense that coefficients corresponding to each group shall be simultaneously either zero or nonzero. To make effective use of such group sparsity structure given a priori, we introduce a novel log-composite regularizer, which can be minimized by an iterative algorithm. In particular, our algorithm iteratively solves for a traditional group least absolute shrinkage and selection operator (LASSO) problem that involves summing up the l(2) norm of each group until convergent. By updating group weights, our approach enforces a group of smaller coefficients from the previous iterate to be more likely to set to zero compared to the group LASSO. Theoretical results include a minimizing property of the proposed model as well as the convergence of the iterative algorithm to a stationary solution under mild conditions. We conduct extensive experiments on synthetic and real datasets, indicating that our method yields a performance that is superior to that of the state-of-the-art methods in linear regression and binary classification.