Abstract
In this survey, we review various regularization techniques that induce different types of sparsity, which have attracted considerable interest recently. We categorize these techniques into additive sparse regularization and structural sparse regularization and discuss optimization algorithms for the two classes. More precisely, we discuss dual augmented Lagrangian (DAL) method for the former. DAL is particularly suited for poorly conditioned problems that may arise from the additivity. For the latter we discuss the alternating direction method of multipliers. This method is particularly attractive because it allows to separate the structure represented by a matrix from the sparsity inducing norms.
