Parallel Algorithms for Big Data Optimization
The authors propose a decomposition framework for the parallel optimization of the sum of a differentiable function and a (block) separable non-smooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Their framework is very flexible and includes both fully parallel Jacobi schemes and Gauss-Seidel (i.e., sequential) ones, as well as virtually all possibilities \"In between\" with only a subset of variables updated at each iteration. Their theoretical convergence results improve on existing ones, and numerical results on LASSO and logistic regression problems show that the new method consistently outperforms existing algorithms.