Variable projection for non-smooth problems

Variable projection solves structured optimization problems by completely minimizing over a subset of the variables while iterating over the remaining variables. Over the past 30 years, the technique has been widely used, with empirical and theoretical results demonstrating both greater efficacy and greater stability compared to competing approaches. Classic examples have exploited closed-form projections and smoothness of the objective function. We extend the approach to problems that include nonsmooth terms, develop an inexact adaptive algorithm that solves projection subproblems inexactly by iterative methods, and analyze its computational complexity. Finally, we illustrate the effectiveness of the adaptive algorithm with numerical examples. Code to reproduce the examples is available at https://github.com/TristanvanLeeuwen/VarProNS