Bayesian Augmented Lagrangian Algorithm for System Identification

Nonlinear Auto-Regressive model with eXogenous input (NARX) is one of the most popular black-box model classes that can describe many nonlinear systems. The structure determination is the most challenging and important part during the system identification. NARX can be formulated as a linear-in-the-parameters model, then the identification problem can be solved to obtain a sparse solution from the viewpoint of the weighted minimization problem. Such an optimization problem not only minimizes the sum squares of model errors but also the sum of reweighted model parameters. In this paper, a novel algorithm named Bayesian Augmented Lagrangian Algorithm (BAL) is proposed to solve the weighted minimization problem, which is able to obtain a sparse solution and enjoys fast computation. This is achieved by converting the original optimization problem into distributed suboptimization problems solved separately and penalizing the overall complex model to avoid overfitting under the Bayesian framework. The regularization parameter is also iteratively updated to obtain a satisfied solution. In particular, a solver with guaranteed convergence is constructed and the corresponding theoretical proof is given. Two numerical examples have been used to demonstrate the effectiveness of the proposed method in comparison to several popular methods.