## Abstract

The construction of a radial basis function (RBF)

network involves the determination of the model size, hidden

nodes, and output weights. Least squares-based subset selection

methods can determine a RBF model size and its parameters

simultaneously. Although these methods are robust, they may

not achieve optimal results. Alternatively, gradient methods are

widely used to optimize all the parameters. The drawback is that

most algorithms may converge slowly as they treat hidden nodes

and output weights separately and ignore their correlations. In

this paper, a new discrete-continuous algorithm is proposed for

the construction of a RBF model. First, the orthogonal least

squares (OLS)-based forward stepwise selection constructs an

initial model by selecting model terms one by one from a

candidate term pool. Then a new Levenberg–Marquardt (LM)-

based parameter optimization is proposed to further optimize

the hidden nodes and output weights in the continuous space. To

speed up the convergence, the proposed parameter optimization

method considers the correlation between the hidden nodes and

output weights, which is achieved by translating the output

weights to dependent parameters using the OLS method. The

correlation is also used by the previously proposed continuous

forward algorithm (CFA). However, unlike the CFA, the new

method optimizes all the parameters simultaneously. In addition,

an equivalent recursive sum of squared error is derived to reduce

the computation demanding for the first derivatives used in the

LM method. Computational complexity is given to confirm the

new method is much more computationally efficient than the

CFA. Different numerical examples are presented to illustrate

the effectiveness of the proposed method. Further, Friedman

statistical tests on 13 classification problems are performed, and

the results demonstrate that RBF networks built by the new

method are very competitive in comparison with some popular

classifiers.

network involves the determination of the model size, hidden

nodes, and output weights. Least squares-based subset selection

methods can determine a RBF model size and its parameters

simultaneously. Although these methods are robust, they may

not achieve optimal results. Alternatively, gradient methods are

widely used to optimize all the parameters. The drawback is that

most algorithms may converge slowly as they treat hidden nodes

and output weights separately and ignore their correlations. In

this paper, a new discrete-continuous algorithm is proposed for

the construction of a RBF model. First, the orthogonal least

squares (OLS)-based forward stepwise selection constructs an

initial model by selecting model terms one by one from a

candidate term pool. Then a new Levenberg–Marquardt (LM)-

based parameter optimization is proposed to further optimize

the hidden nodes and output weights in the continuous space. To

speed up the convergence, the proposed parameter optimization

method considers the correlation between the hidden nodes and

output weights, which is achieved by translating the output

weights to dependent parameters using the OLS method. The

correlation is also used by the previously proposed continuous

forward algorithm (CFA). However, unlike the CFA, the new

method optimizes all the parameters simultaneously. In addition,

an equivalent recursive sum of squared error is derived to reduce

the computation demanding for the first derivatives used in the

LM method. Computational complexity is given to confirm the

new method is much more computationally efficient than the

CFA. Different numerical examples are presented to illustrate

the effectiveness of the proposed method. Further, Friedman

statistical tests on 13 classification problems are performed, and

the results demonstrate that RBF networks built by the new

method are very competitive in comparison with some popular

classifiers.

Original language | English |
---|---|

Pages (from-to) | 1785 |

Number of pages | 1798 |

Journal | I E E E Transactions on Neural Networks and Learning Systems |

Volume | 24 |

Issue number | 11 |

Publication status | Published - 25 Jun 2013 |

## Keywords

- Forward stepwise selection
- Levenberg–Marquardt
- model generalization
- orthogonal least squares
- radial basis function (RBF) networks