A New Discrete-Continuous Algorithm for Radial Basis Function (RBF) Networks Construction

Long Zhang, Kang Li, Haibo He, George W. Irwin

    Research output: Contribution to journalArticlepeer-review


    The construction of a radial basis function (RBF)
    network involves the determination of the model size, hidden
    nodes, and output weights. Least squares-based subset selection
    methods can determine a RBF model size and its parameters
    simultaneously. Although these methods are robust, they may
    not achieve optimal results. Alternatively, gradient methods are
    widely used to optimize all the parameters. The drawback is that
    most algorithms may converge slowly as they treat hidden nodes
    and output weights separately and ignore their correlations. In
    this paper, a new discrete-continuous algorithm is proposed for
    the construction of a RBF model. First, the orthogonal least
    squares (OLS)-based forward stepwise selection constructs an
    initial model by selecting model terms one by one from a
    candidate term pool. Then a new Levenberg–Marquardt (LM)-
    based parameter optimization is proposed to further optimize
    the hidden nodes and output weights in the continuous space. To
    speed up the convergence, the proposed parameter optimization
    method considers the correlation between the hidden nodes and
    output weights, which is achieved by translating the output
    weights to dependent parameters using the OLS method. The
    correlation is also used by the previously proposed continuous
    forward algorithm (CFA). However, unlike the CFA, the new
    method optimizes all the parameters simultaneously. In addition,
    an equivalent recursive sum of squared error is derived to reduce
    the computation demanding for the first derivatives used in the
    LM method. Computational complexity is given to confirm the
    new method is much more computationally efficient than the
    CFA. Different numerical examples are presented to illustrate
    the effectiveness of the proposed method. Further, Friedman
    statistical tests on 13 classification problems are performed, and
    the results demonstrate that RBF networks built by the new
    method are very competitive in comparison with some popular
    Original languageEnglish
    Pages (from-to)1785
    Number of pages1798
    JournalI E E E Transactions on Neural Networks and Learning Systems
    Issue number11
    Publication statusPublished - 25 Jun 2013


    • Forward stepwise selection
    • Levenberg–Marquardt
    • model generalization
    • orthogonal least squares
    • radial basis function (RBF) networks


    Dive into the research topics of 'A New Discrete-Continuous Algorithm for Radial Basis Function (RBF) Networks Construction'. Together they form a unique fingerprint.

    Cite this