Self-organizing mixture networks for probability density estimation

H. Yin, N. M. Allinson

    Research output: Contribution to journalArticlepeer-review


    A self-organizing mixture network (SOMN) is derived for learning arbitrary density functions. The network minimizes the Kullback-Leibler information metric by means of stochastic approximation methods. The density functions are modeled as mixtures of parametric distributions A mixture needs not to be homogenous, i.e., it can have different density profiles. The first layer of the network is similar to Kohonen's self-organizing map (SOM), but with the parameters of the component densities as the learning weights. The winning mechanism is based on maximum posterior probability, and updating of the weights is limited to a small neighborhood around the winner. The second layer accumulates the responses of these local nodes, weighted by the learned mixing parameters. The network possesses a simple structure and computational form, yet yields fast and robust convergence. The network has a generalization ability due to the relative entropy criterion used. Applications to density profile estimation and pattern classification are presented. The SOMN can also provide an insight to the role of neighborhood function used in the SOM.
    Original languageEnglish
    Pages (from-to)405-411
    Number of pages6
    JournalIEEE Transactions on Neural Networks
    Issue number2
    Publication statusPublished - Mar 2001


    • Density estimation
    • Expectation-maximization (EM) algorithm
    • Maximum likelihood
    • Mixture distribution
    • Self-organizing maps
    • Unsupervised learning


    Dive into the research topics of 'Self-organizing mixture networks for probability density estimation'. Together they form a unique fingerprint.

    Cite this