Globally optimal on-line learning rules for multi-layer neural networks

Magnus Rattray; David Saad

doi:10.1088/0305-4470/30/22/005

Globally optimal on-line learning rules for multi-layer neural networks

Research output: Chapter in Book/Conference proceeding › Conference contribution

Abstract

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

Original language	English
Title of host publication	Journal of Physics A: Mathematical and General\|J. Phys. Math. Gen.
Publisher	MIT Press
Pages	L771-L776
Volume	30
DOIs	https://doi.org/10.1088/0305-4470/30/22/005
Publication status	Published - 21 Nov 1997
Event	Advances in Neural Information Processing Systems 10, [NIPS Conference, Denver, Colorado, USA, 1997] - Duration: 1 Jan 1824 → … http://dblp.uni-trier.de/db/conf/nips/nips1997.html#RattrayS97http://dblp.uni-trier.de/rec/bibtex/conf/nips/RattrayS97.xmlhttp://dblp.uni-trier.de/rec/bibtex/conf/nips/RattrayS97

Conference

Conference	Advances in Neural Information Processing Systems 10, [NIPS Conference, Denver, Colorado, USA, 1997]
Period	1/01/24 → …
Internet address	http://dblp.uni-trier.de/db/conf/nips/nips1997.html#RattrayS97http://dblp.uni-trier.de/rec/bibtex/conf/nips/RattrayS97.xmlhttp://dblp.uni-trier.de/rec/bibtex/conf/nips/RattrayS97

Access to Document

10.1088/0305-4470/30/22/005

Cite this

@inproceedings{ea68660db2654f609db3d176b289904e,

title = "Globally optimal on-line learning rules for multi-layer neural networks",

abstract = "We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.",

author = "Magnus Rattray and David Saad",

year = "1997",

month = nov,

day = "21",

doi = "10.1088/0305-4470/30/22/005",

language = "English",

volume = "30",

pages = "L771--L776",

booktitle = "Journal of Physics A: Mathematical and General|J. Phys. Math. Gen.",

publisher = "MIT Press",

address = "United States",

note = "Advances in Neural Information Processing Systems 10, [NIPS Conference, Denver, Colorado, USA, 1997] ; Conference date: 01-01-1824",

url = "http://dblp.uni-trier.de/db/conf/nips/nips1997.html#RattrayS97http://dblp.uni-trier.de/rec/bibtex/conf/nips/RattrayS97.xmlhttp://dblp.uni-trier.de/rec/bibtex/conf/nips/RattrayS97",

}

TY - GEN

T1 - Globally optimal on-line learning rules for multi-layer neural networks

AU - Rattray, Magnus

AU - Saad, David

PY - 1997/11/21

Y1 - 1997/11/21

N2 - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

AB - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

U2 - 10.1088/0305-4470/30/22/005

DO - 10.1088/0305-4470/30/22/005

M3 - Conference contribution

VL - 30

SP - L771-L776

BT - Journal of Physics A: Mathematical and General|J. Phys. Math. Gen.

PB - MIT Press

T2 - Advances in Neural Information Processing Systems 10, [NIPS Conference, Denver, Colorado, USA, 1997]

Y2 - 1 January 1824

ER -

Globally optimal on-line learning rules for multi-layer neural networks

Abstract

Conference

Access to Document

Fingerprint

Cite this