Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker

Xin Jin; Mikel Luján; Luis A. Plana; Alexander D. Rast; Stephen R. Welbourne; Steve B. Furber

doi:10.1145/1787275.1787297

Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker

Xin Jin, Mikel Luján, Luis A. Plana, Alexander D. Rast, Stephen R. Welbourne, Steve B. Furber

Advanced Processor Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper presents an efficient implementation and performance analysis of mapping multilayer perceptron networks with the backpropagation learning rule on SpiNNaker - a massively parallel architecture dedicated for neural network simulation. A new algorithm called pipelined checker-boarding partitioning scheme is proposed for efficient mapping. The new mapping algorithm relies on a checker-board partitioning scheme, but the key advantage comes from introducing a pipelined mode. The six-stage pipelined mode captures the parallelism within each partition of the weight matrix, allowing the overlapping of communication and computation. Not only does the proposed mapping localize communication, but it can also hide a part of or even all the communication for high efficiency. © 2010 author/owner(s).

Original language	English
Title of host publication	CF 2010 - Proceedings of the 2010 Computing Frontiers Conference\|CF - Proc. Comput. Front. Conf.
Place of Publication	New York, USA
Publisher	Association for Computing Machinery
Pages	89-90
Number of pages	1
ISBN (Print)	9781450300445
DOIs	https://doi.org/10.1145/1787275.1787297
Publication status	Published - 2010
Event	7th ACM International Conference on Computing Frontiers, CF'10 - Bertinoro Duration: 1 Jul 2010 → …

Conference

Conference	7th ACM International Conference on Computing Frontiers, CF'10
City	Bertinoro
Period	1/07/10 → …

Keywords

backpropagation
mapping
mlp
parallel
pipeline
spinnaker

Access to Document

10.1145/1787275.1787297

Cite this

@inproceedings{a19c70b7b3404670976417335061b4dc,

title = "Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker",

abstract = "This paper presents an efficient implementation and performance analysis of mapping multilayer perceptron networks with the backpropagation learning rule on SpiNNaker - a massively parallel architecture dedicated for neural network simulation. A new algorithm called pipelined checker-boarding partitioning scheme is proposed for efficient mapping. The new mapping algorithm relies on a checker-board partitioning scheme, but the key advantage comes from introducing a pipelined mode. The six-stage pipelined mode captures the parallelism within each partition of the weight matrix, allowing the overlapping of communication and computation. Not only does the proposed mapping localize communication, but it can also hide a part of or even all the communication for high efficiency. {\textcopyright} 2010 author/owner(s).",

keywords = "backpropagation, mapping, mlp, parallel, pipeline, spinnaker",

author = "Xin Jin and Mikel Luj{\'a}n and Plana, {Luis A.} and Rast, {Alexander D.} and Welbourne, {Stephen R.} and Furber, {Steve B.}",

year = "2010",

doi = "10.1145/1787275.1787297",

language = "English",

isbn = "9781450300445",

pages = "89--90",

booktitle = "CF 2010 - Proceedings of the 2010 Computing Frontiers Conference|CF - Proc. Comput. Front. Conf.",

publisher = "Association for Computing Machinery",

address = "United States",

note = "7th ACM International Conference on Computing Frontiers, CF'10 ; Conference date: 01-07-2010",

}

Jin, X, Luján, M, Plana, LA, Rast, AD, Welbourne, SR & Furber, SB 2010, Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker. in CF 2010 - Proceedings of the 2010 Computing Frontiers Conference|CF - Proc. Comput. Front. Conf.. Association for Computing Machinery, New York, USA, pp. 89-90, 7th ACM International Conference on Computing Frontiers, CF'10, Bertinoro, 1/07/10. https://doi.org/10.1145/1787275.1787297

Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker. / Jin, Xin; Luján, Mikel; Plana, Luis A. et al.
CF 2010 - Proceedings of the 2010 Computing Frontiers Conference|CF - Proc. Comput. Front. Conf.. New York, USA: Association for Computing Machinery, 2010. p. 89-90.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker

AU - Jin, Xin

AU - Luján, Mikel

AU - Plana, Luis A.

AU - Rast, Alexander D.

AU - Welbourne, Stephen R.

AU - Furber, Steve B.

PY - 2010

Y1 - 2010

N2 - This paper presents an efficient implementation and performance analysis of mapping multilayer perceptron networks with the backpropagation learning rule on SpiNNaker - a massively parallel architecture dedicated for neural network simulation. A new algorithm called pipelined checker-boarding partitioning scheme is proposed for efficient mapping. The new mapping algorithm relies on a checker-board partitioning scheme, but the key advantage comes from introducing a pipelined mode. The six-stage pipelined mode captures the parallelism within each partition of the weight matrix, allowing the overlapping of communication and computation. Not only does the proposed mapping localize communication, but it can also hide a part of or even all the communication for high efficiency. © 2010 author/owner(s).

AB - This paper presents an efficient implementation and performance analysis of mapping multilayer perceptron networks with the backpropagation learning rule on SpiNNaker - a massively parallel architecture dedicated for neural network simulation. A new algorithm called pipelined checker-boarding partitioning scheme is proposed for efficient mapping. The new mapping algorithm relies on a checker-board partitioning scheme, but the key advantage comes from introducing a pipelined mode. The six-stage pipelined mode captures the parallelism within each partition of the weight matrix, allowing the overlapping of communication and computation. Not only does the proposed mapping localize communication, but it can also hide a part of or even all the communication for high efficiency. © 2010 author/owner(s).

KW - backpropagation

KW - mapping

KW - mlp

KW - parallel

KW - pipeline

KW - spinnaker

U2 - 10.1145/1787275.1787297

DO - 10.1145/1787275.1787297

M3 - Conference contribution

SN - 9781450300445

SP - 89

EP - 90

BT - CF 2010 - Proceedings of the 2010 Computing Frontiers Conference|CF - Proc. Comput. Front. Conf.

PB - Association for Computing Machinery

CY - New York, USA

T2 - 7th ACM International Conference on Computing Frontiers, CF'10

Y2 - 1 July 2010

ER -

Efficient parallel implementation of multilayer backpropagation networks on SpiNNaker

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this