Gradient-descent-based learning in memristive crossbar arrays

Manu V Nair; Piotr Dudek

doi:10.1109/IJCNN.2015.7280658

Gradient-descent-based learning in memristive crossbar arrays

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

This paper describes techniques to implement gradient-descent-based machine learning algorithms on crossbar arrays made of memristors or other analog memory devices. We introduce the Unregulated Step Descent (USD) algorithm, which is an approximation of the steepest descent algorithm, and discuss how it addresses various hardware implementation issues. We discuss the effect of device parameters and their variability on performance of the algorithm by using artificially generated and real-world datasets. In addition to providing insights on the effect of device parameters on learning, we illustrate how the USD algorithm partially offsets the effect of device variability. Finally, we discuss how the USD algorithm can be implemented in crossbar arrays using a simple 4-phase training scheme. The method allows parallel update of crossbar memory elements and reduces the hardware cost and complexity of the training architecture significantly.

Original language	English
Title of host publication	International Joint Conference on Neural Networks, IJCNN 2015
Publisher	IEEE
Number of pages	7
ISBN (Electronic)	978-1-4799-1960-4
DOIs	https://doi.org/10.1109/IJCNN.2015.7280658
Publication status	Published - Jul 2015
Event	International Joint Conference on Neural Networks, IJCNN 2015 - Duration: 1 Jan 1824 → …

Conference

Conference	International Joint Conference on Neural Networks, IJCNN 2015
Period	1/01/24 → …

Access to Document

10.1109/IJCNN.2015.7280658

Cite this

@inproceedings{44f1f098a64246aaae737cd09f653e1b,

title = "Gradient-descent-based learning in memristive crossbar arrays",

abstract = "This paper describes techniques to implement gradient-descent-based machine learning algorithms on crossbar arrays made of memristors or other analog memory devices. We introduce the Unregulated Step Descent (USD) algorithm, which is an approximation of the steepest descent algorithm, and discuss how it addresses various hardware implementation issues. We discuss the effect of device parameters and their variability on performance of the algorithm by using artificially generated and real-world datasets. In addition to providing insights on the effect of device parameters on learning, we illustrate how the USD algorithm partially offsets the effect of device variability. Finally, we discuss how the USD algorithm can be implemented in crossbar arrays using a simple 4-phase training scheme. The method allows parallel update of crossbar memory elements and reduces the hardware cost and complexity of the training architecture significantly.",

author = "Nair, {Manu V} and Piotr Dudek",

year = "2015",

month = jul,

doi = "10.1109/IJCNN.2015.7280658",

language = "English",

booktitle = "International Joint Conference on Neural Networks, IJCNN 2015",

publisher = "IEEE",

address = "United States",

note = "International Joint Conference on Neural Networks, IJCNN 2015 ; Conference date: 01-01-1824",

}

TY - GEN

T1 - Gradient-descent-based learning in memristive crossbar arrays

AU - Nair, Manu V

AU - Dudek, Piotr

PY - 2015/7

Y1 - 2015/7

N2 - This paper describes techniques to implement gradient-descent-based machine learning algorithms on crossbar arrays made of memristors or other analog memory devices. We introduce the Unregulated Step Descent (USD) algorithm, which is an approximation of the steepest descent algorithm, and discuss how it addresses various hardware implementation issues. We discuss the effect of device parameters and their variability on performance of the algorithm by using artificially generated and real-world datasets. In addition to providing insights on the effect of device parameters on learning, we illustrate how the USD algorithm partially offsets the effect of device variability. Finally, we discuss how the USD algorithm can be implemented in crossbar arrays using a simple 4-phase training scheme. The method allows parallel update of crossbar memory elements and reduces the hardware cost and complexity of the training architecture significantly.

AB - This paper describes techniques to implement gradient-descent-based machine learning algorithms on crossbar arrays made of memristors or other analog memory devices. We introduce the Unregulated Step Descent (USD) algorithm, which is an approximation of the steepest descent algorithm, and discuss how it addresses various hardware implementation issues. We discuss the effect of device parameters and their variability on performance of the algorithm by using artificially generated and real-world datasets. In addition to providing insights on the effect of device parameters on learning, we illustrate how the USD algorithm partially offsets the effect of device variability. Finally, we discuss how the USD algorithm can be implemented in crossbar arrays using a simple 4-phase training scheme. The method allows parallel update of crossbar memory elements and reduces the hardware cost and complexity of the training architecture significantly.

U2 - 10.1109/IJCNN.2015.7280658

DO - 10.1109/IJCNN.2015.7280658

M3 - Conference contribution

BT - International Joint Conference on Neural Networks, IJCNN 2015

PB - IEEE

T2 - International Joint Conference on Neural Networks, IJCNN 2015

Y2 - 1 January 1824

ER -

Gradient-descent-based learning in memristive crossbar arrays

Abstract

Conference

Access to Document

Fingerprint

Cite this