Understanding Neural Reuse: A Case Study on Improving Energy Efficiency of Convolutional Neural Networks

Abstract

Artificial Neural Networks (ANNs) constitute a vital part in the Artificial Intelligence (AI) technology today. Neural networks run in basically every electrical machinery for the purposes of response-time efficiency, machine learning, or simulation. These networks have helped so many advancements in different parts of the areas over the years. However, even the simplest ANN is power-hungry and causes an overhead to the running platform; it is always expensive to benefit from a network. Moreover, separate tasks are handled by different networks in order to make the output produced by the network more accurate which results even more overhead. Our main research question is "How can we make multiple ANNs more energy efficient without losing their effectiveness?". Accordingly, we showcase at least two different tasks can be handled by the same network which is more efficient when the two regular networks are combined. This idea is rooted and inspired by the Neural Reuse theory which indicates the same neural paths are used for more than one task in the brain, and these tasks do not need to be similar at all. We merge two Convolutional Neural Networks (CNNs), one recognises sounds and the other recognises images, then compress them via quantisation and make them even lighter and faster as our case study. This research exploits and employs open-source software resources and pretrained CNNs. The research targets traditional hardware resources, such as a PC with a regular CPU or a mobile phone, and facilitates solutions to this extent. Our ultimate aim is to provide easily accessible, i.e., open-source, and reproducible by everybody with a simple computer software solutions while increasing energy efficiency in multiple-networked systems via this research.

Date of Award	31 Dec 2022
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Steve Furber (Supervisor) & Mikel Luján (Supervisor)