A bio-inspired attentive model to predict perceptual saliency in natural scenes

Abstract

The sensory input from the entire visual field carried through the optic nerve to the visual system could exceed the processing capabilities of the cortex. Focalisation towards specific areas of interest represents the natural coping mechanism of processing exclusively the relevant part of the visual field. State-of-the-art mainstream artificial vision, relying on frame-based cameras and convolutional neural networks, exploits the current availability of computational resources but falls short when the visual system has to be deployed in compact, autonomous systems that cannot rely on external access to computing devices. This leads to the following questions: Can we reduce computational load via bioinspired visual attention mechanisms? Can a robot take advantage of the same attentional mechanisms to quickly interact with the environment? This work addresses these questions by bridging the gap between biologically inspired vision sensors and models of attention and resulted in the implementation of an event-driven model of attention on the humanoid robot iCub. Biologically inspired event-driven vision sensors are loosely inspired by the retina parvo magno-parvo and magnocellular pathway, reacting to changes in the field of view. They reduce the redundancy of the visual signal related to static stimuli and produce a stream of spikes that encode information similarly to biological neurons. Biologically inspired models of visual attention explain which mechanisms drive the selection of salient stimuli in the visual input. To test the assumption that biologically inspired vision sensors coupled with attention models can be exploited to select relevant stimuli for a robot, I selected three main event-driven bottom-up feature extraction channels fed into a biologically plausible saliency model based on the Gestalt theory of perceptual grouping. Intensity, disparity and motion are the first information cues of this project towards a more complex attention model where the channels compete with each other to represent the scene. My work demonstrated the applicability of biologically plausible event-driven saliency-based visual attention models for iCub. These models can run online and on neuromorphic platforms proving the possibility of exploiting fully bioinspired pipelines to determine visual attention cues with low latency.

Date of Award	1 Aug 2023
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Angelo Cangelosi (Supervisor)