Proximity Estimation Using Vision Features Computed on Sensor

Jianing Chen, Yanan Liu, Stephen J. Carey, Piotr Dudek

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review


This paper presents a monocular vision based proximity estimation system using abstract features, such as corner points, blobs and edges, as inputs to a neural network. An experimental vehicle was built using a vision system integrating the SCAMP-5 vision chip, a micro-controller, and an RC model car. The vision chip includes image sensor with embedded 256×256 processor SIMD array. The pixel processor array chip was programmed to capture images and run the feature algorithms directly on the focal plane, and then digest them so that only sparse feature description data were read-out in the form of 40 values. By logging the vision output and the output from three infrared proximity sensors, training data were obtained to train three fully connected layer-recurrent neural networks with fewer than 700 parameters each. The trained neural network was able to estimate the proximity to the level of accuracy sufficient for a reactive collision avoidance behaviour to be achieved. The latency of the control system, from image capture to neural network output, was under 4ms, enabling the vehicles to avoid obstacles while moving at 0.64m/s to 1.8m/s in the experiment.
Original languageEnglish
Title of host publication2020 IEEE International Conference on Robotics and Automation
Place of PublicationParis, France
Number of pages7
ISBN (Print)9781728173955
Publication statusPublished - Jun 2020

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Electronic)2577-087X


Dive into the research topics of 'Proximity Estimation Using Vision Features Computed on Sensor'. Together they form a unique fingerprint.

Cite this