Sensor-level computer vision with pixel processor arrays for agile robots

Piotr Dudek, Thomas Richardson, Laurie Bose, Stephen Carey, Jianing Chen, Yanan Liu, Colin Greatwood, Walterio Mayol-Cuevas

Research output: Contribution to journalReview articlepeer-review

327 Downloads (Pure)

Abstract

Vision processing for control of agile autonomous robots requires low-latency computation, within a limited power and space budget. This is challenging for conventional computing hardware. Parallel processor arrays (PPAs) are a new class of vision sensor devices that exploit advances in semiconductor technology, embedding a processor within each pixel of the image sensor array. Sensed pixel data is processed on the focal plane and only a small amount of relevant information is transmitted out of the vision sensor. This tight integration of sensing, processing, and memory within a massively parallel computing architecture leads to an interesting trade-off between high-performance, low-latency, low-power, low-cost and versatility in a machine vision system. Here, we review the history of image sensing and processing hardware from the perspective of in-pixel computing and outline the key features of a state-of-the-art smart camera system based on a PPA device, through the description of the SCAMP-5 system. We describe several robotic applications for agile ground and aerial vehicles, demonstrating PPA sensing functionalities including high-speed odometry, target tracking, obstacle detection and avoidance. In the conclusions, we provide some insight and perspective on the future development of PPA devices, including their application and benefits within agile, robust, adaptable and lightweight robotics.
Original languageEnglish
Article numbereabl7755
Pages (from-to)eabl7755
Number of pages12
JournalScience Robotics
Volume7
Issue number67
DOIs
Publication statusPublished - 29 Jun 2022

Fingerprint

Dive into the research topics of 'Sensor-level computer vision with pixel processor arrays for agile robots'. Together they form a unique fingerprint.

Cite this