Oral Presentations (OP)

OP-10 Bioinspired intelligent visual attention system for the humanoid robot iCub exploring event-driven sensing and neuromorphic hardware

Abstract

Introduction Visual applications in robotics must meet strict requirements for power efficiency, low latency, and data processing capacity. Despite the remarkable performance achievements of traditional computer vision methods, they struggle to generalise effectively and often rely on vast datasets, increasing data processing and transfer. The proposed system leverages bioinspired visual attention mechanisms to process only relevant parts of the scene, further exploring event-based sensing and neuromorphic computing via Spiking Neural Networks (SNNs).

Aims This scientific challenge aims to connect bioinspired hardware with biologically plausible algorithms, thereby showcasing the potential of spike-based implementations for online robotics visual applications.

Methods The bioinspired saliency-based visual attention model processes events from event-driven cameras on the humanoid robot iCub, running on SpiNNaker neuromorphic hardware. Intensity, disparity, and motion are the bottom-up feature extraction channels competing for scene representation. These cues feed into a biologically plausible saliency-based proto-object model based on Gestalt perceptual grouping theories to detect only relevant scene parts. The model produces saliency maps with salient areas representing regions potentially containing objects, called ‘proto-objects’.

Results The online system accurately generates saliency maps in ~16ms detecting salient proto-objects and disregarding clutter. The system has been qualitatively and quantitatively validated, achieving comparable results to the frame-based implementation, in online simple office scenarios, as well as when compared against the ground truth fixation maps from real human subjects (NUS3D dataset).

Conclusion This project is the first significant step towards more complex real-world robotic applications for vision, where bioinspiration sets the basis for fast, power-efficient online robotic applications and innovative computer vision approaches.

Article metrics
Altmetric data not available for this article.
Dimensionsopen-url