Article Text

Download PDFPDF

OP-10 Bioinspired intelligent visual attention system for the humanoid robot iCub exploring event-driven sensing and neuromorphic hardware
  1. Giulia D’Angelo,
  2. Chiara Bartolozzi
  1. Italian Institute of Technology, Italy

Abstract

Introduction Visual applications in robotics must meet strict requirements for power efficiency, low latency, and data processing capacity. Despite the remarkable performance achievements of traditional computer vision methods, they struggle to generalise effectively and often rely on vast datasets, increasing data processing and transfer. The proposed system leverages bioinspired visual attention mechanisms to process only relevant parts of the scene, further exploring event-based sensing and neuromorphic computing via Spiking Neural Networks (SNNs).

Aims This scientific challenge aims to connect bioinspired hardware with biologically plausible algorithms, thereby showcasing the potential of spike-based implementations for online robotics visual applications.

Methods The bioinspired saliency-based visual attention model processes events from event-driven cameras on the humanoid robot iCub, running on SpiNNaker neuromorphic hardware. Intensity, disparity, and motion are the bottom-up feature extraction channels competing for scene representation. These cues feed into a biologically plausible saliency-based proto-object model based on Gestalt perceptual grouping theories to detect only relevant scene parts. The model produces saliency maps with salient areas representing regions potentially containing objects, called ‘proto-objects’.

Results The online system accurately generates saliency maps in ~16ms detecting salient proto-objects and disregarding clutter. The system has been qualitatively and quantitatively validated, achieving comparable results to the frame-based implementation, in online simple office scenarios, as well as when compared against the ground truth fixation maps from real human subjects (NUS3D dataset).

Conclusion This project is the first significant step towards more complex real-world robotic applications for vision, where bioinspiration sets the basis for fast, power-efficient online robotic applications and innovative computer vision approaches.

https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.