Improving tactile gesture recognition with optical flow

Conference item

Tactile gesture recognition systems play a crucial role in Human-Robot Interaction (HRI) by enabling intuitive communication between humans and robots. The literature mainly addresses this problem by applying machine learning techniques to classify sequences of tactile images encoding the pressure distribution generated when executing the gestures. However, some gestures can be hard to differentiate based on the information provided by tactile images alone. In this paper, we present a simple yet effective way to improve the accuracy of a gesture recognition classifier. Our approach focuses solely on processing the tactile images used as input by the classifier. In particular, we propose to explicitly highlight the dynamics of the contact in the tactile image by computing the dense optical flow. This additional information makes it easier to distinguish between gestures that produce similar tactile images but exhibit different contact dynamics. We validate the proposed approach in a tactile gesture recognition task, showing that a classifier trained on tactile images augmented with optical flow information achieved a 9% improvement in gesture classification accuracy compared to one trained on standard tactile images.

Authors: Zhong, S; Albini, A; Caroleo, G; Cannata, G; Maiolino, P

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: IEEE
• Host title: 2025 34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)
• Pages: 2246-2252
• Publication date: 2025-08-29
• Acceptance date: 2025-06-01
• Event title: 2025 34th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)
• Event location: Eindhoven, Netherlands
• Event website: https://www.ro-man2025.org/
• Event start date: 2025-08-25
• Event end date: 2025-08-29
• DOI: 10.1109/ro-man63969.2025.11217777
• EISSN: 1944-9437
• ISSN: 1944-9445
• EISBN: 9798331587710
• ISBN: 9798331587727
• Language: English
• Keywords: accuracy; torque; shape; human-robot interaction; gesture recognition; machine learning; robot sensing systems; sensors; optical flow; standards