S-ROPE: Spectral Frame Representation of Periodic Events

Garcia Rodriguez, Luis; Konrad, Jonas; Drees, Dominik; Risse, Benjamin

Abstract

In this paper we introduce a novel event surface representation designed for encoding temporal information of dynamic vision sensors (DVS) into a multi-channel frame format. Different representations have been proposed to extract features from event streams. Among these representations, time surfaces have become a popular choice due to their ability to provide a flattened view of event data over a specified time interval. Despite their widespread adaption, these methods suffer from loss of crucial temporal information. This disadvantage is particularly apparent in DVS applications with repetitive, high-frequency intensity changes, as these signals result in continuously elevated event rates and would therefore benefit from efficient, yet information preserving representations. Moreover, integrating this information into state-of-the-art vision-based machine learning models remains challenging. To address these limitations, we propose Spectral frame Representation of Periodic Events (S-ROPE) which leverages information from frequency domain analysis through the application of the Fourier transformation and arranges the resulting frequency, amplitude and phase into spatially consistent frames. By transforming event streams from the temporal domain to the frequency domain, we demonstrate improved preservation of temporal dynamics, especially for objects displaying high frequency oscillations. We evaluate S-ROPE frames on custom and several publicly available datasets and demonstrate the applicability for three distinct use cases: visualisation, compression and machine learning-based image processing. For the latter, we show that S-ROPE proves useful for object detection in a scenery with high frequency events.