当前位置:
X-MOL 学术
›
arXiv.cs.RO
›
论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Towards Anytime Optical Flow Estimation with Event Cameras
arXiv - CS - Robotics Pub Date : 2023-07-11 , DOI: arxiv-2307.05033 Yaozu Ye, Hao Shi, Kailun Yang, Ze Wang, Xiaoting Yin, Yaonan Wang, Kaiwei Wang
arXiv - CS - Robotics Pub Date : 2023-07-11 , DOI: arxiv-2307.05033 Yaozu Ye, Hao Shi, Kailun Yang, Ze Wang, Xiaoting Yin, Yaonan Wang, Kaiwei Wang
Event cameras are capable of responding to log-brightness changes in
microseconds. Its characteristic of producing responses only to the changing
region is particularly suitable for optical flow estimation. In contrast to the
super low-latency response speed of event cameras, existing datasets collected
via event cameras, however, only provide limited frame rate optical flow ground
truth, (e.g., at 10Hz), greatly restricting the potential of event-driven
optical flow. To address this challenge, we put forward a high-frame-rate,
low-latency event representation Unified Voxel Grid, sequentially fed into the
network bin by bin. We then propose EVA-Flow, an EVent-based Anytime Flow
estimation network to produce high-frame-rate event optical flow with only
low-frame-rate optical flow ground truth for supervision. The key component of
our EVA-Flow is the stacked Spatiotemporal Motion Refinement (SMR) module,
which predicts temporally-dense optical flow and enhances the accuracy via
spatial-temporal motion refinement. The time-dense feature warping utilized in
the SMR module provides implicit supervision for the intermediate optical flow.
Additionally, we introduce the Rectified Flow Warp Loss (RFWL) for the
unsupervised evaluation of intermediate optical flow in the absence of ground
truth. This is, to the best of our knowledge, the first work focusing on
anytime optical flow estimation via event cameras. A comprehensive variety of
experiments on MVSEC, DESC, and our EVA-FlowSet demonstrates that EVA-Flow
achieves competitive performance, super-low-latency (5ms), fastest inference
(9.2ms), time-dense motion estimation (200Hz), and strong generalization. Our
code will be available at https://github.com/Yaozhuwa/EVA-Flow.
中文翻译:
使用事件相机进行随时光流估计
事件摄像机能够响应微秒级的对数亮度变化。其仅对变化区域产生响应的特性特别适合光流估计。然而,与事件相机的超低延迟响应速度相比,通过事件相机收集的现有数据集仅提供有限的帧速率光流地面实况(例如,10Hz),极大地限制了事件驱动光流的潜力。为了应对这一挑战,我们提出了一种高帧率、低延迟的事件表示统一体素网格,按顺序将 bin 馈送到网络中。然后,我们提出 EVA-Flow,一种基于事件的随时流估计网络,用于生成高帧率事件光流,仅使用低帧率光流地面实况进行监督。我们的 EVA-Flow 的关键组件是堆叠式时空运动细化(SMR)模块,它可以预测时间密集光流并通过时空运动细化来提高准确性。SMR 模块中使用的时间密集特征扭曲为中间光流提供隐式监控。此外,我们还引入了整流流扭曲损失(RFWL),用于在没有地面事实的情况下对中间光流进行无监督评估。据我们所知,这是第一个专注于通过事件相机进行随时光流估计的工作。对 MVSEC、DESC 和我们的 EVA-FlowSet 进行的全面实验表明,EVA-Flow 实现了具有竞争力的性能、超低延迟(5ms)、最快推理(9.2ms)、时间密集运动估计(200Hz)和概括性强。
更新日期:2023-07-12
中文翻译:
使用事件相机进行随时光流估计
事件摄像机能够响应微秒级的对数亮度变化。其仅对变化区域产生响应的特性特别适合光流估计。然而,与事件相机的超低延迟响应速度相比,通过事件相机收集的现有数据集仅提供有限的帧速率光流地面实况(例如,10Hz),极大地限制了事件驱动光流的潜力。为了应对这一挑战,我们提出了一种高帧率、低延迟的事件表示统一体素网格,按顺序将 bin 馈送到网络中。然后,我们提出 EVA-Flow,一种基于事件的随时流估计网络,用于生成高帧率事件光流,仅使用低帧率光流地面实况进行监督。我们的 EVA-Flow 的关键组件是堆叠式时空运动细化(SMR)模块,它可以预测时间密集光流并通过时空运动细化来提高准确性。SMR 模块中使用的时间密集特征扭曲为中间光流提供隐式监控。此外,我们还引入了整流流扭曲损失(RFWL),用于在没有地面事实的情况下对中间光流进行无监督评估。据我们所知,这是第一个专注于通过事件相机进行随时光流估计的工作。对 MVSEC、DESC 和我们的 EVA-FlowSet 进行的全面实验表明,EVA-Flow 实现了具有竞争力的性能、超低延迟(5ms)、最快推理(9.2ms)、时间密集运动估计(200Hz)和概括性强。