FlowNet3D++: Geometric losses for deep scene flow estimation

Conference item

We present FlowNet3D++, a deep scene flow estimation network. Inspired by classical methods, FlowNet3D++ incorporates geometric constraints in the form of point-toplane distance and angular alignment between individual vectors in the flow field, into FlowNet3D [21]. We demonstrate that the addition of these geometric loss terms improves the previous state-of-art FlowNet3D accuracy from 57.85% to 63.43%. To further demonstrate the effectiveness of our geometric constraints, we propose a benchmark for flow estimation on the task of dynamic 3D reconstruction, thus providing a more holistic and practical measure of performance than the breakdown of individual metrics previously used to evaluate scene flow. This is made possible through the contribution of a novel pipeline to integrate point-based scene flow predictions into a global dense volume. FlowNet3D++ achieves up to a 15.0% reduction in reconstruction error over FlowNet3D, and up to a 35.2% improvement over KillingFusion [32] alone. We will release our scene flow estimation code later.

Authors: Wang, Z; Li, S; Howard-Jenkins, H; Prisacariu, VA; Chen, M

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: IEEE
• Pages: 91-98
• Publication date: 2020-05-14
• Acceptance date: 2019-12-10
• Event title: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV 2020)
• Event location: Snowmass Village, Colorado, USA
• Event start date: 2020-03-01
• Event end date: 2020-03-05
• DOI: 10.1109/WACV45572.2020.9093302
• EISSN: 2642-9381
• ISSN: 2472-6737
• EISBN: 9781728165530
• ISBN: 9781728165547
• Language: English
• Keywords: three-dimensional displays; FFR; two dimensional displays; estimation; measurement; benchmark testing; feature extraction; training