Deep FusionNet for point cloud semantic segmentation

Conference item

Many point cloud segmentation methods rely on transferring irregular points into a voxel-based regular representation. Although voxel-based convolutions are useful for feature aggregation, they produce ambiguous or wrong predictions if a voxel contains points from different classes. Other approaches (such as PointNets and point-wise convolutions) can take irregular points for feature learning. But their high memory and computational costs (such as for neighborhood search and ball-querying) limit their ability and accuracy for large-scale point cloud processing. To address these issues, we propose a deep fusion network architecture (FusionNet) with a unique voxel-based “mini-PointNet” point cloud representation and a new feature aggregation module (fusion module) for large-scale 3D semantic segmentation. Our FusionNet can learn more accurate point-wise predictions when compared to voxel-based convolutional networks. It can realize more effective feature aggregations with lower memory and computational complexity for large-scale point cloud segmentation when compared to the popular point-wise convolutions. Our experimental results show that FusionNet can take more than one million points on one GPU for training to achieve state-of-the-art accuracy on large-scale Semantic KITTI benchmark.The code will be available at https://github.com/feihuzhang/LiDARSeg.

Authors: Zhang, F; Fang, J; Wah, B; Torr, PHS

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Springer International Publishing
• Host title: Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI
• Volume: 12356
• Pages: 644-663
• Series: Lecture Notes in Computer Science
• Publication date: 2020-11-30
• Acceptance date: 2020-07-03
• Event title: European Conference on Computer Vision (ECCV), 2020
• Event location: Virtual event
• Event website: https://eccv2020.eu/
• Event start date: 2020-08-23
• Event end date: 2020-08-28
• DOI: 10.1007/978-3-030-58586-0_38
• ISSN: 0302-9743
• EISBN: 978-3-030-58621-8
• ISBN: 978-3-030-58620-1
• Language: English
• Keywords: FFR