SVP-Classifier: Single-View Point Cloud Data Classifier with Multi-view Hallucination

We address single-view 3D shape classification with partial Point Cloud Data (PCD) inputs. Conventional PCD classifiers achieve the best performance when trained and evaluated with complete 3D object scans. However, they all experience a performance drop when trained and evaluated on partial single-view PCD. We propose a Single-View PCD Classifier (SVP-Classifier), which first hallucinates the features of other viewpoints covering the unseen part of the object with a Conditional Variational Auto-Encoder (CVAE). It then aggregates the hallucinated multi-view features with a multi-level Graph Convolutional Network (GCN) to form a global shape representation that helps to improve the single-view PCD classification performance. With experiments on the single-view PCDs generated from ModelNet40 and ScanObjectNN, we prove that the proposed SVP-Classifier outperforms the best single-view PCD-based methods, after they have been retrained on single-view PCDs, thus reducing the gap between single-view methods and methods that employ complete PCDs. Code and datasets are available: https://github.com/IIT-PAVIS/SVP-Classifier.