2D and 3D Semantic Segmentation for Interpreting and Understanding 3D Heritage Spaces

The 3D digitization of Cultural Heritage (CH) sites has become increasingly requested for documentation, preservation, and analysis applications. Beyond capturing 3D spatial geometry, the semantic interpretation and understanding of digital models are critical for enabling meaningful CH studies and facilitating informed conservation strategies. However, manual annotation and classification of architectural elements and surface pathologies remain labor-intensive and time-consuming, underscoring the need for automated approaches. This study presents a comparative analysis between two distinct semantic segmentation frameworks: (1) a 2D-to-3D pipeline that projects 2D image-based detections onto 3D point clouds produced with V-SLAM data and (2) direct segmentation methods of 3D point clouds acquired with portable LiDAR sensors. These frameworks are evaluated on data acquired using two distinct mobile mapping systems (MMS): (1) a fisheye multi-camera Visual SLAM-based portable system (ATOM-ANT3D) for the 2D-to-3D pipeline; (2) a LiDAR-based MMS (Heron MS Twin Color) for the 3D segmentation methods. Achieved results demonstrate the ability of the proposed frameworks to generate semantically enriched 3D heritage data, with the 2D-to-3D method slightly outperforming the 3D segmentation techniques.