Fusion of lidar and hyperspectral data for semantic segmentation of forest tree species

International audienceAbstract. Hyperspectral images (HI) and Light Detection and Ranging (LiDAR) provide high resolution radiometric and geometric information for monitoring forests at individual tree crown (ITC) level. It has many important applications for sustainable forest management, biodiversity assessment and healthy ecosystem preservation. However, the integration of different remote sensing modalities is a challenging task for tree species classification due to different artifacts such as the lighting variability, the topographic effects and the atmospheric conditions of the data acquisition. The characterization of ITC can benefit from the extraction and selection of robust feature descriptors that solve these issues. This paper aims to investigate the integration of feature descriptors from HI and LiDAR by using the intra-set and inter-set feature importance for the semantic segmentation of forest tree species. A fusion methodology is proposed between high-density LiDAR data – (20 pulses m−2) and VNIR HI – (160 bands and 0.80 m spatial resolution) acquired on French temperate forests along an altitude gradient. The proposed scheme has three inputs: the field inventory information, the HI and the LiDAR data. Our approach can be described in nine stages: polygon projection, non-overlapping pixel selection, vegetation and shadow removal, LiDAR feature extraction, height mask, robust PCA (rPCA), feature reduction and classification. The overall accuracy of tree species classification at pixel-level was 68.9% by using random forest (RF) classifier. Our approach showed that 74.0% of trees were correctly assigned overall, by having conifer species such as Norway Spruce (Picea abies) with a producer’s accuracy of 97.4%