Using Weakly Supervised Semantic Segmentation for Post-Disaster Scene Understanding

Creating semantic segmentation datasets is more time-consuming than the preparation of image-level classification labels. This paper presents a cost-efficient pixel labeling method and experimental results of using Grad-CAM-generated heatmap for weakly supervised post-disaster site semantic segmentation. It starts with training CNN binary-classification models, then removing non-target objects, reducing searching regions according to the distributions of heatmap values, and assigning the pixel-level label to the highest heatmap value regions as segmentations. The proposed CNN model architecture has a 1 × 1 convolutional heatmap layer with a suitable number of channels to fit the target objects. Testing results showed the 16-channel heatmap layer reached the best building-background classification performance (F1-score 0.89), and the 128-channel had the best building-damage classification performance (F1-score 0.78) among the evaluated options of 4, 8, 16, 32, 64, and 128 channels. Furthermore, the proposed pixel labeling method can understand complicated post-disaster scenes by correctly locating target objects (buildings and damages) in aerial images via repetitions of searching the target objects in regions of higher heatmap values.