Sensors, Vol. 26, Pages 638: Multimodal Building Damage Assessment Method Fusing Adaptive Attention Mechanism and State-Space Modeling

Sensors doi: 10.3390/s26020638

Authors:
Rongping Zhu
Xiaoji Lan

Rapid and reliable building damage assessment (BDA) is crucial for post-disaster emergency response. However, existing methods face challenges such as complex background interference, the difficulty in jointly modeling local geometric details and global spatial dependencies, and adverse weather conditions. To address these issues, this paper proposes the Adaptive Difference State-Space Fusion Network (ADSFNet), capable of processing both optical and Synthetic Aperture Radar (SAR) data to alleviate weather-induced limitations. To achieve this, ADSFNet innovatively introduces the Adaptive Difference Attention Fusion (ADAF) module and the Hybrid Selective State-Space Convolution (HSSC) module. Specifically, ADAF integrates pre- and post-disaster features to guide the network to focus on building regions while suppressing background interference. Meanwhile, HSSC synergizes the local texture extraction of CNNs with the global modeling strength of Mamba, enabling the simultaneous capture of cross-building spatial relationships and fine-grained damage details. Experimental results on sub-meter high-resolution MultiModal (BRIGHT) and optical (xBD) datasets demonstrate that ADSFNet attains F1 scores of 71.36% and 73.98%, which are 1.29% and 0.6% higher than the state-of-the-art mainstream methods, respectively. Finally, we leverage the model outputs to construct a disaster-centric knowledge graph and integrate it with Large Language Models to develop an intelligent management system, providing a novel technical pathway for emergency decision-making.

Source link

Rongping Zhu www.mdpi.com