APSIPA Transactions on Signal and Information Processing > Vol 13 > Issue 3

AMBNet: Adaptive Multi-feature Balanced Network for Multimodal Remote Sensing Semantic Segmentation

Xiaochen Xiu, The Chinese University of Hong Kong, China, Xianping Ma, The Chinese University of Hong Kong, China, Man-On Pun, The Chinese University of Hong Kong, China, SimonPun@cuhk.edu.cn , Ming Liu, MizarVision, China
 
Suggested Citation
Xiaochen Xiu, Xianping Ma, Man-On Pun and Ming Liu (2024), "AMBNet: Adaptive Multi-feature Balanced Network for Multimodal Remote Sensing Semantic Segmentation", APSIPA Transactions on Signal and Information Processing: Vol. 13: No. 3, e202. http://dx.doi.org/10.1561/116.00000071

Publication Date: 22 Apr 2024
© 2024 X. Xiu, X. Ma, M. Pun and M. Liu
 
Subjects
 
Keywords
Height estimationFeature fusionMultimodal semantic segmentation
 

Share

Open Access

This is published under the terms of CC BY-NC.

Downloaded: 136 times

In this article:
Introduction 
Proposed AMBNet 
Fusion Quality Metrics 
Experiments Configuration 
Results and Discussions 
Conclusion 
References 

Abstract

This work proposes an Adaptive Multi-feature Balanced network (AMBNet) for semantic segmentation in complex urban remote sensing scenarios. To fully exploit optical images and Digital Surface Models (DSM) data obtained from remote sensing sensors, a Depth Feature Extraction and Balancer (DFEB) module is devised to estimate and balance the depth information of all pixels by capturing detailed structural compositions of the ground surface. After that, a Parallel Multi-Stage Segmentator (PMSS) comprised of a dual-branch Encoder and Decoder with skip connections is constructed to perform effective segmentation by exploiting the balanced DSM (BDSM) and optical information. As a result, the proposed AMBNet can make effective use of optical images to complete depth information, so as to achieve multimodal informationassisted semantic segmentation for complex remote sensing scenes. Comprehensive experiments performed on the ISPRS Vaihingen and Potsdam remote sensing datasets confirm the segmentation performance of the proposed method.

DOI:10.1561/116.00000071

Companion

APSIPA Transactions on Signal and Information Processing Special Issue - Advanced Machine Learning Techniques for Remote Sensing: Algorithms and Applications
See the other articles that are part of this special issue.