遥感图像包含丰富的地物信息,采用多标签学习方法更有利于对遥感图像进行准确分类.近年来,基于深度网络的遥感图像多标签分类已成为研究热点.深度学习方法通常需要在大型数据集上训练模型以获得更好的泛化能力,然而大规模数据集的类别标注费时费力,标注质量无法得到保证,可能会出现缺失标签的情况.标签信息的不完备会误导模型训练过程,影响深度模型分类性能.针对遥感图像的不完备多标签分类问题,提出一种基于局部语义(local semantics,LS)学习的分类方法.首先,通过语义表征学习(semanticrepresentation learning,SRL)模块提取输入图像的深度特征进行标签预测,获得伪标签;接着,通过局部语义学习模块捕获特征中的语义局部相关性,利用局部语义信息对伪标签进行加权增强;最后,以伪标签和增强伪标签的加权组合作为最终的预测标签,采用交叉熵损失函数对网络进行训练.遥感图像不完备多标签分类实验结果表明,与其他多标签分类方法相比,所提方法在不同标签缺失率下具有更好的标签预测性能.
Multilabel learning is highly conducive to the accurate classification of remote sensing (RS) images that contain rich feature information, and the deep-learning-based multilabel classification of RS images has recently gained popularity owing to its exceptional performance. However, the categorical annotation of training samples in large-scale datasets is a time-consuming and labor-intensive task whose quality cannot be guaranteed, leading to the potential occurrence of missing labels. Incomplete label information can mislead the model training process, thereby negatively impacting classification performance. To address this problem, an incomplete multilabel classification method for RS images based on local semantics (LS) is proposed in this paper. First, a semantic representation learning (SRL) module is used to extract deep features from input images, which are then used to generate pseudo-labels. Next, a local semantic learning module is deployed to capture local semantic correlations within the features. These correlations are weighted and subsequently used to enhance the pseudo-labels. Finally, a weighted combination of pseudo-labels and enhanced pseudo-labels is used as the final predicted label, and the network is optimizes using the cross-entropy loss function. The experimental results demonstrate that the proposed method outperforms existing multilabel classification methods with different missing label rates.
[1] Zhu X X, Hu J, Qiu C, et al. So2Sat LCZ42: a benchmark data set for the classiflcation of global local climate zones [J]. IEEE Geoscience and Remote Sensing Magazine, 2020, 8(3): 76-89.
[2] Tan Q, Liu Y, Chen X, et al. Multi-label classiflcation based on low rank representation for image annotation [J]. Remote Sensing, 2017, 9(2): 109.
[3] LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[4] Wang X, Duan L, Ning C. Global context-based multilevel feature fusion networks for multilabel remote sensing image scene classiflcation [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 11179-11196.
[5] Li Y, Chen R, Zhang Y, et al. Multi-label remote sensing image scene classiflcation by combining a convolutional neural network and a graph neural network [J]. Remote Sensing, 2020, 12(23): 4003.
[6] Alshehri A, Bazi Y, Ammour N, et al. Deep attention neural network for multi-label classiflcation in unmanned aerial vehicle imagery [J]. IEEE Access, 2019, 7: 119873-119880.
[7] Ji J, Jing W, Chen G, et al. Multi-label remote sensing image classiflcation with latent semantic dependencies [J]. Remote Sensing, 2020, 12(7): 1110.
[8] Sumbul G, Demir B. A deep multi-attention driven approach for multi-label remote sensing image classiflcation [J]. IEEE Access, 2020, 8: 95934-95946.
[9] Chen Z M, Wei X S, Wang P, et al. Multi-label image recognition with graph convolutional networks [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 5177-5186.
[10] Liu Y, Ni K, Zhang Y, et al. Semantic interleaving global channel attention for multilabel remote sensing image classiflcation [J]. International Journal of Remote Sensing, 2024, 45(2): 393-419.
[11] Huang R, Zheng F, Huang W. Multilabel remote sensing image annotation with multiscale attention and label correlation [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 6951-6961.
[12] Xie M K, Huang S J. Partial multi-label learning [C]//Proceedings of the AAAI Conference on Artiflcial Intelligence. 2018: 4302-4309.
[13] Zhang M L, Fang J P. Partial multi-label learning via credible label elicitation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(10): 3587-3599.
[14] Sun L, Feng S, Wang T, et al. Partial multi-label learning by low-rank and sparse decomposition [C]// Proceedings of the AAAI Conference on Artiflcial Intelligence. 2019: 5016-5023.
[15] Chen T, Pu T, Wu H, et al. Structured semantic transfer for multi-label recognition with partial labels [C]// Proceedings of the AAAI Conference on Artiflcial Intelligence. 2022: 339-346.
[16] Ben-Baruch E, Ridnik T, Friedman I, et al. Multi-label classiflcation with partial annotations using class-aware selective loss [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 4764-4772.
[17] Ma Z, Li L, Mao Q, et al. Label structure preserving contrastive embedding for multi-label learning with missing labels [J/OL]. Computing Research Repository, 2022, https://doi.org/10.48550/arXiv.2209.01314.
[18] Lin J, Yu T, Wang Z J. Rethinking crowdsourcing annotation: partial annotation with salient labels for multilabel aerial image classiflcation [J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-12.
[19] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[20] Hua Y, Mou L, Zhu X X. Relation network for multilabel aerial image classiflcation [J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(7): 4558-4572.
[21] Chaudhuri B, Demir B, Chaudhuri S, et al. Multilabel remote sensing image retrieval using a semi supervised graph-theoretic method [J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(2): 1144-1158.
[22] Hua Y, Mou L, Zhu X X. Label relation inference for multi-label aerial image classiflcation [C]//IGARSS 2019|2019 IEEE International Geoscience and Remote Sensing Symposium. 2019: 5244-5247.
[23] Zhang M L, Zhou Z H. ML-KNN: a lazy learning approach to multi-label learning [J]. Pattern Recognition, 2007, 40(7): 2038-2048.
