Multilabel learning is highly conducive to the accurate classification of remote sensing (RS) images that contain rich feature information, and the deep-learning-based multilabel classification of RS images has recently gained popularity owing to its exceptional performance. However, the categorical annotation of training samples in large-scale datasets is a time-consuming and labor-intensive task whose quality cannot be guaranteed, leading to the potential occurrence of missing labels. Incomplete label information can mislead the model training process, thereby negatively impacting classification performance. To address this problem, an incomplete multilabel classification method for RS images based on local semantics (LS) is proposed in this paper. First, a semantic representation learning (SRL) module is used to extract deep features from input images, which are then used to generate pseudo-labels. Next, a local semantic learning module is deployed to capture local semantic correlations within the features. These correlations are weighted and subsequently used to enhance the pseudo-labels. Finally, a weighted combination of pseudo-labels and enhanced pseudo-labels is used as the final predicted label, and the network is optimizes using the cross-entropy loss function. The experimental results demonstrate that the proposed method outperforms existing multilabel classification methods with different missing label rates.
OU Hanzhi
,
HUANG Rui
. Incomplete multilabel classification of remote sensing images based on local semantics[J]. Journal of Shanghai University, 2026
, 32(2)
: 261
-269
.
DOI: 10.12066/j.issn.1007-2861.2553
[1] Zhu X X, Hu J, Qiu C, et al. So2Sat LCZ42: a benchmark data set for the classiflcation of global local climate zones [J]. IEEE Geoscience and Remote Sensing Magazine, 2020, 8(3): 76-89.
[2] Tan Q, Liu Y, Chen X, et al. Multi-label classiflcation based on low rank representation for image annotation [J]. Remote Sensing, 2017, 9(2): 109.
[3] LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition [J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[4] Wang X, Duan L, Ning C. Global context-based multilevel feature fusion networks for multilabel remote sensing image scene classiflcation [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 11179-11196.
[5] Li Y, Chen R, Zhang Y, et al. Multi-label remote sensing image scene classiflcation by combining a convolutional neural network and a graph neural network [J]. Remote Sensing, 2020, 12(23): 4003.
[6] Alshehri A, Bazi Y, Ammour N, et al. Deep attention neural network for multi-label classiflcation in unmanned aerial vehicle imagery [J]. IEEE Access, 2019, 7: 119873-119880.
[7] Ji J, Jing W, Chen G, et al. Multi-label remote sensing image classiflcation with latent semantic dependencies [J]. Remote Sensing, 2020, 12(7): 1110.
[8] Sumbul G, Demir B. A deep multi-attention driven approach for multi-label remote sensing image classiflcation [J]. IEEE Access, 2020, 8: 95934-95946.
[9] Chen Z M, Wei X S, Wang P, et al. Multi-label image recognition with graph convolutional networks [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 5177-5186.
[10] Liu Y, Ni K, Zhang Y, et al. Semantic interleaving global channel attention for multilabel remote sensing image classiflcation [J]. International Journal of Remote Sensing, 2024, 45(2): 393-419.
[11] Huang R, Zheng F, Huang W. Multilabel remote sensing image annotation with multiscale attention and label correlation [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 6951-6961.
[12] Xie M K, Huang S J. Partial multi-label learning [C]//Proceedings of the AAAI Conference on Artiflcial Intelligence. 2018: 4302-4309.
[13] Zhang M L, Fang J P. Partial multi-label learning via credible label elicitation [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(10): 3587-3599.
[14] Sun L, Feng S, Wang T, et al. Partial multi-label learning by low-rank and sparse decomposition [C]// Proceedings of the AAAI Conference on Artiflcial Intelligence. 2019: 5016-5023.
[15] Chen T, Pu T, Wu H, et al. Structured semantic transfer for multi-label recognition with partial labels [C]// Proceedings of the AAAI Conference on Artiflcial Intelligence. 2022: 339-346.
[16] Ben-Baruch E, Ridnik T, Friedman I, et al. Multi-label classiflcation with partial annotations using class-aware selective loss [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 4764-4772.
[17] Ma Z, Li L, Mao Q, et al. Label structure preserving contrastive embedding for multi-label learning with missing labels [J/OL]. Computing Research Repository, 2022, https://doi.org/10.48550/arXiv.2209.01314.
[18] Lin J, Yu T, Wang Z J. Rethinking crowdsourcing annotation: partial annotation with salient labels for multilabel aerial image classiflcation [J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-12.
[19] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[20] Hua Y, Mou L, Zhu X X. Relation network for multilabel aerial image classiflcation [J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(7): 4558-4572.
[21] Chaudhuri B, Demir B, Chaudhuri S, et al. Multilabel remote sensing image retrieval using a semi supervised graph-theoretic method [J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(2): 1144-1158.
[22] Hua Y, Mou L, Zhu X X. Label relation inference for multi-label aerial image classiflcation [C]//IGARSS 2019|2019 IEEE International Geoscience and Remote Sensing Symposium. 2019: 5244-5247.
[23] Zhang M L, Zhou Z H. ML-KNN: a lazy learning approach to multi-label learning [J]. Pattern Recognition, 2007, 40(7): 2038-2048.
