基于损失加权的实时篮球裁判手势识别系统

doi:10.12066/j.issn.1007-2861.2422

Abstract

Abstract:

To help the audience better understand the meaning of the referee's gesture when watching a live broadcast or a video of a basketball game, or to help video analysts analyze the game video, a real-time basketball referee gesture detection and recognition system, Yolov5-Basketball Referee (Yolov5-BR), was designed. The Yolov5 target detection algorithm was used as the basic model, and the intersection over union (IoU) and complete IoU (CIoU) loss functions of its boundary box were weighted to enhance the robustness of the prediction box. After the C3 module, an attention mechanism was added to generate more distinguishing feature representation and improve the network recognition performance. In addition, an adaptive feature fusion mechanism was incorporated into the head of the detection layer to make full use of the high-level semantic information of the image. Finally, the target confidence loss function was weighted unequally to improve the robustness of small-target detection. On a self-made referee gesture dataset, Yolov5-BR achieved a 95.4% mAP value, with a local video detection rate of 55.5 frame/s, an external camera resolution of $1 280 \times 960$, and a detection rate of 25 frame/s. Experimental results show that, compared with the original model, Yolov5-BR can effectively improve the performance of judging gestures while maintaining high accuracy, stability, and real-time response.

Key words: target detection, gesture recognition, basketball referee, deep learning, loss function

CLC Number:

TP391

LI Zhongyu, SUN Haodong, LI Jiao. Design of real-time basketball referee gesture recognition system based on loss weighting[J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 68-81.

Figures/Tables 18

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Table 1

Table 2

Fig. 11

Fig. 12

Fig. 13

Fig. 14

Table 3

Table 4

References 17

[1]	廖武勋. 上海体育学院篮球裁判员培养模式研究[D]. 上海: 上海体育学院, 2021.
[2]	Pan T Y, Tsai W L, Chang C Y, et al. A hierarchical hand gesture recognition framework for sports referee training-based EMG and accelerometer sensors[J]. IEEE Transactions on Cybernetics, 2022, 52(5): 3172-3183.
[3]	Guyon I, Athitsos V, Jangyodsuk P, et al. The ChaLearn gesture dataset (CGD 2011)[J]. Machine Vision and Applications, 2014, 25(8): 1929-1951.
[4]	Žemgulys J, Raudonis V, Maskeliūnas R, et al. Recognition of basketball referee signals from videos using histogram of oriented gradients (HOG) and support vector machine (SVM)[J]. Procedia Computer Science, 2018, 130: 953-960.
[5]	Žemgulys J, Raudonis V, Maskeliūnas R, et al. Recognition of basketball referee signals from real-time videos[J]. Journal of Ambient Intelligence and Humanized Computing, 2019, 11(3): 979-991.
[6]	Wang X, Shrivastava A, Gupta A. A-Fast-RCNN: hard positive generation via adversary for object detection[C]// 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2017: 3039-3048.
[7]	Redmon J, Farhadi A. YOLO9000: better, faster, stronger[C]// 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2017: 6517-6525.
[8]	韩文静, 罗晓曙, 杨日星. 一种复合型手势识别方法研究[J]. 计算机工程与应用, 2021, 57(4): 108-113.
[9]	郭庆, 何劼恺, 金焰, 等. 基于Faster RCNN 的手势识别[J]. 桂林电子科技大学学报, 2019, 39(6): 490-493.
[10]	范晶晶, 薛皓玮, 吴欣鸿, 等. 引入重影特征映射和通道注意力机制的手势识别算法[J]. 计算机辅助设计与图形学学报, 2022, 34(3): 403-414.
[11]	Jiang B, Luo R, Mao J, et al. Acquisition of localization confidence for accurate object detection[C]// 15th European Conference on Computer Vision (ECCV). 2018: 816-832.
[12]	Rezatofighi H, Tsoi N, Gwak J Y, et al. Generalized inter-section over union: a metric and a loss for bounding box regression[C]// 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 658-666.
[13]	Zheng Z, Wang P, Liu W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]// 34th AAAI Conference on Artificial Intelligence. 2019: 12993-13000.
[14]	Jie H, Li S, Gang S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42(8): 2011-2023.
[15]	Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]// 15th European Conference on Computer Vision (ECCV). 2018: 3-19.
[16]	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42(2): 318-327.
[17]	Zhang H, Wang Y, Dayoub F, et al. VarifocalNet: an IoU-aware dense object detector[C]// IEEE Conference on Computer Vision and Pattern Recognition. 2021: 8510-8519.

Design of real-time basketball referee gesture recognition system based on loss weighting

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 18

References 17

Related Articles 11

Recommended Articles

Metrics

Comments

[1]	YANG Yingjie, WANG Yongfang, ZHANG Han. HDR video reconstruction based on attention and feedback mechanism [J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 56-67.
[2]	WANG Rongrong, XU Shugong, HUANG Jianbo. Image matting based on deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(2): 261-269.
[3]	LI Chengfan, ZHAO Junjuan. Improved approach to detect small sample target based on remote sensing image [J]. Journal of Shanghai University（Natural Science Edition）, 2022, 28(2): 314-323.
[4]	XU Genglin, RAN Feng, DENG Liang, SHI Huakang, GUO Aiying. Application of lightweight neural network and Hash tracking algorithm in embedded face capture system [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(6): 1018-1028.
[5]	XING Yixue, ZHU Yonghua, GAO Haiyan, ZHOU Jin, ZHANG Ke. Distant supervision for relation extraction via attention CNNs [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(5): 983-992.
[6]	SHAO Huixiang, ZENG Dan. Classification and recognition of underwater small targets based on improved YOLOv3 algorithm [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(3): 481-491.
[7]	CHEN Yu, DING Youdong, YU Bing, XU Min. Video colourisation based on voxel flow [J]. Journal of Shanghai University（Natural Science Edition）, 2021, 27(1): 18-27.
[8]	Jiacheng HU, Xiangyang WANG, Han LIU. Defect detection of continuous casting slabs based on deep learning [J]. Journal of Shanghai University（Natural Science Edition）, 2019, 25(4): 445-452.
[9]	XIE Zhifeng, YE Guanhua, YAN Shuqi, HE Shaorong, DING Youdong. HDR image style transfer technique based on generative adversarial networks [J]. Journal of Shanghai University（Natural Science Edition）, 2018, 24(4): 524-534.
[10]	LANG Yue1, ZHOU Jiting1, LIANG Xiaolong2, ZHANG Wenjun1. Automatic annotation for film and Television drama shots and recut system based on face identification [J]. Journal of Shanghai University（Natural Science Edition）, 2017, 23(3): 353-363.
[11]	SHEN Wenfeng, ZHANG Jianlei, ZHOU Dingqian, CHEN Shengbo, QIU Feng. Recognition of Chinese characters on license plates based on big data [J]. Journal of Shanghai University（Natural Science Edition）, 2016, 22(1): 88-96.