基于损失加权的实时篮球裁判手势识别系统

doi:10.12066/j.issn.1007-2861.2422

摘要/Abstract

摘要：

为了方便观众更好地在观看比赛直播和录像时理解裁判手势的含义, 或帮助录像分析师分析比赛视频, 设计了一种实时篮球裁判手势检测与识别系统Yolov5-BR(Yolov5-Basketball Referee). 首先, 采用目标检测中的Yolov5 算法为基础模型, 对其边界框的交并比(intersection over union, IoU) 损失函数完全交并比(complete intersection over union, CIoU) 进行加权处理, 增强预测框的鲁棒性; 其次, 在C3 模块后加入注意力机制, 产生更具分辨性的特征表示, 从而提升网络识别性能; 此外, 在检测层头部融入自适应特征融合机制, 充分利用图像高层语义信息; 最后, 对目标置信度损失函数进行不对等加权处理, 从而提高对小目标检测的鲁棒性. 在自制的裁判手势数据集上, Yolov5-BR 取得了95.4% 的mAP 值, 本地视频检测速率为55.5 帧/s, 外接摄像头分辨率为$1 280\times 960$, 检测速率为25 帧/s. 实验结果表明, Yolov5-BR 相对于原始模型在检测裁判手势的性能上有所提升, 保持了较高的准确率、稳定性与实时性.

关键词: 目标检测, 手势识别, 篮球裁判, 深度学习, 损失函数

Abstract:

To help the audience better understand the meaning of the referee's gesture when watching a live broadcast or a video of a basketball game, or to help video analysts analyze the game video, a real-time basketball referee gesture detection and recognition system, Yolov5-Basketball Referee (Yolov5-BR), was designed. The Yolov5 target detection algorithm was used as the basic model, and the intersection over union (IoU) and complete IoU (CIoU) loss functions of its boundary box were weighted to enhance the robustness of the prediction box. After the C3 module, an attention mechanism was added to generate more distinguishing feature representation and improve the network recognition performance. In addition, an adaptive feature fusion mechanism was incorporated into the head of the detection layer to make full use of the high-level semantic information of the image. Finally, the target confidence loss function was weighted unequally to improve the robustness of small-target detection. On a self-made referee gesture dataset, Yolov5-BR achieved a 95.4% mAP value, with a local video detection rate of 55.5 frame/s, an external camera resolution of $1 280 \times 960$, and a detection rate of 25 frame/s. Experimental results show that, compared with the original model, Yolov5-BR can effectively improve the performance of judging gestures while maintaining high accuracy, stability, and real-time response.

Key words: target detection, gesture recognition, basketball referee, deep learning, loss function

中图分类号:

TP391

李忠雨, 孙浩东, 李娇. 基于损失加权的实时篮球裁判手势识别系统[J]. 上海大学学报(自然科学版), 2023, 29(1): 68-81.

LI Zhongyu, SUN Haodong, LI Jiao. Design of real-time basketball referee gesture recognition system based on loss weighting[J]. Journal of Shanghai University（Natural Science Edition）, 2023, 29(1): 68-81.

图/表 18

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

表1

表2

图11

图12

图13

图14

表3

表4

参考文献 17

[1]	廖武勋. 上海体育学院篮球裁判员培养模式研究[D]. 上海: 上海体育学院, 2021.
[2]	Pan T Y, Tsai W L, Chang C Y, et al. A hierarchical hand gesture recognition framework for sports referee training-based EMG and accelerometer sensors[J]. IEEE Transactions on Cybernetics, 2022, 52(5): 3172-3183.
[3]	Guyon I, Athitsos V, Jangyodsuk P, et al. The ChaLearn gesture dataset (CGD 2011)[J]. Machine Vision and Applications, 2014, 25(8): 1929-1951.
[4]	Žemgulys J, Raudonis V, Maskeliūnas R, et al. Recognition of basketball referee signals from videos using histogram of oriented gradients (HOG) and support vector machine (SVM)[J]. Procedia Computer Science, 2018, 130: 953-960.
[5]	Žemgulys J, Raudonis V, Maskeliūnas R, et al. Recognition of basketball referee signals from real-time videos[J]. Journal of Ambient Intelligence and Humanized Computing, 2019, 11(3): 979-991.
[6]	Wang X, Shrivastava A, Gupta A. A-Fast-RCNN: hard positive generation via adversary for object detection[C]// 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2017: 3039-3048.
[7]	Redmon J, Farhadi A. YOLO9000: better, faster, stronger[C]// 30th IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2017: 6517-6525.
[8]	韩文静, 罗晓曙, 杨日星. 一种复合型手势识别方法研究[J]. 计算机工程与应用, 2021, 57(4): 108-113.
[9]	郭庆, 何劼恺, 金焰, 等. 基于Faster RCNN 的手势识别[J]. 桂林电子科技大学学报, 2019, 39(6): 490-493.
[10]	范晶晶, 薛皓玮, 吴欣鸿, 等. 引入重影特征映射和通道注意力机制的手势识别算法[J]. 计算机辅助设计与图形学学报, 2022, 34(3): 403-414.
[11]	Jiang B, Luo R, Mao J, et al. Acquisition of localization confidence for accurate object detection[C]// 15th European Conference on Computer Vision (ECCV). 2018: 816-832.
[12]	Rezatofighi H, Tsoi N, Gwak J Y, et al. Generalized inter-section over union: a metric and a loss for bounding box regression[C]// 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019: 658-666.
[13]	Zheng Z, Wang P, Liu W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]// 34th AAAI Conference on Artificial Intelligence. 2019: 12993-13000.
[14]	Jie H, Li S, Gang S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42(8): 2011-2023.
[15]	Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]// 15th European Conference on Computer Vision (ECCV). 2018: 3-19.
[16]	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 42(2): 318-327.
[17]	Zhang H, Wang Y, Dayoub F, et al. VarifocalNet: an IoU-aware dense object detector[C]// IEEE Conference on Computer Vision and Pattern Recognition. 2021: 8510-8519.