💡💡💡本文解决什么问题：教会你如何用自己的数据集转换成对应格式的数据集以及如何训练YOLO11-pose关键点检测
1.YOLO11介绍
Ultralytics YOLO11是一款尖端的、最先进的模型，它在之前YOLO版本成功的基础上进行了构建，并引入了新功能和改进，以进一步提升性能和灵活性。YOLO11设计快速、准确且易于使用，使其成为各种物体检测和跟踪、实例分割、图像分类以及姿态估计任务的绝佳选择。
pose官方在COCO数据集上做了更多测试：
结构图如下：
1.1 C3k2
C3k2，结构图如下
C3k2，继承自类C2f，其中通过c3k设置False或者Ture来决定选择使用C3k还是Bottleneck
实现代码ultralytics/nn/modules/block.py
1.2 C2PSA介绍
借鉴V10 PSA结构，实现了C2PSA和C2fPSA，最终选择了基于C2的C2PSA（可能涨点更好？）
实现代码ultralytics/nn/modules/block.py
1.3 11 Detect介绍
分类检测头引入了DWConv（更加轻量级，为后续二次创新提供了改进点），结构图如下（和V8的区别）：
实现代码ultralytics/nn/modules/head.py
2.如何标注自己的关键点数据集
2.1 labelme下载
# 安装labelme pip install labelme
2.2使用labelme下
直接在python环境下运行
labelme
2.3 labelme介绍
关键点标记主要使用
1)Create Rectangle生成矩形框；
2）Create Point生成关键点；
2.4 数据集标注
3.数据集格式转换
3.1标记后的数据格式如下
一张图片对应一个json文件
json部分内容如下：
{ "version": "5.1.1", "flags": {}, "shapes": [ { "label": "Ulna", "points": [ [ 474.71929824561397, 1443.4035087719294 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "Radius", "points": [ [ 639.6315789473683, 1445.1578947368416 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "FMCP", "points": [ [ 832.6140350877192, 1222.3508771929824 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "FPIP", "points": [ [ 1023.8421052631577, 1089.0175438596489 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "FDIP", "points": [ [ 1157.1754385964912, 987.2631578947367 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "MCP5", "points": [ [ 430.85964912280696, 946.9122807017543 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "MCP4", "points": [ [ 543.1403508771929, 910.0701754385964 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "MCP3", "points": [ [ 657.1754385964912, 881.9999999999998 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "MCP2", "points": [ [ 797.5263157894736, 896.035087719298 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "PIP5", "points": [ [ 401.0350877192982, 864.4561403508769 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "PIP4", "points": [ [ 529.1052631578947, 801.2982456140348 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "PIP3", "points": [ [ 678.2280701754386, 773.2280701754385 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "PIP2", "points": [ [ 843.1403508771929, 810.0701754385964 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "MIP5", "points": [ [ 329.1052631578947, 696.035087719298 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "MIP4", "points": [ [ 513.3157894736842, 562.7017543859647 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "MIP3", "points": [ [ 713.3157894736842, 515.3333333333333 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "MIP2", "points": [ [ 944.8947368421052, 608.3157894736839 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "DIP5", "points": [ [ 313.31578947368416, 601.2982456140348 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "DIP4", "points": [ [ 509.8070175438596, 411.8245614035086 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "DIP3", "points": [ [ 727.3508771929824, 345.15789473684185 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "DIP2", "points": [ [ 994.017543859649, 492.52631578947353 ] ], "group_id": null, "shape_type": "point", "flags": {} }, { "label": "hand", "points": [ [ 204.82608695652186, 183.95652173913044 ], [ 1265.695652173913, 1775.2608695652175 ] ], "group_id": null, "shape_type": "rectangle", "flags": {} } ], "imagePath": "1525.png",
3.2 生成适合yolo格式的关键点数据集
labelme2yolo-keypoint
源码见博客:
https://blog.csdn.net/m0_63774211/article/details/142981705
生成的txt内容如下:
0 0.48481 0.47896 0.70079 0.77886 0.31308 0.70597 2 0.42206 0.70695 2 0.54954 0.59785 2 0.67569 0.53278 2 0.76420 0.48288 2 0.28402 0.46282 2 0.35865 0.44521 2 0.43395 0.43102 2 0.52642 0.43836 2 0.26486 0.42270 2 0.34941 0.39188 2 0.44782 0.37818 2 0.55680 0.39628 2 0.21731 0.34051 2 0.33884 0.27495 2 0.47094 0.25196 2 0.62351 0.29746 2 0.20674 0.29403 2 0.33620 0.20108 2 0.48018 0.16879 2 0.65654 0.24070 2
讲解：
第一个0代表：框的类别，因为只有hand一类，所以为0
0.48481 0.47896 0.70079 0.77886 代表：归一化后的 框的中心点横纵坐标、宽、高
0.31308 0.70597 2代表：归一化后的 第一个关键点的横纵坐标、关键点可见性
关键点可见性理解：0代表不可见、1代表遮挡、2代表可见
3.3生成的yolo数据集如下
hand_keypoint： -images: --train: png图片 --val：png图片 -labels： --train: txt文件 --val：txt文件
4.手部关键点训练
4.1 新建data/hand_keypoint.yaml
参考coco8-pose.yaml即可
# Ultralytics YOLO 🚀, AGPL-3.0 license # COCO8-pose dataset (first 8 images from COCO train2017) by Ultralytics # Example usage: yolo train data=coco8-pose.yaml # parent # ├── ultralytics # └── datasets # └── coco8-pose ← downloads here (1 MB) # Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..] path: ./ultralytics-7.31/data/hand_keypoint # dataset root dir train: images/train # train images (relative to 'path') 4 images val: images/val # val images (relative to 'path') 4 images test: # test images (optional) # Keypoints kpt_shape: [21, 3] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible) flip_idx: [0, 2, 1, 4, 3, 6, 5, 8, 7, 10, 9, 12, 11, 14, 13, 16, 15,17,18,19,20] # Classes names: 0: hand # Download script/URL (optional) #download: https://ultralytics.com/assets/coco8-pose.zip
4.2修改ultralytics/cfg/models/11/yolo11-pose.yaml
修改为21个关键点和一个类别nc:1
# Ultralytics YOLO 🚀, AGPL-3.0 license # YOLO11-pose keypoints/pose estimation model. For Usage examples see https://docs.ultralytics.com/tasks/pose # Parameters nc: 1 # number of classes kpt_shape: [21, 3] # number of keypoints, number of dims (2 for x,y or 3 for x,y,visible) scales: # model compound scaling constants, i.e. 'model=yolo11n-pose.yaml' will call yolo11.yaml with scale 'n' # [depth, width, max_channels] n: [0.50, 0.25, 1024] # summary: 344 layers, 2908507 parameters, 2908491 gradients, 7.7 GFLOPs s: [0.50, 0.50, 1024] # summary: 344 layers, 9948811 parameters, 9948795 gradients, 23.5 GFLOPs m: [0.50, 1.00, 512] # summary: 434 layers, 20973273 parameters, 20973257 gradients, 72.3 GFLOPs l: [1.00, 1.00, 512] # summary: 656 layers, 26230745 parameters, 26230729 gradients, 91.4 GFLOPs x: [1.00, 1.50, 512] # summary: 656 layers, 58889881 parameters, 58889865 gradients, 204.3 GFLOPs
4.3默认参数开启训练
import warnings warnings.filterwarnings('ignore') from ultralytics import YOLO if __name__ == '__main__': model = YOLO('ultralytics/cfg/models/11/yolo11-pose.yaml') model.train(data='data/hand_keypoint.yaml', cache=False, imgsz=640, epochs=200, batch=8, close_mosaic=10, device='0', optimizer='SGD', # using SGD project='runs/train-pose', name='exp', )
4.4 训练结果分析
200个epoch以后，训练结果可视化如下：
Pose mAP50 为 0.871
YOLO11-pose summary (fused): 300 layers, 3,199,712 parameters, 0 gradients, 7.8 GFLOPs Class Images Instances Box(P R mAP50 mAP50-95) Pose(P R mAP50 mAP50-95): 100%|██████████| 4/4 [00:04<00:00, 1.23s/it] all 64 64 0.999 1 0.995 0.668 0.922 0.922 0.871 0.638
PosePR_curve.png
预测结果如下：
