计算YOLOv3在COCO数据集上的mAP值

安装darknet

详细步骤和解释请参考YOLOv3的官网，这里给出相应的命令行和需要特别注意的地方

从github上下载项目源码

git clone https://github.com/pjreddie/darknet
cd darknet

对源码进行编译(编译默认运行YOLOv3时是不使用GPU的，我们假设跑YOLOv3时需要使用GPU，所以需要修改编译选项)
1. 只需要将当前文件夹下（darknet/）的Makefile文件中第一行的GPU=0改为GPU=1，改完后的Makefile：
```
GPU=1
CUDNN=0
OPENCV=0
OPENMP=0
DEBUG=0

...
...
...
```
2. 编译
```
make
```
将权重文件（yolov3.weights）下载到当前目录下
```
wget https://pjreddie.com/media/files/yolov3.weights
```

测试

#命令行中的‘-i 0’代表使用第0号显卡，假如不想使用GPU可以用‘-nogpu’替代‘-i 0’
./darknet -i 0 detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights data/dog.jpg

COCO数据集的下载和配置

通过脚本文件（scripts/get_coco_dataset.sh）获取COCO数据集（数据集下载速度会很慢，建议从Function的数据集下载站下载，然后逐行运行scripts/get_coco_dataset.sh中的每一行命令）
```
cp scripts/get_coco_dataset.sh data
cd data
bash get_coco_dataset.sh
```

修改COCO的配置文件（cfg/coco.data）

classes= 80
train  = <path-to-coco>/trainvalno5k.txt # 注意这里的<path-to-coco>需要填入COCO数据集的文件夹路径
valid  = <path-to-coco>/5k.txt # 注意这里的<path-to-coco>需要填入COCO数据集的文件夹路径
names = data/coco.names
backup = backup
eval = coco #注意这行是为了生成json格式的检测结果，官网里并没有说，如果去掉这一行生成的文件是classes(80)类个txt文件

修改YOLOv3模型的配置文件（cfg/yolov3.cfg）

[net]
# Testing
# batch=1
# subdivisions=1
# Training
batch=64 #注意这里的batch有可能使得显存爆掉,这时需要修改,GeForce RTX 2080Ti需要修改为8
subdivisions=8
....

对COCO验证集进行检测

在darknet目录下运行下面的命令行
```
./darknet -i 0 detector valid cfg/coco.data cfg/yolov3.cfg yolov3.weights # '-i 0'代表使用0号GPU
```
运行完上述命令后会生成results/coco_results.json文件，该文件保存了检测结果

计算mAP

方法源于：https://blog.csdn.net/xidaoliang/article/details/88397280

安装pycocotools
```
pip install pycocotools
```

在darknet目录下编写一个python脚本(compute_coco_mAP.py)用于计算mAP（注意根据自己的实际情况修改两个文件路径）

#-*- coding:utf-8 -*-
import matplotlib.pyplot as plt 
from pycocotools.coco import COCO 
from pycocotools.cocoeval import COCOeval 
import numpy as np 
import skimage.io as io 
import pylab,json 
pylab.rcParams['figure.figsize'] = (10.0, 8.0) 
def get_img_id(file_name): 
    ls = [] 
    myset = [] 
    annos = json.load(open(file_name, 'r')) 
    for anno in annos: 
      ls.append(anno['image_id']) 
    myset = {}.fromkeys(ls).keys() 
    return myset 
if __name__ == '__main__': 
    annType = ['segm', 'bbox', 'keypoints']#set iouType to 'segm', 'bbox' or 'keypoints'
    annType = annType[1] # specify type here
    cocoGt_file = 'data/coco/annotations/instances_val2014.json' #需要根据自己的实际情况配置该路径
    cocoGt = COCO(cocoGt_file)#取得标注集中coco json对象
    cocoDt_file = 'results/coco_results.json' #需要根据自己的实际情况配置该路径
    imgIds = get_img_id(cocoDt_file) 
    print(len(imgIds))
    cocoDt = cocoGt.loadRes(cocoDt_file)#取得结果集中image json对象
    imgIds = sorted(imgIds)#按顺序排列coco标注集image_id
    imgIds = imgIds[0:5000]#标注集中的image数据
    cocoEval = COCOeval(cocoGt, cocoDt, annType) 
    cocoEval.params.imgIds = imgIds#参数设置
    cocoEval.evaluate()#评价
    cocoEval.accumulate()#积累
    cocoEval.summarize()#总结

运行compute_coco_mAP.py脚本
```
python compute_coco_mAP.py
```
有可能在运行compute_coco_mAP.py脚本的过程中会报错，可能的解决方法：
1. 升级scikit-image
```
pip install -U scikit-image
```
2. 修改Numpy版本为1.16
```
pip install numpy==1.16
```

结果展示

img_size为608*608（img_size的设置可以通过修改cfg/yolov3.cfg文件中的width和height来实现）时：

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.334
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.585
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.345
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.194
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.365
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.439
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.291
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.446
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.470
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.304
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.502
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.593

img_size为416*416时：

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.314
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.559
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.318
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.142
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.341
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.464
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.278
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.419
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.442
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.239
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.482
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.611

img_size为320*320时：

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.286
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.521
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.284
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.103
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.315
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.449
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.260
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.389
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.408
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.189
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.454
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.601