使用ResNet的caffe實現進行圖片分類

背景

圖片分類是深度學習在圖像領域的初級應用，更高階的應用如目標檢測、圖像分割等。那就從最初級的應用-圖像分類做起吧。

準備數據

準備數據包括訓練集和測試集。

比方說gemfield要做個224分類的應用，那麼先準備好224個子目錄，每個目錄里放的是同一分類的照片：

gemfield@ai:/bigdata/video_annotation_web/annotation_output$ find . -type f./0/wzry223.mp4_7670_1051_385_126_140.jpg./0/wzry110.mp4_2740_562_293_160_148.jpg......./65/wzry223.mp4_7670_1021_385_126_140.jpg./65/wzry110.mp4_2740_572_293_160_148.jpg......./153/wzry61.mp4_1240_561_298_182_150.jpg./153/wzry73.mp4_3660_792_424_280_238.jpg......./223/wzry73.mp4_3510_830_426_252_232.jpg./223/wzry73.mp4_1450_820_434_274_240.jpg

224分類的label是0到223，一定要從0開始，caffe框架決定的。那現在有了224個目錄的照片，怎麼生成對應的label文件呢？可以使用下面的python代碼：

gemfield@ai:/bigdata/video_annotation_web/annotation_output$ find . -type f > ../gemfield.txtgemfield@ai:/bigdata/video_annotation_web/annotation_output$ cd ..gemfield@ai:/bigdata/video_annotation_web$ pythonPython 2.7.13 (default, Nov 23 2017, 15:37:09) [GCC 6.3.0 20170406] on linux2Type "help", "copyright", "credits" or "license" for more information.>>> l = open(gemfield.txt).readlines()>>> ll = [x.strip() + + x.split(/)[1] +
for x in l]>>> import random>>> random.shuffle(ll)>>> with open(gemfield.lst, w) as f:... f.writelines(ll)... >>>

現在label文件也生成了，我們需要將其分為訓練集和測試集，針對本次實驗，gemfield按照6:1的比例劃分了訓練集和測試集；這個劃分過程不用動那224個圖片目錄，而只需要將上面的那個gemfield.txt按照6:1的比例劃分為2個label文件：train.lst和val.lst 文件：

gemfield@ai:/bigdata/video_annotation_web$ tail -24319 gemfield.lst > val.lstgemfield@ai:/bigdata/video_annotation_web$ head -120000 gemfield.lst > train.lst

準備Caffe環境

下載官方的caffe倉庫，修改makefile.config的參數，編譯就ok了。參考本專欄的前述文章。

準備ResNet的網路

Gemfield使用的是ResNet50版本：

KaimingHe/deep-residual-networks

這個下載下來的prototxt文件，拷貝2份出來，一份命名為ResNet-50-deploy-by-gemfield.prototxt，這個不作修改；另一份命名為ResNet-50-train-val-by-gemfield.prototxt，然後這個網路結構文件要進行修改，修改部分如下所示：

第一部分：修改input部分：

ResNet-50-deploy-by-gemfield.prototxt中的

input: "data"input_dim: 1input_dim: 3input_dim: 224input_dim: 224

改為ResNet-50-train-val-by-gemfield.prototxt中的

layer { name: "data" type: "Data" top: "data" top: "label" include { phase: TRAIN } transform_param { mirror: false # crop_size: 224 mean_value: 0 mean_value: 0 mean_value: 0 } data_param { source: "wzry_224_train_lmdb" batch_size: 16 backend: LMDB }}layer { name: "data" type: "Data" top: "data" top: "label" include { phase: TEST } transform_param { mirror: false # crop_size: 224 mean_value: 0 mean_value: 0 mean_value: 0 } data_param { source: "wzry_224_val_lmdb" batch_size: 1 backend: LMDB }}

第二部分：

1，將全連接層的fc1000改為fc224，因為我們的分類是224種；

2，將softmax改為softmax with loss，因為訓練的時候必須有loss作為反饋；

將ResNet-50-deploy-by-gemfield.prototxt中的

layer { bottom: "pool5" top: "fc1000" name: "fc1000" type: "InnerProduct" inner_product_param { num_output: 1000 }}layer { bottom: "fc1000" top: "prob" name: "prob" type: "Softmax"}

改為ResNet-50-train-val-by-gemfield.prototxt中的

layer { bottom: "pool5" top: "fc224" name: "fc224" type: "InnerProduct" inner_product_param { num_output: 224 }}layer { bottom: "fc224" bottom: "label" name: "loss" type: "SoftmaxWithLoss" top: "loss"}layer { bottom: "fc224" bottom: "label" top: "acc/top-1" name: "acc/top-1" type: "Accuracy" include { phase: TEST }}

第三部分：

ResNet的輸入參數是224x224，因此，在下面生成lmdb的時候，要將圖像resize成224x224，除非你的圖片本來就是224x224。

準備solver.prototxt

net: "ResNet-50-train-val-by-gemfield.prototxt"#每次測試跑多少次迭代，照片數量/test的batch_sizetest_iter: 24000#每多少次迭代進行一次測試test_interval: 2000test_initialization: false#初始的學習率base_lr: 0.001#學習率的下降策略lr_policy: "step"gamma: 0.1stepsize: 5000#每迭代多少次列印一次loss信息display: 100#最多迭代多少次max_iter: 75000#梯度下降策略，90%依賴這次，10%依賴上次momentum: 0.9#正則項的比重weight_decay: 0.0001#迭代多少次生成一次中間的caffemodelsnapshot: 5000#caffemodel的前綴snapshot_prefix: "models/resnet_gemfield_cls224"solver_mode: GPU

生成lmdb

使用Caffe自帶的工具convert_imageset來生成lmdb文件（在caffe倉庫目錄里），注意，resnet的輸入都是224x224,所以在製作數據集的時候需要進行resize；其次，convert_imageset會簡單的將下列命令中的root目錄"./"和train.lst中的文件路徑進行直接的字元串拼接；再次，生成的lmdb目錄不能已經存在。

先生成訓練集：

gemfield@ai:~$ convert_imageset -resize_height 224 -resize_width 224 ./ train.lst wzry_224_train_lmdbI0113 12:13:44.050431 27 convert_imageset.cpp:89] A total of 120000 images.E0113 12:13:44.051177 27 common.cpp:114] Cannot create Cublas handle. Cublas wont be available.E0113 12:13:44.051475 27 common.cpp:121] Cannot create Curand generator. Curand wont be available.I0113 12:13:44.051504 27 db_lmdb.cpp:35] Opened lmdb wzry_224_train_lmdbI0113 12:13:45.643923 27 convert_imageset.cpp:147] Processed 1000 files.I0113 12:13:47.270226 27 convert_imageset.cpp:147] Processed 2000 files.I0113 12:13:48.633996 27 convert_imageset.cpp:147] Processed 3000 files.I0113 12:13:50.994860 27 convert_imageset.cpp:147] Processed 4000 files.I0113 12:13:52.172212 27 convert_imageset.cpp:147] Processed 5000 files.I0113 12:13:53.344650 27 convert_imageset.cpp:147] Processed 6000 files.I0113 12:13:55.925756 27 convert_imageset.cpp:147] Processed 7000 files.I0113 12:13:57.467402 27 convert_imageset.cpp:147] Processed 8000 files.I0113 12:13:59.090173 27 convert_imageset.cpp:147] Processed 9000 files.I0113 12:14:00.676419 27 convert_imageset.cpp:147] Processed 10000 files.I0113 12:14:02.851028 27 convert_imageset.cpp:147] Processed 11000 files.I0113 12:14:04.404832 27 convert_imageset.cpp:147] Processed 12000 files.I0113 12:14:05.979625 27 convert_imageset.cpp:147] Processed 13000 files.I0113 12:14:07.152940 27 convert_imageset.cpp:147] Processed 14000 files.I0113 12:14:09.217108 27 convert_imageset.cpp:147] Processed 15000 files.I0113 12:14:10.806496 27 convert_imageset.cpp:147] Processed 16000 files.I0113 12:14:12.268177 27 convert_imageset.cpp:147] Processed 17000 files.I0113 12:14:13.508546 27 convert_imageset.cpp:147] Processed 18000 files.I0113 12:14:15.016147 27 convert_imageset.cpp:147] Processed 19000 files.I0113 12:14:17.189391 27 convert_imageset.cpp:147] Processed 20000 files.I0113 12:14:19.469425 27 convert_imageset.cpp:147] Processed 21000 files.I0113 12:14:20.869881 27 convert_imageset.cpp:147] Processed 22000 files.I0113 12:14:22.358198 27 convert_imageset.cpp:147] Processed 23000 files.I0113 12:14:23.811714 27 convert_imageset.cpp:147] Processed 24000 files.I0113 12:14:25.611598 27 convert_imageset.cpp:147] Processed 25000 files.I0113 12:14:27.188184 27 convert_imageset.cpp:147] Processed 26000 files.I0113 12:14:28.578914 27 convert_imageset.cpp:147] Processed 27000 files.I0113 12:14:30.235432 27 convert_imageset.cpp:147] Processed 28000 files.I0113 12:14:31.867501 27 convert_imageset.cpp:147] Processed 29000 files.I0113 12:14:33.497715 27 convert_imageset.cpp:147] Processed 30000 files.I0113 12:14:34.925644 27 convert_imageset.cpp:147] Processed 31000 files.I0113 12:14:36.906508 27 convert_imageset.cpp:147] Processed 32000 files.I0113 12:14:38.464900 27 convert_imageset.cpp:147] Processed 33000 files.I0113 12:14:40.249969 27 convert_imageset.cpp:147] Processed 34000 files.I0113 12:14:42.774039 27 convert_imageset.cpp:147] Processed 35000 files.I0113 12:14:44.158181 27 convert_imageset.cpp:147] Processed 36000 files.I0113 12:14:45.778302 27 convert_imageset.cpp:147] Processed 37000 files.I0113 12:14:47.252451 27 convert_imageset.cpp:147] Processed 38000 files.I0113 12:14:49.799836 27 convert_imageset.cpp:147] Processed 39000 files.I0113 12:14:51.288296 27 convert_imageset.cpp:147] Processed 40000 files.I0113 12:14:52.484359 27 convert_imageset.cpp:147] Processed 41000 files.I0113 12:14:54.095209 27 convert_imageset.cpp:147] Processed 42000 files.I0113 12:14:56.525655 27 convert_imageset.cpp:147] Processed 43000 files.I0113 12:14:58.033469 27 convert_imageset.cpp:147] Processed 44000 files.I0113 12:14:59.409360 27 convert_imageset.cpp:147] Processed 45000 files.I0113 12:15:00.880760 27 convert_imageset.cpp:147] Processed 46000 files.I0113 12:15:02.602565 27 convert_imageset.cpp:147] Processed 47000 files.I0113 12:15:04.345715 27 convert_imageset.cpp:147] Processed 48000 files.I0113 12:15:05.799033 27 convert_imageset.cpp:147] Processed 49000 files.I0113 12:15:06.995527 27 convert_imageset.cpp:147] Processed 50000 files.I0113 12:15:08.686816 27 convert_imageset.cpp:147] Processed 51000 files.I0113 12:15:11.079591 27 convert_imageset.cpp:147] Processed 52000 files.I0113 12:15:12.673241 27 convert_imageset.cpp:147] Processed 53000 files.I0113 12:15:14.083250 27 convert_imageset.cpp:147] Processed 54000 files.I0113 12:15:15.629169 27 convert_imageset.cpp:147] Processed 55000 files.I0113 12:15:17.027292 27 convert_imageset.cpp:147] Processed 56000 files.I0113 12:15:18.703470 27 convert_imageset.cpp:147] Processed 57000 files.I0113 12:15:20.178828 27 convert_imageset.cpp:147] Processed 58000 files.I0113 12:15:21.541697 27 convert_imageset.cpp:147] Processed 59000 files.I0113 12:15:23.018353 27 convert_imageset.cpp:147] Processed 60000 files.I0113 12:15:24.517725 27 convert_imageset.cpp:147] Processed 61000 files.I0113 12:15:26.077165 27 convert_imageset.cpp:147] Processed 62000 files.I0113 12:15:27.476835 27 convert_imageset.cpp:147] Processed 63000 files.I0113 12:15:28.695413 27 convert_imageset.cpp:147] Processed 64000 files.I0113 12:15:30.552235 27 convert_imageset.cpp:147] Processed 65000 files.I0113 12:15:32.182060 27 convert_imageset.cpp:147] Processed 66000 files.I0113 12:15:33.655457 27 convert_imageset.cpp:147] Processed 67000 files.I0113 12:15:34.957446 27 convert_imageset.cpp:147] Processed 68000 files.I0113 12:15:36.233927 27 convert_imageset.cpp:147] Processed 69000 files.I0113 12:15:37.818439 27 convert_imageset.cpp:147] Processed 70000 files.I0113 12:15:39.956657 27 convert_imageset.cpp:147] Processed 71000 files.I0113 12:15:41.336088 27 convert_imageset.cpp:147] Processed 72000 files.I0113 12:15:42.587082 27 convert_imageset.cpp:147] Processed 73000 files.I0113 12:15:44.044525 27 convert_imageset.cpp:147] Processed 74000 files.I0113 12:15:45.632407 27 convert_imageset.cpp:147] Processed 75000 files.I0113 12:15:47.075654 27 convert_imageset.cpp:147] Processed 76000 files.I0113 12:15:48.567188 27 convert_imageset.cpp:147] Processed 77000 files.I0113 12:15:49.854224 27 convert_imageset.cpp:147] Processed 78000 files.I0113 12:15:51.043192 27 convert_imageset.cpp:147] Processed 79000 files.I0113 12:15:52.586591 27 convert_imageset.cpp:147] Processed 80000 files.I0113 12:15:54.092438 27 convert_imageset.cpp:147] Processed 81000 files.I0113 12:15:55.546066 27 convert_imageset.cpp:147] Processed 82000 files.I0113 12:15:56.811760 27 convert_imageset.cpp:147] Processed 83000 files.I0113 12:15:57.987690 27 convert_imageset.cpp:147] Processed 84000 files.I0113 12:15:59.671317 27 convert_imageset.cpp:147] Processed 85000 files.I0113 12:16:01.283619 27 convert_imageset.cpp:147] Processed 86000 files.I0113 12:16:02.826110 27 convert_imageset.cpp:147] Processed 87000 files.I0113 12:16:04.015786 27 convert_imageset.cpp:147] Processed 88000 files.I0113 12:16:05.246563 27 convert_imageset.cpp:147] Processed 89000 files.I0113 12:16:07.631480 27 convert_imageset.cpp:147] Processed 90000 files.I0113 12:16:09.503032 27 convert_imageset.cpp:147] Processed 91000 files.I0113 12:16:10.790294 27 convert_imageset.cpp:147] Processed 92000 files.I0113 12:16:12.273556 27 convert_imageset.cpp:147] Processed 93000 files.I0113 12:16:14.271823 27 convert_imageset.cpp:147] Processed 94000 files.I0113 12:16:15.744318 27 convert_imageset.cpp:147] Processed 95000 files.I0113 12:16:17.090734 27 convert_imageset.cpp:147] Processed 96000 files.I0113 12:16:18.353124 27 convert_imageset.cpp:147] Processed 97000 files.I0113 12:16:19.863163 27 convert_imageset.cpp:147] Processed 98000 files.I0113 12:16:21.596539 27 convert_imageset.cpp:147] Processed 99000 files.I0113 12:16:23.036448 27 convert_imageset.cpp:147] Processed 100000 files.I0113 12:16:24.533990 27 convert_imageset.cpp:147] Processed 101000 files.I0113 12:16:25.711519 27 convert_imageset.cpp:147] Processed 102000 files.I0113 12:16:27.407729 27 convert_imageset.cpp:147] Processed 103000 files.I0113 12:16:29.793790 27 convert_imageset.cpp:147] Processed 104000 files.I0113 12:16:31.340585 27 convert_imageset.cpp:147] Processed 105000 files.I0113 12:16:32.955618 27 convert_imageset.cpp:147] Processed 106000 files.I0113 12:16:34.523054 27 convert_imageset.cpp:147] Processed 107000 files.I0113 12:16:36.220069 27 convert_imageset.cpp:147] Processed 108000 files.I0113 12:16:37.792441 27 convert_imageset.cpp:147] Processed 109000 files.I0113 12:16:39.185384 27 convert_imageset.cpp:147] Processed 110000 files.I0113 12:16:40.366708 27 convert_imageset.cpp:147] Processed 111000 files.I0113 12:16:41.930851 27 convert_imageset.cpp:147] Processed 112000 files.I0113 12:16:43.605252 27 convert_imageset.cpp:147] Processed 113000 files.I0113 12:16:45.682309 27 convert_imageset.cpp:147] Processed 114000 files.I0113 12:16:47.001965 27 convert_imageset.cpp:147] Processed 115000 files.I0113 12:16:48.201941 27 convert_imageset.cpp:147] Processed 116000 files.I0113 12:16:50.482267 27 convert_imageset.cpp:147] Processed 117000 files.I0113 12:16:53.129992 27 convert_imageset.cpp:147] Processed 118000 files.I0113 12:16:55.065909 27 convert_imageset.cpp:147] Processed 119000 files.I0113 12:16:56.272369 27 convert_imageset.cpp:147] Processed 120000 files.

再生成測試集：

gemfield@ai:~$ convert_imageset -resize_height 224 -resize_width 224 ./ val.lst wzry_224_val_lmdbI0113 12:35:57.135522 28 convert_imageset.cpp:89] A total of 24319 images.E0113 12:35:57.136313 28 common.cpp:114] Cannot create Cublas handle. Cublas wont be available.E0113 12:35:57.136673 28 common.cpp:121] Cannot create Curand generator. Curand wont be available.I0113 12:35:57.136708 28 db_lmdb.cpp:35] Opened lmdb wzry_224_val_lmdbI0113 12:35:58.789322 28 convert_imageset.cpp:147] Processed 1000 files.I0113 12:36:00.379387 28 convert_imageset.cpp:147] Processed 2000 files.I0113 12:36:01.759858 28 convert_imageset.cpp:147] Processed 3000 files.I0113 12:36:02.987604 28 convert_imageset.cpp:147] Processed 4000 files.I0113 12:36:04.426345 28 convert_imageset.cpp:147] Processed 5000 files.I0113 12:36:05.960880 28 convert_imageset.cpp:147] Processed 6000 files.I0113 12:36:07.313741 28 convert_imageset.cpp:147] Processed 7000 files.I0113 12:36:08.496649 28 convert_imageset.cpp:147] Processed 8000 files.I0113 12:36:10.013240 28 convert_imageset.cpp:147] Processed 9000 files.I0113 12:36:11.585314 28 convert_imageset.cpp:147] Processed 10000 files.I0113 12:36:12.984436 28 convert_imageset.cpp:147] Processed 11000 files.I0113 12:36:14.225903 28 convert_imageset.cpp:147] Processed 12000 files.I0113 12:36:15.468725 28 convert_imageset.cpp:147] Processed 13000 files.I0113 12:36:17.084775 28 convert_imageset.cpp:147] Processed 14000 files.I0113 12:36:18.566448 28 convert_imageset.cpp:147] Processed 15000 files.I0113 12:36:19.855185 28 convert_imageset.cpp:147] Processed 16000 files.I0113 12:36:21.040355 28 convert_imageset.cpp:147] Processed 17000 files.I0113 12:36:22.528937 28 convert_imageset.cpp:147] Processed 18000 files.I0113 12:36:23.917870 28 convert_imageset.cpp:147] Processed 19000 files.I0113 12:36:25.164480 28 convert_imageset.cpp:147] Processed 20000 files.I0113 12:36:27.153021 28 convert_imageset.cpp:147] Processed 21000 files.I0113 12:37:18.769785 28 convert_imageset.cpp:147] Processed 22000 files.I0113 12:37:19.995821 28 convert_imageset.cpp:147] Processed 23000 files.I0113 12:37:21.388216 28 convert_imageset.cpp:147] Processed 24000 files.I0113 12:37:21.853869 28 convert_imageset.cpp:153] Processed 24319 files.

值得注意的是，1.5G的圖片生成的lmdb大小為20.5GB！

獲取pretrained的model

pre trained model主要是用來初始化當前網路結構的參數。

gemfield@ai:~$ ls -l ResNet-50-model.caffemodel -rw-r--r-- 1 gemfield gemfield 102462397 1月 13 21:16 ResNet-50-model.caffemodel

開始訓練

gemfield@ai:~$ build/tools/caffe train -solver solver.prototxt -weights ResNet-50-model.caffemodel

一些訓練日誌

I0114 03:52:23.357553 21226 solver.cpp:218] Iteration 73900 (6.61567 iter/s, 15.1156s/100 iters), loss = 0.0242063I0114 03:52:23.357609 21226 solver.cpp:237] Train net output #0: loss = 0.0242095 (* 1 = 0.0242095 loss)I0114 03:52:23.357619 21226 sgd_solver.cpp:105] Iteration 73900, lr = 1e-17I0114 03:52:38.312636 21226 solver.cpp:330] Iteration 74000, Testing net (#0)I0114 03:54:58.760177 21231 data_layer.cpp:73] Restarting data prefetching from start.I0114 03:57:33.356952 21226 solver.cpp:397] Test net output #0: acc/top-1 = 0.969708I0114 03:57:33.357126 21226 solver.cpp:397] Test net output #1: loss = 0.115373 (* 1 = 0.115373 loss)I0114 03:57:33.503705 21226 solver.cpp:218] Iteration 74000 (0.322433 iter/s, 310.142s/100 iters), loss = 0.0952036I0114 03:57:33.503748 21226 solver.cpp:237] Train net output #0: loss = 0.0952069 (* 1 = 0.0952069 loss)I0114 03:57:33.503764 21226 sgd_solver.cpp:105] Iteration 74000, lr = 1e-17I0114 03:57:48.466397 21226 solver.cpp:218] Iteration 74100 (6.68344 iter/s, 14.9624s/100 iters), loss = 0.00265225I0114 03:57:48.466521 21226 solver.cpp:237] Train net output #0: loss = 0.00265556 (* 1 = 0.00265556 loss)I0114 03:57:48.466542 21226 sgd_solver.cpp:105] Iteration 74100, lr = 1e-17I0114 03:58:03.440867 21226 solver.cpp:218] Iteration 74200 (6.67818 iter/s, 14.9741s/100 iters), loss = 0.011983I0114 03:58:03.441133 21226 solver.cpp:237] Train net output #0: loss = 0.0119865 (* 1 = 0.0119865 loss)I0114 03:58:03.441164 21226 sgd_solver.cpp:105] Iteration 74200, lr = 1e-17I0114 03:58:18.470480 21226 solver.cpp:218] Iteration 74300 (6.65372 iter/s, 15.0292s/100 iters), loss = 0.234399I0114 03:58:18.470551 21226 solver.cpp:237] Train net output #0: loss = 0.234403 (* 1 = 0.234403 loss)I0114 03:58:18.470561 21226 sgd_solver.cpp:105] Iteration 74300, lr = 1e-17I0114 03:58:33.493978 21226 solver.cpp:218] Iteration 74400 (6.65636 iter/s, 15.0232s/100 iters), loss = 0.00225647I0114 03:58:33.494228 21226 solver.cpp:237] Train net output #0: loss = 0.0022598 (* 1 = 0.0022598 loss)I0114 03:58:33.494269 21226 sgd_solver.cpp:105] Iteration 74400, lr = 1e-17I0114 03:58:48.538324 21226 solver.cpp:218] Iteration 74500 (6.6472 iter/s, 15.0439s/100 iters), loss = 2.68966e-05I0114 03:58:48.538383 21226 solver.cpp:237] Train net output #0: loss = 3.02486e-05 (* 1 = 3.02486e-05 loss)I0114 03:58:48.538391 21226 sgd_solver.cpp:105] Iteration 74500, lr = 1e-17I0114 03:59:03.605232 21226 solver.cpp:218] Iteration 74600 (6.63717 iter/s, 15.0667s/100 iters), loss = 0.00452782I0114 03:59:03.605480 21226 solver.cpp:237] Train net output #0: loss = 0.00453112 (* 1 = 0.00453112 loss)I0114 03:59:03.605512 21226 sgd_solver.cpp:105] Iteration 74600, lr = 1e-17I0114 03:59:18.693054 21226 solver.cpp:218] Iteration 74700 (6.62804 iter/s, 15.0874s/100 iters), loss = 0.173986I0114 03:59:18.693130 21226 solver.cpp:237] Train net output #0: loss = 0.173989 (* 1 = 0.173989 loss)I0114 03:59:18.693142 21226 sgd_solver.cpp:105] Iteration 74700, lr = 1e-17I0114 03:59:33.762369 21226 solver.cpp:218] Iteration 74800 (6.63611 iter/s, 15.0691s/100 iters), loss = 0.237659I0114 03:59:33.762547 21226 solver.cpp:237] Train net output #0: loss = 0.237662 (* 1 = 0.237662 loss)I0114 03:59:33.762562 21226 sgd_solver.cpp:105] Iteration 74800, lr = 1e-17I0114 03:59:48.851802 21226 solver.cpp:218] Iteration 74900 (6.62731 iter/s, 15.0891s/100 iters), loss = 0.0473921I0114 03:59:48.851860 21226 solver.cpp:237] Train net output #0: loss = 0.0473955 (* 1 = 0.0473955 loss)I0114 03:59:48.851868 21226 sgd_solver.cpp:105] Iteration 74900, lr = 1e-17I0114 04:00:03.242220 21230 data_layer.cpp:73] Restarting data prefetching from start.I0114 04:00:03.808995 21226 solver.cpp:447] Snapshotting to binary proto file models/resnet_gemfield_cls224_iter_75000.caffemodelI0114 04:00:04.100811 21226 sgd_solver.cpp:273] Snapshotting solver state to binary proto file models/resnet_gemfield_cls224_iter_75000.solverstateI0114 04:00:04.314110 21226 solver.cpp:310] Iteration 75000, loss = 0.225416I0114 04:00:04.314160 21226 solver.cpp:315] Optimization Done.I0114 04:00:04.314165 21226 caffe.cpp:259] Optimization Done.

清洗了一次訓練集，把一些模糊的照片去掉後，再次用同樣的參數去訓練，得到的訓練日誌如下：

I0116 01:29:12.525300 61 sgd_solver.cpp:105] Iteration 73800, lr = 1e-17I0116 01:29:45.552590 61 solver.cpp:218] Iteration 73900 (3.02776 iter/s, 33.0277s/100 iters), loss = 0.121681I0116 01:29:45.554165 61 solver.cpp:237] Train net output #0: loss = 0.121683 (* 1 = 0.121683 loss)I0116 01:29:45.554219 61 sgd_solver.cpp:105] Iteration 73900, lr = 1e-17I0116 01:30:18.147012 61 solver.cpp:330] Iteration 74000, Testing net (#0)I0116 01:33:34.854918 67 data_layer.cpp:73] Restarting data prefetching from start.I0116 01:44:08.726112 61 solver.cpp:397] Test net output #0: acc/top-1 = 0.976I0116 01:44:08.726408 61 solver.cpp:397] Test net output #1: loss = 0.0893586 (* 1 = 0.0893586 loss)I0116 01:44:09.004158 61 solver.cpp:218] Iteration 74000 (0.115814 iter/s, 863.456s/100 iters), loss = 0.272815I0116 01:44:09.004226 61 solver.cpp:237] Train net output #0: loss = 0.272817 (* 1 = 0.272817 loss)I0116 01:44:09.004240 61 sgd_solver.cpp:105] Iteration 74000, lr = 1e-17I0116 01:44:41.944175 61 solver.cpp:218] Iteration 74100 (3.03601 iter/s, 32.938s/100 iters), loss = 0.00181933I0116 01:44:41.944440 61 solver.cpp:237] Train net output #0: loss = 0.00182139 (* 1 = 0.00182139 loss)I0116 01:44:41.944473 61 sgd_solver.cpp:105] Iteration 74100, lr = 1e-17I0116 01:45:15.054388 61 solver.cpp:218] Iteration 74200 (3.0204 iter/s, 33.1082s/100 iters), loss = 0.0241835I0116 01:45:15.054733 61 solver.cpp:237] Train net output #0: loss = 0.0241856 (* 1 = 0.0241856 loss)I0116 01:45:15.054776 61 sgd_solver.cpp:105] Iteration 74200, lr = 1e-17I0116 01:45:48.085701 61 solver.cpp:218] Iteration 74300 (3.02762 iter/s, 33.0293s/100 iters), loss = 0.0254915I0116 01:45:48.085886 61 solver.cpp:237] Train net output #0: loss = 0.0254935 (* 1 = 0.0254935 loss)I0116 01:45:48.085896 61 sgd_solver.cpp:105] Iteration 74300, lr = 1e-17I0116 01:46:21.485394 61 solver.cpp:218] Iteration 74400 (2.99422 iter/s, 33.3977s/100 iters), loss = 0.000650348I0116 01:46:21.485787 61 solver.cpp:237] Train net output #0: loss = 0.000652366 (* 1 = 0.000652366 loss)I0116 01:46:21.485879 61 sgd_solver.cpp:105] Iteration 74400, lr = 1e-17I0116 01:46:54.568955 61 solver.cpp:218] Iteration 74500 (3.02283 iter/s, 33.0815s/100 iters), loss = 0.209352I0116 01:46:54.569267 61 solver.cpp:237] Train net output #0: loss = 0.209354 (* 1 = 0.209354 loss)I0116 01:46:54.569316 61 sgd_solver.cpp:105] Iteration 74500, lr = 1e-17I0116 01:47:27.618690 61 solver.cpp:218] Iteration 74600 (3.02593 iter/s, 33.0477s/100 iters), loss = 0.0229688I0116 01:47:27.618928 61 solver.cpp:237] Train net output #0: loss = 0.0229709 (* 1 = 0.0229709 loss)I0116 01:47:27.618969 61 sgd_solver.cpp:105] Iteration 74600, lr = 1e-17I0116 01:48:00.857776 61 solver.cpp:218] Iteration 74700 (3.0085 iter/s, 33.2391s/100 iters), loss = 0.00383524I0116 01:48:00.857947 61 solver.cpp:237] Train net output #0: loss = 0.00383731 (* 1 = 0.00383731 loss)I0116 01:48:00.857959 61 sgd_solver.cpp:105] Iteration 74700, lr = 1e-17I0116 01:48:34.184334 61 solver.cpp:218] Iteration 74800 (3.00065 iter/s, 33.3261s/100 iters), loss = 0.126383I0116 01:48:34.184636 61 solver.cpp:237] Train net output #0: loss = 0.126385 (* 1 = 0.126385 loss)I0116 01:48:34.184814 61 sgd_solver.cpp:105] Iteration 74800, lr = 1e-17I0116 01:49:07.174204 61 solver.cpp:218] Iteration 74900 (3.03142 iter/s, 32.9879s/100 iters), loss = 0.327584I0116 01:49:07.174506 61 solver.cpp:237] Train net output #0: loss = 0.327586 (* 1 = 0.327586 loss)I0116 01:49:07.174561 61 sgd_solver.cpp:105] Iteration 74900, lr = 1e-17I0116 01:49:38.800302 66 data_layer.cpp:73] Restarting data prefetching from start.I0116 01:49:40.076367 61 solver.cpp:447] Snapshotting to binary proto file models/resnet_gemfield_cls224_iter_75000.caffemodelI0116 01:49:40.382776 61 sgd_solver.cpp:273] Snapshotting solver state to binary proto file models/resnet_gemfield_cls224_iter_75000.solverstateI0116 01:49:40.630769 61 solver.cpp:310] Iteration 75000, loss = 0.0174639I0116 01:49:40.630820 61 solver.cpp:315] Optimization Done.I0116 01:49:40.630826 61 caffe.cpp:259] Optimization Done.

嗯，效果還是好了些。