淺入淺出TensorFlow 4 - 訓練CIFAR數據

來自專欄 淺入淺出TensorFlow

一. CIFAR數據集

CIFAR數據集是一個經典的數據集，提供兩個版本的分類樣本，CIFAR-10和CIFAR-100。

CIFAR-10 提供10類標註數據，每類6000張（32*32），其中5000張用於訓練，1000張用於測試。

獲取數據集的方法：

git clone https://github.com/tensorflow/models.git cd models/tutorials/image/cifar10

可以看一下我們從github上down下來的數據，外面不看了，直接進 tutorials/image，教程專用，看來是基礎的不能再基礎了。

裡面提供了幾個典型的數據集的 下載、訓練等介面，方便直接在python里調用。

進入cifar10，能夠看到：

其中文件 cifar10.py 和 cifar10_input.py 就是接下來我們要 import 的。

二. 代碼實現

擼一段 Python 代碼，可以View裡面的注釋講解：

#coding=utf-8 import cifar10,cifar10_input import tensorflow as tf import numpy as np import time # define max_iter_step batch_size max_iter_step = 1000 batch_size = 128 # define variable_with_weight_loss # 和之前定義的weight有所不同， # 這裡定義附帶loss的weight，通過權重懲罰避免部分權重係數過大，導致overfitting def variable_with_weight_loss(shape,stddev,w1): var = tf.Variable(tf.truncated_normal(shape,stddev=stddev)) if w1 is not None: weight_loss = tf.multiply(tf.nn.l2_loss(var),w1,name=weight_loss) tf.add_to_collection(losses,weight_loss) return var # 下載數據集 － 調用cifar10函數下載並解壓 cifar10.maybe_download_and_extract() cifar_dir = /tmp/cifar10_data/cifar-10-batches-bin # 採用 data augmentation進行數據處理 # 生成訓練數據，訓練數據通過cifar10_input的distort變化 images_train, labels_train = cifar10_input.distorted_inputs(data_dir=cifar_dir,batch_size=batch_size) # 測試數據（eval_data 測試數據） images_test,labels_test = cifar10_input.inputs(eval_data=True,data_dir=cifar_dir,batch_size=batch_size) # 創建輸入數據，採用 placeholder x_input = tf.placeholder(tf.float32,[batch_size,24,24,3]) y_input = tf.placeholder(tf.int32,[batch_size]) # 創建第一個卷積層 input:3(channel) kernel:64 size:5*5 weight1 = variable_with_weight_loss(shape=[5,5,3,64],stddev=5e-2,w1=0.0) bias1 = tf.Variable(tf.constant(0.0,shape=[64])) conv1 = tf.nn.conv2d(x_input,weight1,[1,1,1,1],padding=SAME) relu1 = tf.nn.relu(tf.nn.bias_add(conv1,bias1)) pool1 = tf.nn.max_pool(conv1,ksize=[1,3,3,1],strides=[1,2,2,1],padding=SAME) norm1 = tf.nn.lrn(pool1,4,bias=1.0,alpha=0.001/9.0,beta=0.75) # 創建第二個卷積層 input:64 kernel:64 size:5*5 weight2 = variable_with_weight_loss(shape=[5,5,64,64],stddev=5e-2,w1=0.0) bias2 = tf.Variable(tf.constant(0,1,shape=[64])) conv2 = tf.nn.conv2d(norm1,weight2,[1,1,1,1],padding=SAME) relu2 = tf.nn.relu(tf.nn.bias_add(conv2,bias2)) norm2 = tf.nn.lrn(relu2,4,bias=1.0,alpha=0.001/9.0,beta=0.75) pool2 = tf.nn.max_pool(norm2,ksize=[1,3,3,1],strides=[1,2,2,1],padding=SAME) # 創建第三個層－全連接層 output:384 reshape = tf.reshape(pool2,[batch_size,-1]) dim = reshape.get_shape()[1].value weight3 = variable_with_weight_loss(shape=[dim,384],stddev=0.04,w1=0.004) bias3 = tf.Variable(tf.constant(0.1,shape=[384])) local3 = tf.nn.relu(tf.matmul(reshape,weight3)+bias3) # 創建第四個層－全連接層 output:192 weight4 = variable_with_weight_loss(shape=[384,192],stddev=0.04,w1=0.004) bias4 = tf.Variable(tf.constant(0.1,shape=[192])) # 最後一層 output:10 weight5 = variable_with_weight_loss(shape=[192,10],stddev=1/192.0,w1=0.0) bias5 = tf.Variable(tf.constant(0.0,shape=[10])) results = tf.add(tf.matmul(local4,weight5),bias5) # 定義loss def loss(results,labels): labels = tf.cast(labels,tf.int64) cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=results,labels=labels,name=cross_entropy_per_example) cross_entropy_mean = tf.reduce_mean(cross_entropy,name=cross_entropy) tf.add_to_collection(losses,cross_entropy_mean) return tf.add_n(tf.get_collection(losses),name=total_loss) # 計算loss loss = loss(results,y_input) train_op = tf.train.AdamOptimizer(1e-3).minimize(loss) # Adam top_k_op = tf.nn.in_top_k(results,y_input,1) # top1 準確率 sess = tf.InteractiveSession() # 創建session tf.global_variable_initializer().run() # 初始化全部模型 tf.train.start_queue_runners() # 啟動多線程加速 # 開始訓練 for step in range(max_steps): start_time = time.time() image_batch,label_batch = sess.run([images_train,labels_train]) _,loss_value = sess.run([train_op,loss], feed_dict={x_input:image_batch, y_input:label_batch}) duration = time.time() - start_time if step % 10 == 0: examples_per_sec = batch_size/duration sec_per_batch = float(duration) format_str = (step %d,loss=%.2f (%.1f examples/sec; %.3f sec/batch) print(format_str % (step,loss_value,examples_per_sec,sec_per_batch)) # 評測模型在測試集上的準確度 num_examples = 10000 import math num_iter = int(math.ceil(num_examples/batch_size)) true_count = 0 total_sample_count = num_iter * batch_size step = 0 while step < num_iter: image_batch,label_batch = sess.run([images_test,labels_test]) predictions = sess.run([top_k_op],feed_dict={x_input:image_batch,y_input:label_batch}) true_count += np.sum(predictions) step += 1 # 列印結果 precision = true_count / total_sample_count print(precision @ 1 = %.3f % precision)

注意，這裡與前面不一樣的地方在於引入了權值懲罰，另外，top_k的用法也是第一次，將代碼另存為 .py文件，copy到models/tutorials/image/cifar10目錄下調用，觀察下載數據及訓練過程，然後再Review代碼，相信會有新的收穫！

推薦閱讀：