深度學習筆記12：卷積神經網路的Tensorflow實現

深度學習筆記12：卷積神經網路的Tensorflow實現

來自專欄程序猿數據愛好者4 人贊了文章

文章出處：深度學習筆記11：利用numpy搭建一個卷積神經網路

免費視頻課程：Hellobi Live | 從數據分析師到機器學習(深度學習)工程師的進階之路

在上一講中，我們學習了如何利用 numpy 手動搭建卷積神經網路。但在實際的圖像識別中，使用 numpy 去手寫 CNN 未免有些吃力不討好。在 DNN 的學習中，我們也是在手動搭建之後利用 Tensorflow 去重新實現一遍，一來為了能夠對神經網路的傳播機制能夠理解更加透徹，二來也是為了更加高效使用開源框架快速搭建起深度學習項目。本節就繼續和大家一起學習如何利用 Tensorflow 搭建一個卷積神經網路。

我們繼續以 NG 課題組提供的 sign 手勢數據集為例，學習如何通過 Tensorflow 快速搭建起一個深度學習項目。數據集標籤共有零到五總共 6 類標籤，示例如下：

先對數據進行簡單的預處理並查看訓練集和測試集維度：

X_train = X_train_orig/255.

X_test = X_test_orig/255.

Y_train = convert_to_one_hot(Y_train_orig, 6).T

Y_test = convert_to_one_hot(Y_test_orig, 6).T

print ("number of training examples = " + str(X_train.shape[0]))

print ("number of test examples = " + str(X_test.shape[0]))

print ("X_train shape: " + str(X_train.shape))

print ("Y_train shape: " + str(Y_train.shape))

print ("X_test shape: " + str(X_test.shape))

print ("Y_test shape: " + str(Y_test.shape))

可見我們總共有 1080 張 64643 訓練集圖像，120 張 64643 的測試集圖像，共有 6 類標籤。下面我們開始搭建過程。

創建 placeholder

首先需要為訓練集預測變數和目標變數創建佔位符變數 placeholder ，定義創建佔位符變數函數：

def create_placeholders(n_H0, n_W0, n_C0, n_y):

"""

Creates the placeholders for the tensorflow session.

Arguments:

n_H0 -- scalar, height of an input image

n_W0 -- scalar, width of an input image

n_C0 -- scalar, number of channels of the input

n_y -- scalar, number of classes

Returns:

X -- placeholder for the data input, of shape [None, n_H0, n_W0, n_C0] and dtype "float"

Y -- placeholder for the input labels, of shape [None, n_y] and dtype "float"

"""

X = tf.placeholder(tf.float32, shape=(None, n_H0, n_W0, n_C0), name=X)

Y = tf.placeholder(tf.float32, shape=(None, n_y), name=Y)

return X, Y

參數初始化

然後需要對濾波器權值參數進行初始化：

def initialize_parameters():

"""

Initializes weight parameters to build a neural network with tensorflow.

Returns:

parameters -- a dictionary of tensors containing W1, W2

"""

tf.set_random_seed(1)

W1 = tf.get_variable("W1", [4,4,3,8], initializer = tf.contrib.layers.xavier_initializer(seed = 0))

W2 = tf.get_variable("W2", [2,2,8,16], initializer = tf.contrib.layers.xavier_initializer(seed = 0))

parameters = {"W1": W1,

"W2": W2}

return parameters

執行卷積網路的前向傳播過程

前向傳播過程如下所示：

CONV2D -> RELU -> MAXPOOL -> CONV2D -> RELU -> MAXPOOL -> FLATTEN -> FULLYCONNECTED

可見我們要搭建的是一個典型的 CNN 過程，經過兩次的卷積-relu激活-最大池化，然後展開接上一個全連接層。利用 Tensorflow 搭建上述傳播過程如下：

def forward_propagation(X, parameters):

"""

Implements the forward propagation for the model

Arguments:

X -- input dataset placeholder, of shape (input size, number of examples)

parameters -- python dictionary containing your parameters "W1", "W2"

the shapes are given in initialize_parameters

Returns:

Z3 -- the output of the last LINEAR unit

"""

# Retrieve the parameters from the dictionary "parameters"

W1 = parameters[W1]

W2 = parameters[W2]

# CONV2D: stride of 1, padding SAME

Z1 = tf.nn.conv2d(X,W1, strides = [1,1,1,1], padding = SAME)

# RELU

A1 = tf.nn.relu(Z1)

# MAXPOOL: window 8x8, sride 8, padding SAME

P1 = tf.nn.max_pool(A1, ksize = [1,8,8,1], strides = [1,8,8,1], padding = SAME)

# CONV2D: filters W2, stride 1, padding SAME

Z2 = tf.nn.conv2d(P1,W2, strides = [1,1,1,1], padding = SAME)

# RELU

A2 = tf.nn.relu(Z2)

# MAXPOOL: window 4x4, stride 4, padding SAME

P2 = tf.nn.max_pool(A2, ksize = [1,4,4,1], strides = [1,4,4,1], padding = SAME)

# FLATTEN

P2 = tf.contrib.layers.flatten(P2)

Z3 = tf.contrib.layers.fully_connected(P2, 6, activation_fn = None)

return Z3

計算當前損失

在 Tensorflow 中計算損失函數非常簡單，一行代碼即可：

def compute_cost(Z3, Y):

"""

Computes the cost

Arguments:

Z3 -- output of forward propagation (output of the last LINEAR unit), of shape (6, number of examples)

Y -- "true" labels vector placeholder, same shape as Z3

Returns:

cost - Tensor of the cost function

"""

cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=Z3, labels=Y))

return cost

定義好上述過程之後，就可以封裝整體的訓練過程模型。可能你會問為什麼沒有反向傳播，這裡需要注意的是 Tensorflow 幫助我們自動封裝好了反向傳播過程，無需我們再次定義，在實際搭建過程中我們只需將前向傳播的網路結構定義清楚即可。

封裝模型

def model(X_train, Y_train, X_test, Y_test, learning_rate = 0.009,

num_epochs = 100, minibatch_size = 64, print_cost = True):

"""

Implements a three-layer ConvNet in Tensorflow:

CONV2D -> RELU -> MAXPOOL -> CONV2D -> RELU -> MAXPOOL -> FLATTEN -> FULLYCONNECTED

Arguments:

X_train -- training set, of shape (None, 64, 64, 3)

Y_train -- test set, of shape (None, n_y = 6)

X_test -- training set, of shape (None, 64, 64, 3)

Y_test -- test set, of shape (None, n_y = 6)

learning_rate -- learning rate of the optimization

num_epochs -- number of epochs of the optimization loop

minibatch_size -- size of a minibatch

print_cost -- True to print the cost every 100 epochs

Returns:

train_accuracy -- real number, accuracy on the train set (X_train)

test_accuracy -- real number, testing accuracy on the test set (X_test)

parameters -- parameters learnt by the model. They can then be used to predict.

"""

ops.reset_default_graph()

tf.set_random_seed(1)

seed = 3

(m, n_H0, n_W0, n_C0) = X_train.shape

n_y = Y_train.shape[1]

costs = []

# Create Placeholders of the correct shape

X, Y = create_placeholders(n_H0, n_W0, n_C0, n_y)

# Initialize parameters

parameters = initialize_parameters()

# Forward propagation

Z3 = forward_propagation(X, parameters)

# Cost function

cost = compute_cost(Z3, Y)

# Backpropagation

optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(cost) # Initialize all the variables globally

init = tf.global_variables_initializer()

# Start the session to compute the tensorflow graph

with tf.Session() as sess:

# Run the initialization

sess.run(init)

# Do the training loop

for epoch in range(num_epochs):

minibatch_cost = 0.

num_minibatches = int(m / minibatch_size)

seed = seed + 1

minibatches = random_mini_batches(X_train, Y_train, minibatch_size, seed)

for minibatch in minibatches:

# Select a minibatch

(minibatch_X, minibatch_Y) = minibatch

_ , temp_cost = sess.run([optimizer, cost], feed_dict={X: minibatch_X, Y: minibatch_Y})

minibatch_cost += temp_cost / num_minibatches

# Print the cost every epoch

if print_cost == True and epoch % 5 == 0:

print ("Cost after epoch %i: %f" % (epoch, minibatch_cost))

if print_cost == True and epoch % 1 == 0:

costs.append(minibatch_cost)

# plot the cost

plt.plot(np.squeeze(costs))

plt.ylabel(cost)

plt.xlabel(iterations (per tens))

plt.title("Learning rate =" + str(learning_rate))

plt.show() # Calculate the correct predictions

predict_op = tf.argmax(Z3, 1)

correct_prediction = tf.equal(predict_op, tf.argmax(Y, 1))

# Calculate accuracy on the test set

accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

print(accuracy)

train_accuracy = accuracy.eval({X: X_train, Y: Y_train})

test_accuracy = accuracy.eval({X: X_test, Y: Y_test})

print("Train Accuracy:", train_accuracy)

print("Test Accuracy:", test_accuracy)

return train_accuracy, test_accuracy, parameters

對訓練集執行模型訓練：

_, _, parameters = model(X_train, Y_train, X_test, Y_test)

訓練迭代過程如下：

我們在訓練集上取得了 0.67 的準確率，在測試集上的預測準確率為 0.58 ，雖然效果並不顯著，模型也有待深度調優，但我們已經學會了如何用 Tensorflow 快速搭建起一個深度學習系統了。

註：本深度學習筆記系作者學習 Andrew NG 的 deeplearningai 五門課程所記筆記，其中代碼為每門課的課後assignments作業整理而成。

參考資料：

https://www.coursera.org/learn/machine-learning

https://www.deeplearning.ai/

往期精彩：

深度學習筆記10：三維卷積、池化與全連接

深度學習筆記9：卷積神經網路（CNN）入門

深度學習筆記8：利用Tensorflow搭建神經網路

深度學習筆記7：Tensorflow入門

深度學習筆記6：神經網路優化演算法之從SGD到Adam

數據分析入行半年之經驗、感悟與思考

談談過擬合

一個統計方向畢業生的2017年數據科學從業之路總結

推薦閱讀：