學習筆記TF054:TFLearn、Keras

元框架(metaframework)。

TFLearn。模塊化深度學習框架，更高級API，快速實驗，完全透明兼容。

TFLearn實現AlexNet。

https://github.com/tflearn/tflearn/blob/master/examples/images/alexnet.py

牛津大學鮮花數據集(Flower Dataset)。Visual Geometry Group Home Page 。提供17個類別鮮花數據，每個類別80張圖片，有大量姿態、光變化。

# -*- coding: utf-8 -*-

""" AlexNet.

Applying Alexnet to Oxfords 17 Category Flower Dataset classification task.

References:

- Alex Krizhevsky, Ilya Sutskever & Geoffrey E. Hinton. ImageNet

Classification with Deep Convolutional Neural Networks. NIPS, 2012.

- 17 Category Flower Dataset. Maria-Elena Nilsback and Andrew Zisserman.

Links:

- [AlexNet Paper](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf)

- [Flower Dataset (17)](Visual Geometry Group Home Page)

"""

from __future__ import division, print_function, absolute_import

import tflearn

from tflearn.layers.core import input_data, dropout, fully_connected

from tflearn.layers.conv import conv_2d, max_pool_2d

from tflearn.layers.normalization import local_response_normalization

from tflearn.layers.estimator import regression

import tflearn.datasets.oxflower17 as oxflower17

# 載入數據

X, Y = oxflower17.load_data(one_hot=True, resize_pics=(227, 227))

# Building AlexNet 構建網路模型

network = input_data(shape=[None, 227, 227, 3])

network = conv_2d(network, 96, 11, strides=4, activation=relu)

network = max_pool_2d(network, 3, strides=2)

network = local_response_normalization(network)

network = conv_2d(network, 256, 5, activation=relu)

network = max_pool_2d(network, 3, strides=2)

network = local_response_normalization(network)

network = conv_2d(network, 384, 3, activation=relu)

network = conv_2d(network, 384, 3, activation=relu)

network = conv_2d(network, 256, 3, activation=relu)

network = max_pool_2d(network, 3, strides=2)

network = local_response_normalization(network)

network = fully_connected(network, 4096, activation=tanh)

network = dropout(network, 0.5)

network = fully_connected(network, 4096, activation=tanh)

network = dropout(network, 0.5)

network = fully_connected(network, 17, activation=softmax)

network = regression(network, optimizer=momentum,

loss=categorical_crossentropy,

learning_rate=0.001)

# Training 訓練模型 載入AlexNet模型檢查點文件

model = tflearn.DNN(network, checkpoint_path=model_alexnet,

max_checkpoints=1, tensorboard_verbose=2)

model.fit(X, Y, n_epoch=1000, validation_set=0.1, shuffle=True,

show_metric=True, batch_size=64, snapshot_step=200,

snapshot_epoch=False, run_id=alexnet_oxflowers17)

Keras。高級Python神經網路框架。https://keras.io/。TensorFlow默認框架。快速搭建原型。兼容Theano和TensorFlow。Keras高度封裝，適合新手，代碼更新快，示例代碼多，文檔、討論區完善。自動調用GPU並行計算。模塊化，模型神經層、成本函數、優化器、初始化、激活函數、規範化模塊，組合創建模型。極簡。易擴展，容易添加新模塊。Python語言。

Keras模型。Keras核心數據結構是模型。模型組織網路層。Sequential模型，網路層順序構成棧，單輸入單輸出，層間只有相鄰關係，簡單模型。Model模型建立複雜模型。

Sequential模型。載入完數據，X_train,Y_train,X_test,Y_test。構建模型：

from keras.models import Sequential

from keras.layers.core import Dense, Dropout,Activation

model = Sequential()

model.add(Dense(output_dim=64, input_dim=100))

model.add(Activation(relu))

model.add(Dense(output_dim=10))

model.add(Activation(softmax))

model.compile(loss=categorical_crossentropy, optimizer=sgd, metrics=accuracy)

model.fit(X_train, Y_train, batch_size=32, nb_epoch=5)

loss_and_metrics = model.evaluate(X_test, Y_test, batch_size=32)

Keras源碼示例 fchollet/keras ：CIFAR10-圖片分類(CNN、實時數據)、IMDB-電影評論觀點分類(LSTM)、Reuters-新聞主題分類(多層感知器)、MNIST-手寫數字識別(多層感知器、CNN)、OCR-識別字元級文本生成(LSTM)。

安裝。pip install keras 。選擇依賴後端，~/.keras/keras.json 修改最後一行"backend":"fensorflow" 。

Keras實現卷積神經網路。

https://github.com/fchollet/blob/master/examples/mnist_cnn.py 。

#!/usr/bin/python

# -*- coding:utf8 -*-

Trains a simple convnet on the MNIST dataset.

Gets to 99.25% test accuracy after 12 epochs

(there is still a lot of margin for parameter tuning).

16 seconds per epoch on a GRID K520 GPU.

from __future__ import print_function

import keras

from keras.datasets import mnist

from keras.models import Sequential

from keras.layers import Dense, Dropout, Flatten

from keras.layers import Conv2D, MaxPooling2D

from keras import backend as K

batch_size = 128

num_classes = 10 # 分類數

epochs = 12 # 訓練輪數

# input image dimensions 輸入圖片維度

img_rows, img_cols = 28, 28

# the data, shuffled and split between train and test sets

# 載入數據

(x_train, y_train), (x_test, y_test) = mnist.load_data()

if K.image_data_format() == channels_first:

#使用Theano順序:(conv_dim1,channels,conv_dim2,conv_dim3)

x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)

x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)

input_shape = (1, img_rows, img_cols)

else:

#使用TensorFlow順序:(conv_dim1conv_dim2,conv_dim3,channels,)

x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)

x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)

input_shape = (img_rows, img_cols, 1)

x_train = x_train.astype(float32)

x_test = x_test.astype(float32)

x_train /= 255

x_test /= 255

print(x_train shape:, x_train.shape)

print(x_train.shape[0], train samples)

print(x_test.shape[0], test samples)

# convert class vectors to binary class matrices 將類向量轉換為二進位類矩陣

y_train = keras.utils.to_categorical(y_train, num_classes)

y_test = keras.utils.to_categorical(y_test, num_classes)

#模型構建:2個卷積層、1個池化層、2個全連接層

model = Sequential()

model.add(Conv2D(32, kernel_size=(3, 3),

activation=relu,

input_shape=input_shape))

model.add(Conv2D(64, (3, 3), activation=relu))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Dropout(0.25))

model.add(Flatten())

model.add(Dense(128, activation=relu))

model.add(Dropout(0.5))

model.add(Dense(num_classes, activation=softmax))

# 模型編譯

model.compile(loss=keras.losses.categorical_crossentropy,

optimizer=keras.optimizers.Adadelta(),

metrics=[accuracy])

# 模型訓練

model.fit(x_train, y_train,

batch_size=batch_size,

epochs=epochs,

verbose=1,

validation_data=(x_test, y_test))

# 模型評估

score = model.evaluate(x_test, y_test, verbose=0)

print(Test loss:, score[0])

print(Test accuracy:, score[1])

模型載入保存。 https://github.com/fchollet/keras/blob/master/tests/test_model_saving.py 。

Keras的save_model、load_model方法將模型、權重保存到HDF5文件。包括模型結構、權重、訓練配置(損失函數、優化器)。方便中斷後再繼續訓練。

from keras.models import save_model, load_model

def test_sequential_model_saving():

model = Sequential()

model.add(Dense(2, input_shape=(3,)))

model.add(RepeatVector(3))

model.add(TimeDistributed(Dense(3)))

model.compile(loss=losses.MSE,

optimizer=optimizers.RMSprop(lr=0.0001),

metrics=[metrics.categorical_accuracy],

sample_weight_mode=temporal)

x = np.random.random((1, 3))

y = np.random.random((1, 3, 3))

model.train_on_batch(x, y)

out = model.predict(x)

_, fname = tempfile.mkstemp(.h5) # 創建HDFS 5文件

save_model(model, fname)

new_model = load_model(fname)

os.remove(fname)

out2 = new_model.predict(x)

assert_allclose(out, out2, atol=1e-05)

# test that new updates are the same with both models

# 檢測新保存的模型和之前定義的模型是否一致

x = np.random.random((1, 3))

y = np.random.random((1, 3, 3))

model.train_on_batch(x, y)

new_model.train_on_batch(x, y)

out = model.predict(x)

out2 = new_model.predict(x)

assert_allclose(out, out2, atol=1e-05)

只保存模型結構。

json_string = model.to_jsion()

yaml_string = model.to_yaml()

手動編輯。

from keras.models import model_from_json

model = model_from_json(json_string)

model = model_from_yaml(yaml_string)

只保存模型權重。

model.save_weights(my_model_weights.h5)

model.load_weights(my_model_weights.h5)

參考資料：

《TensorFlow技術解析與實戰》

歡迎推薦上海機器學習工作機會，我的微信：qingxingfengzi