學習筆記TF022:產品環境模型部署、Docker鏡像、Bazel工作區、導出模型、伺服器、客戶端

產品環境模型部署，創建簡單Web APP，用戶上傳圖像，運行Inception模型，實現圖像自動分類。

搭建TensorFlow服務開發環境。安裝Docker，https://docs.docker.com/engine/installation/ 。用配置文件在本地創建Docker鏡像，docker build --pull -t $USER/tensorflow-serving-devel https://raw.githubusercontent.com/tensorflow/serving/master/tensorflow_serving/tools/docker/Dockerfile.devel 。鏡像運行容器，docker run -v $HOME:/mnt/home -p 9999:9999 -it $USER/tensorflow-serving-devel ，在home目錄載入到容器/mnt/home路徑，在終端工作。用IDE或編輯器編輯代碼，用容器運行構建工具，主機通過9999埠訪問，構建伺服器。exit命令退出容器終端，停止運行。

TensorFlow服務程序C++寫，使用Google的Bazel構建工具。容器運行Bazel。Bazel代碼級管理第三方依賴項。Bazel自動下載構建。項目庫根目錄定義WORKSPACE文件。TensorFlow模型庫包含Inception模型代碼。

TensorFlow服務在項目作為Git子模塊。mkdir ~/serving_example，cd ~/serving_example，git init，git submodule add tensorflow/serving ，tf_serving，git submodule update --init --recursive 。

WORKSPACE文件local_repository規則定義第三方依賴為本地存儲文件。項目導入tf_workspace規則初始化TensorFlow依賴項。

workspace(name = "serving")

local_repository(

name = "tf_serving",

path = __workspace_dir__ + "/tf_serving",

)

local_repository(

name = "org_tensorflow",

path = __workspace_dir__ + "/tf_serving/tensorflow",

)

load(//tf_serving/tensorflow/tensorflow:workspace.bzl, tf_workspace)

tf_workspace("tf_serving/tensorflow/", "@org_tensorflow")

bind(

name = "libssl",

actual = "@boringssl_git//:ssl",

)

bind(

name = "zlib",

actual = "@zlib_archive//:zlib",

)

local_repository(

name = "inception_model",

path = __workspace_dir__ + "/tf_serving/tf_models/inception",

)

導出訓練好的模型，導出數據流圖及變數，給產品用。模型數據流圖，必須從佔位符接收輸入，單步推斷計算輸出。Inception模型(或一般圖像識別模型)，JPEG編碼圖像字元串輸入，與從TFRecord文件讀取輸入不同。定義輸入佔位符，調用函數轉換佔位符表示外部輸入為原始推斷模型輸入格式，圖像字元串轉換為各分量位於[0, 1]內像素張量，縮放圖像尺寸，符合模型期望寬度高度，像素值變換到模型要求區間[-1, 1]內。調用原始模型推斷方法，依據轉換輸入推斷結果。

推斷方法各參數賦值。從檢查點恢復參數值。周期性保存模型訓練檢查點文件，文件包含學習參數。最後一次保存訓練檢查點文件包含最後更新模型參數。下去載預訓練檢查點文件：http://download.tensorflow.org/models/imagenet/inception-v3-2016-03-01.tar.gz 。在Docker容器中，cd /tmp, curl -0 http://download.tensorflow.org/models/image/imagenet/inception-v3-2016-03-01.tar.gz, tar -xzf inception-v3-2016-03-01.tar.gz 。

tensorflow_serving.session_bundle.exporter.Exporter類導出模型。傳入保存器實例創建實例，用exporter.classification_signature創建模型簽名。指定input_tensor、輸出張量。classes_tensor 包含輸出類名稱列表、模型分配各類別分值(或概率)socres_tensor。類別數多模型，配置指定僅返田大口tf.nntop_k選擇類別，模型分配分數降序排列前K個類別。調用exporter.Exporter.init方法簽名，export方法導出模型，接收輸出路徑、模型版本號、會話對象。Exporter類自動生成代碼存在依賴，Doker容器內部使用中bazel運行導出器。代碼保存到bazel工作區exporter.py。

import time

import sys

import tensorflow as tf

from tensorflow_serving.session_bundle import exporter

from inception import inception_model

NUM_CLASSES_TO_RETURN = 10

def convert_external_inputs(external_x):

image = tf.image.convert_image_dtype(tf.image.decode_jpeg(external_x, channels=3), tf.float32)

images = tf.image.resize_bilinear(tf.expand_dims(image, 0), [299, 299])

images = tf.mul(tf.sub(images, 0.5), 2)

return images

def inference(images):

logits, _ = inception_model.inference(images, 1001)

return logits

external_x = tf.placeholder(tf.string)

x = convert_external_inputs(external_x)

y = inference(x)

saver = tf.train.Saver()

with tf.Session() as sess:

ckpt = tf.train.get_checkpoint_state(sys.argv[1])

if ckpt and ckpt.model_checkpoint_path:

saver.restore(sess, sys.argv[1] + "/" + ckpt.model_checkpoint_path)

else:

print("Checkpoint file not found")

raise SystemExit

scores, class_ids = tf.nn.top_k(y, NUM_CLASSES_TO_RETURN)

classes = tf.contrib.lookup.index_to_string(tf.to_int64(class_ids),

mapping=tf.constant([str(i) for i in range(1001)]))

model_exporter = exporter.Exporter(saver)

signature = exporter.classification_signature(

input_tensor=external_x, classes_tensor=classes, scores_tensor=scores)

model_exporter.init(default_graph_signature=signature, init_op=tf.initialize_all_tables())

model_exporter.export(sys.argv[1] + "/export", tf.constant(time.time()), sess)

一個構建規則BUILD文件。在容器命令運行導出器，cd /mnt/home/serving_example, hazel run:export /tmp/inception-v3 ，依據/tmp/inception-v3提到的檢查點文件在/tmp/inception-v3/{currenttimestamp}/創建導出器。首次運行要對TensorFlow編譯。load從外部導入protobuf庫，導入cc_proto_library規則定義，為proto文件定義構建規則。通過命令bazel run :server 9999 /tmp/inception-v3/export/{timestamp}，容器運行推斷伺服器。

py_binary(

name = "export",

srcs = [

"export.py",

],

deps = [

"@tf_serving//tensorflow_serving/session_bundle:exporter",

"@org_tensorflow//tensorflow:tensorflow_py",

"@inception_model//inception",

],

)

load("@protobuf//:protobuf.bzl", "cc_proto_library")

cc_proto_library(

name="classification_service_proto",

srcs=["classification_service.proto"],

cc_libs = ["@protobuf//:protobuf"],

protoc="@protobuf//:protoc",

default_runtime="@protobuf//:protobuf",

use_grpc_plugin=1

)

cc_binary(

name = "server",

srcs = [

"http://server.cc",

],

deps = [

":classification_service_proto",

"@tf_serving//tensorflow_serving/servables/tensorflow:session_bundle_factory",

"@grpc//:grpc++",

],

)

定義伺服器介面。TensorFlow服務使用gRPC協議(基於HTTP/2二進位協議)。支持創建伺服器和自動生成客戶端存根各種語言。在protocol buffer定義服務契約，用於gRPC IDL(介面定義語言)和二進位編碼。接收JPEG編碼待分類圖像字元串輸入，返回分數排列推斷類別列表。定義在classification_service.proto文件。接收圖像、音頻片段、文字服務可用可一介面。proto編譯器轉換proto文件為客戶端和伺服器類定義。bazel build:classification_service_proto可行構建，通過bazel-genfiles/classification_service.grpc.pb.h檢查結果。推斷邏輯，ClassificationService::Service介面必須實現。檢查bazel-genfiles/classification_service.pb.h查看request、response消息定義。proto定義變成每種類型C++介面。

syntax = "proto3";

message ClassificationRequest {

// bytes input = 1;

float petalWidth = 1;

float petalHeight = 2;

float sepalWidth = 3;

float sepalHeight = 4;

};

message ClassificationResponse {

repeated ClassificationClass classes = 1;

};

message ClassificationClass {

string name = 1;

float score = 2;

}

service ClassificationService {

rpc classify(ClassificationRequest) returns (ClassificationResponse);

}

實現推斷伺服器。載入導出模型，調用推斷方法，實現ClassificationService::Service。導出模型，創建SessionBundle對象，包含完全載入數據流圖TF會話對象，定義導出工具分類簽名元數據。SessionBundleFactory類創建SessionBundle對象，配置為pathToExportFiles指定路徑載入導出模型，返回創建SessionBundle實例unique指針。定義ClassificationServiceImpl，接收SessionBundle實例參數。

載入分類簽名，GetClassificationSignature函數載入模型導出元數據ClassificationSignature，簽名指定所接收圖像真實名稱的輸入張量邏輯名稱，以及數據流圖輸出張量邏輯名稱映射推斷結果。將protobuf輸入變換為推斷輸入張量，request參數複製JPEG編碼圖像字元串到推斷張量。運行推斷，sessionbundle獲得TF會話對象，運行一次，傳入輸入輸出張量推斷。推斷輸出張量變換protobuf輸出，輸出張量結果複製到ClassificationResponse消息指定形狀response輸出參數格式化。設置gRPC伺服器，SessionBundle對象配置，創建ClassificationServiceImpl實例樣板代碼。

#include <iostream>

#include <memory>

#include <string>

#include <grpc++/grpc++.h>

#include "classification_service.grpc.pb.h"

#include "tensorflow_serving/servables/tensorflow/session_bundle_factory.h"

using namespace std;

using namespace tensorflow::serving;

using namespace grpc;

unique_ptr<SessionBundle> createSessionBundle(const string& pathToExportFiles) {

SessionBundleConfig session_bundle_config = SessionBundleConfig();

unique_ptr<SessionBundleFactory> bundle_factory;

SessionBundleFactory::Create(session_bundle_config, &bundle_factory);

unique_ptr<SessionBundle> sessionBundle;

bundle_factory->CreateSessionBundle(pathToExportFiles, &sessionBundle);

return sessionBundle;

}

class ClassificationServiceImpl final : public ClassificationService::Service {

private:

unique_ptr<SessionBundle> sessionBundle;

public:

ClassificationServiceImpl(unique_ptr<SessionBundle> sessionBundle) :

sessionBundle(move(sessionBundle)) {};

Status classify(ServerContext* context, const ClassificationRequest* request,

ClassificationResponse* response) override {

ClassificationSignature signature;

const tensorflow::Status signatureStatus =

GetClassificationSignature(sessionBundle->meta_graph_def, &signature);

if (!signatureStatus.ok()) {

return Status(StatusCode::INTERNAL, signatureStatus.error_message());

}

tensorflow::Tensor input(tensorflow::DT_STRING, tensorflow::TensorShape());

input.scalar<string>()() = request->input();

vector<tensorflow::Tensor> outputs;

const tensorflow::Status inferenceStatus = sessionBundle->session->Run(

{{signature.input().tensor_name(), input}},

{signature.classes().tensor_name(), signature.scores().tensor_name()},

{},

&outputs);

if (!inferenceStatus.ok()) {

return Status(StatusCode::INTERNAL, inferenceStatus.error_message());

}

for (int i = 0; i < outputs[0].NumElements(); ++i) {

ClassificationClass *classificationClass = response->add_classes();

classificationClass->set_name(outputs[0].flat<string>()(i));

classificationClass->set_score(outputs[1].flat<float>()(i));

}

return Status::OK;

}

};

int main(int argc, char** argv) {

if (argc < 3) {

cerr << "Usage: server <port> /path/to/export/files" << endl;

return 1;

}

const string serverAddress(string("0.0.0.0:") + argv[1]);

const string pathToExportFiles(argv[2]);

unique_ptr<SessionBundle> sessionBundle = createSessionBundle(pathToExportFiles);

ClassificationServiceImpl classificationServiceImpl(move(sessionBundle));

ServerBuilder builder;

builder.AddListeningPort(serverAddress, grpc::InsecureServerCredentials());

builder.RegisterService(&classificationServiceImpl);

unique_ptr<Server> server = builder.BuildAndStart();

cout << "Server listening on " << serverAddress << endl;

server->Wait();

return 0;

}

通過伺服器端組件從webapp訪問推斷服務。運行Python protocol buffer編譯器，生成ClassificationService Python protocol buffer客戶端：pip install grpcio cython grpcio-tools, python -m grpc.tools.protoc -I. --python_out=. --grpc_python_out=. classification_service.proto。生成包含調用服務stub classification_service_pb2.py 。伺服器接到POST請求，解析發送表單，創建ClassificationRequest對象 。分類伺服器設置一個channel，請求提交，分類響應渲染HTML，送回用戶。容器外部命令python client.py，運行伺服器。瀏覽器導航http://localhost:8080 訪問UI。

from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler

import cgi

import classification_service_pb2

from grpc.beta import implementations

class ClientApp(BaseHTTPRequestHandler):

def do_GET(self):

self.respond_form()

def respond_form(self, response=""):

form = """

<html><body>

<h1>Image classification service</h1>

<form enctype="multipart/form-data" method="post">

<div>Image: <input type="file" name="file" accept="image/jpeg"></div>

<div><input type="submit" value="Upload"></div>

</form>

%s

</body></html>

"""

response = form % response

self.send_response(200)

self.send_header("Content-type", "text/html")

self.send_header("Content-length", len(response))

self.end_headers()

self.wfile.write(response)

def do_POST(self):

form = cgi.FieldStorage(

fp=self.rfile,

headers=self.headers,

environ={

REQUEST_METHOD: POST,

CONTENT_TYPE: self.headers[Content-Type],

})

request = classification_service_pb2.ClassificationRequest()

request.input = form[file].file.read()

channel = implementations.insecure_channel("127.0.0.1", 9999)

stub = classification_service_pb2.beta_create_ClassificationService_stub(channel)

response = stub.classify(request, 10) # 10 secs timeout

self.respond_form("<div>Response: %s</div>" % response)

if __name__ == __main__:

host_port = (0.0.0.0, 8080)

print "Serving in %s:%s" % host_port

HTTPServer(host_port, ClientApp).serve_forever()

產品準備，分類伺服器應用產品。編譯伺服器文件複製到容器永久位置，清理所有臨時構建文件。容器中，mkdir /opt/classification_server, cd /mnt/home/serving_example, cp -R bazel-bin/. /opt/classification_server, bazel clean 。容器外部，狀態提交新Docker鏡像，創建記錄虛擬文件系統變化快照。容器外，docker ps, dock commit <container id>。圖像推送到自己偏好docker服務雲，服務。

參考資料：

《面向機器智能的TensorFlow實踐》

歡迎付費諮詢(150元每小時)，我的微信：qingxingfengzi