Docker 部署tensorflow_model_server過程記錄

檢視CUDA版本：

cat /usr/local/cuda/version.txt

檢視檔案連結到哪裡：

ls -al libcuda.so.1

docker 下 tensorflow_model_server t2t部署

1.安裝docker

見菜鳥教程：http://www.runoob.com/docker/docker-tutorial.html

2.下載下傳serving鏡像：

docker pull tensorflow/serving:latest-devel（檔案較大3G多，下載下傳時間較長）

3.用serving鏡像建立容器：

• docker run -it -p 9000:9000 tensorflow/serving:latest-devel  --privileged=true（調用GPU）
             

4.将模型拷貝到容器中：（新開個指令視窗）

docker cp [模型檔案所在目錄] 容器ID:/[容器中目錄]

如：

• docker cp E:/model/export 0f087sdf8sf:/model  
             

5.容器中運作tensorflow_model_server服務

tensorflow_model_server --port=9000 --model_name=nmt --model_base_path=/model

6.t2t連接配接

• t2t-query-server --server=*.*.*.*:9000 --servable_name=nmt --problem=nmt_zhen --data_dir=/home/data --t2t_usr_dir=/home/script
             

  （...:9000位址同伺服器位址，或docker啟動時預設位址）

GPU版本問題：

1）tensorflow_model_server: error while loading shared libraries: libcuda.so.1: cannot open shared object file: No such file or directory

解決方法：将libcuda.so.1檔案放到對應目錄下

1）

cp /usr/local/cuda-10.0/compat/libcud* /usr/local/cuda/lib64/

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64"

export CUDA_HOME=/usr/local/cuda

2）

更新apt-get: apt-get update

安裝vim： apt-get install vim

vi ./root/.bashrc

添加：

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda-10.0/compat"

/usr/local/cuda-10.0/lib64/stubs

export CUDA_HOME=/usr/local/cuda

執行：

source ./root/.bashrc

3）

安裝nvidia-docker 解決問題

/usr/local/cuda-10.0/lib64/stubs

加載單模型：

docker run -p 8501:8501

–mount type=bind,source=/path/to/my_model/,target=/models/my_model

-e MODEL_NAME=my_model -t tensorflow/serving

docker run -p 9098:8500 --mount type=bind,source=/opt/data/D_NMT/translate_enzh/export/V1.0/,target=/models/nmt_enzh -e MODEL_NAME=nmt_enzh -t tensorflow/serving

加載多模型：

docker run --runtime=nvidia -p 8500:8500 -p 8501:8501

–mount type=bind,source=/path/to/my_model/,target=/models/my_model

–mount type=bind,source=/path/to/my/models.config,target=/models/models.config

-t tensorflow/serving:latest-gpu --model_config_file=/models/models.config &

安裝nvidai-docker問題：

安裝教程：https://github.com/NVIDIA/nvidia-docker#quick-start

version `XZ_5.1.2alpha’ not found (required by /lib64/librpmio.so.3)

下載下傳：liblzma.so.5.2.2到/opt/anaconda3/envs/py36/lib目錄

執行軟連接配接：sudo ln -s -f liblzma.so.5.2.2 liblzma.so.5

解決問題！

加載多模型：

sudo docker run -d -p 8500:8500 --mounttype=bind,source=/path/to/source_models/model1/,target=/models/model1 --mounttype=bind,source=/path/to/source_models/model2/,target=/models/model2 --mounttype=bind,source=/path/to/source_models/model3/,target=/models/model3 --mounttype=bind,source=/path/to/source_models/model.config,target=/models/model.config -t --name ner tensorflow/serving --model_config_file=/models/model.config

docker run --runtime=nvidia -p 9000:8500 --mount type=bind,source=/opt/data/models/nmt_enzh,target=/models/nmt_enzh --mount type=bind,source=/opt/data/models/nmt_zhen/,target=/models/nmt_zhen --mount type=bind,source=/opt/data/models/model.config,target=/models/model.config -t tensorflow/serving:latest-gpu --model_config_file=/models/model.config

[[email protected] /]# cd /usr/local/cuda-10.0/lib64

[[email protected] lib64]# ls -al libcuda.so.1

lrwxrwxrwx 1 root root 43 Mar 15 14:24 libcuda.so.1 -> /usr/local/cuda-10.0/lib64/stubs/libcuda.so