Docker倉庫
一、Docker單機倉庫
Docker Registry作為Docker的核心元件之一負責單主機的鏡像内容的存儲與分發，用戶端的docker pull以及push指令都将直接與registry進行互動,最初版本的registry 由Python實作，由于設計初期在安全性，性能以及API的設計上有着諸多的缺陷，該版本在0.9之後停止了開發，由新項目distribution（新的docker register被稱為Distribution）來重新設計并開發下一代registry，新的項目由go語言開發，所有的API，底層存儲方式，系統架構都進行了全面的重新設計已解決上一代registry中存在的問題，2016年4月份registry 2.0正式釋出，docker 1.6版本開始支援registry 2.0，而八月份随着docker 1.8 釋出，docker hub正式啟用2.1版本registry全面替代之前版本 registry，新版registry對鏡像存儲格式進行了重新設計并和舊版不相容，docker 1.5和之前的版本無法讀取2.0的鏡像，另外，Registry 2.4版本之後支援了資源回收筒機制，也就是可以删除鏡像了，在2.4版本之前是無法支援删除鏡像的，是以如果你要使用最好是大于Registry 2.4版本的
官方文檔位址: https://docs.docker.com/registry/
官方github 位址: https://github.com/docker/distribution
官方部署文檔: https://github.com/docker/docker.github.io/blob/master/registry/deploying.md
- 下載下傳docker registry 鏡像
[root@localhost docker]# docker pull registry:2.7.1
2.7.1: Pulling from library/registry
79e9f2f55bf5: Pull complete
0d96da54f60b: Pull complete
5b27040df4a2: Pull complete
e2ead8259a04: Pull complete
3790aef225b9: Pull complete
Digest: sha256:169211e20e2f2d5d115674681eb79d21a217b296b43374b8e39f97fcf866b375
Status: Downloaded newer image for registry:2.7.1
docker.io/library/registry:2.7.1
[root@localhost docker]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry 2.7.1 b8604a3fe854 3 months ago 26.2MB
1、建立單機倉庫
1.1 建立賬号啟動docker registry容器
[root@localhost docker]# yum -y install httpd
[root@localhost docker]# htpasswd -Bbn test 123456 > /etc/docker/auth/registry
[root@localhost docker]# cat /etc/docker/auth/registry
test:$2y$05$ecpX/anNOrNRe7xgenNZVOqmA1DWlb8e1AAUWD2LGFrV5pFnBtMs2
[root@localhost docker]# docker run -d -p 5000:5000 --restart=always --name registry -v /etc/docker/auth:/auth -e "REGISTRY_AUTH=htpasswd" -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/registry registry:2.7.1
a218fcf93146d0d2cf9888fc8bd65ef6a753ddb985996acf3a70487c469c037d
[root@localhost docker]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a218fcf93146 registry:2.7.1 "/entrypoint.sh /etc…" About a minute ago Up About a minute 0.0.0.0:5000->5000/tcp registry
[root@localhost docker]# ss -tln
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:5000 *:*
LISTEN 0 128 *:22 *:*
LISTEN 0 128 [::]:22 [::]:*
1.2 登入倉庫
#docker login 預設使用https登入,而docker registry為http,是以預設登入失敗
[root@localhost docker]# docker login 192.168.187.10:5000
Username: test
Password:
Error response from daemon: Get "https://192.168.187.10:5000/v2/": http: server gave HTTP response to HTTPS client
- 将registry倉庫位址加入到service單元檔案
[root@localhost docker]# cat /etc/docker/daemon.json
{
"registry-mirrors": ["https://boqr6s5g.mirror.aliyuncs.com"],
"insecure-registry": ["192.168.187.10:5000"]
}
#或者
[root@localhost docker]# cat /usr/lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket containerd.service
[Service]
Type=notify
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry 192.168.187.10:5000
[root@localhost docker]#systemctl daemon-reload
[root@localhost docker]#systemctl restart docker
- 登入
[root@localhost docker]# docker login 192.168.187.10:5000
Username: test
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
1.3 上傳下載下傳鏡像
#需要先登入（dcoker login）
[root@dockerserver2 ~]# docker tag centos:centos7.8.2003 192.168.187.10:5000/centos7:v1
[root@dockerserver2 ~]# docker push 192.168.187.10:5000/centos7:v1
The push refers to repository [192.168.187.10:5000/centos7]
fb82b029bea0: Pushed
v1: digest: sha256:50b9a3bc27378889210f88d6d0695938e45a912aa99b3fdacfb9a0fef511f15a size: 529
#下載下傳鏡像
[root@dockerserver2 ~]# docker pull 192.168.187.10:5000/centos7:v1
v1: Pulling from centos7
9b4ebb48de8d: Pull complete
Digest: sha256:50b9a3bc27378889210f88d6d0695938e45a912aa99b3fdacfb9a0fef511f15a
Status: Downloaded newer image for 192.168.187.10:5000/centos7:v1
192.168.187.10:5000/centos7:v1
[root@dockerserver2 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
192.168.187.10:5000/centos7 v1 afb6fca791e0 21 months ago 203MB
[root@dockerserver2 ~]# docker run -it --rm 192.168.187.10:5000/centos7:v1
[root@14fd3b4ad4b8 /]# cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)
二、Docker分布式倉庫Harbor
1、 Harhor
Harbor是一個用于存儲和分發Docker鏡像的企業級Registry伺服器，由VMware開源，其通過添加一些企業必需的功能特性，例如安全、辨別和管理等，擴充了開源 Docker Distribution。作為一個企業級私有Registry伺服器，Harbor 提供了更好的性能和安全。提升使用者使用Registry建構和運作環境傳輸鏡像的效率。Harbor支援安裝在多個Registry節點的鏡像資源複制，鏡像全部儲存在私有 Registry 中，確定資料和知識産權在公司内部網絡中管控，另外，Harbor也提供了進階的安全特性，諸如使用者管理，通路控制和活動審計等
vmware 官方開源服務: https://vmware.github.io/
harbor 官方github 位址: https://github.com/vmware/harbor
harbor 官方網址: https://goharbor.io/
harbor 官方文檔: https://goharbor.io/docs/
github文檔: https://github.com/goharbor/harbor/tree/master/docs
官方功能介紹
- 基于角色的通路控制: 使用者與Docker鏡像倉庫通過“項目”進行組織管理，一個使用者可以對多個鏡像倉庫在同一命名空間（project）裡有不同的權限
- 鏡像複制: 鏡像可在多個Registry執行個體中複制（同步）。尤其适合于負載均衡，高可用，混合雲和多雲的場景
- 圖形化使用者界面: 使用者可以通過浏覽器來浏覽，檢索目前Docker鏡像倉庫，管理項目和命名空間
- AD/LDAP 支: Harbor可以內建企業内部已有的AD/LDAP，用于鑒權認證管理
- 審計管理: 所有針對鏡像倉庫的操作都可以被記錄追溯，用于審計管理
- 國際化: 已擁有英文、中文、德文、日文和俄文的本地化版本。更多的語言将會添加進來
- RESTful API: 提供給管理者對于Harbor更多的操控, 使得與其它管理軟體內建變得更容易
- 部署簡單: 提供線上和離線兩種安裝工具， 也可以安裝到vSphere平台(OVA方式)虛拟裝置
Harbor組成
- Proxy: 對應啟動元件nginx。它是一個nginx反向代理，代理Notary client（鏡像認證）、Dockerclient（鏡像上傳下載下傳等）和浏覽器的通路請求（Core Service）給後端的各服務
-
UI（Core Service）: 對應啟動元件harbor-ui。底層資料存儲使用mysql資料庫，主要提供了四個
子功能:
- UI: 一個web管理頁面ui
- API: Harbor暴露的API服務
- Auth: 使用者認證服務，decode後的token中的使用者資訊在這裡進行認證；auth後端可以接db、ldap、uaa三種認證實作
- Token服務（上圖中未展現）: 負責根據使用者在每個project中的role來為每一個dockerpush/pull指令釋出一個token，如果從docker client發送給registry的請求沒有帶token，registry會重定向請求到token服務建立token
- Registry: 對應啟動元件registry。負責存儲鏡像檔案，和處理鏡像的pull/push指令。Harbor對鏡像進行強制的通路控制，Registry會将用戶端的每個pull、push請求轉發到token服務來擷取有效的token
-
Admin Service: 對應啟動元件harbor-adminserver。是系統的配置管理中心附帶檢查存儲用量，
ui和jobserver啟動時候需要加載adminserver的配置
- Job Sevice: 對應啟動元件harbor-jobservice。負責鏡像複制工作的，他和registry通信，從一個registry pull鏡像然後push到另一個registry，并記錄job_log
-
Log Collector: 對應啟動元件harbor-log。日志彙總元件，通過docker的log-driver把日志彙總到
一起
-
DB: 對應啟動元件harbor-db，負責存儲project、 user、 role、replication、image_scan、
access等的metadata資料
2、安裝Harbor
下載下傳位址: https://github.com/vmware/harbor/releases
安裝文檔: https://github.com/goharbor/harbor/blob/master/docs/install-config/_index.md
- 安裝docker ---- 濾過
- 安裝docker compose
[root@dockerserver2 ~]# curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 664 100 664 0 0 901 0 --:--:-- --:--:-- --:--:-- 900
100 12.1M 100 12.1M 0 0 4631k 0 0:00:02 0:00:02 --:--:-- 9.7M
[root@dockerserver2 ~]#
[root@dockerserver2 ~]# ll /usr/local/bin/docker-compose
-rw-r--r--. 1 root root 12737304 Feb 18 08:32 /usr/local/bin/docker-compose
[root@dockerserver2 ~]# chmod +x /usr/local/bin/docker-compose
[root@dockerserver2 ~]# ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
[root@dockerserver2 ~]# docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019
- 安裝Harbor
以下使用 harbor 穩定版本v1.10.10安裝包
下載下傳離線完整安裝包,推薦使用
[root@dockerserver2 ~]#wget https://github.com/goharbor/harbor/releases/download/v1.10.10/harbor-offline-installer-v1.10.10.tgz
[root@dockerserver2 src]# mkdir /apps
[root@dockerserver2 src]# tar -zxf harbor-offline-installer-v1.10.10.tgz
- 編輯harbor配置檔案
最新文檔: https://github.com/goharbor/harbor/blob/master/docs/install-config/configure-yml-file.
md
[root@dockerserver2 ~]#vim /apps/harbor/harbor.cfg
#隻需要修改下面兩行
hostname = 192.168.187.11 #修改此行,指向目前主機IP 或 FQDN
harbor_admin_password = 123456 #修改此行指定harbor登入使用者admin的,預設使用者:admin/Harbor12345
#可選項
ui_url_protocol = http #預設即可,如果修改為https,需要指定下面證書路徑
ssl_cert = /data/cert/server.crt #預設即可,https時，需指定下面證書檔案路徑
ss_cert_key = /data/cert/server.key #預設即可,https時，需指定下面私鑰檔案路徑
- 運作安裝腳本
[root@dockerserver2 harbor]# yum -y install python
[root@dockerserver2 harbor]# ./install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 20.10.12
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 1.29.2
[Step 2]: loading Harbor images ...
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /apps/harbor
/usr/src/app/utils/configs.py:100: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
configs = yaml.load(f)
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
/usr/src/app/utils/configs.py:90: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
versions = yaml.load(f)
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[Step 5]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating harbor-log ... done
Creating registryctl ... done
Creating redis ... done
Creating registry ... done
Creating harbor-db ... done
Creating harbor-portal ... done
Creating harbor-core ... done
Creating harbor-jobservice ... done
Creating nginx ... done
✔ ----Harbor has been installed and started successfully.----
#安裝harbor後會自動開啟很多相關容器
[root@dockerserver2 harbor]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1834217183fb goharbor/nginx-photon:v1.10.10 "nginx -g 'daemon of…" About a minute ago Up About a minute (healthy) 0.0.0.0:80->8080/tcp nginx
a726b631c455 goharbor/harbor-jobservice:v1.10.10 "/harbor/harbor_jobs…" About a minute ago Up About a minute (healthy) harbor-jobservice
d93cd0e5c4c0 goharbor/harbor-core:v1.10.10 "/harbor/harbor_core" About a minute ago Up About a minute (healthy) harbor-core
62f4feeea29f goharbor/harbor-portal:v1.10.10 "nginx -g 'daemon of…" About a minute ago Up About a minute (healthy) 8080/tcp harbor-portal
7b4100dfc0be goharbor/harbor-db:v1.10.10 "/docker-entrypoint.…" About a minute ago Up About a minute (healthy) 5432/tcp harbor-db
9ede3db336cb goharbor/registry-photon:v1.10.10 "/home/harbor/entryp…" About a minute ago Up About a minute (healthy) 5000/tcp registry
0a86c46beb82 goharbor/redis-photon:v1.10.10 "redis-server /etc/r…" About a minute ago Up About a minute (healthy) 6379/tcp redis
d8cc10dda240 goharbor/harbor-registryctl:v1.10.10 "/home/harbor/start.…" About a minute ago Up About a minute (healthy) registryctl
3a1e7e0eb2bc goharbor/harbor-log:v1.10.10 "/bin/sh -c /usr/loc…" About a minute ago Up About a minute (healthy) 127.0.0.1:1514->10514/tcp harbor-log
- 開機啟動harbor
[root@dockerserver2 harbor]# vim /lib/systemd/system/harbor.service
[Unit]
Description=Harbor
After=docker.service systemd-networkd.service systemd-resolved.service
Requires=docker.service
Documentation=http://github.com/vmware/harbor
[Service]
Type=simple
Restart=on-failure
RestartSec=5
ExecStart=/usr/bin/docker-compose -f /apps/harbor/docker-compose.yml up
ExecStop=/usr/bin/docker-compose -f /apps/harbor/docker-compose.yml down
[Install]
WantedBy=multi-user.target
[root@dockerserver2 harbor]# systemctl daemon-reload
[root@dockerserver2 harbor]# systemctl enable harbor
Created symlink from /etc/systemd/system/multi-user.target.wants/harbor.service to /usr/lib/systemd/system/harbor.service.
3、使用Harbor
通路：http://192.168.187.11/
- 賬号：admin/123456
- 使用單機harbor
- 建立項目
harbor上必須先建立項目，才能上傳鏡像
- 登入harbor上傳鏡像
[root@ubuntu1804 ~]#vim /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
--insecure-registry 10.0.0.101 --insecure-registry 192.168.187.11
[root@dockerserver2 src]# systemctl daemon-reload
[root@dockerserver2 src]# systemctl restart docker
[root@dockerserver2 src]# docker login 192.168.187.11
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
[root@dockerserver2 ~]# cat .docker/config.json
{
"auths": {
"192.168.187.10:5000": {
"auth": "dGVzdDoxMjM0NTY="
},
"192.168.187.11": {
"auth": "YWRtaW46MTIzNDU2"
}
}
}
#上傳鏡像
#修改 images 的名稱，不修改成指定格式無法将鏡像上傳到 harbor 倉庫
#格式： Harbor主機IP/項目名/image名字:版本
[root@dockerserver2 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu 20.04 ba6acccedd29 4 months ago 72.8MB
192.168.187.10:5000/centos7 v1 afb6fca791e0 21 months ago 203MB
[root@dockerserver2 ~]# docker tag ubuntu:20.04 192.168.187.11/test/ubuntu-base:v1
[root@dockerserver2 ~]# docker push 192.168.187.11/test/ubuntu-base:v1
The push refers to repository [192.168.187.11/test/ubuntu-base]
9f54eef41275: Pushed
v1: digest: sha256:7cc0576c7c0ec2384de5cbf245f41567e922aab1b075f3e8ad565f508032df17 size: 529
- 驗證是否成功上傳
**注意：**如果不事先建立項目，上傳鏡像會失敗
- 可檢視日志記錄
- 下載下傳鏡像
下載下傳前必須修改docker的service 檔案，加入harbor伺服器的位址才可以下載下傳
[root@ubuntu1804 ~]#vim /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
--insecure-registry 10.0.0.101 --insecure-registry 192.168.187.11
[root@dockerserver2 src]# systemctl daemon-reload
[root@dockerserver2 src]# systemctl restart docker
[root@dockerserver1 src]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry 2.7.1 b8604a3fe854 3 months ago 26.2MB
[root@dockerserver1 src]# docker pull 192.168.187.11/test/ubuntu-base:v1
v1: Pulling from test/ubuntu-base
7b1a6ab2e44d: Pull complete
Digest: sha256:7cc0576c7c0ec2384de5cbf245f41567e922aab1b075f3e8ad565f508032df17
Status: Downloaded newer image for 192.168.187.11/test/ubuntu-base:v1
192.168.187.11/test/ubuntu-base:v1
[root@dockerserver1 src]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry 2.7.1 b8604a3fe854 3 months ago 26.2MB
192.168.187.11/test/ubuntu-base v1 ba6acccedd29 4 months ago 72.8MB
- 建立自動上傳鏡像腳本
[root@ubuntu1804 ~]#cd /data/dockerfile/web/nginx/1.16.1-alpine/
[root@ubuntu1804 1.16.1-alpine]#vim build.sh
[root@ubuntu1804 1.16.1-alpine]#cat build.sh
#!/bin/bash
TAG=$1
docker build -t 10.0.0.101/example/nginx-alpine:1.16.1-${TAG} .
docker push 10.0.0.101/example/nginx-alpine:1.16.1-${TAG}
docker rmi -f 10.0.0.101/example/nginx-alpine:1.16.1-${TAG}
[root@ubuntu1804 1.16.1-alpine]#bash build.sh v1
- 修改harbor配置
#後期如果修改harbor配置，比如: 修改IP位址等，可執行以下步驟生效
[root@ubuntu1804 ~]#cd /apps/harbor/
[root@ubuntu1804 harbor]#docker-compose stop
Stopping nginx ... done
Stopping harbor-portal ... done
Stopping harbor-jobservice ... done
Stopping harbor-core ... done
Stopping harbor-adminserver ... done
Stopping harbor-db ... done
Stopping registryctl ... done
Stopping registry ... done
Stopping redis ... done
Stopping harbor-log ...
#修改harbor配置
[root@dockerserver2 harbor]# vim /apps/harbor/harbor.yml
#更新配置
[root@dockerserver2 harbor]#/apps/harbor/prepare
#重新啟動docker compose
[root@dockerserver2 harbor]#docker-compose start
4、Harbor 高可用
Harbor支援基于政策的Docker鏡像複制功能，這類似于MySQL的主從同步，其可以實作不同的資料中心、不同的運作環境之間同步鏡像，并提供友好的管理界面，大大簡化了實際運維中的鏡像管理工作，已經有用很多網際網路公司使用harbor搭建内網docker倉庫的案例，并且還有實作了雙向複制功能
- 安裝第二台harbor主機
- 注意：harbor.cfg中配置 hostname = 192.168.187.12
- 建立相同的項目
- 參考第一台主機資訊,建立複制（同步）目标資訊,将第一台主機設為複制的目标
- 輸入第一台主機資訊
- 第二台harbor上建立複制規則實作到第一台harbor的單向複制
- 在第一台harbor主機上重複上面操作，在第一台harbor上再執行下面操作，才實作雙向同步
- 确認同步成功
5、Harbor https 配置
#安裝docker步驟省略
#生成私鑰和證書
[root@dockerserver2 harbor]#touch /root/.rnd
[root@dockerserver2 harbor]#mkdir /apps/harbor/certs/
[root@dockerserver2 harbor]#cd /apps/harbor/certs/
#生成CA憑證
[root@dockerserver2 harbor]#openssl req -newkey rsa:4096 -nodes -sha256 -keyout
ca.key -x509 -subj "/CN=ca.org" -days 365 -out ca.crt
#生成harbor主機的證書申請
[root@dockerserver2 harbor]#openssl req -newkey rsa:4096 -nodes -sha256 -subj
"/CN=harbor.magedu.org" -keyout harbor.org.key -out harbo.org.csr
#給harbor主機頒發證書
[root@dockerserver2 harbor]#openssl x509 -req -in harbor.org.csr -CA ca.crt -
CAkey ca.key -CAcreateserial -out harbor.org.crt
[root@dockerserver2 harbor]#tree /apps/harbor/certs
/apps/harbor/certs
├── ca.crt
├── ca.key
├── ca.srl
├── harbor.org.crt
├── harbor.org.csr
└── harbor.org.key
0 directories, 6 files
[root@dockerserver2 harbor]#vim /apps/harbor/harbor.cfg
hostname = harbor.magedu.org
ui_url_protocol = https
ssl_cert = /apps/harbor/certs/harbor.org.crt
ssl_cert_key = /apps/harbor/certs/harbor.org.key
harbor_admin_password = 123456
[root@dockerserver2 harbor]#apt -y install python
[root@dockerserver2 harbor]#/apps/harbor/install.sh
- 上傳下載下傳鏡像需要在用戶端下載下傳證書，否則會報錯
[root@ubuntu1804 ~]#mkdir -pv /etc/docker/certs.d/