一、简介
1.官方已经提供registry镜像为什么还需要用harbor
1)registry缺少镜像清理机制，可以push但是不能删除，耗费空间
2)registry缺乏相应的扩展机制
3)harbor特点：
- Cloud native registry:云本地环境 With support for both container images and Helm charts, Harbor serves as registry for cloud native environments like container runtimes and orchestration platforms.
- Role based access control:RBAC基于角色的权限控制 Users and repositories are organized via ‘projects’ and a user can have different permission for images or Helm charts under a project.
- Policy based replication: 基于策略的镜像复制 Images and charts can be replicated (synchronized) between multiple registry instances based on policies with multiple filters (repository, tag and label). Harbor automatically retries a replication if it encounters any errors. Great for load balancing, high availability, multi-datacenter, hybrid and multi-cloud scenarios.
- Vulnerability Scanning: 漏洞扫描 Harbor scans images regularly and warns users of vulnerabilities.
- LDAP/AD support: LDAP/AD支持 Harbor integrates with existing enterprise LDAP/AD for user authentication and management, and supports importing LDAP groups into Harbor and assigning proper project roles to them.
- OIDC support: 身份验证 Harbor leverages OpenID Connect (OIDC) to verify the identity of users authenticated by an external authorization server or identity provider. Single sign-on can be enabled to log into the Harbor portal.
- Image deletion & garbage collection: 镜像删除和垃圾清理 Images can be deleted and their space can be recycled.
- Notary: 镜像签名 Image authenticity can be ensured.
- Graphical user portal: 用户界面 User can easily browse, search repositories and manage projects.
- Auditing: 审计 All the operations to the repositories are tracked.
- RESTful API: RESTful api RESTful APIs for most administrative operations, easy to integrate with external systems. An embedded Swagger UI is available for exploring and testing the API.
- Easy deployment: 安装简单 Provide both an online and offline installer. In addition, a Helm Chart can be used to deploy Harbor on Kubernetes.
2.什么是harbor
VMware公司开源的企业级registry项目，基于docker registry开发的，harbor是一个用于存储和分发docker镜像的企业级registry服务器，通过添加需要的功能如安全性、身份认证、管理来扩展了源Docker Distribution，提升了镜像的传输效率，支持registry之间复制镜像，还提供了更高级的安全功能，比如：漏洞分析、用户管理、访问控制、活动审计等。该项目已经在github上获得超过了4600颗星。
官方网址：https://goharbor.io/
github安装指南：https://github.com/goharbor/harbor/blob/master/docs/installation_guide.md
下载：https://github.com/goharbor/harbor/releases
注意：这里直接使用offline版本即可
3.harbor图标
二、harbor架构原理
1.架构图
2.主要包含7个组件
Proxy：harbor的registry、UI、token server等组件都在反向代理后面，代理来自浏览器和Docker客户端的请求转发到各种后端服务
Registry：负责存储Docker镜像和处理docker推/拉命令，由于harbor需要强制执行对镜像的访问控制，因此registry将引导客户端使用令牌服务，以便于每个请i去提供有效的令牌
Core services：harbor的核心服务，主要提供以下服务：
UI：图形用户界面，可以帮助用户管理注册表的图形
Webhook：webhook是在注册表中配置的一种机制，因此可以将registry中的镜像状态更改填充到harbor的webhook端点。harbor使用webhook更新日志，启动复制和其他一些功能。
Token service：令牌服务负责更加项目用户的角色未每个docker push/pull命令发布令牌。如果从Docker客户端发送的请求中没有令牌，则registry将请求重定向到令牌服务。
Database：数据库存储，项目、用户、角色、复制策略和镜像的元数据。
Job services：用户镜像的复制，可以将本地镜像复制(同步)到其他harbor实例。
Log collector：负责在一个地方收集其他模块的日志。
redis：用于存储session。
3.实验架构图
环境软件版本：docker engine 19.03.2
docker-compose version 1.18.0
CentOS Linux release 7.7.1908 (Core)
harbor.v1.9.1
三、harbor的安装
1.安装方法
可以有两种方式安装：online installer/offline installer
online installer ：从docker hub下载安装，下载地址请见安装步骤章节
offline installer：无internet时，下载离线安装包安装
还可以使用helm chart 在kubernetes上部署
2.安装要求
1)Hardware
ResourceCapacityDescriptionCPUminimal 2 CPU4 CPU is preferredMemminimal 4GB8GB is preferredDiskminimal 40GB160GB is preferred
2)Software
SoftwareVersionDescriptionDocker engineversion 17.06.0-ce+ or higherFor installation instructions, please refer to: docker engine docDocker Composeversion 1.18.0 or higherFor installation instructions, please refer to: docker compose docOpenssllatest is preferredGenerate certificate and keys for Harbor
3)Network ports
PortProtocolDescription443HTTPSHarbor portal and core API will accept requests on this port for https protocol, this port can change in config file4443HTTPSConnections to the Docker Content Trust service for Harbor, only needed when Notary is enabled, This port can change in config file80HTTPHarbor portal and core API will accept requests on this port for http protocol
3.官方安装步骤
The installation steps boil down to the following
Download the installer;
下载地址：https://github.com/goharbor/harbor/releases
Configure harbor.yml;
主要修改hostname主机名及hoarbor_admin_password初始密码
Run install.sh to install and start Harbor;
运行安装脚本
4.开始部署
1)部署说明：harbor支持docker-compose和kubernetes的部署方式，默认是docker-compose单机部署
2)安装docker步骤省略
请参考Docker4-docker私库的搭建及常用方法-docker-registry方式
systemctl start docker
systemctl enable docker
3)下载安装包
创建自定义harbor目录：
mkdir /harbor
cd /harbor
把下载的包上传到此自定义的harbor目录
解压：
tar -zxvf harbor-offline-installer-v1.9.0.tgz
4)修改harbor.yml
暂时必须修改的两个值：
hostname必须指定
初始密码自己改一个，这里改成harbor
默认的用户名/密码为admin / Harbor12345
[[email protected] harbor]# cat harbor.yml |grep hostname# The IP address or hostname to access admin UI and registry service.hostname: 192.168.216.52 #---标准域名或ip# And when it enabled the hostname will no longer used[[email protected] harbor]# cat harbor.yml |grep harbor_ad*harbor_admin_password: harbor #---初始密码
5)安装docker-compose
[[email protected] harbor]# ll total 607872-rw-r--r-- 1 root root 622428100 Sep 27 14:52 harbor.v1.9.1.tar.gz-rw-r--r-- 1 root root 5798 Oct 9 15:55 harbor.yml-rwxr-xr-x 1 root root 5088 Sep 27 14:52 install.sh-rw-r--r-- 1 root root 11347 Sep 27 14:52 LICENSE-rwxr-xr-x 1 root root 1748 Sep 27 14:52 prepare[[email protected] harbor]# ./install.sh #---先运行安装脚本，提示环境需要compose[Step 0]: checking installation environment ...Note: docker version: 19.03.2?.Need to install docker-compose(1.18.0+) by yourself first and run this script again. #---提示安装compose
安装compose
yum -y install python-pippip install --upgrade pippip install docker-composedocker-compose version
[[email protected] harbor]# docker-compose version docker-compose version 1.18.0, build 8dd22a9docker-py version: 2.6.1CPython version: 3.6.8OpenSSL version: OpenSSL 1.0.2k-fips 26 Jan 2017[[email protected] harbor]#
6)运行安装脚本
./install.sh[Step 0]: checking installation environment ...Note: docker version: 19.03.2Note: docker-compose version: 1.18.0[Step 1]: loading Harbor images ...b80136ee24a4: Loading layer [> ] 360.4kB/34.25MBb80136ee24a4: Loading layer [====> ] 2.884MB/34.25MBb80136ee24a4: Loading layer [=========> ] 6.488MB/34.25MBb80136ee24a4: Loading layer [==============> ] 10.09MB/34.25MBb80136ee24a4: Loading layer [======================> ] 15.14MB/34.25MB。。。。。。。。。。。。。。。。。。。。。。。。。。。。省略loading步骤。。。。。。。。。。。。。。。。。。。。✔ ----Harbor has been installed and started successfully.---- #---安装成功Now you should be able to visit the admin portal at http://192.168.216.52. For more details, please visit https://github.com/goharbor/harbor .
到这里就安装完成了，此时有9个容器运行如下：
[[email protected] harbor]# docker psCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESe70ce2270a2b goharbor/nginx-photon:v1.9.1 "nginx -g 'daemon of?? 30 minutes ago Up 30 minutes (healthy) 0.0.0.0:80->8080/tcp nginxf8c165eb8f4e goharbor/harbor-jobservice:v1.9.1 "/harbor/harbor_jobs?? 30 minutes ago Up 30 minutes (healthy) harbor-jobserviceba46b285ff14 goharbor/harbor-core:v1.9.1 "/harbor/harbor_core" 30 minutes ago Up 30 minutes (healthy) harbor-core5179d37b0029 goharbor/harbor-db:v1.9.1 "/docker-entrypoint.?? 30 minutes ago Up 30 minutes (healthy) 5432/tcp harbor-db8d210e049b95 goharbor/harbor-portal:v1.9.1 "nginx -g 'daemon of?? 30 minutes ago Up 30 minutes (healthy) 8080/tcp harbor-portal9bb9ea0b891b goharbor/harbor-registryctl:v1.9.1 "/harbor/start.sh" 30 minutes ago Up 30 minutes (healthy) registryctl6af51f3478c5 goharbor/redis-photon:v1.9.1 "redis-server /etc/r?? 30 minutes ago Up 30 minutes (healthy) 6379/tcp redis93656b06f470 goharbor/registry-photon:v2.7.1-patch-2819-2553-v1.9.1 "/entrypoint.sh /etc?? 30 minutes ago Up 30 minutes (healthy) 5000/tcp registrybe49e0941ce4 goharbor/harbor-log:v1.9.1 "/bin/sh -c /usr/loc?? 30 minutes ago Up 30 minutes (healthy) 127.0.0.1:1514->10514/tcp harbor-log[[email protected] harbor]#
7)访问web界面
四、如何使用harbor-registry
1.客户端登陆，使用http协议需要修改不安全的注册来用允许http链接
注意：客户端在login之前需要添加 “–insecure-registry” 不安全的注册。 即通信使用 http 协议。如果使用安全的通信，就使用 https
注意：在测试过程中，我客户端是低版本docker所以在修改/etc/docker/daemon.json 文件的时候写成了”insecure-registries”:[“192.168.216.52”]，一直没有成功login，后来更新了新版本docker后半部分写成了[“http://192.168.216.52”]，一下子就成功了，不知道是版本还是格式错误，这个问题之后测试再来补上，总之下面可以顺利进行了。
不修改登陆报错：
[[email protected] docker]# docker login 192.168.216.52Authenticating with existing credentials...Login did not succeed, error: Error response from daemon: Get https://192.168.216.52/v2/: dial tcp 192.168.216.52:443: connect: connection refusedUsername (admin): admin^H^H^HPassword: Error response from daemon: Get https://192.168.216.52/v2/: dial tcp 192.168.216.52:443: connect: connection refused
修改/etc/docker/daemon.json文件，如没有就创建一个：(修改的是客户端的)
[[email protected] docker]# pwd/etc/docker[[email protected] docker]# ll total 8-rw-r--r-- 1 root root 52 Oct 10 17:42 daemon.json.bak-rw------- 1 root root 244 Jul 25 11:16 key.json[[email protected] docker]# mv daemon.json.bak daemon.json[[email protected] docker]# cat daemon.json {"insecure-registries":["http://192.168.216.52"]}#---然后重启docker[[email protected] docker]# systemctl daemon-reload&&systemctl restart docker
登陆：
docker login 192.168.216.52
注：这里其实是需要用户名密码的，就用之前修改harbor.yml里面默认密码admin/harbor,下面实例是因为已经登陆过所以没有提示密码
[[email protected] docker]# docker login 192.168.216.52Authenticating with existing credentials...WARNING! Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded[[email protected] docker]#
也可以直接加用户密码参数登陆
[[email protected] docker]# docker login -u admin -p harbor 192.168.216.52 WARNING! Using --password via the CLI is insecure. Use --password-stdin.WARNING! Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded[[email protected] docker]#
2.上传镜像
使用docker push 命令
docker push 192.168.216.52/library/zxg/centos_nginx:v1
1)打标签
[[email protected] docker]# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEzxg/centos_nginx v1 7c6604cacec1 7 weeks ago 698MBatlassian/jira-software latest c4b90dede4f3 7 weeks ago 624MBzxg/my_nginx v1 b164f4c07c64 2 months ago 126MBzxg/my_nginx latest f07837869dfc 2 months ago 126MBnginx latest e445ab08b2be 2 months ago 126MBalpine latest b7b28af77ffe 3 months ago 5.58MBcentos latest 9f38484d220f 6 months ago 202MB[[email protected] docker]# docker tag 7c6604cacec1 192.168.216.52/library/zxg/centos_nginx:v1 [[email protected] docker]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE 192.168.216.52/library/zxg/centos_nginx v1 7c6604cacec1 7 weeks ago 698MB
2)上传
[[email protected] docker]# docker push 192.168.216.52/library/zxg/centos_nginx:v1The push refers to repository [192.168.216.52/library/zxg/centos_nginx]7b4de0c97fbb: Pushed d69483a6face: Pushed v1: digest: sha256:2654d7a4fbab3b1be85ca177ac08ce9e13177f9ad45b827ca3ed1e1629050078 size: 742
3)检查是否成功
3.验证daemon.json格式
1)把刚才客户端的daemon.json文件发送到当前节点
[[email protected] docker]# scp daemon.json [email protected]:/etc/docker/The authenticity of host '192.168.216.52 (192.168.216.52)' can't be established.ECDSA key fingerprint is SHA256:kvAeuWOn6RFSXvl5qFIszQEx9gLizuZER+I4VJkpAso.ECDSA key fingerprint is MD5:b7:ef:e0:3c:8f:97:01:c2:5c:9a:2e:fc:4d:e2:99:83.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added '192.168.216.52' (ECDSA) to the list of known [email protected]'s password: daemon.json 100% 52 9.4KB/s 00:00 [[email protected] docker]#
2)修改一下格式看看行不行
[[email protected] harbor]# cat /etc/docker/daemon.json {"insecure-registries":["192.168.216.52"] ##之前的格式是["http://192.168.216.52"]}[[email protected] harbor]#
修改后，需要重启docker及compose
systemctl daemon-reload&&systemctl restart dockerdocker-compose down -vdocker-compose up -d
3)login测试是否可以登陆
[[email protected] harbor]# docker login 192.168.216.52Username: adminPassword: WARNING! Your password will be stored unencrypted in /root/.docker/config.json.Configure a credential helper to remove this warning. Seehttps://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded #登陆正常看来不是格式问题，应该就是版本问题了，低版本应该是需要修改别的配置文件
4.下载镜像
docker pull 192.168.216.52/library/zxg/centos_nginx:v1
[[email protected] harbor]# docker pull 192.168.216.52/library/zxg/centos_nginx:v1v1: Pulling from library/zxg/centos_nginx8ba884070f61: Pull complete 75754525faad: Pull complete Digest: sha256:2654d7a4fbab3b1be85ca177ac08ce9e13177f9ad45b827ca3ed1e1629050078Status: Downloaded newer image for 192.168.216.52/library/zxg/centos_nginx:v1192.168.216.52/library/zxg/centos_nginx:v1[[email protected] harbor]# docker imagesREPOSITORY TAG IMAGE ID CREATED SIZEgoharbor/chartmuseum-photon v0.9.0-v1.9.1 0aa7451af9b8 13 days ago 131MBgoharbor/harbor-migrator v1.9.1 a83f1be1ec94 13 days ago 362MBgoharbor/redis-photon v1.9.1 4d8d79a557df 13 days ago 110MBgoharbor/clair-photon v2.0.9-v1.9.1 98b318ca3cb0 13 days ago 165MBgoharbor/notary-server-photon v0.6.1-v1.9.1 78dac4ed14d8 13 days ago 138MBgoharbor/notary-signer-photon v0.6.1-v1.9.1 5d9f413e14a8 13 days ago 135MBgoharbor/harbor-registryctl v1.9.1 f4b2b72cdf71 13 days ago 99.6MBgoharbor/registry-photon v2.7.1-patch-2819-2553-v1.9.1 d460d658f383 13 days ago 82.3MBgoharbor/nginx-photon v1.9.1 0db1e12b9d30 13 days ago 43.9MBgoharbor/harbor-log v1.9.1 368dd79ef99f 13 days ago 82.6MBgoharbor/harbor-jobservice v1.9.1 71d4a3eaff94 13 days ago 141MBgoharbor/harbor-core v1.9.1 d105210d9924 13 days ago 155MBgoharbor/harbor-portal v1.9.1 2dced1823043 13 days ago 51.3MBgoharbor/harbor-db v1.9.1 91784692a954 13 days ago 147MBgoharbor/prepare v1.9.1 44775181c88d 13 days ago 148MB192.168.216.51:5000/busybox v1 19485c79a9bb 5 weeks ago 1.22MB192.168.216.51:5000/busybox v2 19485c79a9bb 5 weeks ago 1.22MB192.168.216.52:5000/busybox latest 19485c79a9bb 5 weeks ago 1.22MB192.168.216.52:5000/busybox v2 19485c79a9bb 5 weeks ago 1.22MBbusybox v1 19485c79a9bb 5 weeks ago 1.22MBbusybox v2 19485c79a9bb 5 weeks ago 1.22MB192.168.216.52/library/zxg/centos_nginx v1 7c6604cacec1 7 weeks ago 698MB[[email protected] harbor]#
五、其他设定
1.使用https访问配置harbor
请参考https://github.com/goharbor/harbor/blob/master/docs/configure_https.md
2.管理harbor的生命周期
1)停止
docker-compose stop
2)重新启动
docker-compose start
3)修改配置
停止harbor—》更新harbor.yml—》运行prepare脚本填充配置—》启动harbor
docker-compose down -v #—此命令关闭harbor同时保存镜像数据及harbor的数据库文件在文件系统上
vim harbor.yml
docker-compose up -d
4)删除harbor的数据库和图像数据(进行重新安装)
rm -r /data/database
rm -r /data/registry
3.安装notary(公正服务)
./install.sh –with-notry
必要设置:ui_url_protocol HTTPS
4.安装clair服务
./install.sh –with-clair
5.安装chart repository服务
./install.sh –with-chartmuseum
6.如果都安装必须使用同一个命令
./install.sh –with-notary –with-clair –with-chartmuseum
7.docker-compose命令帮助
https://docs.docker.com/compose/reference/
8.数据及日志文件
默认在/data/目录中，可以修改harbor.yml更改配置
9.外部数据库
目前harbor只支持postgreSQL数据库，使用外部数据库，需要取消注释external_database部分，然后首先应该创建harbor core,clair、notaryserver、notary signer这四个数据库
10.管理harbor用户配置命令行配置请见下面链接：
https://github.com/goharbor/harbor/blob/master/docs/configure_user_settings.md
总结流程：
安装docker—>安装docker-compose—>下载harbor-offline压缩包—>上传到linux—>解压harbor-offline-installer-v1.9.1-rc1.tgz—>编辑解压的harbor文件里的harbor.yml文件—>修改关键值(域名/初始密码)—>执行./install.sh脚本开始安装—>修改客户端的daemon.json文件，添加非信任授权—>客户端登陆执行docker pull/push命令。