基于harbor搭建企业docker镜像仓库

背景

docker中要使用镜像，一般会从本地、docker Hup公共仓库和其它第三方公共仓库中下载镜像，一般出于安全和外网(墙)资源下载速率的原因考虑企业级上不会轻易使用。那么有没有一种办法可以存储自己的镜像又有安全认证的仓库呢? 

—-> 企业级环境中基于Harbor搭建自己的安全认证仓库。

Harbor是VMware公司最近开源的企业级Docker Registry项目, 其目标是帮助用户迅速搭建一个企业级的Docker registry服务。

安装Harbor

harbor需要安装docker和docker-compose才能使用，安装docker步骤省略，

安装docker-dompose

docker-dompose安装步骤如下：

下载最新版的docker-compose文件 

1
$ curl -L https://github.com/docker/compose/releases/download/1.23.2/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose

添加可执行权限

1
$ chmod +x /usr/local/bin/docker-compose

验证版本

1
2
$ docker-compose -v
docker-compose version 1.23.2, build 1110ad01

获取Harbor软件包

1
https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.1.tgz

解压

1
tar -xf harbor-offline-installer-v1.7.1.tgz -C /usr/local/

编辑配置文件

1
2
3
4
5
6
7
8
9
10
$ cd /usr/local/harbor
$ vim harbor.cfg
hostname = reg.for-k8s.com 		  # 本机外网IP或域名，该地址供用户通过UI进行访问，不要使用127.0.0.1
ui_url_protocol = https             # 用户访问私仓时使用的协议，默认时http，配置成https
db_password = root123           　　 # 指定mysql数据库管理员密码
harbor_admin_password：Harbor12345   # harbor的管理员账户密码
ssl_cert = /data/cert/reg.for-k8s.com.crt 　　# 设置证书文件路径
ssl_cert_key = /data/cert/reg.for-k8s.com.key  # 设置证书密钥文件路径

# 其他配置选项按需填写即可

生成ssl证书

生成根证书

1
2
3
$ cd /dada/cert/

$ openssl req  -newkey rsa:4096 -nodes -sha256 -keyout ca.key -x509 -days 365 -out ca.crt -subj "/C=CN/L=Shanghai/O=harbor/CN=harbor-registry"

生成一个证书签名, 设置访问域名为 reg.for-k8s.com

1
$ openssl req -newkey rsa:4096 -nodes -sha256 -keyout reg.for-k8s.com.key -out server.csr -subj "/C=CN/L=Shanghai/O=harbor/CN=reg.for-k8s.com"

生成主机证书

1
$ openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out reg.for-k8s.com.crt

通过自带脚本一键安装

1
2
3
4
5
6
7
8
9
$ cd /usr/local/harbor/
./install.sh
......
......
......
✔ ----Harbor has been installed and started successfully.----

Now you should be able to visit the admin portal at https://reg.for-k8s.com.
For more details, please visit https://github.com/goharbor/harbor .

然后绑定hosts访问即可:

默认账号密码admin / Harbor12345

ok 那上面的私有仓库服务已经搭建完毕了，该怎么使用呢？

  1. 首先在harbor上创建一个项目myproject(我这里不使用默认的libary)

    这里我选择私有仓库, pull/push都需要在主机上面执行docker login才行;

  2. 当我通过Dockerfile构建一个新镜像的时候, 直接指明registry和标签, 比如:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ docker build -t reg.for-k8s.com/myproject/mydocker-image:v1.0.1 .
Sending build context to Docker daemon  97.21MB
Step 1/12 : FROM  1and1internet/ubuntu-16
 ---> dbf985f1f449
Step 2/12 : MAINTAINER guomaoqiu <guomaoqiu@gmail.com>
 ---> Using cache
 ---> 598894333db9
......
......
Successfully built b190966f3773
Successfully tagged reg.for-k8s.com/myproject/mydocker-image:v1.0.1


$ docker images | grep myproject
reg.for-k8s.com/myproject/mydocker-image  v1.0.1   b190966f3773   44 seconds ago    482MB
  1. 加入当你从别处获取的镜像想上传到私有仓库呢？就是打个tag就行啦, 比如我想把从官网的这个nginx镜像放到我的仓库:
1
2
3
4
$ docker tag nginx reg.for-k8s.com/myproject/mynginx:latest
$ docker images | grep myproject
reg.for-k8s.com/myproject/mydocker-image    v1.0.1 b190966f3773  2 minutes ago       482MB
reg.for-k8s.com/myproject/mynginx           latest 568c4670fa80  5 weeks ago         109MB
  1. 登录仓库
1
2
3
4
5
6
7
8
$ docker login -u admin -p Harbor12345 reg.for-k8s.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
  1. 最后把本地的镜像push到仓库
    当我执行这个的时候报错了：
    1
    2
    docker push reg.for-k8s.com/myproject/mynginx:latest
    Error response from daemon: Get https://reg.for-k8s.com/v2/: x509: certificate signed by unknown authority

解决办法就是如果不在客户端部署证书，那么在Docker启动时设置参数 “–insecure-registry IP/仓库域名”,然后重载服务重启docker进程；注意的是我这里使用的这个域名是自定义的，那么需要在需要上传下载镜像的机器上，同样需要修改docker进程参数，并且绑定hosts,否则即使配置了参数，这个域名没法解析也是push/pull不到镜像的。

  1. 再次执行push操作:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ docker push reg.for-k8s.com/myproject/mynginx:latest
The push refers to repository [reg.for-k8s.com/myproject/mynginx]
b7efe781401d: Pushed
c9c2a3696080: Pushed
7b4e562e58dc: Pushed
latest: digest: sha256:e2847e35d4e0e2d459a7696538cbfea42ea2d3b8a1ee8329ba7e68694950afd3 size: 948

$ [root@k8s-m1 kubectl-terminal-ubuntu]# docker push reg.for-k8s.com/myproject/mydocker-image:v1.0.1
The push refers to repository [reg.for-k8s.com/myproject/mydocker-image]
96dca48ee72c: Pushed
fa879b69764c: Pushed
4d823b00e6b7: Pushed
6bf6e96da4a0: Pushed
eedda540c6a8: Pushed
f2a971e53afa: Pushed
3ee1a3b3fd18: Pushed
8a225cfa6dea: Pushed
428c1ba11354: Pushed
b097f5edab7b: Pushed
27712caf4371: Pushed
8241afc74c6f: Pushed
v1.0.1: digest: sha256:a20629f62d73cff93bf73b31958878a1d76c2dd42e36ebb2cb6d0ac294a46da7 size: 2826

以上push成功；

测试pull
那为了测试pull并且能成功运行，我这里通过kuernetes运行一个DaemonSet，镜像采用: mynginx ,并且设置镜像pull策略为Always, 然后创建一个服务在集群内部通过ClusterIP能够访问, yaml如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
$ cat >> test.yaml << EOF
apiVersion: v1
kind: Service
metadata:
  labels:
    app: mynginx-service
  name: mynginx-service
spec:
  ports:
  - name: 80-80
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: mynginx
  type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    run: mynginx
  name: mynginx
spec:
  selector:
    matchLabels:
      run: mynginx
  template:
    metadata:
      labels:
        run: mynginx
    spec:
      containers:
      - image: reg.for-k8s.com/myproject/mynginx:latest
        imagePullPolicy: Always
        name: mynginx
EOF

$  kubectl apply -f daemonset.yaml
service/mynginx-service created
daemonset.extensions/mynginx create

由于我刚才创建仓库的时候设置的仓库隐私性为私有的
需要docker login 登录成功之后，k8s kubectl create 就拉取不了镜像；
如果设置为公开，那么久不需要配置这一步骤。只需要docker login 登录成功之后，k8s kubectl create 就可以拉取镜像; 但是我不想让其为公开的；所以还需要配置如下步骤：

配置一个私有仓库harbor的secret：

1
2
3
kubectl create secret docker-registry registry-secret --namespace=default \
--docker-server=https://reg.for-k8s.com --docker-username=admin \
--docker-password=Harbor12345

部署时指定imagePullSecrets, 修改在上面的yaml中添加这个选项:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
$ cat >> test.yaml << EOF
apiVersion: v1
kind: Service
metadata:
  labels:
    app: mynginx-service
  name: mynginx-service
spec:
  ports:
  - name: 80-80
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: mynginx
  type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  labels:
    run: mynginx
  name: mynginx
spec:
  selector:
    matchLabels:
      run: mynginx
  template:
    metadata:
      labels:
        run: mynginx
    spec:
      containers:
      - image: reg.for-k8s.com/myproject/mynginx:latest
        imagePullPolicy: Always
        name: mynginx
      imagePullSecrets:
        - name: registry-secret
EOF

$  kubectl apply -f daemonset.yaml
service/mynginx-service created
daemonset.extensions/mynginx create