天天看點

k8s 證書過期處理

k8s dashboard無法登入,從dashboard查到apiserver 再查到etcd 最後是發現是master證書問題,自簽證書有效1期年,服務初始由kubeadm搭建

2020-07-16 09:31:21.660802 I | raft: aa548f97cfe05a3f is starting a new election at term 7829

2020-07-16 09:31:21.660842 I | raft: aa548f97cfe05a3f became candidate at term 7830

2020-07-16 09:31:21.660855 I | raft: aa548f97cfe05a3f received MsgVoteResp from aa548f97cfe05a3f at term 7830

2020-07-16 09:31:21.660877 I | raft: aa548f97cfe05a3f [logterm: 7109, index: 66329229] sent MsgVote request to 9585571875104827 at term 7830

2020-07-16 09:31:21.660890 I | raft: aa548f97cfe05a3f [logterm: 7109, index: 66329229] sent MsgVote request to f800f3113703eccb at term 7830

2020-07-16 09:31:21.664434 I | embed: rejected connection from "" (error "tls: failed to verify client's certificate: x509: certificate has expired or is not yet valid", ServerName "")

檢視節點

NAME           STATUS   ROLES    AGE    VERSION
bj-github-1   Ready    master   365d   v1.15.0
bj-github-2   Ready    master   365d   v1.15.0
bj-github-3    Ready    master   349d   v1.15.0
bj-github-4    Ready    master   349d   v1.15.0
bj-github-5    Ready    master   349d   v1.15.0
bj-github-6    Ready    master   365d   v1.15.0
bj-github-7    Ready    master   365d   v1.15.0
bj-github-8    Ready    master   365d   v1.15.0
bj-github-9    Ready    master   365d   v1.15.0
           

檢視舊證書,果然過期了

https://github.com/kubernetes/kubeadm/issues/581 官方也有類似的issue

openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep ' Not '
            Not Before: Jul 17 08:49:08 2019 GMT
            Not After : Jul 16 08:49:08 2020 GMT
           

按操作流程,備份原證書,建立新證書,但,

kubeadm alpha phase certs apiserver

指令不存在

kubeadm alpha phase certs apiserver --apiserver-advertise-address ${MASTER_API_SERVER_IP}
           

猜測是kubeadm的版本不一緻,一級一級檢視指令說明,存在certs子指令

kubeadm alpha --help
kubeadm alpha --help
Kubeadm experimental sub-commands
Usage:
  kubeadm alpha [command]

Available Commands:
  certs       Commands related to handling kubernetes certificates
  kubeconfig  Kubeconfig file utilities
  kubelet     Commands related to handling the kubelet
  selfhosting Make a kubeadm cluster self-hosted
           
kubeadm alpha certs --help
kubeadm alpha certs --help
Commands related to handling kubernetes certificates

Usage:
  kubeadm alpha certs [command]

Aliases:
  certs, certificates

Available Commands:
  certificate-key  Generate certificate keys
  check-expiration Check certificates expiration for a Kubernetes cluster
  renew            Renew certificates for a Kubernetes cluster
           
kubeadm alpha certs renew --help
This command is not meant to be run on its own. See list of available subcommands.
Usage:
  kubeadm alpha certs renew [flags]
  kubeadm alpha certs renew [command]
Available Commands:
  admin.conf               Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself
  all                      Renew all available certificates
  apiserver                Renew the certificate for serving the Kubernetes API
  apiserver-etcd-client    Renew the certificate the apiserver uses to access etcd
  apiserver-kubelet-client Renew the certificate for the API server to connect to kubelet
  controller-manager.conf  Renew the certificate embedded in the kubeconfig file for the controller manager to use
  etcd-healthcheck-client  Renew the certificate for liveness probes to healthcheck etcd
  etcd-peer                Renew the certificate for etcd nodes to communicate with each other
  etcd-server              Renew the certificate for serving etcd
  front-proxy-client       Renew the certificate for the front proxy client
  scheduler.conf           Renew the certificate embedded in the kubeconfig file for the scheduler manager to use
           

執行

kubeadm alpha certs renew all

即可

驗證新證書

openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -text |grep ' Not '
            Not Before: Jul 17 08:49:08 2019 GMT
            Not After : Jul 16 10:05:36 2021 GMT
           

end