使用Prometheus對系統進行監控，使用Grafana進行資料展示。我的環境是K8S，我的部署方式是通過 yaml檔案進行部署。已經部署檔案上傳到了Github 位址：https://github.com/zeyangli/devops-on-k8s.
1
部署node-exporter對叢集進行監控
部署node-expoter對k8s叢集監控
- 使用DaemonSet方式
- 使用最新鏡像：node-exporter:v0.18.1
部署描述檔案如下：
在這裡我建立了一個DaemonSet類型的部署，會在每個node節點上運作。運作的Namespace是在kube-system中。建立service 使用nodePort方式映射端口。
--- apiVersion: apps/v1 kind: DaemonSet metadata: name: node-exporter namespace: kube-system labels: k8s-app: node-exporter spec: selector: matchLabels: name: node-exporter template: metadata: labels: name: node-exporter spec: containers: - name: node-exporter image: prom/node-exporter:v0.18.1 ports: - containerPort: 9100 protocol: TCP name: http --- apiVersion: v1 kind: Service metadata: labels: k8s-app: node-exporter name: node-exporter namespace: kube-system spec: ports: - name: http port: 9100 nodePort: 31672 protocol: TCP type: NodePort selector: k8s-app: node-exporter 2
部署Prometheus服務
部署prometheus服務
deploy.yaml
這裡我建立了一個應用部署檔案deployment對象，挂載configmap配置檔案和volume存儲持久化資料。
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
name: prometheus-deployment
name: prometheus
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: prometheus
template:
metadata:
labels:
app: prometheus
spec:
containers:
- image: prom/prometheus:v2.15.2
name: prometheus
command:
- "/bin/prometheus"
args:
- "--config.file=/etc/prometheus/prometheus.yml"
- "--storage.tsdb.path=/prometheus"
- "--storage.tsdb.retention=24h"
ports:
- containerPort: 9090
protocol: TCP
volumeMounts:
- mountPath: "/prometheus"
name: data
- mountPath: "/etc/prometheus"
name: config-volume
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 500m
memory: 2500Mi
serviceAccountName: prometheus
volumes:
- name: data
hostPath:
path: /data/devops/prometheus
type: Directory
- name: config-volume
configMap:
name: prometheus-config
service.yaml
這裡我建立了一個service對象，nodePort方式暴露30003端口進行通路。
--- kind: Service apiVersion: v1 metadata: labels: app: prometheus name: prometheus namespace: kube-system spec: type: NodePort ports: - port: 9090 targetPort: 9090 nodePort: 30003 selector: app: prometheus
RBAC Setting（配置應用程式權限）
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: prometheus rules: - apiGroups: [""] resources: - nodes - nodes/proxy - services - endpoints - pods verbs: ["get", "list", "watch"] - apiGroups: - extensions resources: - ingresses verbs: ["get", "list", "watch"] - nonResourceURLs: ["/metrics"] verbs: ["get"] --- apiVersion: v1 kind: ServiceAccount metadata: name: prometheus namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: prometheus roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: prometheus subjects: - kind: ServiceAccount name: prometheus namespace: kube-system 3
部署Grafana服務
deployment
apiVersion: apps/v1 kind: Deployment metadata: name: grafana-core namespace: kube-system labels: app: grafana component: core spec: selector: matchLabels: app: grafana replicas: 1 template: metadata: labels: app: grafana component: core spec: containers: - image: grafana/grafana:6.5.3 name: grafana-core imagePullPolicy: IfNotPresent env: # The following env variables set up basic auth twith the default admin user and admin password. - name: GF_AUTH_BASIC_ENABLED value: "true" - name: GF_AUTH_ANONYMOUS_ENABLED value: "false" # - name: GF_AUTH_ANONYMOUS_ORG_ROLE # value: Admin # does not really work, because of template variables in exported dashboards: # - name: GF_DASHBOARDS_JSON_ENABLED # value: "true" readinessProbe: httpGet: path: /login port: 3000 # initialDelaySeconds: 30 # timeoutSeconds: 1 volumeMounts: - name: grafana-persistent-storage mountPath: /var volumes: - name: grafana-persistent-storage hostPath: path: /data/devops/grafana type: Directory service
apiVersion: v1 kind: Service metadata: name: grafana namespace: kube-system labels: app: grafana component: core spec: type: NodePort ports: - port: 3000 nodePort: 30011 selector: app: grafana component: core 4
通路驗證
我們先看下dashboard中pod運作狀态
然後看下pometheus中是否有資料
通路Grafana
到此我們就成功的将Prometheus+Grafana部署到了Kubernetes系統中。接下來我們來配置grafana展示叢集中的監控資料。
5
配置Grafana 面闆
添加資料源
導入面闆
https://grafana.com/grafana/dashboards/315
最終效果