容器開啟資料服務之旅系列（四）
Kubernetes QoS 助力線上運用與大資料離線運用的帶寬控制和磁盤控制
概述
本文是2018年大資料峰會上的一些分享，關于線上業務，離線業務在ACK（阿裡雲容器服務Kubernetes）的平台上通過對bandwidth, disk quota的靈活組合完成線上，離線業務場景的混合部署，來提高總體資源的使用率，以及帶寬，本地盤資源的動态配置設定調整，來控制離線，線上資源水位。
離線大資料運算帶寬控制
QoS for bandwidth
annotations:
kubernetes.io/egress-bandwidth: 10M
kubernetes.io/ingress-bandwidth: 5M 命名空間隔離和流量資源控制
ingress-bandwidth: 5M
egress-bandwidth: 10M Pod級别的流量控制
ingress-bandwidth: 10M
egress-bandwidth: 10M Name: batch
Labels: <none>
Annotations: kubernetes.io/egress-bandwidth=10M
kubernetes.io/ingress-bandwidth=5M
Status: Active
Resource Quotas
Name: quota
Resource Used Hard
-------- --- ---
configmaps 0 100
cpu 100m 4
memory 256Mi 32Gi
persistentvolumeclaims 0 100
pods 1 100
replicationcontrollers 0 10
requests.storage 0 1024G
secrets 1 100
services 0 10
Resource Limits
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container cpu - - 100m 1 -
Container memory - - 256Mi 16Gi -![image] 帶寬限制測試
[ 4] 0.00-10.00 sec 93.7 MBytes 78.6 Mbits/sec receiver
離線大資料運算本地磁盤用量控制
QoS for rootfs
limits:
ephemeral-storage: 100Mi
requests:
ephemeral-storage: 100Mi 命名空間隔離和rootfs disk資源控制
limits:
- default:
cpu: 1
memory: 16Gi
ephemeral-storage: 100Mi
defaultRequest:
cpu: 100m
memory: 256Mi
ephemeral-storage: 100Mi
type: Container Pod級别的rootfs disk控制
limits:
ephemeral-storage: 100Mi
requests:
ephemeral-storage: 100Mi 本地磁盤用量限制
apiVersion: v1
kind: LimitRange # set default request/limit for your containers
metadata:
name: limits
# namespace: users-namespace # specify your namespace
spec:
limits:
- default:
cpu: 1
memory: 16Gi
ephemeral-storage: 100Mi
defaultRequest:
cpu: 100m
memory: 256Mi
ephemeral-storage: 100Mi
type: Container -
在容器内部測試本地盤限制
