背景
在k8s里面搭建HPA时。HPA需要metric-server进行采集数据,在过程中猜到两个坑,经过最后的折腾总算是解决了问题。所以记录一下方便其他人在遇到问题时可以快速解决。
metric-server 修改日志级别查看问题
在metric-server搭建完成后,可以通过 kubectl logs -f podName 查看metric-server输出的日志。但是有的时候查看日志看不出来问题的话。可以调整一下日志级别。这样可以输出更多有信息的日志。如下图
–v 表示日志级别。数字越大,输出的日志越多。
![](https://img.laitimes.com/img/__Qf2AjLwojIjJCLyojI0JCLiAzNfRHLGZkRGZkRfJ3bs92YsYTMfVmepNHLzkleOdXRq1EMJpHW4Z0MMBjVtJWd0ckW65UbM5WOHJWa5kHT20ESjBjUIF2X0hXZ0xCMx81dvRWYoNHLrdEZwZ1Rh5WNXp1bwNjW1ZUba9VZwlHdssmch1mclRXY39CXldWYtlWPzNXZj9mcw1ycz9WL49zZuBnL5ITN3UTN1IjM3ITMxAjMwIzLc52YucWbp5GZzNmLn9Gbi1yZtl2Lc9CX6MHc0RHaiojIsJye.png)
HPA not able to find CPU 问题排查
当HPA的event里面有如下输出日志的时候,这种问题是HPA的target的Deployment 管理的Pod没有设置CPU的request。设置就好了。
kubectl describe hpa hpa-demo
Name: hpa-demo
Namespace: default
Labels: <none>
Annotations: <none>
CreationTimestamp: Tue, 19 Nov 2019 17:14:56 +0800
Reference: Deployment/hpa-demo
Metrics: ( current / target )
resource cpu on pods (as a percentage of request): <unknown> / 10%
Min replicas: 1
Max replicas: 10
Deployment pods: 1 current / 0 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale
ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: missing request for cpu
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetResourceMetric 14s (x4 over 60s) horizontal-pod-autoscaler missing request for cpu
Warning FailedComputeMetricsReplicas 14s (x4 over 60s) horizontal-pod-autoscaler invalid metrics (1 invalid out of 1), first error is: failed to get cpu utilization: missing request for cpu