以下是针对 kube-state-metrics 组件版本升级( 1.9.7 – 2.7.0)带来的指标变化以及兼容方案的说明。
相关产品
Prometheus 监控服务 产品页,含版本规格、价格与购买入口。
一、serviceaccount 相关指标
| kube_serviceaccount_info |
| kube_serviceaccount_created |
| kube_serviceaccount_deleted |
| kube_serviceaccount_secret |
| kube_serviceaccount_image_pull_secret |
| kube_serviceaccount_annotations |
| kube_serviceaccount_labels |
二、clusterrole 相关指标
| kube_clusterrole_annotations |
| kube_clusterrole_labels |
| kube_clusterrole_info |
| kube_clusterrole_created |
| kube_clusterrole_metadata_resource_version |
三、role 相关指标
| kube_role_annotations |
| kube_role_labels |
| kube_role_info |
| kube_role_created |
| kube_role_metadata_resource_version |
四、rolebinding 相关指标
| kube_rolebinding_annotations |
| kube_rolebinding_labels |
| kube_rolebinding_info |
| kube_rolebinding_created |
| kube_rolebinding_metadata_resource_version |
五、certificatesigningrequest 相关指标
| kube_certificatesigningrequest_annotations |
| kube_certificatesigningrequest_created |
| kube_certificatesigningrequest_condition |
| kube_certificatesigningrequest_labels |
| kube_certificatesigningrequest_cert_length |
六、ingressclass 相关指标
| kube_ingressclass_annotations |
| kube_ingressclass_info |
| kube_ingressclass_labels |
| kube_ingressclass_created |
七、其他指标
| kube_daemonset_annotations |
| kube_configmap_annotations |
| kube_configmap_labels |
| kube_cronjob_annotations |
| kube_cronjob_status_last_successful_time |
| kube_cronjob_metadata_resource_version |
| kube_cronjob_spec_successful_job_history_limit |
| kube_cronjob_spec_failed_job_history_limit |
| kube_deployment_annotations |
| kube_deployment_status_replicas_ready |
| kube_endpoint_annotations |
| kube_endpoint_address_not_ready |
| kube_endpoint_address_available |
| kube_endpoint_ports |
| kube_endpoint_address |
| kube_horizontalpodautoscaler_info |
| kube_horizontalpodautoscaler_annotations |
| kube_horizontalpodautoscaler_status_target_metric |
| kube_ingress_annotations |
| kube_ingress_info |
| kube_job_annotations |
| kube_namespace_annotations |
| kube_networkpolicy_annotations |
| kube_poddisruptionbudget_annotations |
| kube_poddisruptionbudget_labels |
| kube_replicaset_annotations |
| kube_secret_annotations |
| kube_statefulset_annotations |
| kube_statefulset_status_replicas_available |
| kube_statefulset_status_replicas_available |
| kube_storageclass_annotations |
八、变更指标
| 变更前 |
变更后 |
| kube_pod_container_resource_requests_cpu_cores |
kube_pod_container_resource_requests{resource="cpu", unit="core"} |
| kube_pod_container_resource_limits_cpu_cores |
kube_pod_container_resource_limits{resource="cpu", unit="core"} |
| kube_pod_container_resource_requests_memory_bytes |
kube_pod_container_resource_requests{resource="memory", unit="byte"} |
| kube_pod_container_resource_limits_memory_bytes |
kube_pod_container_resource_limits{resource="memory", unit="byte"} |
| kube_node_status_capacity_pods |
kube_node_status_capacity{resource="pods", unit="integer"} |
| kube_node_status_capacity_cpu_cores |
kube_node_status_capacity{resource="cpu", unit="core"} |
| kube_node_status_capacity_memory_bytes |
kube_node_status_capacity{resource="memory", unit="byte"} |
| kube_node_status_allocatable_pods |
kube_node_status_allocatable{resource="pods", unit="integer"} |
| kube_node_status_allocatable_cpu_cores |
kube_node_status_allocatable{resource="cpu", unit="core"} |
| kube_node_status_allocatable_memory_bytes |
kube_node_status_allocatable{resource="memory", unit="byte"} |
九、兼容方案
目前已将变更指标中的8个免费指标添加入指标列,如存在旧实例升级的需求,可以参考如下两种方案,视需求选择一种即可:
十、采集解决方案
如集群较少,可以直接修改集群内的 kube-state-metrics 采集任务,名称为 kube-system/kube-state-metrics,类型为 servicemonitor,在 spec.endpoints[0].metricRelabelings 下面添加如下 relabel 规则,将新指标替换为旧指标。
– action: replace regex: kube_pod_container_resource_requests;cpu;core targetLabel: __name__ replacement: kube_pod_container_resource_requests_cpu_cores sourceLabels: [__name__, resource, unit] – action: replace regex: kube_pod_container_resource_limits;cpu;core targetLabel: __name__ replacement: kube_pod_container_resource_limits_cpu_cores sourceLabels: [__name__, resource, unit] – action: replace regex: kube_pod_container_resource_requests;memory;byte targetLabel: __name__ replacement: kube_pod_container_resource_requests_memory_bytes sourceLabels: [__name__, resource, unit] – action: replace regex: kube_pod_container_resource_limits;memory;byte targetLabel: __name__ replacement: kube_pod_container_resource_limits_memory_bytes sourceLabels: [__name__, resource, unit] – action: replace regex: kube_node_status_capacity;cpu;core targetLabel: __name__ replacement: kube_node_status_capacity_cpu_cores sourceLabels: [__name__, resource, unit] – action: replace regex: kube_node_status_capacity;memory;byte targetLabel: __name__ replacement: kube_node_status_capacity_memory_bytes sourceLabels: [__name__, resource, unit] – action: replace regex: kube_node_status_allocatable;cpu;core targetLabel: __name__ replacement: kube_node_status_allocatable_cpu_cores sourceLabels: [__name__, resource, unit] – action: replace regex: kube_node_status_allocatable;memory;byte targetLabel: __name__ replacement: kube_node_status_allocatable_memory_bytes sourceLabels: [__name__, resource, unit]
十一、预聚合解决方案
如实例中绑定了多个集群,不便于修改每个集群中的 kube-state-metrics 的采集配置时,可以考虑如下预聚合的方式,还原旧指标。
注意:如下预聚合规则中的间隔 interval 要和采集任务对齐,如用户无任何修改则默认为15s。
name: kube-state-metrics-compatibility.rulesinterval: 15srules: – expr: kube_pod_container_resource_requests{resource="cpu", unit="core"} record: kube_pod_container_resource_requests_cpu_cores – expr: kube_pod_container_resource_limits{resource="cpu", unit="core"} record: kube_pod_container_resource_limits_cpu_cores – expr: kube_pod_container_resource_requests{resource="memory", unit="byte"} record: kube_pod_container_resource_requests_memory_bytes – expr: kube_pod_container_resource_limits{resource="memory", unit="byte"} record: kube_pod_container_resource_limits_memory_bytes – expr: kube_node_status_capacity{resource="cpu", unit="core"} record: kube_node_status_capacity_cpu_cores – expr: kube_node_status_capacity{resource="memory", unit="byte"} record: kube_node_status_capacity_memory_bytes – expr: kube_node_status_allocatable{resource="cpu", unit="core"} record: kube_node_status_allocatable_cpu_cores – expr: kube_node_status_allocatable{resource="memory", unit="byte"} record: kube_node_status_allocatable_memory_bytes
如果预聚合如无数据,请检查是否有采集对应新指标。默认采集覆盖这两个指标的功能目前正在发布,存量可能存在未采集的情况。