#prometheus (2022-11)

prometheus

Archive: https://archive.sweetops.com/prometheus/

2022-11-07

ghostface avatar
ghostface
01:38:56 PM

i’m using the kube-prometheus-stack chart and i’m trying to get my pods labels into the metrics.

see my pod labels below.

Labels:       app=xx-failing-nginx
              app.kubernetes.io/instance=xx-failing-nginx
              app.kubernetes.io/name=xx-failing-nginx
              xx.net/dataclassification=confidential
              xx.net/environment=uat
              xx.net/networkposition=internal
              xx.net/owner=edge
              xx.net/priority=P3
              xx.net/product=hft
              xx.net/service=xx-failing
              env=uat
              helm.sh/chart=xx-service-0.0.33

i added the below to the values file for the chart, in order to bring all of the labels from the pod into the metrics.

kube-state-metrics:
  prometheus:
    monitor:
      relabelings:
        - action: labelmap
          regex: __meta_kubernetes_pod_label_(.+)

but this hasn’t worked…the only difference it’s made is the below:

kube_pod_container_status_waiting_reason{app_kubernetes_io_component="metrics", app_kubernetes_io_instance="kube-prometheus-stack", app_kubernetes_io_managed_by="Helm", app_kubernetes_io_name="kube-state-metrics", app_kubernetes_io_part_of="kube-state-metrics", app_kubernetes_io_version="2.6.0", container="app", endpoint="http", helm_sh_chart="kube-state-metrics-4.20.2", instance="10.1.1.xx:xx", job="kube-state-metrics", namespace="uat", pod="xx-failing-nginx-5cb58d8977-8r46t", pod_template_hash="646b9498dd", reason="CrashLoopBackOff", release="kube-prometheus-stack", service="kube-prometheus-stack-kube-state-metrics"}

it seems to have added only labels and the values from the kube-state-metrics service monitor instead of the labels from my pod above.

    keyboard_arrow_up