#kubecost (2019-08)

https://www.kubecost.com/

Kubernetes resource and cost management

Archive: https://archive.sweetops.com/kubecost/

2019-08-19

Jan avatar

any one ever run into the cost-analyzer-frontend container spitting the following error ?

[emerg] host not found in upstream "cost-analyzer-grafana.default.svc.cluster.local" in /etc/nginx/conf.d/default.conf:11
Jan avatar

ah oki found teh issue

Jan avatar

I hand global.grafana.enabled: false yet was passing a custom domain

Jan avatar

mmm no thats not it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

do you have a service called cost-analyzer-grafana in the default namespace?

Jan avatar

we also dont use the built in Prometheus and grafana as we have our own running

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy G (Cloud Posse) has had pretty good success. we have it all up and running on our prometheus and grafana. @Ajay Tripathy @webb have been a huge help.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

our helmfiles are up to date

webb avatar

Thanks, Eric. @Jan, happy to connect if we can be useful!

Jan avatar

Thanks @Erik Osterman (Cloud Posse) I will take a look, @webb I will shout

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jan are you still using helmfile?

Jan avatar

Will need to check

Jan avatar

Tomorrow

Jan avatar

We don’t use helm file at all

2019-08-20

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)
04:06:49 PM

@Jeremy G (Cloud Posse) has joined the channel

2019-08-22

Jan avatar

So despite setting the grafana.domainName

Jan avatar

the nginx-conf config map still uses

upstream grafana {
        server cost-analyzer-grafana.default.svc.cluster.local;
Jan avatar

how do I change these settings from the chart?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

The settings are under global. You need to set

global:
  grafana:
    enabled: false
    domainName: <external domain name of your Grafana>
    scheme: <http or https for accessing your Grafana>

https://github.com/cloudposse/helmfiles/blob/103b1c3b5ab68b568307017f159f07431c07d8b3/releases/kubecost.yaml#L45-L48

cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jan

2019-08-23

Jan avatar

So I have tried those, at least in our case, I can’t seem to populate those parameters from passing arguments to the helm chart

Jan avatar

We deploy the chart passing a values.yaml file and additional override specific things (grafana address etc)

Jan avatar

By setting specific parameters as arguments

Jan avatar

Even after doing so if I edit the Nginx-conf config map I see the upstream grafana value isn’t set

webb avatar

@Jan, any chance you can share your values.yaml file? I can try to reproduce.

Jan avatar

I can do in a few hours

Jan avatar

When my kids are asleep

Jan avatar

Will need to show you how we are installing the chart

Jan avatar

so we are installing the chart via terraform and setting two parameters while installing as well as providing two values files

Jan avatar
data "helm_repository" "kubecost" {
  url  = "<https://kubecost.github.io/cost-analyzer>"
  name = "kubecost"
}

resource "helm_release" "kubecost" {
  name       = "kubecost"
  namespace  = "monitoring"
  chart      = "kubecost/cost-analyzer"
  repository = "${data.helm_repository.kubecost.name}"

  set {
    name  = "ingress.hosts[0]"
    value = "kubecost.${local.cluster_domain}"
  }

  set {
    name  = "grafana.domainName[0]"
    value = "grafana.${local.cluster_domain}"
  }

  values = [
    "${file("defaults.yaml")}",
    "${file("stage.yaml")}",
  ]
}
Jan avatar

so ingress.hosts works 100% as expected

Jan avatar

grafana.domainName doesn not as I would expect

Jan avatar

defaults.yaml:

global:
  prometheus:
    enabled: false
    fqdn: <http://promop-prometheus-operator-prometheus.monitoring.svc.cluster.local:9090>

  grafana:
    enabled: false
    scheme: "http"

  notifications:
    alertmanager:
      fqdn: <http://promop-prometheus-operator-alertmanager.monitoring.svc.cluster.local:9093>

# <https://kubecost.com/install>
# This token is not really secret and can be exchanged at any time
kubecostToken: xxxxxx
serviceMonitor:
  enabled: true

kubecost:
  limits:
    cpu: 300m
    memory: 128Mi

kubecostFrontend:
  limits:
    cpu: 30m
    memory: 56Mi

kubecostModel:
  limits:
    cpu: 400m
    memory: 128Mi

ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: traefik

grafana:
  sidecar:
    dashboards:
      enabled: true
    datasources:
      enabled: true
Jan avatar

stage.yaml is only used to enable slack notifications for specific accounts

Jan avatar
global:
  notifications:
    slack:
      enabled: true
      webhook: <https://hooks.slack.com/services/xxxxxx>
Jan avatar

that does almost not seem to be working currently

Jan avatar

well it doesn’t get set

Jan avatar

works if i set it via the kubecost site after

webb avatar

Thanks for sharing! You’re meaning to be using a ServiceMonitor correct? That looks like the only part I haven’t tested recently. @Jeremy G (Cloud Posse) actually has a similar config I believe.

Jan avatar

so everything else is working fine

Jan avatar

I just dont seem to be able to preset those parameters (slack and grafana address)

Jan avatar

that said I have not yet pulled the chart apart

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@Jan As I explained before, the setting you need is global.grafana.domainName but you are only setting grafana.domainName

1
Jan avatar

ah yea!

Jan avatar

which I should probably just do

Jan avatar

any idea on the slack notification where im going wrong?

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

The slack notification is a feature of AlertManager. If you are using your own Prometheus then you are also using your own AlertManager and have to configure Slack there.

Jan avatar

ah I see

Jan avatar

cheers mate

Jan avatar

much appreciated!

webb avatar

Jan, looks like you got this resolved but let me know if we can help with anything else! Our product can actually integrate directly with Slack too, but we have not exposed this integration via Helm yet.

Jan avatar

Thanks bro, I think I have everything yea

2019-08-26

2019-08-27

    keyboard_arrow_up