#helmfile (2020-04)

https://github.com/roboll/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles Archive: https://archive.sweetops.com/helmfile/

2020-04-08

Vadim Bauer avatar
Vadim Bauer

I have a question, the sample helmfile reference on https://github.com/roboll/helmfile#configuration do the values for the keys represent defaults?

or where can I find all the defaults values? if they aren’t mentioned in the example? eg:

 verify: true
    # wait for k8s resources via --wait. Defaults to `false`
    wait: true
    # time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, and waits on pod/pvc/svc/deployment readiness) (default 300)
    timeout: 60
    # performs pods restart for the resource if applicable
    recreatePods: true
    # forces resource update through delete/recreate if needed
    force: true
    # set `false` to uninstall on sync
    installed: true
    # restores previous state in case of failed release
    atomic: true
    # when true, cleans up any new resources created during a failed release
    cleanupOnFail: false

its kind of confusing especially with the comments in

helmDefaults:

\# defaults for verify, wait, force, timeout and recreatePods under releases[]
  verify: true
  wait: true
  timeout: 600
  recreatePods: true
  force: true

@mumoshu do you have a reference?

mumoshu avatar
mumoshu

well sorry i dont fully understand your question.

yes, its basically default values for each entry under releases[]

Vadim Bauer avatar
Vadim Bauer

ok, the question is what are the defaults for each section?

for

helmDefaults

and for

releases
mumoshu avatar
mumoshu

ah okay i believe we don’t have a refence for that

Vadim Bauer avatar
Vadim Bauer

because I have the feeling the eg.

releases:
 atomic: true #default is actually false
mumoshu avatar
mumoshu

yeah, it is atomic: false by default

mumoshu avatar
mumoshu

it mostly correspond to helm flags with the similar names

Vadim Bauer avatar
Vadim Bauer

can I make a pull request and start adding defaults? because I was under the impression that the values are all defaults. especially for ( all booleans)

mumoshu avatar
mumoshu

so the default would be no flags passed to helm

Vadim Bauer avatar
Vadim Bauer

ok, so the reference here would be Helm and not Helmfile right?

mumoshu avatar
mumoshu

which means

helmDefaults:
  #tillerNamespace:  
  tillerless: false
  kubeContext: ""
  cleanupOnFail: false
  args:
  verify: false
  wait: false
  timeout: 300
  recreatePods: false
  force: false
  tls: false
  #tlsCACert: "$HELM_HOME/ca.pem"
  #tlsCert: "$HELM_HOME/cert.pem"
  #tlsKey: "$HELM_HOME/key.pem"
  historyMax: 10
mumoshu avatar
mumoshu


ok, so the reference here would be Helm and not Helmfile right?
correct. and translating helm defaults would produce something like https://sweetops.slack.com/archives/CE5NGCB9Q/p1586342938045600?thread_ts=1586342576.043500&cid=CE5NGCB9Q

which means

helmDefaults:
  tillerNamespace: tiller-namespace  #dedicated default key for tiller-namespace
  tillerless: false                  #dedicated default key for tillerless
  kubeContext: kube-context          #dedicated default key for kube-context (--kube-context)
  cleanupOnFail: false               #dedicated default key for helm flag --cleanup-on-fail
  # additional and global args passed to helm
  args:
    - "--set k=v"
  verify: false
  wait: false
  timeout: 300
  recreatePods: false
  force: true
  # enable TLS for request to Tiller
  tls: true
  # path to TLS CA certificate file (default "$HELM_HOME/ca.pem")
  tlsCACert: "path/to/ca.pem"
  # path to TLS certificate file (default "$HELM_HOME/cert.pem")
  tlsCert: "path/to/cert.pem"
  # path to TLS key file (default "$HELM_HOME/key.pem")
  tlsKey: ""
  # limit the maximum number of revisions saved per release. Use 0 for no limit (default 10)
  historyMax: 10
Vadim Bauer avatar
Vadim Bauer

could we update the docs with that values

mumoshu avatar
mumoshu

how would you update it? just replacing the whole helmDefaults in the example?

Vadim Bauer avatar
Vadim Bauer

it would make life easyier fo so many just by looking at your example. i take the example as a reference

mumoshu avatar
mumoshu

okay that makes sense

mumoshu avatar
mumoshu

but would you also leave the information about how you could customize it

Vadim Bauer avatar
Vadim Bauer

yes, or making comments eg-

 historyMax: 1 # dafaults 10

mumoshu avatar
mumoshu

like


\# When set to true, enable TLS for request to Tiller
  tls: false
mumoshu avatar
mumoshu

yeah that would work too

Vadim Bauer avatar
Vadim Bauer

for me and my team mates this file is the reference, i usually only look at this one: If I need more I look at the description If I am still clueless I look at helm.sh docs

mumoshu avatar
mumoshu

if you prefer the uncommented line to denote a default, im fine with

```

# Number of hostorical releases to be retained in the cluster. Defaults to 10. Set it to e.g. 1 if you want only 1 release to be retained historyMax: 10

Vadim Bauer avatar
Vadim Bauer

in this order, that why i call it the reference

Vadim Bauer avatar
Vadim Bauer

this is also fine

mumoshu avatar
mumoshu

got it. that makes sense

mumoshu avatar
mumoshu

would you mind giving it a shot?

Vadim Bauer avatar
Vadim Bauer

ok,

Vadim Bauer avatar
Vadim Bauer

i’ll make a PR wait

mumoshu avatar
mumoshu

i mean would you mind submitting a pr for that as the foundation

mumoshu avatar
mumoshu

thanks a lot!

Vadim Bauer avatar
Vadim Bauer
update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

Vadim Bauer avatar
Vadim Bauer

There are some inconsistencies in the docs. Some defaults are written on separate line some other behind the values. I added defaults as a comment behind the values.

However I am open to make adaptions.

An alternative might be.

  1. if a comment above KV exist add the default in there.
  2. If no comment exist append default in the same line as the value
Vadim Bauer avatar
Vadim Bauer

What do you think @mumoshu? or is it ok as is?

mumoshu avatar
mumoshu

I prefer the option 1 and making it consistent in all places(incl. helmDefaults and releases[]) if possible

mumoshu avatar
mumoshu

thanks anyway! just reviewed it

Vadim Bauer avatar
Vadim Bauer

@mumoshu so only 1. not the combination of 1+2?

mumoshu avatar
mumoshu

yes that’s my intention.

just curious, but do you have any motivation to mix 1 and 2? (i had no intention to do that myself - i just didn’t read that part of readme that much these days

Vadim Bauer avatar
Vadim Bauer


2. If no comment exist append default in the same line as the value
I would put the comments and explanation into helmDefaults and keep the release clean as much as possible

Vadim Bauer avatar
Vadim Bauer

also the defaults in releasedepend on helmDefaults if you dont set wait it will fallback to helmDefault

adding a default in there might be not correct as it depends on helmDefaults

mumoshu avatar
mumoshu

agreed. so how about only doing the option 1 for helmDefaults and removing comments for defaults under releases[]?

:--1:1
mumoshu avatar
mumoshu

I mean comments in this part of the example does look like confusing:

https://github.com/roboll/helmfile/pull/1189/files#diff-04c6e90faac2675aa89e2176d2eec7d8R156-R160

update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

mumoshu avatar
mumoshu
update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

mumoshu avatar
mumoshu
update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

mumoshu avatar
mumoshu

https://github.com/roboll/helmfile/pull/1189/files#diff-04c6e90faac2675aa89e2176d2eec7d8L151 should be just wait for k8s resources via --wait..

Defaults to false` in the end of the line looks confusing

update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

Vadim Bauer avatar
Vadim Bauer

ok.

Vadim Bauer avatar
Vadim Bauer

I guess I understand. let me give it another try

mumoshu avatar
mumoshu

thank you so much!

Vadim Bauer avatar
Vadim Bauer

regarding https://github.com/roboll/helmfile/pull/1189/files#diff-04c6e90faac2675aa89e2176d2eec7d8L149 I would prefer to have the comment in helmDefault section, as the values also references to helmDefault. This way there is we can keep it a bit DRY

Vadim Bauer avatar
Vadim Bauer

@mumoshu what do you think now?

mumoshu avatar
mumoshu

the reference is the readme section you linked

2020-04-07

2020-04-06

2020-04-05

Jim avatar

Hi. I’ve been struggling with a problem that I hope you guys may be able to help with. I need a way for a values file used by multiple releases to substitute different values, based on what the release is.

Jim avatar

E.g. I might have a helmfile.yaml that contains:

releases:
    - name: app-init
      namespace: app
      chart: ./helm/chart/app-init
      installed: true
      values:
        - helm/vars/app.yaml.gotmpl

    - name: app
      namespace: app
      chart: ./helm/chart/app
      installed: true
      values:
        - helm/vars/app.yaml.gotmpl
Jim avatar

And helm/vars/app.yaml.gotmpl may contain something like the following (that doesn’t work):

configFiles:
  environment_file: |
    {{- if eq .Release.Name "app-init" }
    DB_USER={{ requiredEnv "ADMIN_DB_USER" }}
    DB_PASS={{ requiredEnv "ADMIN_DB_PASS" }}
    {{- else }}
    DB_USER={{ requiredEnv "DB_USER" }}
    DB_PASS={{ requiredEnv "DB_PASS" }}
    {{- end }}
Jim avatar

The intention above being to populate some values using different environment variables depending on which release is going out. The values file is unable to access .Release.Name and so the above doesn’t work.

I’ve also tried changing the values: file in the release example above to something like:

      values:
        - HelmReleaseName: app-init
        - helm/vars/app.yaml.gotmpl

and:

      values:
        - HelmReleaseName: app
        - helm/vars/app.yaml.gotmpl

And changing the test in the values file to be:

{{- if eq .Values.HelmReleaseName "app-init" }}

But again this doesn’t work.

Jim avatar

Is there a way to have a test in the values file be able to see something unique based on the release being installed?

mumoshu avatar
mumoshu

@ Release.Name isn’t available within the gotmpl file. It’s only available in helmfile.yaml template. This is to avoid misuses(like “hiding” the release specific values)

And - you don’t need to complicate things with the if-else for your specific example.

I’d suggest this:

releases:
    - name: app-init
      namespace: app
      chart: ./helm/chart/app-init
      installed: true
      values:
        - helm/vars/app-init.yaml.gotmpl

    - name: app
      namespace: app
      chart: ./helm/chart/app
      installed: true
      values:
        - helm/vars/app.yaml.gotmpl

where app-init.yaml.gotmpl is:

configFiles:
  environment_file: |
    DB_USER={{ requiredEnv "ADMIN_DB_USER" }}
    DB_PASS={{ requiredEnv "ADMIN_DB_PASS" }

app.yaml.gotmpl:

configFiles:
  environment_file: |
    DB_USER={{ requiredEnv "DB_USER" }}
    DB_PASS={{ requiredEnv "DB_PASS" }}
Jim avatar

Thanks @mumoshu but that isn’t going to work for us too well :disappointed: I’ve simplified the configuration a lot for this example. The values files (of which there are multiple being passed) contain large configuration files that will end up in Kubernetes secrets and directly mounted into the file-system within the containers. I’m trying to avoid having to duplicate those configuration files multiple times(in app-init.yaml.gotmpl and again in app.yaml.gotmpl)

mumoshu avatar
mumoshu

So you have a large duplication within the single string values(configFiles.environment_file) for app and app-init?

Jim avatar

Yeah (for this example, again it doesn’t really match reality). We have a number of configuration files below configFiles: and each one is for a different component (PHP, Apache, the Application itself) and some of those configuration files are quite large. We have about 5 variables that are used (sometimes once, sometimes across multiple files) that need to differ for one release vs the other, where both releases use the same configuration files. So I was trying to solve that by using some helmfile conditional logic as part of templating out the config files.

mumoshu avatar
mumoshu

How about aggregating all the common env vars into a single yaml file common.env and reusing it?

app-init.yaml.gotmpl:

configFiles:
  environment_file: |
    DB_USER={{ requiredEnv "ADMIN_DB_USER" }}
    DB_PASS={{ requiredEnv "ADMIN_DB_PASS" }
    {{ readFile "common.env" | nindent 4 }}

app.yaml.gotmpl

configFiles:
  environment_file: |
    DB_USER={{ requiredEnv "DB_USER" }}
    DB_PASS={{ requiredEnv "DB_PASS" }}
    {{ readFile "common.env" | nindent 4 }}
mumoshu avatar
mumoshu

the more control syntax you introduce, the more your config template gets unreadable/buggy.

so i would prefer avoiding using if whenever possible

mumoshu avatar
mumoshu

otherwise - filing a feature request to add support for Release.Name within values.yaml.gotmpl might be needed

Jim avatar

Ok. I’m not sure I can easily do what you’ve suggested, but I’ll give it a shot. Thanks for your help.

But the take-away is that currently there is no way me to test anything defined in the release form within the values file right?

Jim avatar

This looks like a related ticket: https://github.com/roboll/helmfile/issues/760

Unable to use .Release.Name in values.gotmpl · Issue #760 · roboll/helmfile

I have a use case where I want the .Release.Name to be used as a variable in a values.gotmpl file. However, no matter what I do, the values.gotmpl file will not render .Release.Name. Is this expect…

mumoshu avatar
mumoshu

yes. i thought i intentionally decided to not implement that because it seemed to become a maintainance/user support nightmare

mumoshu avatar
mumoshu

awesome! I couldn’t recall that

Jim avatar

If I have trouble implementing the solution you’ve got above, would you consider allowing us to shoot ourselves in the foot?

mumoshu avatar
mumoshu

do you mean you still need .Release.Name within values.yaml.gotmpl?

mumoshu avatar
mumoshu

if so - yeah i would definitely consider implementing it. i think that feature is requested too many times to be ignored

Jim avatar

Yeah, if I can’t break up the configuration files like you’ve shown above, it would be nice if I could test for .Release.Name as an alternative.

mumoshu avatar
mumoshu

undestood. would you mind adding your comment on the issue saying “i need this too”?

Jim avatar

Yes, I’ll do that now. Thank you for your help.

mumoshu avatar
mumoshu

thanks for your support too!

Andrey Nazarov avatar
Andrey Nazarov

@mumoshu , btw, does double render work for values.yaml.gotmpl? Like {{ {{ .Release.Name }} }} . I always get confused with it.

mumoshu avatar
mumoshu

unfortunately no

Jim avatar

I attempted that.

mumoshu avatar
mumoshu

and double rendering only render helmfile.yaml twice

mumoshu avatar
mumoshu

it doesn’t happen (and unnecessary) for values.yaml.gotmpl

Andrey Nazarov avatar
Andrey Nazarov

Ok, good to know. Thanks!

mumoshu avatar
mumoshu
{{` {{ ... }} `}}

is for release templates

mumoshu avatar
mumoshu

np!

mumoshu avatar
mumoshu
roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

Andrey Nazarov avatar
Andrey Nazarov

We’ve never used gotmpl values files, so I don’t know any details. But we are using this double-render trick in our main helmfile.yaml

mumoshu avatar
mumoshu

yes, that makes sense. you should avoid gotmpl values whenever possible.

Andrey Nazarov avatar
Andrey Nazarov

Yes, thanks). I’m aware of how it works for releases. But thank you anyway)

:--1:1
mumoshu avatar
mumoshu

(i got to think that the complication from release templates vs values go tmpl vs double rendering all coming from the fact that we’re abusing yaml templates too much…

mumoshu avatar
mumoshu

regarding that i’ve recently talked with @Erik Osterman (Cloud Posse) about a dream: hcl2-based dsl for helmfile

https://sweetops.slack.com/archives/CFFQ9GFB5/p1585873245019000

@mumoshu this begs the question - have you considered HCL2 for Helmfile?

Andrey Nazarov avatar
Andrey Nazarov

On the other hand, sometimes you can do some crazy things with helmfile)). But the readability suffers a lot)

:--1:1
Jim avatar

FYI. I got this technique to work. Oh but the horrible hacks I had to do in order to get it to work makes it pretty unreadable. Unfortunately the order of the content of the files in question mattered (they’re not just simple env files) and the variables were spread across them, and so the files had to be broken up into multiple parts. Having to change content of the file means finding its multiple parts, plus some bits in multiple helmfile values files. Think of a values file like:

configFiles:
  config_file_1: |
    {{- readFile "start_of_file" | nindent 4 }}
    VAR1={{ requiredEnv "X" }}
    VAR2={{ requiredEnv "Y" }}
    {{- readFile "bit_more_of_file" | nindent 4 }}
    VAR3={{ requiredEnv "Z" }}
    {{- readFile "end_of_file" | nindent 4 }}

  # The same sort of butchery in the next few config files too...
  config_file_2: | 

Then having to have lots of values files because there are different combinations of the VARs that can happen for different releases. So I ended up with lots of file “parts” and 4x values files to drive different releases that all used the same config files, but with the values varying.

Jim avatar

If .Release.Name was available, then I would just have 1 file per configuration file, with some statements like the following within it:

{{- if eq .Release.Name "X" }}
VAR1=a
VAR2=b
{{- else if .Release.Name "Y" }}
VAR1=x
VAR2=y
{{- else }}
VAR1=m
VAR2=n
{{- end }}

Although messy… I still think it’s heaps cleaner than what I had to do.

2020-04-02

2020-04-01

hari avatar

Hello Everyone, How can we differentiate the configmaps with different environments (Dev,Int,Prod). We don’t use the helmfile what is the recommended approach ? should we allow helm to create the configmaps by keeping the configmaps under template directory? for example if i keep the configmap-dev.yaml, configmap-int.yaml, configmap-prod.yaml under template directory it will try to create the configmap object for all 3 config files with same name as the spec will be as below

kind: ConfigMap
metadata:
name: envconfig-configmap-{{ .Release.Name }}
hari avatar

may i know recommended approach to sort it out please

    keyboard_arrow_up