#helmfile (2019-05)

hello there, I am launching an helmfile apply on this

releases: 
- name: redis
  chart: stable/redis
  hooks:
    - events: ["cleanup"]
      command: "echo"
      args: ["foo bar"]

And expect to see “foo bar” on my console but nothing appears ?

what am I doing wrong ?

Do you guys want the new config bases: ["mybase.yaml"] added to helmfile, even though it won’t support templating, and yaml anchors across the border? https://github.com/roboll/helmfile/issues/388#issuecomment-489633839

Started to think we should require .gotpl extension to enable the whole-file templating of helmfile.yaml https://github.com/roboll/helmfile/pull/573#issuecomment-491142465

@SeB @Erik Osterman (Cloud Posse) I’m interested in your opinion!

Also cc/ @Shane

https://github.com/roboll/helmfile/issues/584

Wait I thought we currently required a extension

Yaml files don’t get templates

What am I missing

@Shane Yep, for values.yaml that’s true

I’m proposing the same for helmfile.yaml

Ahh

(I think we’ve discussed about that long before… but at that time we thought it’s better to just use .yaml as helm templates do the same

I typically don’t put anything in my helmfile except for chart references

Oh!

Hence never thinking about the templating of helmfile

Do you use another tools to make it work against various envs, repositories, and so on?

I keep pushing our teams to leverage it as little as possible if we control the chart

i.e. generating helmfile.yaml with a tool like jsonnet

All values are kept in separate values files

So helmfile is a list of charts and value references

that’s a neat idea

so you maintain one helmfile.yaml per environment

Almost yes

In dev we have a helmfile per team also

makes sense!

i think the helmfile.yaml template is used (1)when people want to create a single helmfile.yaml that is the entrypoint to all the flavors of environments or (2)make each helmfile.yaml DRY when one has to maintain many of them, without introducing another tools like jsonnet

so use-case like yours completely make sense to me

Ya tons of options

Hello, @mumoshu 1) could you explain in detail why we can’t consider helmfile.yaml as a template file and why we need to have both extensions. 2) Also why not using the same extension as values (gotmpl vs gotpl) or vise-versa otherwise this will be confusing.

For 1), it doesn’t work well with release templating.

For 2), gotmpl and gotpl should be merged. I think I meant gotmpl

An example related to 1) can be seen at https://github.com/roboll/helmfile/pull/573

releases:
  - name: redis-test
    chart: stable/redis
    values:
    - fake_key:
        redis_pass_east: {{ ssm "east1:redis_password" }}

{{ ssm "east1:redis_password" }} is evaluated twice in the above example.

Another example for release templates, is that you can’t write:

releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

It must be:

releases:
  - name: redis-test
    values:
    - releases/{{` {{ .Release.Name }} `}}/values.yaml

Due to that helmfile.yaml itself is a go template.

thank @mumoshu for your quick answer, but change the extension to gotmpl whould change what in the above examples ?

@SeB What I suggested was to “add” .gotmpl ext. to explicitly enable helmfile.yaml templating. And release templates are always enabled. That being said

but why not consider always helmfile.yaml a templates ?

Making helmfile require .gotmpl to enable helmfile.yaml templating allows you to write a regular YAML file named helmfile.yaml

So that

releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

THis is a valid helmfile.yaml with just release templaets

Note that it isn’t a valid helmfile.yaml if helmfile.yaml itself is a go template

This happens because there’s two template contexts. When templating helmfile.yaml as a whole, you have no access to {{ .Release }}

When templating helmfile.yaml as a whole, we have no {{ .Release }} in the template context available, because helmfile.yaml is basically a template that “produces YAML” . No way to know what releases are defined in the helmfile.yaml before rendering and parsing it as YAML…

ok, I think I got it

And no way to know which {{ .Release }} it is rendering while templating helmfile.yaml as a whole

// {{ .Release }} becomes available only when helmfile iterates over each release defined in the YAML

so what you propose is to have 2 extensions : 1) helmfile.yaml

releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

2) helmfie.gotmpl

releases:
  - name: redis-test
    values:
    - releases/{{` {{ .Release.Name }} `}}/values.yaml

to have the same result

Exactly!

I am afraid this is too complicated for the end-user.

Yep it is

I’m thinking that it is already complicated and we’d better migrate towards explicit file ext. towards helmfile v1

so that it becomes simpler

I beleive this could be just much better explained in the documentation, this is what it is lacking today. The of precise documentation make pleople to hack/try/fail to have their helmfile working

For advanced users I’d still suggest using helmfile.gotmpl. But as I’ve explained in the issue, you can fully achieve the same result even with helmfile.yaml without gotmpl, when combined with bases

Hm, maybe in that case I completely remove helmfile.yaml and only helmfile.gotmpl supported.

so that there’s no confusion anyway.

And the docs you’ve suggested would help people understand why one needs backticks sometimes

yep

Great! Thanks for all the feedbacks

So we can proceed that way

And discuss about re-introducing helmfile.yaml but as a regular YAML file this time separately in longer term, maybe after docs improvements, if it is really people want

Summarized our discussion in https://github.com/roboll/helmfile/issues/584#issuecomment-491177956

It is interesting that helmfile.yaml, rather than helmfile.gotmpl, started to look like for advanced users

is anyone successfully using helmfile with keel?

@Ufou Hey! Unfortunately I have never tried that myself. And I think you’d need to enhance keelhq a bit to “fully” make it work with helmfile

Keel updates your container images referenced by your apps by calling helm on its own.

I thought what you want would be keel calls helmfile to update container image tags of your apps. That’s not how keel works at the moment

Anyway, in case you really want keel just for updating your container image tags out of helmfile, just add keel-related values to your values.yaml managed by helmfile.

https://keel.sh/v1/guide/documentation.html#Helm-example

@mumoshu thanks! Yeah, I worked out as much, I just didn’t get it working yet and wondered if others were successful with it…

@Shane I hope you like this https://twitter.com/mumoshu/status/1128218333315100673

@mumoshu are you gonna make helmfile leverage helm-x to work with plain manifests or kustomize’ed stuff? I’m joking, but have you ever had these thoughts - to manage not only helm via a tool like helmfile. Actually, once I had a request about how to provide management of objects with one tool while some services were helmified, but others were not migrated yet.

@mumoshu keep up the good work

@mumoshu intriguing

Running helmfile from helm/account/dev


in ./helmfile.yaml: in .helmfiles[0]: in helm/account/helmfile.yaml: failed to read helmfile.yaml: helm exited with status 1:
  error loading config: no matching creation rules found
  Error: plugin "secrets" exited with error
 less ./helmfile.yaml                                                                                                                             
 less helm/account/helmfile.yaml                                                                                                                    
 ls -larth helm/account/helmfile.yaml                                                                                                               
-rw-r--r--  1 sstarcher  staff   4.5K May  8 17:50 helm/account/helmfile.yaml
 ls -larth ./helmfile.yaml                                                                                                                          
-rw-r--r--  1 sstarcher  staff   422B May 13 15:51 ./helmfile.yaml

so it’s the secrets plugin choking

Has something changed with the directory structure. It is not longer finding my .sops.yaml for sub helm charts.

nope… something on my end has had to of changed

So ya, probably certainly seems to be with secret decryption, but I can’t find out what would of changed. My .sops.yaml has always been at the root. Did it by chance change to run the secret decryption rooted from the helmfile instead of the root helmfile?

Ya, seems something has changed related to sops. I now require a .sops.yaml in each of my directories that have a helmfile.

Sops had a release 26 days ago so I also changed to the older sops release 3.2.0 from 2018. Both seem to have the same issue with helmfile.

Well… I can fix it by changing ../../_environment/secrets.yaml to ../../../dev/_environment/secrets.yaml, but that does not explain why it just started failing.

well I think I find a viable workaround… but then I caught things on even worse fire

helmfile version v0.60.1

well fixed the 2 errors i was getting - https://github.com/roboll/helmfile/pull/594

@mumoshu sounds great!

set the channel topic: https://github.com/roboll/helmfile

@mumoshu seeing some side effects from the new lockfile addition - https://github.com/roboll/helmfile/issues/598

While my trial on replicated/ship, I came up with how we can potentially leverage kustomize to patch helm releases, but still installing it as a helm chart.

https://github.com/mumoshu/helm-x/issues/2

I think we can extend this to helmfile, so that helmfile can be used for applying adhoc “patches” to a helm chart without forking. The patches can be used for injecting init/sidecar containers, modifying other pod spec, adding configmap/secret required but not included in the chart, etc…

Btw, the latest version of helmfile available via homebrew is v0.54.2.

@Andrew Nazarov @mumoshu Was just coming online to ping you about the homebrew version Are we going to see an update for it in the future? I’m asking before I manually install it from git and there are some breaking changes that make my life hell.

And a side (noobish) question: We’re trying to update some of our config maps via helm upgrade commands using --set but we’re unable to target the chart as it’s part of our helmfile. I’d normally use sync but I want to try a few direct changes. Am I being an idiot or would we have to move to individual chart files to be able to do this?

Hey,

How can I manage deployments of crd’s with helmfile? I am currently migrating parts of our infra to helmfile stored in a repo and I need to deploy cert-manager as well

does helmfile remove the need to manually bump chart versions etc in the repo?

you have to specify the chart version directly in the helmfile, either by hardcoding it (e.g. https://github.com/cloudposse/helmfiles/blob/master/releases/cert-manager.yaml#L30) , or using ENV var

thanks @Andriy Knysh (Cloud Posse)

@btai here’s an example of using the raw chart

https://github.com/cloudposse/helmfiles/blob/master/releases/external-storage.yaml

e.g. if you don’t have the chart for the CRDs, but still want to use helmfile

good morning, is it possible for helmfile to merge a map coming from environment yaml files so that I can take 2 maps and merge them together? eg if my values file looks something like:

default_config:
  resources:
    limits:
      cpu: 0.1
      memory: 200Mi
    requests:
      cpu: 0.1
      memory: 200Mi

service_config_a:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70
  resources:
    limits:
      cpu: 0.3
      memory: 400Mi
    requests:
      cpu: 0.3
      memory: 400Mi

service_config_b:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70

I’d like service_config_b to inherit values from default_config

Does using YAML anchor work?

default_config: &default_config
  resources:
    limits:
      cpu: 0.1
      memory: 200Mi
    requests:
      cpu: 0.1
      memory: 200Mi

service_config_a:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70
  resources:
    limits:
      cpu: 0.3
      memory: 400Mi
    requests:
      cpu: 0.3
      memory: 400Mi

service_config_b:
  <<: *default_config
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70

hmm, I did not consider that, it may well do!

@mumoshu lovely stuff, thanks works beautifully

one other question - is the work for bases released yet?

@Ufou yep! https://github.com/roboll/helmfile/releases/tag/v0.68.0 is the latest release including the bases feature, which all the known issues related to bases fixed

thanks!

@mumoshu what is the release schedule for helmfile if there is one?

I’m gonna reorganise things related to helmfile, try new features and stuff. @mumoshu Should I avoid set in my helmfile.yaml? I noticed somewhere you stated, that it was added for backward compatibility.

anyone see how far they could get with helm3-alpha?

https://github.com/helm/helm/releases/tag/v3.0.0-alpha.1

Can I define several templates? Something like

templates:
  - first: &first
    snip1
  - second: &second
    snip2

Why helm-tiller is not bundled in a helmfile’s Docker image?

I was just lazy to do so! PR welcomed