#helmfile (2019-07)

One more question about {{'{{}}'}} syntax. Say, I set something like this in helmfile.yaml

values:
  - hostname: {{`{{.Release.Namespace}}`}}.my-domain.com

But this causes the error during helmfile lint: reading document at index 1: yaml: line 241: did not find expected key. Is this an expected behaviour? Btw, I’m on quite old version: v0.69.0

does helmfile support creation of standalone kubernetes resources? for example if i add istio as a release and want to install a gateway, but without it being contained in a helmchart

@Mical Helmfile just calls helm. But you’re in luck…

There is a “raw” chart that does exactly what you want

Here is a example: https://github.com/cloudposse/helmfiles/blob/master/releases/external-storage.yaml

And we used it also to install an istio gateway

Here is an example of that: https://github.com/cloudposse/example-app/blob/master/deploy/releases/istio.yaml

@Erik Osterman (Cloud Posse) thanks I’ll check it out!

What’s the road map for 1.0.0? We’re looking at different tools to use for decentralizing helm chart development while having a centralized way to test, package and continuously deploy systems containing a variety of charts. So far we’ve been looking at flux and helmfile. Flux is great but very opinionated while helmfile gives us the freedom of control since it is stateless. The fact that helmfile is still <1.0 is an issue for us for obvious reasons.

I think it’s unfair to judge based on the release version being pre 1.0. Terraform is 0.12 is in use in production by enterprises/banks/etc. Obviously the uptick in traction is a wee bit less for helmfile. From my POV, better indicators to look at is how often the software is released (aka maintained), how responsive are the maintainers to issues, the overall traction of the project/community.

when you look at these characteristics, then helmfile shines.

btw, @mumoshu has now written an HelmfileOperator so you can do flux-like things with helmfile (that is a proof of concept though.

https://github.com/mumoshu/helmfile-operator

@Erik Osterman (Cloud Posse) that’s a valid point.

Thanks Erik!

@Mical If I have anything to add, there would be only two things:

(1) Many orgs including my company uses helmfile in production

https://github.com/roboll/helmfile/blob/master/USERS.md

(2) Even though it is pre-1.0, Helmfile has never introduced breaking changes to existing feature for a year or so.

We do introduce breaking changes to experimental features but even those happen only after prior discussions with the known users of the feature.

can i get the exit code from hooks to propagate so that helmfile sync doesn’t exit with 0 on failure?

hey, just joined the slack, but been using helmfile for a 6+ months; thanks alot for making it

i’ve got a quick question: are environments: [] propagated to children helmfiles: [] ? looks that they aren’t from my experiments, and not sure if its by design

also trying stuff out with bases: [], but i’m probably getting something wrong

Hi, Hoping someone knows off the top of their head why this wouldn’t work

templates:
  dataproduct: &dataproduct
    namespace: dataproduct
    chart: chartmuseum/cdp-chart
    version: 1.0.1
    values:
      - app
          name: {{`"{{.Release.Name}}"`}}
      - {{ .Environment.Name }}.yaml

releases:
  - name: my-dp
    <<: *dataproduct

  - name: another-dp
    <<: *dataproduct

Specifically

- app:
  name:
    {{`"{{.Release.Name}}"`}}

It seems to be an issue with nesting values because if I change it to

values:
  - appName: {{`"{{.Release.Name}}"`}}

it doesn’t fail

The error is YAML parse error on cdp-chart/templates/rbac.yaml: error converting YAML to JSON: yaml: invalid map key: map[interface {}]interface {}{".Release.Name":interface {}(nil)}

@Ben could you copy-paste your actual template once again? I see a trailing : is missing after app in https://sweetops.slack.com/archives/CE5NGCB9Q/p1563858986015700

anyways this seems to work without emitting such error on my machine:

templates:
  dataproduct: &dataproduct
    namespace: dataproduct
    chart: chartmuseum/cdp-chart
    version: 1.0.1
    values:
      - app:
          name: {{`"{{.Release.Name}}"`}}
      - {{ .Environment.Name }}.yaml

releases:
- name: myapp
  chart: stable/mysql
  namespace: myapp
  <<: *dataproduct

@mumoshu Sorry, yes the original has the colon

environments:
  default:
    values:
    - default.yaml
  production:
    values:
    - production.yaml

templates:
  dataproduct: &dataproduct
    namespace: dataproduct
    chart: chartmuseum/cdp-chart
    version: 1.0.0
    values:
      - app:
          name: {{`"{{.Release.Name}}"`}}
      - {{ .Environment.Name }}.yaml

releases:
  - name: california-sos
    <<: *dataproduct

  - name: rdc
    <<: *dataproduct

@Ben Which helmfile version are you using?

thx

I’m doubting something is wrong with your cdp-chart/templates/rbac.yaml. Could you share it?

Also - enabling debug logs like helmfile --log-level=debug allows you to see the yaml after rendering the template, which would help debugging

Actually the error states the error occurs in serviceaccount.yaml which looks like

{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
  name: {{ .Values.app.name }}
  namespace: {{ .Values.app.namespace }}
  labels:
    app: {{ .Values.app.name }}
{{- end -}}

ah sry I missed https://sweetops.slack.com/archives/CE5NGCB9Q/p1563859105017700

indeed this can be a bug in helmfile!

ahh it isn’t rendering nested strings.. i’ll open an issue for that

@Ben Is this a blocker to you? (I’ll prioritize this accordingly if so

Thanks @mumoshu It’s not a blocker but would definitely remove a lot of boilerplate config for us

Do you have any knowledge of how people are doing continuous deployment (or nearly CD with some manual gates before running sync) using helmfile? On the surface it seems simple; build, test, publish docker image, update helmfile with new image tag (or chart version if publishing a new chart) and run sync. However, it gets much more complicated once you have to deploy to multiple environments/tenants and also stage helmfile changes. Just wondering if you’ve seen any good approaches?

We run Helmfile under atlantis

We centralize our Helmfiles in a repo

Then use remote Helmfiles to pull them in pinned to a release (kind of like terraform)

Thanks @Erik Osterman (Cloud Posse) The thing I’m struggling with is how to update centralised helmfiles in an automated way. At the moment I can only see one option; Each microservice git project’s build pipeline creates/publishes docker image, clones the helmfile repo, creates a branch, using a script (sed, etc) updates the chart version or values, commits changes. Then another pipeline checks for helmfile repo changes and deploys to an environment based on branch name (feature/nnn, release/nnn, master). Once the helmfile sync is finished, a tester tests the system in a test env and if happy merges the helmfile repo branch into master. The same pipeline detects the change in git and deploys to prod because changes are now on master. Does that seem reasonable? How does your process differ from this?

@Ben We’ve got a single helmfile with all the releases for all the environments defined. It’s placed in a separate repo, not within the main codebase. Since we’ve got a set of stages (dev, several stagings, prod) and per client installations we are leveraging environments: feature extensively to make it DRY. The workflow is pretty manual indeed, one has to set a new version of the docker image and/or helm chart for the certain environment. To control things pull-request/merge-request workflow could be considered. Not the best approach for sure, but kinda work for us right now.

@Erik Osterman (Cloud Posse) Out of curiosity, what is this Atlantis thing?

Would be nice to have condition: boolean on hooks because sometimes you might want to run a hook only if installed: boolean is true on another release.. if course it can be wrapped by other logic but would be cleaner to have a flag for it

bugs me that helm test --cleanup does not output the logs before removal on error

Hiya. Currently I’m doing this:

releases:
- name: abc-namespace
  chart: stable/magic-namespace
  set:
  - name: tiller.image.tag
    value: v{{ .Values.tillerVersion }}
- name: xyz-namespace
  chart: stable/magic-namespace
  set:
  - name: tiller.image.tag
    value: v{{ .Values.tillerVersion }}

Is there a way to set tiller.image.tag as a default for all releases so that I don’t have to specify that set every time? I’ve seen you can use release templates, but I wonder if there’s an even more general way.

can someone tell me if i’m not understanding environments: correctly and how it applies to releases. i have 2 files in helmfile.d: istio.yaml

environments:
  staging:

releases:
  ...

app.yaml

environments:
  default:
  staging:

releases:
  ...

• if i run helmfile sync i expect only app.yaml releases to be synced
• if i run helmfile -e staging sync i expect both to be synced.

don’t want to wrap istio releases with {{ if eq .Environment.Name "staging" }}

or is it that default applies to all implicitly?

Basically you want to set values under envs and use it like:

environments:
  dev:
    values:
     - foo: aaa
  prod:
    values:
     - foo: bbb

releases:
  something: {{ .Environment.Values.foo }}

for helmfile -e dev apply something would be aaa for helmfile -e prod apply something would be bbb

yeah that i know, was wondering about the default environment if that is applied to all releases

because if i replace default with dev in my case and run helmfile -e dev sync then it only applies to app.yaml releases

i expect i was abusing the default

but thanks for input @Andrew Nazarov (chose some bad names in my example btw.. updated)

Actually, I thought environments don’t affect releases until you define something like mentioned {{ if eq .Environment.Name "app" }}. What you are talking about is something new to me. I mean, that helmfile -e dev syncs only releases in the app.yaml. It it documented somewhere?

Not sure if it is, but it works that way at least

I hope @mumoshu will comment.

but i need {{ if ne .Environment.Name "default" }} around my istio releases to exclude them from the default env

Ah, then it’s expected)). At first I thought you somehow managed to do it without if .Environment.Name since you wrote “don’t want to wrap istio releases ”

Thanks for the clarification.

I meant that i don’t want to wrap it with {{ if eq .Environment.Name "xxx" }}

not {{ if ne .... }}

would prefer not having to add the not equals conditional though

Hey, just started to experiment with Helmfile and besides some hiccups with helm-diff, I was wondering, is it possible to capture directly stdout from a command and use it in a template? E.g. something like {{ requiredEnv "PLATFORM_ENV" }}, maybe {{ getOutput "echo Hello" }}?

@Yannis is that not what {{ exec }} does? (I’m also new to Helmfile)

https://github.com/roboll/helmfile/blob/master/README.md#importing-values-from-any-source

@Emanuel I must have been blind! Let’s test it then

I think I don’t fully understand how environments work. If I have

helmfiles:
- ./*/helmfile.yaml

And I want to use helmfile -e production apply then does each of those subhelmfiles have to define the environment? Else you’d get err: no releases found that matches specified selector() and environment(production), in any helmfile

So what would that look like? Do I put this code at the top of each subhelmfile? There has to be a better way!

bases:
{{- range $_, $file := ( exec "sh" (list "-c" "echo ../environments/*yaml") | splitList " " ) }}
- {{ trim $file }}
{{ end }}

If environments are global, I’d expect to define them in the parent helmfile only.

A --no-hooks option to helmfile would be nice.