#helmfile (2021-05)

I believe so, try:

{{ template "x" (merge (dict "value" .Values.data.user "key" "region") .) }}

I think I found it. my problem was I use kubectl to get data if env not exists and I just passed the value to the jsonpath and it did not work, so I hjad to pass the full –output… part and it works.

ah, nvm, i see helmfile support this already https://github.com/roboll/helmfile/blob/ae942c5288895c84c79171e5446773e4cb41c4ce/pkg/helmexec/exec.go#L277

The goal here is to version our environment values same way our app charts are versioned, and promote them in lockstep.

So that the same commit that adds a new value to a chart can also populate it in the environment values files (and likewise for removal or other changes).

Would be super simple if we could just include the environment values files in the chart itself, instead of trying to download them from Git or something

interesting idea, I wonder to hear if there is some solution

Yes, we use it everywhere. Best to show the error message.

This is the error message:

no matches for path: <https://raw.githubusercontent.com/cloudposse/helmfiles/master/releases/external-dns/helmfile.yaml>
There are no repositories defined in your helmfile.yaml.
This means helmfile cannot update your dependencies or create a lock file.
See <https://github.com/roboll/helmfile/issues/878> for more information.
err: no releases found that matches specified selector() and environment(default), in any helmfile

Also reported it as a GH issue https://github.com/roboll/helmfile/issues/1842

Just to add I don’t mean “Git over HTTPS” but direct “plain” HTTPS.

aha, good distinction. I see we’re using this:

helmfiles:
- path: "git::<https://github.com/cloudposse/helmfiles.git@releases/efs-provisioner.yaml?ref=0.112.0>"

Which is “Git over HTTPS”, and I don’t know if we’ve tested the “plain” HTTPS.

The testing field won’t be available in that scope if it is defined in a values file. You would need to move it to another yaml file that you would import as an environment (default one or specific one)

e.g: environment.yaml

testing: yes

helmfile.yaml

environments:
  default:
    values:
      - ./environment.yaml

releases:
  - name: namespace-dev
    chart: ../../../../charts/namespace
    version: 0.0.1
    createNamespace: false
    values:
      - values.yaml
      - test: {{ .Values.testing }}
      - namespace: dev

I’ve read somewhere some time ago that ArgoCD has some support or can be hacked to process Helmfiles from git repos :)

helmfile apply inside CI/CD pipelines does this for about 1y in one of dev envs

You also could find this thread useful: https://github.com/roboll/helmfile/issues/153

Flux, Argo and helmfile-operator are mentioned there

Probably it’s not intentional

Maybe no. But would you mind giving me an ideal and imaginary configuration example that you have in your mind? I was thinking that you can just use {{ readFile }} to include some file under releases

@Tencho Tenev Hey! Thanks for bringing it up. This sounds like a good chance to contribute.

https://github.com/databus23/helm-diff/issues/253#issuecomment-842733991

I think the first step would be to try to build your own helm-diff after changing this line, so that helm-diff use helm upgrade --dry-run instead of template

https://github.com/databus23/helm-diff/blob/master/cmd/helm3.go#L141

more context in https://github.com/databus23/helm-diff/issues/253#issuecomment-842733991

Hi, so I did the first step and have helm diff using upgrade dry-run instead of template. This produces diffs showing the next revision which makes more sense.

However, the perfect behaviour for me is to produce no diff if the only reason for change is the revision changing. This is not so trivial, in fact, I think it requires a change in helm itself.

After looking at a few projects, I feel like using release revision in all charts is not a best practice. I will suggest changes to the charts my team owns to remove the use of revisions. I may still raise a PR on helm diff with the change for upgrade instead of template because at least the result is more intuitive.

@Tencho Tenev ah, gotcha! just fixing the chart makes sense in that case. good.

In case you can’t fix the chart for whatever reason, I think it’s still a good idea to raise a feature request to helm-diff

to add something like ignoreDifferences in ArgoCD https://argoproj.github.io/argo-cd/user-guide/diffing/

It seems to be implemneted by removing the keys at the jsonpaths before computing diff https://github.com/argoproj/argo-cd/blob/b37eee1054e42c873699460dd5e2447c2f9fe5a6/util/argo/normalizers/diff_normalizer.go#L28-L63

helm-diff does textual diffing here https://github.com/databus23/helm-diff/blob/d42b177462211ce1f15db07f6309359818a005f9/diff/diff.go#L157

so adding some logic for ignoreDiefferences here would work

again, if fixing the chart works, that’s okay

just my two cents

probably not https://github.com/roboll/helmfile/issues/805

haven’t tried that myself. what kind of “exception”s d o you actually see?

We are successfully using subsequent postsync hooks:

    hooks:
      - events: ["postsync"]
        showlogs: true
        command: "sleep"
        args: ["30s"]
      - events: ["postsync"]
        command: "kubectl"
        args: ["annotate", "--overwrite", "ns", ...]
      - events: ["postsync"]
        command: "kubectl"
        args: ["label", "--overwrite", "ns", ...]

I got an exception about invalid index. apparently I added unnecessary spaces

@Andrew Nazarov thank you for the example. I fixed it and now it works

I’m cheekily using mergeOverwrite at the minute, just for a current “workaround”

b.yaml should override a.yaml

ah cool, cheers.

hey! it should just work, as you can indeed use go template expressions like {{ env }} in your helmfile.yaml template.

how did you confirm it doesn’t work? have you tried running helmfile build to see what the rendered helfile.yaml template looks like?

have you tried running it with --debug like helmfile --debug template to see what helmfile sees?

well the helmfile is in a container that I dont have access to

and in this case, I have added the GITLAB_RUNNER_REGISTRATION_TOKEN as a CICD variable in gitlab which means its available to the environment. but the token doesnt get picked up and the pods fail to register — this is how I know it doesnt work

ah okay… then the only thing i could suggest would be

try running something like echo $GITLAB_RUNNER_REGISTRATION_TOKEN | shasunm on CI and compare the output with that of the same command run on your local machine

to see if the envvar is really set or not

helmfile should just read the env and that’s all i can say from helmfile’s perspective

environments and release templates?

environments: https://github.com/roboll/helmfile#environment release templates: https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md#release-template--conventional-directory-structure

I think that might be on the right track. A better way of stating it– can helmfile be used to manage inserting cluster-specific FILES in addition to chart values? A good example of this in our repository is the use of .crt and .pem files for specific sub-charts. These live in directories adjacent to the microservice’s chart, and there is a template for a config map that takes everything in these directories as data for the config map ({{ (.Files.Glob "securityconfig/*").AsConfig | indent 2 }}). Say I wanted to deploy to my dev cluster, could I use the helmfile to include the desired .crt and .pem files for that environment?

Don’t know about the first point, we never needed this. As for the second we have a organisational solution. If one wants to remove a release or something else from the helmfile it should be done in stages: first, installed should be set to false for corresponding releases, and second, only after this configuration is applied a block can be removed from a helmfile. We don’t do gitops, nonetheless I believe the same principle can be applied.

Yep, seems like it’s the only solution for now

Regarding the first point, there’s no built-in functionality that helps it. you need to script it

For the second, right that installed: false is the only solution. That way you are extra sure helmfile would ever remove releases that are explicitly marked so.

We’ve once discussed about storing some helmfile “state” in the k8s cluster so that we can track what are already installed onto the cluster but now missing in the definitions. But it turned out to be defeat one of helmfile goals of being a “thin” wrapper around helm.

(It’s arguable if helmfile is really “thin” today but it would still be a great idea to keep it as simple as possible

hey! thanks for sharing

yeah i think i have more understanding of what you’re trying to achieve now

replied to your issue based on that https://github.com/roboll/helmfile/issues/1817#issuecomment-851724588