#helmfile (2022-06)

https://github.com/helmfile/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles

Archive: https://archive.sweetops.com/helmfile/

2022-06-03

2022-06-07

KimJohn Quinn avatar
KimJohn Quinn

Two questions regarding Helmfile:

  1. Should we be switching to the new repo and using that (Helmfile and Docker image)?
  2. Running into an issue where a chart (opentelemetry) has a JSON schema which is causing a validation issue with the merged values. Using the original Helmfile, can I disable that validation? It does not seem like turning off the validation options works.
bradym avatar
  1. Yes, but no new releases have been made on the new repo yet.

  2. What version of helm-diff do you have installed? From the helmfile readme:

    disableValidation: false
    # passes --disable-validation to helm 3 diff plugin, this requires diff plugin >= 3.1.2
KimJohn Quinn avatar
KimJohn Quinn

3.3.1

KimJohn Quinn avatar
KimJohn Quinn

My release looks like this:

  - name: opentelemetry-collector
    chart: open-telemetry/opentelemetry-collector
    installed: {{ env "OPENTELEMETRY_INSTALLED" | default .Values.installed }}
    namespace: gitlab-managed-apps
    version: 0.18.0
    disableValidation: true
    wait: true
    values:
      - values.gotmpl
KimJohn Quinn avatar
KimJohn Quinn

I’ve tried every “verify” flag in the docs and just tried the one you copied above…same issue

bradym avatar

Can you share the error message you’re getting?

KimJohn Quinn avatar
KimJohn Quinn
COMBINED OUTPUT:
  WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
  WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
  Error: values don't meet the specifications of the schema(s) in the following chart(s):
  opentelemetry-collector:
  - (root): Additional property cluster is not allowed
KimJohn Quinn avatar
KimJohn Quinn
Templating release=opentelemetry-collector, chart=open-telemetry/opentelemetry-collector
in ./helmfile.yaml: in .helmfiles[13]: in charts/opentelemetry/helmfile.yml: command "/usr/local/bin/helm" exited with non-zero status:

PATH:
  /usr/local/bin/helm

ARGS:
  0: helm (4 bytes)
  1: template (8 bytes)
  2: opentelemetry-collector (23 bytes)
  3: open-telemetry/opentelemetry-collector (38 bytes)
  4: --version (9 bytes)
  5: 0.18.0 (6 bytes)
  6: --namespace (11 bytes)
  7: develop (7 bytes)
  8: --values (8 bytes)
  9: /tmp/helmfile3119307560/develop-opentelemetry-collector-values-66f5fbc7fd (73 bytes)
  10: --output-dir (12 bytes)
  11: generated/templates (19 bytes)
KimJohn Quinn avatar
KimJohn Quinn

I dont see any flags being passed to helm from the output…

bradym avatar

You wouldn’t, the disableValidation option passes a flag to helm-diff not helm.

KimJohn Quinn avatar
KimJohn Quinn

yup, youre right. didnt even think about that.

bradym avatar

I haven’t run into this particular issue, but in your place I’d look into the options for helm template and see if there’s one that will disable this validation. Then you could use the --args flag on helmfile template to use it.

KimJohn Quinn avatar
KimJohn Quinn

Correct me if i am wrong but that wont help me when i “apply” the charts?

bradym avatar

helmfile apply also has the --args flag.

bradym avatar

And whatever is set in --args gets sent to the helm command as args. So you’d need to modify the commands you use for template and apply to include the --args command.

KimJohn Quinn avatar
KimJohn Quinn

This is more work than its worth to just disable the validation. If it is passing it down to each of the helm commands (template, diff, etc.) and they take in different args then it doesnt look like it will work.

bradym avatar

Is there a reason you can’t remove the property the validation is complaining about?

KimJohn Quinn avatar
KimJohn Quinn

The issue is that I have the values needed for the chart and an “region” not an environment that needs to be applied to it. So, the values is the combination of what the chart needs plus some “region” specific stuff.

KimJohn Quinn avatar
KimJohn Quinn

Sort of like this for the release:

    values:
      - ../{{ .Values.region }}.yml
      - values.gotmpl
KimJohn Quinn avatar
KimJohn Quinn

Unless there is a way to use variables in the environment yml (which would be ideal)

KimJohn Quinn avatar
KimJohn Quinn

As a note, it is just this chart (so far) but it would be nice to have the consistency of being able to augment the values with a region in the helmfile insted of use it here but not here type of thing.

KimJohn Quinn avatar
KimJohn Quinn

The full helmfile looks like this:

bases:
  - ../common.yml
  - ../environments.yml
---
bases:
  - environments.yml
---
repositories:
  - name: open-telemetry
    url: <https://open-telemetry.github.io/opentelemetry-helm-charts>

releases:
  - name: opentelemetry-collector
    chart: open-telemetry/opentelemetry-collector
    installed: {{ env "OPENTELEMETRY_INSTALLED" | default .Values.installed }}
    namespace: gitlab-managed-apps
    version: 0.18.0
    disableValidation: true
    wait: true
    values:
      - ../{{ .Values.region }}.yml
      - values.gotmpl
bradym avatar

Have you tried naming your region.yml files with the .gotmpl extension instead and referencing state variables in there?

KimJohn Quinn avatar
KimJohn Quinn

yes, that’s actually what it is now….

bradym avatar

I see values.gotmpl above, but ../{{ .Values.region }}.yml

bradym avatar

Have you changed it since posting that snippet?

KimJohn Quinn avatar
KimJohn Quinn

i just typed that in to demonstrate what im trying to do…

Basically:

  1. I have environments - stage, prod, etc. and those are across the board
  2. I have regions which could be in the environments but a pita to maintain and all they do is “augment” slightly certain variables across the charts.
KimJohn Quinn avatar
KimJohn Quinn

Actually, what I am trying to do is different from the main problem - it wont matter too much any solution, and i have a messy hack i can do in ci, as long as the schema is being validated and the values are being merged, short of manually maintaining the chart, it will still error out

bradym avatar

Well, you just lost me.

The only thing I can suggest at this point is structuring your values.gotpml so that the opentelemetry chart won’t see the thing that’s failing validation. Though to be honest I’m still confused why you want a value in there that can’t be used by the chart.

KimJohn Quinn avatar
KimJohn Quinn

Your sort of right. I dont “need” to do this but now i have one helmfile that isnt consistent with the pattern we have.

There is a top-level “environment” yaml that gets loaded - basically, like “globals” but not specific to the charts themselves…this works:

  staging:
    values:
      - installed: false
      - ../globals.gotmpl
      - ../staging.gotmpl
      - ../us-east-2.gotmpl
      - defaults.yml

What I would like to do is “pick” the environment at this level prior to rendering the charts. I could use environments but then I would have staging-us-east-2, staging-us-west-2, etc. and that would mean, as far as I know, i at least have to include those same environments in each chart.

KimJohn Quinn avatar
KimJohn Quinn

I was under the assumption I could “merge” top-level values in at the helmfile level - which for the most part works except in this scenario where the schema is being validated.

KimJohn Quinn avatar
KimJohn Quinn

In short, i want a consistent way to build up a set of variables i can use across any chart. The environments works but in this case it is not really an environment but a slight adaption of the variables…

bradym avatar


a set of variables i can use across any chart
That’s the part I was missing. I’ve never tried that. We just have a variables file for each app we want to deploy.

bradym avatar

I can see why it would be useful, but I honestly don’t know how to make that work.

KimJohn Quinn avatar
KimJohn Quinn

haha, i do really appreciate the help at least - that is the story of my life with helmfile

bradym avatar

Oh I hear that. Every time I work on our helmfile stuff I have to re-learn where variables can be used in templates

KimJohn Quinn avatar
KimJohn Quinn

i know right? i spend more time bending helmfile to my will than i do in the charts haha

this1
Alexander avatar
Alexander

hello, is it possible to access helm builtin variables such as appVersion / chartVersion from helmfile?

Alexander avatar
Alexander

want to set annotation with version

bradym avatar

Yep. You can reference the builtin objects as explained on https://helm.sh/docs/chart_template_guide/builtin_objects/ just by using them in your templates.

Built-in Objects

Built-in objects available to templates.

Alexander avatar
Alexander

if i use {{ .Chart.Version }} in values i get

template: stringTemplate:23:28: executing "stringTemplate" at <.Chart.Version>: can't evaluate field Chart in type *state.EnvironmentTemplateData
Alexander avatar
Alexander

block looks like this:

values:
  - podLabels:
      app: appname
      version: {{ .Chart.Version }}

2022-06-28

mr.shayv avatar
mr.shayv

Hey How do i get helm values yaml output from a specific yaml template file ?

2022-06-30

Brandon avatar
Brandon

Can helm --set args be passed on a per-release basis in helmfile? Based on the README, I only see how to set these as helmDefaults

Chris Dobbyn avatar
Chris Dobbyn

Nothing native that I’m aware of. You’d have to create a release specific values file and use -f. Or continue using set.

Brandon avatar
Brandon

Thanks!

steenhoven avatar
steenhoven

Is there a way to automatically add a custom label to all resources, like werf does? Something like commit=9aeee03d607c1eed133166159fbea3bad5365c57.

Annotating and labeling of chart resources / Deploy process / Helm / Advanced | werfattachment image

Consistent delivery tool. Git as a single source of truth. Build, deploy to Kubernetes, stay in sync.

mumoshu avatar
mumoshu

there’s no streamlined way to do that as far as i know. but adding such functionality to helmfile shouldn’t be so hard!

helmfile already have a feature called “chartify” which is supposed to run kustomize on the chart to apply various kustomizations that isn’t possible with the vanilla helm + chart

https://helmfile.readthedocs.io/en/latest/advanced-features/#adhoc-kustomization-of-helm-charts

kustomize has commonLabels that allows you to add specific label(s) to all the resources so adding a release field (like releases[].commonResourceLabels) to the helmfile.yaml syntax and letting helmfile translate it to kustomize commonLabels would do the job

https://github.com/kubernetes-sigs/kustomize/blob/master/examples/transformerconfigs/README.md#annotations-transformer

Annotating and labeling of chart resources / Deploy process / Helm / Advanced | werfattachment image

Consistent delivery tool. Git as a single source of truth. Build, deploy to Kubernetes, stay in sync.

mumoshu avatar
mumoshu

if you’re interested in helmfile to add such feature, i’d appreciate it if you could write up a feature request in https://github.com/helmfile/helmfile/discussions/new?category=ideas thanks!

1
    keyboard_arrow_up