#helmfile (2022-06)
Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/
2022-06-03
2022-06-07
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
Two questions regarding Helmfile:
- Should we be switching to the new repo and using that (Helmfile and Docker image)?
- Running into an issue where a chart (opentelemetry) has a JSON schema which is causing a validation issue with the merged values. Using the original Helmfile, can I disable that validation? It does not seem like turning off the validation options works.
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
-
Yes, but no new releases have been made on the new repo yet.
-
What version of
helm-diff
do you have installed? From the helmfile readme:
disableValidation: false
# passes --disable-validation to helm 3 diff plugin, this requires diff plugin >= 3.1.2
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
3.3.1
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
My release looks like this:
- name: opentelemetry-collector
chart: open-telemetry/opentelemetry-collector
installed: {{ env "OPENTELEMETRY_INSTALLED" | default .Values.installed }}
namespace: gitlab-managed-apps
version: 0.18.0
disableValidation: true
wait: true
values:
- values.gotmpl
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
I’ve tried every “verify” flag in the docs and just tried the one you copied above…same issue
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Can you share the error message you’re getting?
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
COMBINED OUTPUT:
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
Error: values don't meet the specifications of the schema(s) in the following chart(s):
opentelemetry-collector:
- (root): Additional property cluster is not allowed
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
Templating release=opentelemetry-collector, chart=open-telemetry/opentelemetry-collector
in ./helmfile.yaml: in .helmfiles[13]: in charts/opentelemetry/helmfile.yml: command "/usr/local/bin/helm" exited with non-zero status:
PATH:
/usr/local/bin/helm
ARGS:
0: helm (4 bytes)
1: template (8 bytes)
2: opentelemetry-collector (23 bytes)
3: open-telemetry/opentelemetry-collector (38 bytes)
4: --version (9 bytes)
5: 0.18.0 (6 bytes)
6: --namespace (11 bytes)
7: develop (7 bytes)
8: --values (8 bytes)
9: /tmp/helmfile3119307560/develop-opentelemetry-collector-values-66f5fbc7fd (73 bytes)
10: --output-dir (12 bytes)
11: generated/templates (19 bytes)
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
I dont see any flags being passed to helm from the output…
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
You wouldn’t, the disableValidation
option passes a flag to helm-diff
not helm
.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
yup, youre right. didnt even think about that.
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
I haven’t run into this particular issue, but in your place I’d look into the options for helm template
and see if there’s one that will disable this validation. Then you could use the --args
flag on helmfile template
to use it.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
Correct me if i am wrong but that wont help me when i “apply” the charts?
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
helmfile apply
also has the --args
flag.
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
And whatever is set in --args
gets sent to the helm command as args. So you’d need to modify the commands you use for template and apply to include the --args
command.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
This is more work than its worth to just disable the validation. If it is passing it down to each of the helm commands (template, diff, etc.) and they take in different args then it doesnt look like it will work.
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Is there a reason you can’t remove the property the validation is complaining about?
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
The issue is that I have the values needed for the chart and an “region” not an environment that needs to be applied to it. So, the values is the combination of what the chart needs plus some “region” specific stuff.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
Sort of like this for the release:
values:
- ../{{ .Values.region }}.yml
- values.gotmpl
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
Unless there is a way to use variables in the environment yml (which would be ideal)
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
As a note, it is just this chart (so far) but it would be nice to have the consistency of being able to augment the values with a region in the helmfile insted of use it here but not here type of thing.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
The full helmfile looks like this:
bases:
- ../common.yml
- ../environments.yml
---
bases:
- environments.yml
---
repositories:
- name: open-telemetry
url: <https://open-telemetry.github.io/opentelemetry-helm-charts>
releases:
- name: opentelemetry-collector
chart: open-telemetry/opentelemetry-collector
installed: {{ env "OPENTELEMETRY_INSTALLED" | default .Values.installed }}
namespace: gitlab-managed-apps
version: 0.18.0
disableValidation: true
wait: true
values:
- ../{{ .Values.region }}.yml
- values.gotmpl
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Have you tried naming your region.yml files with the .gotmpl
extension instead and referencing state variables in there?
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
yes, that’s actually what it is now….
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
I see values.gotmpl
above, but ../{{ .Values.region }}.yml
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Have you changed it since posting that snippet?
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
i just typed that in to demonstrate what im trying to do…
Basically:
- I have environments - stage, prod, etc. and those are across the board
- I have regions which could be in the environments but a pita to maintain and all they do is “augment” slightly certain variables across the charts.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
Actually, what I am trying to do is different from the main problem - it wont matter too much any solution, and i have a messy hack i can do in ci, as long as the schema is being validated and the values are being merged, short of manually maintaining the chart, it will still error out
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Well, you just lost me.
The only thing I can suggest at this point is structuring your values.gotpml so that the opentelemetry chart won’t see the thing that’s failing validation. Though to be honest I’m still confused why you want a value in there that can’t be used by the chart.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
Your sort of right. I dont “need” to do this but now i have one helmfile that isnt consistent with the pattern we have.
There is a top-level “environment” yaml that gets loaded - basically, like “globals” but not specific to the charts themselves…this works:
staging:
values:
- installed: false
- ../globals.gotmpl
- ../staging.gotmpl
- ../us-east-2.gotmpl
- defaults.yml
What I would like to do is “pick” the environment at this level prior to rendering the charts. I could use environments but then I would have staging-us-east-2, staging-us-west-2, etc. and that would mean, as far as I know, i at least have to include those same environments in each chart.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
I was under the assumption I could “merge” top-level values in at the helmfile level - which for the most part works except in this scenario where the schema is being validated.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
In short, i want a consistent way to build up a set of variables i can use across any chart. The environments works but in this case it is not really an environment but a slight adaption of the variables…
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
a set of variables i can use across any chart
That’s the part I was missing. I’ve never tried that. We just have a variables file for each app we want to deploy.
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
I can see why it would be useful, but I honestly don’t know how to make that work.
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
haha, i do really appreciate the help at least - that is the story of my life with helmfile
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Oh I hear that. Every time I work on our helmfile stuff I have to re-learn where variables can be used in templates
![KimJohn Quinn avatar](https://avatars.slack-edge.com/2022-03-08/3208504745842_7b7ed70be92a0a431efd_72.jpg)
i know right? i spend more time bending helmfile to my will than i do in the charts haha
![this](/assets/images/custom_emojis/this.png)
![Alexander avatar](https://secure.gravatar.com/avatar/395cdeac0e179f2fd46cd9527d4fea0e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0008-72.png)
hello, is it possible to access helm builtin variables such as appVersion / chartVersion from helmfile?
![Alexander avatar](https://secure.gravatar.com/avatar/395cdeac0e179f2fd46cd9527d4fea0e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0008-72.png)
want to set annotation with version
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Yep. You can reference the builtin objects as explained on https://helm.sh/docs/chart_template_guide/builtin_objects/ just by using them in your templates.
Built-in objects available to templates.
![Alexander avatar](https://secure.gravatar.com/avatar/395cdeac0e179f2fd46cd9527d4fea0e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0008-72.png)
if i use {{ .Chart.Version }} in values i get
template: stringTemplate:23:28: executing "stringTemplate" at <.Chart.Version>: can't evaluate field Chart in type *state.EnvironmentTemplateData
![Alexander avatar](https://secure.gravatar.com/avatar/395cdeac0e179f2fd46cd9527d4fea0e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0008-72.png)
block looks like this:
values:
- podLabels:
app: appname
version: {{ .Chart.Version }}
2022-06-28
![mr.shayv avatar](https://secure.gravatar.com/avatar/284d976b98e338bf3447485ecceb09a4.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0009-72.png)
Hey How do i get helm values yaml output from a specific yaml template file ?
2022-06-30
![Brandon avatar](https://secure.gravatar.com/avatar/17c631c59e0b012f6aeef896323eeb77.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0011-72.png)
Can helm --set
args be passed on a per-release basis in helmfile? Based on the README, I only see how to set these as helmDefaults
![Chris Dobbyn avatar](https://avatars.slack-edge.com/2021-03-01/1814062195588_ba73798ef7efdbd3021e_72.jpg)
Nothing native that I’m aware of. You’d have to create a release specific values file and use -f. Or continue using set.
![Brandon avatar](https://secure.gravatar.com/avatar/17c631c59e0b012f6aeef896323eeb77.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0011-72.png)
Thanks!
![steenhoven avatar](https://secure.gravatar.com/avatar/4a4fd52352a6a141acf07c2c02d966df.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0001-72.png)
Is there a way to automatically add a custom label to all resources, like werf does? Something like commit=9aeee03d607c1eed133166159fbea3bad5365c57
.
![attachment image](https://werf.io/assets/images/share.png)
Consistent delivery tool. Git as a single source of truth. Build, deploy to Kubernetes, stay in sync.
![mumoshu avatar](https://secure.gravatar.com/avatar/8e045bf747ca7a90b1d955dc30217271.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0015-72.png)
there’s no streamlined way to do that as far as i know. but adding such functionality to helmfile shouldn’t be so hard!
helmfile already have a feature called “chartify” which is supposed to run kustomize
on the chart to apply various kustomizations that isn’t possible with the vanilla helm + chart
https://helmfile.readthedocs.io/en/latest/advanced-features/#adhoc-kustomization-of-helm-charts
kustomize has commonLabels
that allows you to add specific label(s) to all the resources so adding a release
field (like releases[].commonResourceLabels
) to the helmfile.yaml syntax and letting helmfile translate it to kustomize commonLabels
would do the job
![attachment image](https://werf.io/assets/images/share.png)
Consistent delivery tool. Git as a single source of truth. Build, deploy to Kubernetes, stay in sync.
![mumoshu avatar](https://secure.gravatar.com/avatar/8e045bf747ca7a90b1d955dc30217271.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0015-72.png)
if you’re interested in helmfile to add such feature, i’d appreciate it if you could write up a feature request in https://github.com/helmfile/helmfile/discussions/new?category=ideas thanks!