#helmfile (2022-06)
Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/
2022-06-03
2022-06-06
2022-06-07
Two questions regarding Helmfile:
- Should we be switching to the new repo and using that (Helmfile and Docker image)?
- Running into an issue where a chart (opentelemetry) has a JSON schema which is causing a validation issue with the merged values. Using the original Helmfile, can I disable that validation? It does not seem like turning off the validation options works.
-
Yes, but no new releases have been made on the new repo yet.
-
What version of
helm-diff
do you have installed? From the helmfile readme:
disableValidation: false
# passes --disable-validation to helm 3 diff plugin, this requires diff plugin >= 3.1.2
3.3.1
My release looks like this:
- name: opentelemetry-collector
chart: open-telemetry/opentelemetry-collector
installed: {{ env "OPENTELEMETRY_INSTALLED" | default .Values.installed }}
namespace: gitlab-managed-apps
version: 0.18.0
disableValidation: true
wait: true
values:
- values.gotmpl
I’ve tried every “verify” flag in the docs and just tried the one you copied above…same issue
Can you share the error message you’re getting?
COMBINED OUTPUT:
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /root/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /root/.kube/config
Error: values don't meet the specifications of the schema(s) in the following chart(s):
opentelemetry-collector:
- (root): Additional property cluster is not allowed
Templating release=opentelemetry-collector, chart=open-telemetry/opentelemetry-collector
in ./helmfile.yaml: in .helmfiles[13]: in charts/opentelemetry/helmfile.yml: command "/usr/local/bin/helm" exited with non-zero status:
PATH:
/usr/local/bin/helm
ARGS:
0: helm (4 bytes)
1: template (8 bytes)
2: opentelemetry-collector (23 bytes)
3: open-telemetry/opentelemetry-collector (38 bytes)
4: --version (9 bytes)
5: 0.18.0 (6 bytes)
6: --namespace (11 bytes)
7: develop (7 bytes)
8: --values (8 bytes)
9: /tmp/helmfile3119307560/develop-opentelemetry-collector-values-66f5fbc7fd (73 bytes)
10: --output-dir (12 bytes)
11: generated/templates (19 bytes)
I dont see any flags being passed to helm from the output…
You wouldn’t, the disableValidation
option passes a flag to helm-diff
not helm
.
yup, youre right. didnt even think about that.
I haven’t run into this particular issue, but in your place I’d look into the options for helm template
and see if there’s one that will disable this validation. Then you could use the --args
flag on helmfile template
to use it.
Correct me if i am wrong but that wont help me when i “apply” the charts?
helmfile apply
also has the --args
flag.
And whatever is set in --args
gets sent to the helm command as args. So you’d need to modify the commands you use for template and apply to include the --args
command.
This is more work than its worth to just disable the validation. If it is passing it down to each of the helm commands (template, diff, etc.) and they take in different args then it doesnt look like it will work.
Is there a reason you can’t remove the property the validation is complaining about?
The issue is that I have the values needed for the chart and an “region” not an environment that needs to be applied to it. So, the values is the combination of what the chart needs plus some “region” specific stuff.
Sort of like this for the release:
values:
- ../{{ .Values.region }}.yml
- values.gotmpl
Unless there is a way to use variables in the environment yml (which would be ideal)
As a note, it is just this chart (so far) but it would be nice to have the consistency of being able to augment the values with a region in the helmfile insted of use it here but not here type of thing.
The full helmfile looks like this:
bases:
- ../common.yml
- ../environments.yml
---
bases:
- environments.yml
---
repositories:
- name: open-telemetry
url: <https://open-telemetry.github.io/opentelemetry-helm-charts>
releases:
- name: opentelemetry-collector
chart: open-telemetry/opentelemetry-collector
installed: {{ env "OPENTELEMETRY_INSTALLED" | default .Values.installed }}
namespace: gitlab-managed-apps
version: 0.18.0
disableValidation: true
wait: true
values:
- ../{{ .Values.region }}.yml
- values.gotmpl
Have you tried naming your region.yml files with the .gotmpl
extension instead and referencing state variables in there?
yes, that’s actually what it is now….
I see values.gotmpl
above, but ../{{ .Values.region }}.yml
Have you changed it since posting that snippet?
i just typed that in to demonstrate what im trying to do…
Basically:
- I have environments - stage, prod, etc. and those are across the board
- I have regions which could be in the environments but a pita to maintain and all they do is “augment” slightly certain variables across the charts.
Actually, what I am trying to do is different from the main problem - it wont matter too much any solution, and i have a messy hack i can do in ci, as long as the schema is being validated and the values are being merged, short of manually maintaining the chart, it will still error out
Well, you just lost me.
The only thing I can suggest at this point is structuring your values.gotpml so that the opentelemetry chart won’t see the thing that’s failing validation. Though to be honest I’m still confused why you want a value in there that can’t be used by the chart.
Your sort of right. I dont “need” to do this but now i have one helmfile that isnt consistent with the pattern we have.
There is a top-level “environment” yaml that gets loaded - basically, like “globals” but not specific to the charts themselves…this works:
staging:
values:
- installed: false
- ../globals.gotmpl
- ../staging.gotmpl
- ../us-east-2.gotmpl
- defaults.yml
What I would like to do is “pick” the environment at this level prior to rendering the charts. I could use environments but then I would have staging-us-east-2, staging-us-west-2, etc. and that would mean, as far as I know, i at least have to include those same environments in each chart.
I was under the assumption I could “merge” top-level values in at the helmfile level - which for the most part works except in this scenario where the schema is being validated.
In short, i want a consistent way to build up a set of variables i can use across any chart. The environments works but in this case it is not really an environment but a slight adaption of the variables…
a set of variables i can use across any chart
That’s the part I was missing. I’ve never tried that. We just have a variables file for each app we want to deploy.
I can see why it would be useful, but I honestly don’t know how to make that work.
haha, i do really appreciate the help at least - that is the story of my life with helmfile
Oh I hear that. Every time I work on our helmfile stuff I have to re-learn where variables can be used in templates
i know right? i spend more time bending helmfile to my will than i do in the charts haha
hello, is it possible to access helm builtin variables such as appVersion / chartVersion from helmfile?
want to set annotation with version
Yep. You can reference the builtin objects as explained on https://helm.sh/docs/chart_template_guide/builtin_objects/ just by using them in your templates.
Built-in objects available to templates.
if i use {{ .Chart.Version }} in values i get
template: stringTemplate:23:28: executing "stringTemplate" at <.Chart.Version>: can't evaluate field Chart in type *state.EnvironmentTemplateData
block looks like this:
values:
- podLabels:
app: appname
version: {{ .Chart.Version }}
2022-06-28
Hey How do i get helm values yaml output from a specific yaml template file ?
2022-06-30
Can helm --set
args be passed on a per-release basis in helmfile? Based on the README, I only see how to set these as helmDefaults
Nothing native that I’m aware of. You’d have to create a release specific values file and use -f. Or continue using set.
Thanks!
Is there a way to automatically add a custom label to all resources, like werf does? Something like commit=9aeee03d607c1eed133166159fbea3bad5365c57
.
Consistent delivery tool. Git as a single source of truth. Build, deploy to Kubernetes, stay in sync.
there’s no streamlined way to do that as far as i know. but adding such functionality to helmfile shouldn’t be so hard!
helmfile already have a feature called “chartify” which is supposed to run kustomize
on the chart to apply various kustomizations that isn’t possible with the vanilla helm + chart
https://helmfile.readthedocs.io/en/latest/advanced-features/#adhoc-kustomization-of-helm-charts
kustomize has commonLabels
that allows you to add specific label(s) to all the resources so adding a release
field (like releases[].commonResourceLabels
) to the helmfile.yaml syntax and letting helmfile translate it to kustomize commonLabels
would do the job
Consistent delivery tool. Git as a single source of truth. Build, deploy to Kubernetes, stay in sync.
if you’re interested in helmfile to add such feature, i’d appreciate it if you could write up a feature request in https://github.com/helmfile/helmfile/discussions/new?category=ideas thanks!