#helmfile (2023-03)
Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/
2023-03-02
![William avatar](https://secure.gravatar.com/avatar/b147680122499c870602bcd935c4b7c8.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0020-72.png)
Hi everyone!
Is there anyway that helmfile apply will do an upgrade of a release even when helm diff does not detect changes in that release?
Currently what we see is that we have a release A and B.
- Helm diff detects changes in release A but not in B
- We get the information that release A is updated from helm diff.
- Afterwards, we get the output in terminal “Affected releases are: B Updated”
- Checking the cluster afterwards in helm list - we can see the version number has incremented for both releases
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
2023-03-07
![Phil avatar](https://secure.gravatar.com/avatar/2a94940d5c0ccde6dec60bd116102cdc.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
Hey guys!
Is there anyway to “package” a helmfile? It’s already possible to point to a remote helmfile, but what I’d like to do is to add some custom manifests into it (like argocd application, config file, etc), but I’m not sure what’s the best way to achieve that. The simplest way I can think of would be to just write all those manifests into the helmfile itself, but looking at the full helmfile specification I don’t think it’s possible to do that.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
@Brandon you can post a feature request.
2023-03-09
![Viacheslav avatar](https://avatars.slack-edge.com/2023-02-21/4846260547329_aab47a8274143e053527_72.jpg)
Hi everyone,
As a part of the automation pipeline, I generate stateValues.yaml
, and my helmfiles located in helmfile.d/
. But when I run apply or template, I got this error:
helmfile --state-values-file stateValues.yaml template -e asa -l app=kubeflow-core -i --skip-deps
in helmfile.d/00-cluster-addons.yaml: environment values file matching "stateValues.yaml" does not exist in "."
As I understand, it happens because the working directory changed to helmfile.d/
when Helmfile proceeded multiple helmfiles, but my stateValues.yaml locates in the root folder, so Helmfile tries to find helmfile.d/satetValues.yaml
and fails, because a path to this file is relative.
My question: is there any applicable solution for this issue, besides providing a state values file as ${PWD}/stateValues.yaml
? Maybe some change of folders structure (environments, bases, helmfile.d) may help?
I’ll be glad to hear any advice.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
please provide your folder structure now.
![Viacheslav avatar](https://avatars.slack-edge.com/2023-02-21/4846260547329_aab47a8274143e053527_72.jpg)
@yxxhero
bases
environments.yaml
helmdefaults.yaml
repositories.yaml
environments
product
asa
values.yaml
stateValues.yaml
helmfile.d
00-cluster-addons.yaml
01-kubeflow.yaml
templates
3rd-party
<chartname>
values.gotmpl
bases.yaml
all references in files points to helmfile.d/*yaml
location, for example ./bases.yaml references starts with double dots:
bases:
- ../bases/helmdefaults.yaml
- ../bases/environments.yaml
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
ok. Thank you.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
@Viacheslav maybe you can use subhelmfile feature. do you mind showing the context of helmfile.d.
![Viacheslav avatar](https://avatars.slack-edge.com/2023-02-21/4846260547329_aab47a8274143e053527_72.jpg)
Sure, @yxxhero. The one of files under helmfile.d looks like:
{{ readFile "../bases.yaml" }}
---
repositories:
- name: banzaicloud-stable
url: <https://kubernetes-charts.banzaicloud.com>
templates:
infra: &infra
values:
- ../templates/3rd-party/{{`{{ .Release.Name }}`}}/values.gotmpl
releases:
- name: mychart
<<: *infra
chart: mychart
version: 0.0.1
namespace: default
labels:
app: '{{`{{ .Release.Name }}`}}'
Others are similar but for different software sets. And base.yaml has a reference to base/environments.yaml, which contains:
environments:
{{ .Environment.Name }}:
values:
- ../environments/project/{{ .Environment.Name }}/values.yaml
secrets:
- ../environments/project/{{ .Environment.Name }}/secrets.yaml
And helmfile accepts this structure when I pass absolute path to stateValues.yaml
.
About subhelmfiles, does it mean to use a single helmfile.yaml, but specify all helmfiles that are currently located under helmfile.d as a list?
helmfiles:
- path: helmfiles/00-cluster-addons.yaml
- path: helmfiles/01-kubeflow.yaml
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
yeah. you are right.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
give you a try.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
helmfiles:
- path: 00-cluster-addons.yaml
- path: 01-kubeflow.yaml
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
or set the location of stateValues.yaml
generated to helmfile.d every time?
![Viacheslav avatar](https://avatars.slack-edge.com/2023-02-21/4846260547329_aab47a8274143e053527_72.jpg)
@yxxhero thanks a lot :slightly_smiling_face: , I will check it. Now I see some variables become missing (product name) when I did this change, so I’ll try to debug this.
could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read ../bases/environments.yaml.part.0: environment values file matching "../environments//asa/values.yaml" does not exist in "."
error in first-pass rendering: result of "../bases/environments.yaml.part.0":
0: environments:
1: asa:
2: values:
2: - ../environments/<no value>/asa/values.yaml
3: secrets:
5: - ../environments/<no value>/asa/secrets.yaml
6:
Regarding forced location of stateValues - it’s possible, but undesirable. Automation tool is atmos and to do this, it requires to split helmfile apply command
in two steps: generate file, and the apply with this file. The same approach can be used to make ${PWD}/stateValues.yaml
which also solve this issue, but I wish to avoid using multiple commands to make the life of an Operator easier.
I will update a thread when check helmfiles:
as you suggested. Have a good weekend!
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
![Andrew Nazarov avatar](https://avatars.slack-edge.com/2021-06-09/2146832855878_fbb84e3b2832cc494a93_72.jpg)
Hi. A question about the helmfile’s get
function. I’m wondering if it supports nesting. For example if I want to use as default a value of .Values.myAnotherValue
, but if this doesn’t exist - some another default. Couldn’t figure out the syntax. The naive assumption would be something like:
myVal: {{ .Value | get "myValue" (get "myAnotherValue" "1" ".Value") }}
But it definitely won’t work. Or what would be a proper way to achieve something like this?
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Have you tried something like this?
myVal: {{ .Values.myValue | default .Values.myAnotherValue }}
![Andrew Nazarov avatar](https://avatars.slack-edge.com/2021-06-09/2146832855878_fbb84e3b2832cc494a93_72.jpg)
But here if I don’t have myValue
and myAnotherValue
keys defined I’ll get an error.
![Andrew Nazarov avatar](https://avatars.slack-edge.com/2021-06-09/2146832855878_fbb84e3b2832cc494a93_72.jpg)
It seems I’ve found out why my approach didn’t work. Will try out one thing.
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Ahh, yes. Good point.
![Andrew Nazarov avatar](https://avatars.slack-edge.com/2021-06-09/2146832855878_fbb84e3b2832cc494a93_72.jpg)
It seems
myVal: {{ .Values | get "myValue" (get "myAnotherValue" "1" .Values) }}
works. The problem was the type of the third argument - it shouldn’t be string (note the absence of quotes).
(the typo about plural .Values wasn’t the case, in the real code everything was correct regarding this)
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
awesome works.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
good example.
2023-03-10
2023-03-13
![Brandon avatar](https://secure.gravatar.com/avatar/17c631c59e0b012f6aeef896323eeb77.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0011-72.png)
where can I find docs explaining all the ways to use ref+
strings? I want to try and use them to refer to a non-secret file.
![z0rc3r avatar](https://avatars.slack-edge.com/2021-02-04/1719550791252_acb06a9721107cb77740_72.png)
Helm-like configuration values loader with support for various sources
![Brandon avatar](https://secure.gravatar.com/avatar/17c631c59e0b012f6aeef896323eeb77.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0011-72.png)
@z0rc3r perfect. Thank you
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
@Brandon welcome to post doc PR for this.
2023-03-15
![z0rc3r avatar](https://avatars.slack-edge.com/2021-02-04/1719550791252_acb06a9721107cb77740_72.png)
It here a way in helmfile.yaml to set --skip-crds
for one specific deployment? I have one chart deployment that includes CRDs that won’t be needed in my case, but rest of deployments in the same helmfile must install crds.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
set –skip-crds in a release? right?
![z0rc3r avatar](https://avatars.slack-edge.com/2021-02-04/1719550791252_acb06a9721107cb77740_72.png)
@yxxhero if I understood you correctly, then yes.
AFAIU --skip-crds
can be passed via cli for the whole helmfile. But I want to have this option to be available per release in helmfile.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
@z0rc3r yeah. you got me. would you mind posting an issue in helmfile/helmfile. so we can discuss the feature together.
![z0rc3r avatar](https://avatars.slack-edge.com/2021-02-04/1719550791252_acb06a9721107cb77740_72.png)
sure, will do
![z0rc3r avatar](https://avatars.slack-edge.com/2021-02-04/1719550791252_acb06a9721107cb77740_72.png)
There are/was hacky ways to pass this arg in cli for whole helmfile, like #713
I have a use case, where I want to pass this argument to single release in helmfile, but let other releases in the same helmfile to allow installing CRDs as usual.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
@z0rc3r do you have an interest in doing that?
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
![z0rc3r avatar](https://avatars.slack-edge.com/2021-02-04/1719550791252_acb06a9721107cb77740_72.png)
I do, but i’m bit time constrained now, my golang expriense is kinda low and i’m not versed in helmfile codebase. So at best it will be a drag.
If i can allocate some time, i’ll submit a PR. But for now please don’t expect this to happen
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
2023-03-16
![Ivo avatar](https://secure.gravatar.com/avatar/a8f515d8ebbe962c037f7f2a6a025aa2.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0018-72.png)
I’m just getting started with Helmfile. I have this basic helmfile:
releases:
- name: argo-cd
namespace: argocd
chart: ../../../charts/argo-cd/5.23.3
values:
- ./values.yaml
The ArgoCD charts exists locally on my filesystem. When I run helmfile template
I get:
Building dependency release=argo-cd, chart=../../../charts/argo-cd/5.23.3
in ./helmfile.yaml: [building dependencies of local chart: command "helm" exited with non-zero status:
PATH:
helm
ARGS:
0: helm (4 bytes)
1: dependency (10 bytes)
2: build (5 bytes)
3: ../../../charts/argo-cd/5.23.3 (30 bytes)
ERROR:
exit status 1
EXIT STATUS
1
STDERR:
Error: no repository definition for <https://dandydeveloper.github.io/charts/>. Please add the missing repos via 'helm repo add'
COMBINED OUTPUT:
Error: no repository definition for <https://dandydeveloper.github.io/charts/>. Please add the missing repos via 'helm repo add']
Is Helmfile not able to resolve transitive dependencies? Do I need to specify this manually?
![Ivo avatar](https://secure.gravatar.com/avatar/a8f515d8ebbe962c037f7f2a6a025aa2.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0018-72.png)
dandydeveloper is a dependency of the ArgoCD Helm chart
![Ivo avatar](https://secure.gravatar.com/avatar/a8f515d8ebbe962c037f7f2a6a025aa2.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0018-72.png)
Can the repo be added automatically by Helmfile?
![voron avatar](https://avatars.slack-edge.com/2020-05-28/1150336641139_5ed259bc80d6dde0a682_72.jpg)
Is Helmfile not able to resolve transitive dependencies? Do I need to specify this manually?
it’s helm, not helmfile.
![Ivo avatar](https://secure.gravatar.com/avatar/a8f515d8ebbe962c037f7f2a6a025aa2.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0018-72.png)
ok
2023-03-17
![Gadi Dor avatar](https://secure.gravatar.com/avatar/f3d1c30526f2ffe15bfa3a60d7f42d5f.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0025-72.png)
Hi All
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
welcome
![Gadi Dor avatar](https://secure.gravatar.com/avatar/f3d1c30526f2ffe15bfa3a60d7f42d5f.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0025-72.png)
dose helmfile work with argocd?
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
yeah.
![Gadi Dor avatar](https://secure.gravatar.com/avatar/f3d1c30526f2ffe15bfa3a60d7f42d5f.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0025-72.png)
is there plugin ?
2023-03-22
![Dominic Cardellino avatar](https://secure.gravatar.com/avatar/43ee8c03a57fd35af4c652f47206f352.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0011-72.png)
Hello,
I have the problem that when I use helmfile --environment k8s-dev diff --suppress-secrets
I still see my provided secrets (via vault) in the output:
loki:
storage_config:
aws:
bucketnames: k8s-{{ .Environment.Values.envPrefix }}-loki
endpoint: s3.eu-central-1.amazonaws.com
region: eu-central-1
access_key_id: <ref+vault://secret/cluster-logging/{{> .Environment.Values.envPrefix }}/aws/s3?proto=https#/aws-access-key-id
secret_access_key: <ref+vault://secret/cluster-logging/{{> .Environment.Values.envPrefix }}/aws/s3?proto=https#/aws-secret-access-key
insecure: false
sse_encryption: false
s3forcepathstyle: true
I thought --suppress-secrets
should hide them?
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Try secretref
as the prefix, ie: <secretref+vault://secret/cluster-logging>
![Dominic Cardellino avatar](https://secure.gravatar.com/avatar/43ee8c03a57fd35af4c652f47206f352.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0011-72.png)
Doesn‘t work either. But I think I know why, because Loki uses a configmap for this configuration and therefore the secrets are shown as plain text
![svil avatar](https://avatars.slack-edge.com/2023-04-05/5065287346530_9551578a8a4ac9b5a71b_72.png)
I’ve used to reference vault secrets like this before in the helm files, but then after implementing vault-sidecar injector leaking secrets became a thing of the past
2023-03-23
2023-03-28
![Nathanael avatar](https://secure.gravatar.com/avatar/97b674a6146eaa2be2ad5a8116886f76.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
Hi there, i try to get started with helmfile. is there a way to add individual yaml resources to a helmchart in helmfile? I tried to add a new (dummy)release and just referenced a normal subfolder with some yamls files. - in fact that worked.
repositories:
- name: cilium
url: <https://helm.cilium.io/>
releases:
- name: cilium
namespace: mgmt-cni-cilium
chart: cilium/cilium
version: 1.13.0
values:
- values.yaml
- name: resources
chart: ./resources
values:
- values2.yaml # <-- does not work :(
My next problem is: how can i use dynamic values with gotmpl on that subfolder? ( not realy a chart more like a bunch of gotmpl-yamls )
I noticed that {{ .Values.foo }}
is not available in this subfolder release but something like {{ requiredEnv "NAME" }}
dos work.
thanks for the help
![Nathanael avatar](https://secure.gravatar.com/avatar/97b674a6146eaa2be2ad5a8116886f76.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
i found that “chartify” is creating a dummy Chart in the tmp folder and is doing some wierd things. 1. gotmpl (my actual file) is put into the main tmp-dir. 2. it gets renderd to files/template
(ignoring my values) and 3. then is Files.Get
from the template
folder…
how can i get my variable in to the file? the gotmpl, seams to ignore the values and regular helm-templating is ignored because of the Files.Get
![Nathanael avatar](https://secure.gravatar.com/avatar/97b674a6146eaa2be2ad5a8116886f76.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0013-72.png)
environments:
default:
values:
- foo: baaaaaaaaaaaa
this does work. So it looks like environment vars can be used but regular releases.values:
not. why is that?
2023-03-29
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)