#helmfile (2020-10)

https://github.com/helmfile/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles

Archive: https://archive.sweetops.com/helmfile/

2020-10-01

Release notes from helmfile avatar
Release notes from helmfile
11:46:36 PM

v0.130.1 Fix: forceNamespace needs shouldRun is true (#1510)

Release v0.130.1 · roboll/helmfile

Fix: forceNamespace needs shouldRun is true (#1510)

Fix: forceNamespace needs shouldRun is true by mdaisuke · Pull Request #1510 · roboll/helmfile

Hi, I found releases[].forceNamespace doesn&#39;t work okay, so tried to fix it. how it works $ cat <<EOF > sample-helmfile.yaml releases: - name: scaler forceNamespace: this-is-forced-n…

Release notes from helmfile avatar
Release notes from helmfile
12:06:25 AM

v0.130.1 954673d (HEAD, tag: v0.130.1, origin/master, origin/HEAD, master) Fix: forceNamespace needs shouldRun is true (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”712637508” data-permission-text=”Title is…

Fix: forceNamespace needs shouldRun is true (#1510) · roboll/helmfile@954673d

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

2020-10-09

Padarn avatar

I’m trying to understand why you would choose values over set or vice versa

Roderik van der Veer avatar
Roderik van der Veer

Helmfile in one of the more recent values got these super nice debug messages helm:pfRFE> wait.go:225: [debug] Deployment is not ready: staging-lavender-five/ingress-staging-lavender-five-ingress-controller. 0 out of 2 expected pods are ready but they are just thrown into the stdout or stderr (not sure). Is there a way these can be outputted in a computer parseable format? I would like to display them (cleaned up a bit) in an applications logging screen.

voron avatar

Message you provided is a helm message actually, helmfile gets it from helm and outputs to stdout/stderr.

2020-10-11

2020-10-12

Dang Vo avatar
Dang Vo

anybody here uses helmfile with Flux, i mean generate HelmRelease file for Flux deployment by helmfile template

voron avatar

IMHO helmfile and flux&helm-operator perform the same thing. helm-operator looks a little bit young project, as well as tied to flux. I don’t think it’s a good idea to create a chart ( or use incubator/raw chart) just to generate HelmRelease CRD. I don’t see a great secrets solution as well with HelmRelease, as I wanna stay in deploy-via-helm-only and I don’t wanna spread deploy to multiple moving parts.

Matt Gowie avatar
Matt Gowie

Yeah, I think Flux / HO vs Helmfile is a “Pick one or the other” decision.

HelmRelease files are not very complicated and templating them via Helmfile wouldn’t make a lot of sense I don’t think. If you needed to template your HelmRelease files (because you have dozens and dozens of them then I would think Kustomize might be worth looking into.

Dang Vo avatar
Dang Vo

there are any solution ?? because i also want to keep my gitops tool

Release notes from helmfile avatar
Release notes from helmfile
06:27:18 AM

v0.130.2: Fix concurrent-map-iteration-and-write errors while running release h… …ooks (#1534) Fixes <a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”706251141” data-permission-text=”Title is private”…

Release notes from helmfile avatar
Release notes from helmfile
06:47:10 AM

v0.130.2: Fix concurrent-map-iteration-and-write errors while running release h… ab9fb2c (HEAD, tag: v0.130.2, origin/master, origin/HEAD, master) Fix concurrent-map-iteration-and-write errors while running release hooks (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”719761479”…

2020-10-13

Release notes from helmfile avatar
Release notes from helmfile
01:06:23 AM

v0.130.3: Bump vals (#1537) To incorporate unexpected SSM secret exposure issue when the parameter version is specified. See variantdev/vals#37

Release v0.130.3: Bump vals (#1537) · roboll/helmfile

To incorporate unexpected SSM secret exposure issue when the parameter version is specified. See variantdev/vals#37

Release notes from helmfile avatar
Release notes from helmfile
01:17:19 AM

v0.130.3: Bump vals (#1537) 563fce4 (HEAD, tag: v0.130.3, origin/master, origin/HEAD, master) Bump vals (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”721048331” data-permission-text=”Title is private”…

Bump vals (#1537) · roboll/helmfile@563fce4

To incorporate unexpected SSM secret exposure issue when the parameter version is specified. See https://github.com/variantdev/vals/pull/37

2020-10-14

vixus0 avatar

Bumping this PR because it would be really useful for our use case: https://github.com/roboll/helmfile/pull/1503

Fix set flag for write-values by vixus0 · Pull Request #1503 · roboll/helmfile

The –set flag for write-values is currently a no-op. I&#39;m guessing this is because we can&#39;t simply pass the flag through to helm like we do for the other commands. I&#39;ve worked around th…

mumoshu avatar
mumoshu

Sry for the delay! Good to know that there is strvals package to support this. I’ve reviewed it and it looks generally good.

Fix set flag for write-values by vixus0 · Pull Request #1503 · roboll/helmfile

The –set flag for write-values is currently a no-op. I&#39;m guessing this is because we can&#39;t simply pass the flag through to helm like we do for the other commands. I&#39;ve worked around th…

mumoshu avatar
mumoshu

If you could fix the test failure, that would be great. Otherwise I’ll fix it myself in a few days

vixus0 avatar

I’m looking into it now - interestingly I get a different error on my local machine, but I’m guessing the root cause is similar (helm not being on the PATH seen by tests):

vixus0 avatar
--- FAIL: TestWriteValues_WithSetFlag (0.10s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1afe00b]

goroutine 2964 [running]:
testing.tRunner.func1.1(0x1d99a00, 0x3486ca0)
        /home/linuxbrew/.linuxbrew/Cellar/go/1.15.2/libexec/src/testing/testing.go:1076 +0x46a
testing.tRunner.func1(0xc00053a300)
        /home/linuxbrew/.linuxbrew/Cellar/go/1.15.2/libexec/src/testing/testing.go:1079 +0x636
panic(0x1d99a00, 0x3486ca0)
        /home/linuxbrew/.linuxbrew/Cellar/go/1.15.2/libexec/src/runtime/panic.go:975 +0x3e9
github.com/roboll/helmfile/pkg/state.(*HelmState).generateVanillaValuesFiles(0xc0003c7680, 0xc000167080, 0xc0006b21c0, 0x7f6f9dd1ae40, 0xc0006b21c0, 0xc000600000, 0x20)
        /home/anshul/git/helmfile/pkg/state/state.go:2323 +0x2eb
github.com/roboll/helmfile/pkg/state.(*HelmState).generateValuesFiles(0xc0003c7680, 0x2baf360, 0xc000479ad0, 0xc000167080, 0x0, 0x47534a, 0xc000046800, 0xc0007ded50, 0x474cdc, 0xc0006ddb00)
        /home/anshul/git/helmfile/pkg/state/state.go:2401 +0x65
github.com/roboll/helmfile/pkg/state.(*HelmState).WriteReleasesValues(0xc0003c7680, 0x2baf360, 0xc000479ad0, 0x34d2790, 0x0, 0x0, 0xc0007df608, 0x1, 0x1, 0x0, ...)
        /home/anshul/git/helmfile/pkg/state/state.go:1207 +0x2b8
github.com/roboll/helmfile/pkg/app.(*App).writeValues(0xc0003bf680, 0xc00007e3f0, 0x2b984c0, 0xc000693b30, 0xc00007e6f0, 0x34d2790, 0x4470f0, 0xc00007e5d0)
        /home/anshul/git/helmfile/pkg/app/app.go:1491 +0xa96
github.com/roboll/helmfile/pkg/app.(*App).WriteValues.func1.1()
        /home/anshul/git/helmfile/pkg/app/app.go:247 +0xd1
github.com/roboll/helmfile/pkg/app.(*Run).withPreparedCharts(0xc00007e3f0, 0x1f8718b, 0xc, 0xc000000000, 0xc0007e0530, 0x0, 0x0)
        /home/anshul/git/helmfile/pkg/app/run.go:78 +0x596
github.com/roboll/helmfile/pkg/app.(*App).WriteValues.func1(0xc00007e3f0, 0x2baf300, 0x0, 0x0, 0x0)
        /home/anshul/git/helmfile/pkg/app/app.go:243 +0x1c8
github.com/roboll/helmfile/pkg/app.(*App).ForEachState.func1(0xc0003c7680, 0xc000000001, 0xffffffff00000087, 0x0, 0xc0007e0890)
        /home/anshul/git/helmfile/pkg/app/app.go:788 +0xc4
github.com/roboll/helmfile/pkg/app.(*App).visitStatesWithSelectorsAndRemoteSupport.func1.1(0xc0003c7680, 0x1, 0xc0000000bc, 0xc0007e0a18)
        /home/anshul/git/helmfile/pkg/app/app.go:897 +0x6d
github.com/roboll/helmfile/pkg/app.processFilteredReleases(0xc0003c7680, 0x2baf360, 0xc000479ad0, 0xc0007e1048, 0x0, 0x0, 0x0, 0x0)
        /home/anshul/git/helmfile/pkg/app/app.go:946 +0x956
github.com/roboll/helmfile/pkg/app.(*App).visitStatesWithSelectorsAndRemoteSupport.func1(0xc0003c7680, 0xc000000001, 0xa3, 0x0, 0xc0003f7b80)
        /home/anshul/git/helmfile/pkg/app/app.go:896 +0xdf
github.com/roboll/helmfile/pkg/app.(*App).visitStates.func1(0x1f87f22, 0xd, 0xc0007009f0, 0x8, 0x0, 0x0)
        /home/anshul/git/helmfile/pkg/app/app.go:736 +0xbcd
github.com/roboll/helmfile/pkg/app.(*App).visitStateFiles.func1(0x3461268, 0xc000000001)
        /home/anshul/git/helmfile/pkg/app/app.go:570 +0x1a7
github.com/roboll/helmfile/pkg/app.(*App).within(0xc0003bf680, 0x1f7b835, 0x1, 0xc0009117d8, 0xc000911798, 0x2)
        /home/anshul/git/helmfile/pkg/app/app.go:512 +0x9b7
github.com/roboll/helmfile/pkg/app.(*App).visitStateFiles(0xc0003bf680, 0x1f87f22, 0xd, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/anshul/git/helmfile/pkg/app/app.go:564 +0x62c
github.com/roboll/helmfile/pkg/app.(*App).visitStates(0xc0003bf680, 0x1f87f22, 0xd, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /home/anshul/git/helmfile/pkg/app/app.go:648 +0x185
github.com/roboll/helmfile/pkg/app.(*App).visitStatesWithSelectorsAndRemoteSupport(0xc0003bf680, 0x1f87f22, 0xd, 0xc00006bca8, 0xc00006bd18, 0x1, 0x1, 0x1, 0xc0007009d0)
        /home/anshul/git/helmfile/pkg/app/app.go:903 +0x90e
github.com/roboll/helmfile/pkg/app.(*App).ForEachState(0xc0003bf680, 0xc00006bd28, 0xc00006bd18, 0x1, 0x1, 0x43e5c0, 0xc00006bd48)
        /home/anshul/git/helmfile/pkg/app/app.go:784 +0x112
github.com/roboll/helmfile/pkg/app.(*App).WriteValues(0xc0003bf680, 0x2b984c0, 0xc000693b30, 0xc000693b30, 0xc0001361e8)
        /home/anshul/git/helmfile/pkg/app/app.go:240 +0xe9
github.com/roboll/helmfile/pkg/app.TestWriteValues_WithSetFlag(0xc00053a300)
        /home/anshul/git/helmfile/pkg/app/app_test.go:4265 +0x55b
testing.tRunner(0xc00053a300, 0x1fe52d0)
        /home/linuxbrew/.linuxbrew/Cellar/go/1.15.2/libexec/src/testing/testing.go:1127 +0x203
created by testing.(*T).Run
        /home/linuxbrew/.linuxbrew/Cellar/go/1.15.2/libexec/src/testing/testing.go:1178 +0x797
FAIL    github.com/roboll/helmfile/pkg/app      1.916s
?       github.com/roboll/helmfile/pkg/app/version      [no test files]
vixus0 avatar
vixus0 avatar

oh no maybe not, this is something to do with HelmState.valsRuntime.Eval

Vadim Bauer avatar
Vadim Bauer

I would like to pass values and secrets as argument to the helmfile. The flag --state-values-file  works only for values not secrets . I couldn’t find the equivalent cli argument for the secret. How would I pass secrets files?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(fwiw, and alternatively speaking, helmfile natively supports retrieving secrets from many different backends like SSM, vault, etc)

mumoshu avatar
mumoshu
How would I pass secrets files to helmfile via CLI? · Issue #1539 · roboll/helmfile

I would like to pass values and secrets as argument to the helmfile apply. The flag –state-values-file works only for values not secrets . I couldn&#39;t find the equivalent cli argument for the s…

Release notes from helmfile avatar
Release notes from helmfile
01:27:29 AM

v0.131.0: Bump chartify to 0.4.4 (#1540) To incorporate the fix for unconventional chart templates like seen in the Datadog chart variantdev/chartify@5443ca1

Release v0.131.0: Bump chartify to 0.4.4 (#1540) · roboll/helmfile

To incorporate the fix for unconventional chart templates like seen in the Datadog chart variantdev/chartify@5443ca1

Release notes from helmfile avatar
Release notes from helmfile
01:37:25 AM

v0.131.0: Bump chartify to 0.4.4 (#1540) 080080f (HEAD, tag: v0.131.0, origin/master, origin/HEAD, master) Bump chartify to 0.4.4 (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”721884071” data-permission-text=”Title is private”…

2020-10-18

Release notes from helmfile avatar
Release notes from helmfile
12:16:19 AM

v0.132.0: Minor README improvement (#1543) 9ec4a85 (HEAD, tag: v0.132.0, origin/master, origin/HEAD, master) Minor README improvement (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”723045991” data-permission-text=”Title is private”…

2020-10-19

2020-10-20

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Checkout what @mumoshu whipped up! https://github.com/mumoshu/waypoint-plugin-helmfile

mumoshu/waypoint-plugin-helmfile

Helmfile deployment plugin for HashiCorp Waypoint. Contribute to mumoshu/waypoint-plugin-helmfile development by creating an account on GitHub.

1
Jonathan avatar
Jonathan

Man, the speed at which he creates things like this is insane!

mumoshu/waypoint-plugin-helmfile

Helmfile deployment plugin for HashiCorp Waypoint. Contribute to mumoshu/waypoint-plugin-helmfile development by creating an account on GitHub.

2
2
3
1

2020-10-21

Trevor Hartman avatar
Trevor Hartman

hi, new helmfile user here. can i install manifests outside of charts using helmfile? for example, i want to apply some CRDs before installing a chart

Andrew Nazarov avatar
Andrew Nazarov

Yes, there are different approaches. I’ll try to find a discussion related to this

1
Trevor Hartman avatar
Trevor Hartman

You can do everything with helmfile hooks, although I find that unwieldy for all but the simplest commands, so for more complicated stuff, like installing, upgrading, and removing cert-manager CRDs, I prefer to use the hook to run a custom script (which mainly runs kubectl commands, because until last week, jetstack was warning that helm still had issues with CRDs, apparently just resolved in the past few days with the release of helm 3.3.1).

For cert-manager specifically, I have a presync hook that calls the script to install or upgrade the CRDs, a postuninstall script (actually the same script with different arguments) to remove the CRDs, and a postsync hook that does nothing but sleep so that cert-manager has time to set up its CA and webhooks before the next release runs.

The next release, BTW, is in the same helmfile.yaml and “needs” the first one, installs ClusterIssuers. It also has to have

disableValidation: true

because otherwise helmfile apply will fail to install the releases because the CRDs are not yet installed and so the ClusterIssuers are not valid resources yet.

Andrew Nazarov avatar
Andrew Nazarov

Yep, this is exactly it

1
Trevor Hartman avatar
Trevor Hartman

attempting to apply a simple helmfile with:

repositories:

  - name: itscontained
    url: <https://charts.itscontained.io>

helmDefaults:
  verify: false
  force: true

releases:

  - name: secret-manager
    labels:
      group: vault
    namespace: infra
    chart: itscontained/secret-manager
    version: 0.3.0
    atomic: true
    kubeContext: "{{ .Environment.Name }}"

this results in

Error: Failed to render chart: exit status 1: Error: failed to download "itscontained/secret-manager" (hint: running `helm repo update` may help)

maybe this is masking another problem? because the chart repo and chart are correct (i installed it via plain helm already)

Trevor Hartman avatar
Trevor Hartman
Confusing error message on chart verification error · Issue #467 · roboll/helmfile

I&#39;m facing the following issue. I defined a bunch of external services in my helmfile. Then I tried to sync everything. Doing helm sync I got Upgrading stable/cert-manager Upgrading gitlab/gitl…

Trevor Hartman avatar
Trevor Hartman

oof, my bad, i mixed app version and chart version

Shreyank Sharma avatar
Shreyank Sharma

Hello helm users. have some questions on helm for Prometheus. 1 ) what is the difference between Prometheus-operator and kube-prometheus (what will be installed as part of Prometheus-operator and kube-prometheus ) 2 ) I want to install prometheus-operator with version 0.29.0. in Chart.yaml undersources its pointing to https://github.com/coreos/prometheus-operator. but there is no value.yaml file. am i checking the right repo? 3) if i want to install prometheus-operator using local helm repo using custom value.yaml, which repo i should download.

Thank you

Andrew Nazarov avatar
Andrew Nazarov
prometheus-community/helm-charts

Prometheus community Helm charts. Contribute to prometheus-community/helm-charts development by creating an account on GitHub.

Andrew Nazarov avatar
Andrew Nazarov
helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

Shreyank Sharma avatar
Shreyank Sharma

thank you @Andrew Nazarov i used this -> https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack to replicate the same things but i showed Error: Chart requires kubernetesVersion: >=1.16.0-0 which is incompatible with Kubernetes v1.14.10 i will try the deprecated link which you provided

Andrew Nazarov avatar
Andrew Nazarov

It’s possible to use not the latest version of kube-prometheus-stack. We have one cluster with k8s 1.15.12 and we deployed 9.4.10 there.

2020-10-22

Mark Henneman avatar
Mark Henneman

Does anyone know how to iterate over files in a certain folder with helmfile?

Raymond Liu avatar
Raymond Liu

I have met the problem, and @mumoshu give me a great solution. https://github.com/roboll/helmfile/issues/724#issuecomment-609039954

Allow .Files like helm does · Issue #724 · roboll/helmfile

I need to glob prometheus rule files rules/* into a yaml block. So I was hoping .Files to be available to do so: additionalPrometheusRules: {{- range $file, $bytes := .Files.Glob &quot;rules/*.yaml…

1
Raymond Liu avatar
Raymond Liu

My usage is just like your case

{{- range $_, $file := ( exec "bash" (list "-c" "echo -n dashboards/general/*.json") | splitList " " ) }}
{{ trimSuffix (ext $file) (base $file) }}:
json: |
    {{- readFile $file | nindent 10 }}
{{- end }}
Mark Henneman avatar
Mark Henneman
  {{- $root := . -}}
  {{- range $path, $bytes := .Files.Glob .Values.filePattern }}
  {{ base $path }}: |
    {{ $root.Files.Get $path | b64enc }}
  {{- end }}

helm style seems not to work

Mark Henneman avatar
Mark Henneman
in ./helmfile.yaml: failed to render values files "config/0-default/values.yaml.gotmpl": failed to render [config/0-default/values.yaml.gotmpl], because of template: stringTemplate:17:29: executing "stringTemplate" at <$.Files.Glob>: can't evaluate field Files in type state.releaseTemplateData

is the error I get

vixus0 avatar

@Mark Henneman I don’t think that’s possible, what are you trying to achieve?

Mark Henneman avatar
Mark Henneman

@vixus0 I have unknown number of files in a folder, which I want to read into a secret, where each filename is the key and the contents is a base64 string (secret volume).

vixus0 avatar

hmm, you could possibly use the incubator/raw helm chart to do that, using the templates: [] value.

vixus0 avatar

that runs through the helm templating engine

vixus0 avatar

oh although I don’t know what path it would read the files from, so not sure that would work.

Mark Henneman avatar
Mark Henneman

Hmm, I guess that’s not what I need. The problem with Helm is that it doesnt read files outside of the helm root folder. And my value files (chart values and environment values) are outside the chart folder

Mark Henneman avatar
Mark Henneman

fixed it by

{{- range $_, $file := ( exec "bash" (list "-c" "echo -n secrets-volume/**") | splitList " " ) }}
{{ $bytes := readFile $file }}
{{- end }}
1
vixus0 avatar

Oh yeah, I always forget about exec

2020-10-23

vixus0 avatar

I’m really struggling with this for some reason… Here’s my problem:

• I am managing multiple k8s clusters, let’s call them “environments”.

• Each environment has some global config (AWS account ID, domain name, etc.) that gets used in release values.

• I have a common set of releases that should be applied to all clusters.

• I want to allow the possibility for each cluster to specify its own additional set of releases.

• Each release has values shared across all the clusters and also environment-specific values. What’s the best way to achieve this with helmfile?

Mark Henneman avatar
Mark Henneman

Maybe you could layer your value files en env files. Maybe this article could help you on how to set it up https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md

roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

2020-10-24

Release notes from helmfile avatar
Release notes from helmfile
02:36:21 AM

v0.132.1: Disable dependency update while running helm-x/chartify in more cases… … (#1548) helm dep up is now skipped while running helm-x/chartify when the chart/directory is obtained by running go-getter, or skipDeps is configured…

Release v0.132.1: Disable dependency update while running helm-x/chartify in more cases… · roboll/helmfile

… (#1548) helm dep up is now skipped while running helm-x/chartify when the chart/directory is obtained by running go-getter, or skipDeps is configured using a command-line flag, helmDefaults, or r…

Release notes from helmfile avatar
Release notes from helmfile
02:46:14 AM

v0.132.1: Disable dependency update while running helm-x/chartify in more cases… 0663831 (HEAD, tag: v0.132.1, origin/master, origin/HEAD, master) Disable dependency update while running helm-x/chartify in more cases (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”724182522”…

Disable dependency update while running helm-x/chartify in more cases… · roboll/helmfile@0663831

… (#1548) helm dep up is now skipped while running helm-x/chartify when the chart/directory is obtained by running go-getter, or skipDeps is configured using a command-line flag, helmDefaults,…

Joaquin Menchaca avatar
Joaquin Menchaca

Kustomize question. I was using the helmify.sh script, but one question, how do I inject namespace? I tried to use this method with an operator that uses ClusterRoleBinding . I ran into this:

ClusterRoleBinding.rbac.authorization.k8s.io "run-nfs-provisioner" is invalid: subjects[0].namespace: Required value

Where:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: rook-nfs-operator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: rook-nfs-operator
subjects:
- kind: ServiceAccount
  name: rook-nfs-operator
  namespace: <MY-NAME-SPACE-NEEDS-TO-MATCH>
Joaquin Menchaca avatar
Joaquin Menchaca

I am not sure on the strategy to handle this scenario…

Joaquin Menchaca avatar
Joaquin Menchaca

It seems I have to use a static namespace

Joaquin Menchaca avatar
Joaquin Menchaca

Is there a way to have helmfile apply gotemplating on some of kustomize files before I helmify.sh it? Kustomize is so anti-DRY anti-cloud-native IMO, but I have to use it for many use cases where operators are used and there’s no helm chart to use the operator.

Vugar avatar

Hi @Joaquin Menchaca Hm… are you sure it is a good idea to mix helmfile and kustomize? Especially if you don’t find approach with overlays too DRY or too attractive? If you prefer working with templates… would you rather consider raw chart instead?

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

Joaquin Menchaca avatar
Joaquin Menchaca

There’s no other choice. In the community at large, *operators* are all the rage, and often there’s no helm chart around installing or using the *operator*. So if I want to use helmfile with the operator and things that use the CRD, I have to use ./helmify.sh.

Now I could instead write a helm chart for every operator that doesn’t support helm charts, but by the time I finished, I would be fired and unemployed, for spending too much time writing helm charts for other projects, rather than adding features to my company’s project.

The last two I came across are jaeger-operator (which can use kafka, spark, cassandra, elasticsearch operators) and rook ceph operator.

Vugar avatar

Well… I haven’t used ‘helmify.sh’ myself. I hope someone else could assist you with this! I can only add that in case if you are on fire and running out of time maybe consider using presync and postsync event hooks to execute script with direct call to kustomize… I believe you should be able to pass required variables. Or create a lazy chart with all of the values hardcoded… and template it later when you have more time :-)

Joaquin Menchaca avatar
Joaquin Menchaca

That sounds like an option. kustomize edit can inject values, very limited. I am not sure how I could down pass values from helmfile environment, like namespace or values.

Joaquin Menchaca avatar
Joaquin Menchaca

For a workaround, I am creating kustomize values with "{{ .Values.nfs.path }}", which are then consumed by values I inject to the final rendered helm chart from helmfile. The downside, is that this is (1) overly complex to people that my use it, (2) brittle.

I would like to inject a set of default values, such as a value.yaml into the final rendered chart from helmify.sh. That way I can continue to use factory pattern method documented for helmfile.

Joaquin Menchaca avatar
Joaquin Menchaca

Side note, I noticed that helmfile is not able to inject a namespace into the rendered helmchart. I’m not sure why/how this is. So I added a namespace inside the kustomize.yaml to get the same namespace. helmfile at least creates the namespace if it doesn’t exist.

The kustomize was smart enough to inject the namespace to metadata.namespace where appropriate, and even change the ClusterRoleBinding subjects[0].namespace as well.

mumoshu avatar
mumoshu

@Joaquin Menchaca helmfile delegates setting metadat.aname in the helm-template result to… helm.

helm is only able to set metadata.name in the template output if and only if the chart supports it.

not sure this was what you were struggling. but hope it helps!

mumoshu avatar
mumoshu

For example, cluster-autoscaler doesn’t support setting metadat.anamespace in the template output, cuz it doesn’t have namespace: {{ .Release.Namespace }} in the template https://github.com/helm/charts/blob/b9278fa98cef543f5473eb55160eaf45833bc74e/stable/cluster-autoscaler/templates/deployment.yaml#L8

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

mumoshu avatar
mumoshu

cert-manager is correct. It has namespace: {{ .Release.Namespace }} in the template https://github.com/helm/charts/blob/10ddb0c607525b323d0b0ceb301ca82bbd31b6f9/stable/cert-manager/templates/deployment.yaml#L5

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

mumoshu avatar
mumoshu

Fortunately theres’ a workaround. Helmfile has the forceNamespace setting to modify the helm-template output and sets metadata.namespae regardless of the chart supports it or not

https://github.com/roboll/helmfile/pull/1444

Add experimental "forceNamespace" in helmfile.yaml by mumoshu · Pull Request #1444 · roboll/helmfile

This is an experimental feature to support a potential use-case that you need to set namespaces in manifests rendered by helmfile template, WHEN the chart is unconventional hence does not have name…

mumoshu avatar
mumoshu

:point_up: FYI everyone, helmfile has forceNamespace that allows you to set metadata.namespace in the helm-template output even if the chart doesn’t support it

1
Joaquin Menchaca avatar
Joaquin Menchaca

The helmify.sh script from the gist, didn’t have this in there, as it took the final raw k8s manifests.

Joaquin Menchaca avatar
Joaquin Menchaca

One thing it would be nice is to actually inject values into the kustomize scripts on the fly, such as namespace (like a gotemplated kustomize), then run the helmify.sh script.

mumoshu avatar
mumoshu

is your helmify.sh is something similar to https://gist.github.com/mumoshu/f9d0bd98e0eb77f636f79fc2fb130690?

Joaquin Menchaca avatar
Joaquin Menchaca

It’s exactly that one

mumoshu avatar
mumoshu

ah ok

mumoshu avatar
mumoshu

i haven’t read throughout all this thread yet, but have you tried the kustomize integration?

https://github.com/roboll/helmfile/blob/master/test/advanced/helmfile.yaml#L7

roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

Joaquin Menchaca avatar
Joaquin Menchaca

If I could have the kustomize.yaml.gotemplate rendered, then run kustomize… that would be good workflow

mumoshu avatar
mumoshu

tl;dr; you can point your kustomization directory in chart:

mumoshu avatar
mumoshu

so that helmfile automatically turns the kustomization into a temporary chart

mumoshu avatar
mumoshu

also you can use chart values to set certain parameters on the kustomize project. see

https://github.com/variantdev/chartify/blob/master/testdata/kustomize/input/values.yaml

variantdev/chartify

Convert K8s manifests/Kustomization into Helm Chart - variantdev/chartify

Joaquin Menchaca avatar
Joaquin Menchaca

I didn’t know that there was direct support, if I am reading the example correctly.

Joaquin Menchaca avatar
Joaquin Menchaca

I was using the hook method style with the helmify-kustomize script

Joaquin Menchaca avatar
Joaquin Menchaca

example:

  - name: jaeger
    namespace: monitoring
    chart: ./jaeger
    needs:
      - monitoring/jaeger-operator
    hooks:
      - events:
          - prepare
          - cleanup
        command: ./helmify.sh
        args:
          - "{{`{{if eq .Event.Name \"prepare\"}}build{{else}}clean{{end}}`}}"
          - "{{`{{.Release.Chart}}`}}"
          - badger
    disableValidation: true
mumoshu avatar
mumoshu

gotcha. i certainly forgot to update the doc about that

Joaquin Menchaca avatar
Joaquin Menchaca

BTW. Side note, thank you so much for disableValidation and needs.

mumoshu avatar
mumoshu

Joaquin Menchaca avatar
Joaquin Menchaca

Lifesaver

mumoshu avatar
mumoshu

glad to hear

Joaquin Menchaca avatar
Joaquin Menchaca

When I had an exact problem, you had an exact solution

1
1
Joaquin Menchaca avatar
Joaquin Menchaca

operators + crds w/ kustomize becoming popular, so this was quite useful…

Joaquin Menchaca avatar
Joaquin Menchaca

So chartify is more robust kustomize-2-chart

mumoshu avatar
mumoshu

yeah.. it was pain for me as well to manage kustomize/helm mix

mumoshu avatar
mumoshu

i believe so

mumoshu avatar
mumoshu

i think you can now rewrite your example with

releases:
  - name: jaeger
    namespace: monitoring
    chart: ./jaeger
    needs:
      - monitoring/jaeger-operator
mumoshu avatar
mumoshu

or with predefined values

releases:
  - name: jaeger
    namespace: monitoring
    chart: ./jaeger
    values:
    - namespace: whatever #=> kustomize edit set namespace=whatever
    needs:
      - monitoring/jaeger-operator
Joaquin Menchaca avatar
Joaquin Menchaca

That’s gorgeous

Joaquin Menchaca avatar
Joaquin Menchaca

So chartify is apart of helmify now, as a library?

Joaquin Menchaca avatar
Joaquin Menchaca

My overlay kustomize is:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

bases:
  - ../../base
patches:
  - storage.yaml
  - ingress.yaml
Joaquin Menchaca avatar
Joaquin Menchaca

I didn’t overlay w/ environment, but rather features I wanted to toggle on/off

mumoshu avatar
mumoshu

basically yes. chartify is integrated into helmfile

Joaquin Menchaca avatar
Joaquin Menchaca

the jaeger has so many backend features, badger, cassandra, elasticsearch for storage, and spark/kafka integration features, so these would be alternative overlays.

Joaquin Menchaca avatar
Joaquin Menchaca

That’s awesome.

Joaquin Menchaca avatar
Joaquin Menchaca

Some examples, or unit tests, would be nice, I’ll blog about this stuff, add some docs… this is beyond awesome, needs to be shared.

mumoshu avatar
mumoshu

ah gotcha. that sounds like a valid use-case from jaeger authors perspective :slightly_smiling_face:

from helmfile perspective it might be a bit annoying to manage depending on reqs

i believe someone would want to dynamically generate patches array in the kustomization.yaml from helmfile values. that’s impossible today

mumoshu avatar
mumoshu

a workaround for that pattern would be to prepare a few variation of kustomization.yaml, and switch that depending on helmfile values

releases:
  - name: jaeger
    namespace: monitoring
    chart: ./jaeger-{{ .Values.jaegarVarigationName }}
    needs:
      - monitoring/jaeger-operator
mumoshu avatar
mumoshu

other than that the integrated chartify feature should work perfectly

Joaquin Menchaca avatar
Joaquin Menchaca

I am in a quasi devops, solutions architect, developer support role, so I create code snippets, docs, for tools (monitoring, observability, logs) that could integrate to our product, so that’s why I code various options for customers, they setup something like env.sh, chose what features.

mumoshu avatar
mumoshu

that does make sense a lot

Joaquin Menchaca avatar
Joaquin Menchaca

BTW, separate topic, what do you think about kustomize. Helm is quite different approach.

mumoshu avatar
mumoshu

for me both are fine. but being locked-in to one of them would never be best..

Joaquin Menchaca avatar
Joaquin Menchaca

kustomize folks are opinionated in what they think is best practices is, which seems particular implementation of gitops. From their docs.

mumoshu avatar
mumoshu

yeah. and that “opinion” seems generally ok to me

mumoshu avatar
mumoshu

in the context of gitops, for me, it’s just kustomize build vs helm template, and which format the specific user wants read/write, kustomization or helm chart

mumoshu avatar
mumoshu

imho helm chart template is much powerful as you can work on text, not only yaml. but great power comes with great responsibiliy

Joaquin Menchaca avatar
Joaquin Menchaca

Yep

Joaquin Menchaca avatar
Joaquin Menchaca

Crazy question (another topic), ever thought of doing something similar with docker-compose? Right now, it is pretty inflexible, gotemplating on it sounds nice. It’s becoming a standard (https://www.compose-spec.io/), and supposedly would be an alternative to kubectl manifests.

Joaquin Menchaca avatar
Joaquin Menchaca

In the mean time I was thinking of using gomplate for this.

hairyhenderson/gomplate

A flexible commandline tool for template rendering. Supports lots of local and remote datasources. - hairyhenderson/gomplate

mumoshu avatar
mumoshu

variant2 and https://github.com/cloudposse/atmos should be a great alternative for that cc/ @Erik Osterman (Cloud Posse)

cloudposse/atmos

Universal Tool for DevOps and Cloud Automation. Contribute to cloudposse/atmos development by creating an account on GitHub.

this1
Joaquin Menchaca avatar
Joaquin Menchaca

This is orchestrator orchestrator orchestrator orchestrator…

Joaquin Menchaca avatar
Joaquin Menchaca

I just gave a demo of helmfile to sres and qa in another group today, they were so excited and full of imagination on the possibilities now.

2020-10-25

2020-10-27

Shreyank Sharma avatar
Shreyank Sharma

Hi Helm-Users,

When i do        helm fetch --untar stable/velero this will fetch the helm latest stable helm chart, what command do i  have use to download the specific version of helm chart????

Thank You

voron avatar

Just add --version

      --version string             specify the exact chart version to use. If this is not specified, the latest version is used

Type helm fetch --help to see possible options

Nicolás de la Torre avatar
Nicolás de la Torre

Hi! When using helmfile_release terraform resource, how do you handle helm repositories?

2020-10-28

Shreyank Sharma avatar
Shreyank Sharma

Hello Helm Users in general if i fetch a helm chart repo using helm fetch then, i write a custom value file i.e my-value.yaml and if do,

helm install ./velero -f ./velero/my-value.yaml --name velero --namespace velero

will it refer the default value.yaml which alreday is present or it will ignore the value.yaml ?

Thank you

voron avatar

Hello, helm will merge values from in-chart values.yaml and my-value.yaml , and values from my-value.yaml will override corresponding existing values from values.yaml . You may check official manual too

Values Files

Instructions on how to use the –values flag.

2020-10-29

2020-10-30

    keyboard_arrow_up