#helmfile (2024-04)

https://github.com/helmfile/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles

Archive: https://archive.sweetops.com/helmfile/

2024-04-04

mikaelec avatar
mikaelec

Hi there, Is it possible to refer to values files packaged with the chart?

Similarly to what I can do with Argo and Flux.

E.g. for Argo: https://argo-cd.readthedocs.io/en/stable/user-guide/multiple_sources/#helm-value-files-from-external-git-repository Here values references are implicitly from the chart package, while you need to use $values for sourced files.

Or for Flux: https://fluxcd.io/flux/components/helm/api/v2beta2/#helm.toolkit.fluxcd.io/v2beta2.HelmChartTemplateSpec

yxxhero avatar
yxxhero

you can use filepath.

mikaelec avatar
mikaelec

@yxxhero Sorry, that answer makes no sense to me. Searching through docs and GitHub repo for ‘filepath’ does not help me either.

Erik Jagyugya avatar
Erik Jagyugya

@mikaelec probably you are looking for this: https://helmfile.readthedocs.io/en/latest/paths/

mikaelec avatar
mikaelec

That did not help either. It only mentions paths relative to the helmfile manifest. What I would like is to refer to a file in the Helm chart - i. e.

helm pull "oci://$REGISTRY/$REPOSITORY" --version "$VERSION"
tar -xf *.tgz values.2.yaml --strip-components=1

Except that I was hoping for a builtin way of doing this.

2024-04-05

2024-04-06

2024-04-15

Maks avatar

Hi all, I have a problem understanding helm diff and removing crds from kubernetes cluster when helm apply I have a helmfile which contains 2 charts, the first chart contains crds and other templates and the second one contains crunchy-postgres cluster configuration description, I have clustered these charts in the cluster and everything was working, but there was a need to update and remove crds in a separate chart, for this I released a new tag for the first chart which includes only template files, and released a new chart that includes only the new crds, after applying helmfile -e cpdev diff helm complained that it expects the release-name field with the value crunchy-postgres-operator in the new crds, which was logical since the new chart was now called crunchy-postgres-operator-crds and the current release did not manage it, it was decided to patch the old crd with the command kubectl patch crds [postgresclusters.postgres-operator.crunchydata.com](http://postgresclusters.postgres-operator.crunchydata.com) -p '{"metadata":{"annotations":{"meta.helm. sh/release-name": "crunchy-postgres-operator-crds", "[meta.helm.sh/release-namespace](http://meta.helm.sh/release-namespace)": "crunchy-postgres"}, "labels":{"[app.kubernetes.io/managed-by](http://app.kubernetes.io/managed-by)": "Helm"}}}'' After the patch I saw a normal diff that showed that it removes the old crd and applies the new one, I did helmfile apply and got the crunchy-postgres-operator update, but when updating the operator there were errors and I had to roll back to the old crd and operator version while keeping crunchy-postgres working, I commented out the lines that are associated with the separate chart that only includes crds and changed the tag in the first chart to the old one that includes the template files and the crds files, when applying the helmfile apply I saw that the helmfile expects the old name in the release-name field with the value crunchy-postgres-operator. I patched the new crds to the old fields so that the helmfile would manage it. kubectl patch crds [postgresclusters.postgres-operator.crunchydata.com](http://postgresclusters.postgres-operator.crunchydata.com) -p '{"metadata":{"annotations":{"meta.helm. sh/release-name": "crunchy-postgres-operator", "[meta.helm.sh/release-namespace](http://meta.helm.sh/release-namespace)": "crunchy-postgres"}, "labels":{"[app.kubernetes.io/managed-by](http://app.kubernetes.io/managed-by)": "Helm"}}}'' I applied helmfile diff and saw that old crds are not deleted but only new crds come in, ok I thought, I did helmfile apply and crd was deleted, and since crd deletion deletes all crd related resources my crunchy-postgres cluster with all bases was deleted, why helmfile deleted crd even though it didn’t show it in helmfile diff? How helmfile works with crds and why this situation happened?

2024-04-18

William avatar
William

Hello everyone! I am just starting out with Helmfile and what I would like to do is have one helmfile per tenant in which I define a default template that I can use for all environments, but then be able to define the environments with different versions of charts, but also be able to use a gotmpl to generate a file. Something like this:

values.yaml

type: deployment 
appName: app1

values.yaml.gotmpl

{{ if eq .Values.type "deployment" }}
deployment:
  annotations: {}
  labels: {}
  replicaCount: 1
...
{{-end }}

helmfile.yaml

templates:
  default:
    chart: .
    namespace: '{{`{{ .Release.Namespace }}`}}'
    # This prevents helmfile exiting when it encounters a missing file
    # Valid values are "Error", "Warn", "Info", "Debug". The default is "Error"
    # Use "Debug" to make missing files errors invisible at the default log level(--log-level=INFO)
    missingFileHandler: Warn
    createNamespace: true
    wait: true
    waitForJobs: true
    timeout: 60
    # limit the maximum number of revisions saved per release. Use 0 for no limit (default 10)
    historyMax: 10
    # When set to `true`, skips running `helm dep up` and `helm dep build` on this release's chart.
    skipDeps: false
    values:
    - ./{{`{{ .Environment.Name }}`}}.yaml
    - values.yaml.gotmpl

repositories:
- name: repo
  url: us-central1-docker.pkg.dev/registry/helm-charts
  oci: true
---
environments:
  # The "default" environment is available and used when `helmfile` is run without `--environment NAME`.
  default:
  development:
---
releases:
  - name: tenant-a
    namespace: tenant-a
    inherit:
      - template: default

This only seems to work if I put a values: section under each environment. That kind of defeats the purpose of the template. Am I missing something here?

2024-04-19

Balazs Varga avatar
Balazs Varga

hello all, I have a secret that contains %39 when I do a helmfile apply. how could I escape it ?

William avatar
William

Does anyone know why defaults don’t work in a gotmpl values file when using Helmfile?

yxxhero avatar
yxxhero

please show some example?

William avatar
William

Actually figured this one out. Had to use get. Would love help on the above issue I posted though

1

2024-04-22

William avatar
William

Is anyone around here?

William avatar
William

This has to be the quietest 7600 people I have ever seen.

2
Marvin de Bruin avatar
Marvin de Bruin

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’re not quiet; we’re just communicating through telepathy.

1
William avatar
William

Haha, maybe I need to learn that. Is that a Cloud Posse offering?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Absolutely. Next-gen stuff!

2024-04-23

2024-04-25

Jason avatar

Hey, #helmfile community. I’m encountering an intriguing issue with a helmfile. I’m attempting to pull the Bitnami helm chart from its OCI repo, but I’m observing some peculiar behavior.

This is my helmfile.yaml file:

 repositories:
  - name: bitnami
    url: <https://charts.bitnami.com/bitnami>
    oci: true
  - name: ingress-nginx
    url: <https://kubernetes.github.io/ingress-nginx>
  - name: jetstack
    url: <https://charts.jetstack.io>

releases:
  - name: external-dns
    namespace: kube-addons
    chart: <oci://registry-1.docker.io/bitnamicharts/external-dns>
    version: "7.2.0"

This is the error I am getting. I can’t work out what it means:

helmfile apply
Adding repo ingress-nginx <https://kubernetes.github.io/ingress-nginx>
"ingress-nginx" has been added to your repositories

Adding repo jetstack <https://charts.jetstack.io>
"jetstack" has been added to your repositories

Pulling registry-1.docker.io/bitnamicharts/external-dns:7.2.0
Comparing release=external-dns, chart=c:\Users\jcontent\AppData\Local\Temp\helmfile2956667309\kube-addons\external-dns\external-dns\7.2.0\external-dns
in ./helmfile.yaml: command "C:\\ProgramData\\chocolatey\\bin\\helm.exe" exited with non-zero status:

PATH:
  C:\ProgramData\chocolatey\bin\helm.exe

ARGS:
  0: helm (4 bytes)
  1: diff (4 bytes)
  2: upgrade (7 bytes)
  3: --allow-unreleased (18 bytes)
  4: external-dns (12 bytes)
  5: C:\Users\jcontent\AppData\Local\Temp\helmfile2956667309\kube-addons\external-dns\external-dns\7.2.0\external-dns (112 bytes)
  6: --version (9 bytes)
  7: 7.2.0 (5 bytes)
  8: --namespace (11 bytes)
  9: kube-addons (11 bytes)
  10: --detailed-exitcode (19 bytes)
  11: --color (7 bytes)
  12: --reset-values (14 bytes)

ERROR:
  exit status 1

EXIT STATUS
  1

STDERR:
  Error: unknown command "diff" for "helm"
  Run 'helm --help' for usage.

COMBINED OUTPUT:
  Error: unknown command "diff" for "helm"
  Run 'helm --help' for usage.
bradym avatar

Error: unknown command "diff" for "helm" – you need to install helm-diff https://github.com/databus23/helm-diff

databus23/helm-diff

A helm plugin that shows a diff explaining what a helm upgrade would change

Jason avatar

ahhh

Jason avatar

thanks so ive tried to deploy my full helmfile but I get this error:

STDERR:
  Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "ingress-nginx-controller" namespace: "kube-addons" from "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
  ensure CRDs are installed first
  Error: plugin "diff" exited with error

COMBINED OUTPUT:
  ********************
        Release was not present in Helm.  Diff will show entire contents as new.
  ********************
  Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "ingress-nginx-controller" namespace: "kube-addons" from "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
  ensure CRDs are installed first
  Error: plugin "diff" exited with error

I followed this tutorial to install the CRDs but it made no difference: https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-manifests/

This is my full helmfile:

repositories:
  - name: bitnami
    url: <https://charts.bitnami.com/bitnami>
    oci: true
  - name: ingress-nginx
    url: <https://kubernetes.github.io/ingress-nginx>
  - name: jetstack
    url: <https://charts.jetstack.io>

releases:
  - name: azure-external-dns
    namespace: kube-addons
    chart: <oci://registry-1.docker.io/bitnamicharts/external-dns>
    version: "7.2.0"
    values:
      - provider: azure
        azure:
          resourceGroup: "jc-testing-dns"
          tenantId: "1f166b6b-6491-4686-ba72-dafa4209cadb"
          subscriptionId: "4ef163e9-0a17-4ee5-bba0-ec60bab6f992"
          useManagedIdentityExtension: true
        logLevel:  "debug"
        domainFilters:
          - "private.jcdnstesting.azure.com"
        txtOwnerId: external-dns

  - name: ingress-nginx
    namespace: kube-addons
    chart: ingress-nginx/ingress-nginx
    version: 3.34.0
    values:
      - controller:
          replicaCount: 2
          nodeSelector:
            kubernetes.io/os: linux
          admissionWebhooks:
            patch:
              nodeSelector:
                kubernetes.io/os: linux
          service:
            externalTrafficPolicy: Local
        defaultBackend:
          nodeSelector:
            kubernetes.io/os: linux

  - name: cert-manager
    namespace: kube-addons
    chart: jetstack/cert-manager
    version: 1.4.0
    values:
      - installCRDs: true
        extraArgs:
          - --cluster-resource-namespace=kube-addons
        global:
          logLevel: 2
Installation with Manifests

This guide explains how to install NGINX Ingress Controller in a Kubernetes cluster using manifests. In addition, it provides instructions on how to set up role-based access control, create both common and custom resources, and uninstall NGINX Ingress Controller.

yxxhero avatar
yxxhero

what’s your k8s version?

Jason avatar

Ohh I solved this. I updated the version of the Nginx helm chart I was using.

Jason avatar

Have you ever used it’scontained? Or know how to configure yaml like settings with Helm?

Jason avatar

ive installed nginx in other clusters before using helm and its never moaned at me to install crds…

Jason avatar

erm dw i fixed it by upgrading the chart version

Chris Waldt avatar
Chris Waldt

would using the environments feature be the best way to conditionally install a chart on certain environments? i.e., only installing a chart in dev but not prod lets say. is there a better way of doing this or is the simplest/recommended way?

Jason avatar

ahh im only testing to see if I can get my head around cert-manager im binning off this cluster once I have finished this tutorial im following

Jason avatar

Has anyone used: itscontained before? https://charts.itscontained.io

Jason avatar

There website is down and this tutorial wants me to deploy this helm file with helmfile using itscontained.

repositories:
  - name: itscontained
    url: <https://charts.itscontained.io>
    
releases:
  - name: cert-manager-issuers
    chart: itscontained/raw
    namespace: kube-addons
    version:  0.2.5
    ## only required if releases included in same helmfile
    ## otherwise, comment out
    # needs: 
    #   - kube-addons/cert-manager
    disableValidation: true
    values:
      - resources:
          - apiVersion: cert-manager.io/v1
            kind: ClusterIssuer
            metadata:
              name: letsencrypt-staging
            spec:
              acme:
                server: <https://acme-staging-v02.api.letsencrypt.org/directory>
                email: {{ requiredEnv "ACME_ISSUER_EMAIL" }}
                privateKeySecretRef:
                  name: letsencrypt-staging
                solvers:
                  - dns01:
                      azureDNS:
                        subscriptionID: {{ requiredEnv "AZ_SUBSCRIPTION_ID" }}
                        resourceGroupName: {{ requiredEnv "AZ_RESOURCE_GROUP" }}
                        hostedZoneName: {{ requiredEnv "AZ_DNS_DOMAIN" }}
                        environment: AzurePublicCloud

          - apiVersion: cert-manager.io/v1
            kind: ClusterIssuer
            metadata:
              name: letsencrypt-prod
            spec:
              acme:
                server: <https://acme-v02.api.letsencrypt.org/directory>
                email: {{ requiredEnv "ACME_ISSUER_EMAIL" }}
                privateKeySecretRef:
                  name: letsencrypt-prod
                solvers:
                  - dns01:
                      azureDNS:
                        subscriptionID: {{ requiredEnv "AZ_SUBSCRIPTION_ID" }}
                        resourceGroupName: {{ requiredEnv "AZ_RESOURCE_GROUP" }}
                        hostedZoneName: {{ requiredEnv "AZ_DNS_DOMAIN" }}
                        environment: AzurePublicCloud
Jason avatar

what does itscontained do?

Jason avatar

and is there an alternative?

2024-04-26

2024-04-29

Balazs Varga avatar
Balazs Varga

hello all, I have a pre-delete-hook and I would like to use wait for helm… how can I force the helm to wait until hook finsihed. I use a job. Job deployed, but not finished… helm does not wait for it.

Shin Imai avatar
Shin Imai

Hi community, I’m new here. Just a quick question. Is there a way to run a test pod (helmfile test) without updating charts? The chart I want to run a test pod doesn’t contain test templates. I googled around but I couldn’t find a way.

2024-04-30

Brandon avatar
Brandon

helmfile template is quite slow when using the helmfile-of-helmfiles pattern. How can I speed it up? I already use --skip-deps.

    keyboard_arrow_up