#helmfile (2024-04)
Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles
Archive: https://archive.sweetops.com/helmfile/
2024-04-04
![mikaelec avatar](https://secure.gravatar.com/avatar/0b824cdb297e544d6938923d5933e742.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0003-72.png)
Hi there, Is it possible to refer to values files packaged with the chart?
Similarly to what I can do with Argo and Flux.
E.g. for Argo: https://argo-cd.readthedocs.io/en/stable/user-guide/multiple_sources/#helm-value-files-from-external-git-repository Here values references are implicitly from the chart package, while you need to use $values for sourced files.
Or for Flux: https://fluxcd.io/flux/components/helm/api/v2beta2/#helm.toolkit.fluxcd.io/v2beta2.HelmChartTemplateSpec
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
you can use filepath.
![mikaelec avatar](https://secure.gravatar.com/avatar/0b824cdb297e544d6938923d5933e742.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0003-72.png)
@yxxhero Sorry, that answer makes no sense to me. Searching through docs and GitHub repo for ‘filepath’ does not help me either.
![Erik Jagyugya avatar](https://secure.gravatar.com/avatar/8d0e1e1a2c6f662e90f0adc2c1cac0e0.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0016-72.png)
@mikaelec probably you are looking for this: https://helmfile.readthedocs.io/en/latest/paths/
![mikaelec avatar](https://secure.gravatar.com/avatar/0b824cdb297e544d6938923d5933e742.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0003-72.png)
That did not help either. It only mentions paths relative to the helmfile manifest. What I would like is to refer to a file in the Helm chart - i. e.
helm pull "oci://$REGISTRY/$REPOSITORY" --version "$VERSION"
tar -xf *.tgz values.2.yaml --strip-components=1
Except that I was hoping for a builtin way of doing this.
2024-04-05
2024-04-06
2024-04-15
![Maks avatar](https://secure.gravatar.com/avatar/8d28b8c7f87bfe9a8ae518edb29e3005.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
Hi all, I have a problem understanding helm diff and removing crds from kubernetes cluster when helm apply
I have a helmfile which contains 2 charts, the first chart contains crds and other templates and the second one contains crunchy-postgres cluster configuration description, I have clustered these charts in the cluster and everything was working, but there was a need to update and remove crds in a separate chart, for this I released a new tag for the first chart which includes only template files, and released a new chart that includes only the new crds, after applying helmfile -e cpdev diff
helm complained that it expects the release-name
field with the value crunchy-postgres-operator
in the new crds, which was logical since the new chart was now called crunchy-postgres-operator-crds
and the current release did not manage it, it was decided to patch the old crd with the command
kubectl patch crds [postgresclusters.postgres-operator.crunchydata.com](http://postgresclusters.postgres-operator.crunchydata.com) -p '{"metadata":{"annotations":{"meta.helm. sh/release-name": "crunchy-postgres-operator-crds", "[meta.helm.sh/release-namespace](http://meta.helm.sh/release-namespace)": "crunchy-postgres"}, "labels":{"[app.kubernetes.io/managed-by](http://app.kubernetes.io/managed-by)": "Helm"}}}''
After the patch I saw a normal diff that showed that it removes the old crd and applies the new one, I did helmfile apply and got the crunchy-postgres-operator update, but when updating the operator there were errors and I had to roll back to the old crd and operator version while keeping crunchy-postgres working, I commented out the lines that are associated with the separate chart that only includes crds and changed the tag in the first chart to the old one that includes the template files and the crds files, when applying the helmfile apply I saw that the helmfile expects the old name in the release-name
field with the value crunchy-postgres-operator
.
I patched the new crds to the old fields so that the helmfile would manage it.
kubectl patch crds [postgresclusters.postgres-operator.crunchydata.com](http://postgresclusters.postgres-operator.crunchydata.com) -p '{"metadata":{"annotations":{"meta.helm. sh/release-name": "crunchy-postgres-operator", "[meta.helm.sh/release-namespace](http://meta.helm.sh/release-namespace)": "crunchy-postgres"}, "labels":{"[app.kubernetes.io/managed-by](http://app.kubernetes.io/managed-by)": "Helm"}}}''
I applied helmfile diff and saw that old crds are not deleted but only new crds come in, ok I thought, I did helmfile apply and crd was deleted, and since crd deletion deletes all crd related resources my crunchy-postgres cluster with all bases was deleted, why helmfile deleted crd even though it didn’t show it in helmfile diff? How helmfile works with crds and why this situation happened?
2024-04-18
![William avatar](https://secure.gravatar.com/avatar/457ed395ac6444eda5a8528373a510bb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0009-72.png)
Hello everyone! I am just starting out with Helmfile and what I would like to do is have one helmfile per tenant in which I define a default template that I can use for all environments, but then be able to define the environments with different versions of charts, but also be able to use a gotmpl
to generate a file. Something like this:
values.yaml
type: deployment
appName: app1
values.yaml.gotmpl
{{ if eq .Values.type "deployment" }}
deployment:
annotations: {}
labels: {}
replicaCount: 1
...
{{-end }}
helmfile.yaml
templates:
default:
chart: .
namespace: '{{`{{ .Release.Namespace }}`}}'
# This prevents helmfile exiting when it encounters a missing file
# Valid values are "Error", "Warn", "Info", "Debug". The default is "Error"
# Use "Debug" to make missing files errors invisible at the default log level(--log-level=INFO)
missingFileHandler: Warn
createNamespace: true
wait: true
waitForJobs: true
timeout: 60
# limit the maximum number of revisions saved per release. Use 0 for no limit (default 10)
historyMax: 10
# When set to `true`, skips running `helm dep up` and `helm dep build` on this release's chart.
skipDeps: false
values:
- ./{{`{{ .Environment.Name }}`}}.yaml
- values.yaml.gotmpl
repositories:
- name: repo
url: us-central1-docker.pkg.dev/registry/helm-charts
oci: true
---
environments:
# The "default" environment is available and used when `helmfile` is run without `--environment NAME`.
default:
development:
---
releases:
- name: tenant-a
namespace: tenant-a
inherit:
- template: default
This only seems to work if I put a values:
section under each environment. That kind of defeats the purpose of the template.
Am I missing something here?
2024-04-19
![Balazs Varga avatar](https://secure.gravatar.com/avatar/944e59f1543dc43935bda4d7b9be7f85.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0019-72.png)
hello all,
I have a secret that contains %39
when I do a helmfile apply. how could I escape it ?
![William avatar](https://secure.gravatar.com/avatar/457ed395ac6444eda5a8528373a510bb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0009-72.png)
Does anyone know why defaults don’t work in a gotmpl values file when using Helmfile?
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
please show some example?
![William avatar](https://secure.gravatar.com/avatar/457ed395ac6444eda5a8528373a510bb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0009-72.png)
Actually figured this one out. Had to use get
. Would love help on the above issue I posted though
2024-04-22
![William avatar](https://secure.gravatar.com/avatar/457ed395ac6444eda5a8528373a510bb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0009-72.png)
Is anyone around here?
![William avatar](https://secure.gravatar.com/avatar/457ed395ac6444eda5a8528373a510bb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0009-72.png)
This has to be the quietest 7600 people I have ever seen.
![Marvin de Bruin avatar](https://secure.gravatar.com/avatar/af508d6253f59476e0c1e098bf29340c.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0004-72.png)
![Erik Osterman (Cloud Posse) avatar](https://secure.gravatar.com/avatar/88c480d4f73b813904e00a5695a454cb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0023-72.png)
![William avatar](https://secure.gravatar.com/avatar/457ed395ac6444eda5a8528373a510bb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0009-72.png)
Haha, maybe I need to learn that. Is that a Cloud Posse offering?
![Erik Osterman (Cloud Posse) avatar](https://secure.gravatar.com/avatar/88c480d4f73b813904e00a5695a454cb.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0023-72.png)
Absolutely. Next-gen stuff!
2024-04-23
2024-04-25
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
Hey, #helmfile community. I’m encountering an intriguing issue with a helmfile. I’m attempting to pull the Bitnami helm chart from its OCI repo, but I’m observing some peculiar behavior.
This is my helmfile.yaml file:
repositories:
- name: bitnami
url: <https://charts.bitnami.com/bitnami>
oci: true
- name: ingress-nginx
url: <https://kubernetes.github.io/ingress-nginx>
- name: jetstack
url: <https://charts.jetstack.io>
releases:
- name: external-dns
namespace: kube-addons
chart: <oci://registry-1.docker.io/bitnamicharts/external-dns>
version: "7.2.0"
This is the error I am getting. I can’t work out what it means:
helmfile apply
Adding repo ingress-nginx <https://kubernetes.github.io/ingress-nginx>
"ingress-nginx" has been added to your repositories
Adding repo jetstack <https://charts.jetstack.io>
"jetstack" has been added to your repositories
Pulling registry-1.docker.io/bitnamicharts/external-dns:7.2.0
Comparing release=external-dns, chart=c:\Users\jcontent\AppData\Local\Temp\helmfile2956667309\kube-addons\external-dns\external-dns\7.2.0\external-dns
in ./helmfile.yaml: command "C:\\ProgramData\\chocolatey\\bin\\helm.exe" exited with non-zero status:
PATH:
C:\ProgramData\chocolatey\bin\helm.exe
ARGS:
0: helm (4 bytes)
1: diff (4 bytes)
2: upgrade (7 bytes)
3: --allow-unreleased (18 bytes)
4: external-dns (12 bytes)
5: C:\Users\jcontent\AppData\Local\Temp\helmfile2956667309\kube-addons\external-dns\external-dns\7.2.0\external-dns (112 bytes)
6: --version (9 bytes)
7: 7.2.0 (5 bytes)
8: --namespace (11 bytes)
9: kube-addons (11 bytes)
10: --detailed-exitcode (19 bytes)
11: --color (7 bytes)
12: --reset-values (14 bytes)
ERROR:
exit status 1
EXIT STATUS
1
STDERR:
Error: unknown command "diff" for "helm"
Run 'helm --help' for usage.
COMBINED OUTPUT:
Error: unknown command "diff" for "helm"
Run 'helm --help' for usage.
![bradym avatar](https://avatars.slack-edge.com/2023-06-21/5464816405572_dd21bed1bf537acb6539_72.jpg)
Error: unknown command "diff" for "helm"
– you need to install helm-diff
https://github.com/databus23/helm-diff
A helm plugin that shows a diff explaining what a helm upgrade would change
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
ahhh
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
thanks so ive tried to deploy my full helmfile but I get this error:
STDERR:
Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "ingress-nginx-controller" namespace: "kube-addons" from "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
ensure CRDs are installed first
Error: plugin "diff" exited with error
COMBINED OUTPUT:
********************
Release was not present in Helm. Diff will show entire contents as new.
********************
Error: Failed to render chart: exit status 1: Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "ingress-nginx-controller" namespace: "kube-addons" from "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
ensure CRDs are installed first
Error: plugin "diff" exited with error
I followed this tutorial to install the CRDs but it made no difference: https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-manifests/
This is my full helmfile:
repositories:
- name: bitnami
url: <https://charts.bitnami.com/bitnami>
oci: true
- name: ingress-nginx
url: <https://kubernetes.github.io/ingress-nginx>
- name: jetstack
url: <https://charts.jetstack.io>
releases:
- name: azure-external-dns
namespace: kube-addons
chart: <oci://registry-1.docker.io/bitnamicharts/external-dns>
version: "7.2.0"
values:
- provider: azure
azure:
resourceGroup: "jc-testing-dns"
tenantId: "1f166b6b-6491-4686-ba72-dafa4209cadb"
subscriptionId: "4ef163e9-0a17-4ee5-bba0-ec60bab6f992"
useManagedIdentityExtension: true
logLevel: "debug"
domainFilters:
- "private.jcdnstesting.azure.com"
txtOwnerId: external-dns
- name: ingress-nginx
namespace: kube-addons
chart: ingress-nginx/ingress-nginx
version: 3.34.0
values:
- controller:
replicaCount: 2
nodeSelector:
kubernetes.io/os: linux
admissionWebhooks:
patch:
nodeSelector:
kubernetes.io/os: linux
service:
externalTrafficPolicy: Local
defaultBackend:
nodeSelector:
kubernetes.io/os: linux
- name: cert-manager
namespace: kube-addons
chart: jetstack/cert-manager
version: 1.4.0
values:
- installCRDs: true
extraArgs:
- --cluster-resource-namespace=kube-addons
global:
logLevel: 2
This guide explains how to install NGINX Ingress Controller in a Kubernetes cluster using manifests. In addition, it provides instructions on how to set up role-based access control, create both common and custom resources, and uninstall NGINX Ingress Controller.
![yxxhero avatar](https://avatars.slack-edge.com/2022-03-26/3311927616433_06648f4d626111baaac9_72.jpg)
what’s your k8s version?
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
Ohh I solved this. I updated the version of the Nginx helm chart I was using.
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
Have you ever used it’scontained? Or know how to configure yaml like settings with Helm?
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
ive installed nginx in other clusters before using helm and its never moaned at me to install crds…
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
erm dw i fixed it by upgrading the chart version
![Chris Waldt avatar](https://secure.gravatar.com/avatar/92ff6ca2100dbf55068f7651f3ada303.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0003-72.png)
would using the environments feature be the best way to conditionally install a chart on certain environments? i.e., only installing a chart in dev
but not prod
lets say. is there a better way of doing this or is the simplest/recommended way?
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
ahh im only testing to see if I can get my head around cert-manager im binning off this cluster once I have finished this tutorial im following
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
Has anyone used: itscontained before? https://charts.itscontained.io
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
There website is down and this tutorial wants me to deploy this helm file with helmfile using itscontained.
repositories:
- name: itscontained
url: <https://charts.itscontained.io>
releases:
- name: cert-manager-issuers
chart: itscontained/raw
namespace: kube-addons
version: 0.2.5
## only required if releases included in same helmfile
## otherwise, comment out
# needs:
# - kube-addons/cert-manager
disableValidation: true
values:
- resources:
- apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
server: <https://acme-staging-v02.api.letsencrypt.org/directory>
email: {{ requiredEnv "ACME_ISSUER_EMAIL" }}
privateKeySecretRef:
name: letsencrypt-staging
solvers:
- dns01:
azureDNS:
subscriptionID: {{ requiredEnv "AZ_SUBSCRIPTION_ID" }}
resourceGroupName: {{ requiredEnv "AZ_RESOURCE_GROUP" }}
hostedZoneName: {{ requiredEnv "AZ_DNS_DOMAIN" }}
environment: AzurePublicCloud
- apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: <https://acme-v02.api.letsencrypt.org/directory>
email: {{ requiredEnv "ACME_ISSUER_EMAIL" }}
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- dns01:
azureDNS:
subscriptionID: {{ requiredEnv "AZ_SUBSCRIPTION_ID" }}
resourceGroupName: {{ requiredEnv "AZ_RESOURCE_GROUP" }}
hostedZoneName: {{ requiredEnv "AZ_DNS_DOMAIN" }}
environment: AzurePublicCloud
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
what does itscontained do?
![Jason avatar](https://secure.gravatar.com/avatar/ff4ceac5b91b00c5969226d048ab4d4e.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
and is there an alternative?
2024-04-26
2024-04-29
![Balazs Varga avatar](https://secure.gravatar.com/avatar/944e59f1543dc43935bda4d7b9be7f85.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0019-72.png)
hello all, I have a pre-delete-hook and I would like to use wait for helm… how can I force the helm to wait until hook finsihed. I use a job. Job deployed, but not finished… helm does not wait for it.
![Shin Imai avatar](https://secure.gravatar.com/avatar/c68b2347ef98fae7a2b58e1834c7c0f0.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0022-72.png)
Hi community, I’m new here. Just a quick question. Is there a way to run a test pod (helmfile test) without updating charts? The chart I want to run a test pod doesn’t contain test templates. I googled around but I couldn’t find a way.
2024-04-30
![Brandon avatar](https://secure.gravatar.com/avatar/17c631c59e0b012f6aeef896323eeb77.jpg?s=72&d=https%3A%2F%2Fa.slack-edge.com%2Fdf10d%2Fimg%2Favatars%2Fava_0011-72.png)
helmfile template
is quite slow when using the helmfile-of-helmfiles pattern. How can I speed it up?
I already use --skip-deps
.