#helmfile (2021-07)

https://github.com/helmfile/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles

Archive: https://archive.sweetops.com/helmfile/

2021-07-01

2021-07-03

Vladimir Avdoshka avatar
Vladimir Avdoshka

Hello! I’m wondering how do you develop/test things locally with Helmfile? I’m looking for some kind of seamless (automated) process of having the feedback loop on dev environment, wether it be triggered by some change of the code or helm chart or Helmfile. Similar to what one can do with the skaffold.

I’ve opened an issue in Helmfile - https://github.com/roboll/helmfile/issues/1903 to discuss this. Thank you for sharing!

Best practices of local development with Helmfile? · Issue #1903 · roboll/helmfileattachment image

I guess one of the greatest features of the Helmfile is the reusability of the deployment logic across local/ci/prod environments, and it seems there are tons of choices of how one can use Helmfile…

2021-07-04

Amos avatar

Hi folks! I’m trying to loop over same release but every iteration needs a different set of values per release iteration.

The helmfile looks like this:

releases:
{{ range $i, $releases := until .Values.RELEASE_COUNTER }}
{{ $chartNumber := add1 $releases | toString }}
- name: release-number-{{ $chartNumber }}
  <<: *default
  labels:
    app: '{{`{{ .Release.Name }}`}}'
    chart: some-chart
    tier: core
{{- end }}

What is the best way to achieve that with helmfile ? https://github.com/roboll/helmfile/issues/1906

[ Question ] Iterate over same release with different values · Issue #1906 · roboll/helmfileattachment image

Hi! I&#39;m trying to loop over same release but every iteration needs a different set of values per release iteration. The helmfile looks like this: releases: {{ range $i, $releases := until .Valu…

2021-07-08

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

Hi all the project readme shows passCredentials parameter for OCI repos But when using it I received an error

in ./helmfile.yaml: failed to read helmfile.yaml: reading document at index 1: yaml: unmarshal errors:
  line 7: field passCredentials not found in type state.RepositorySpec

helmfile version

$ helmfile version
helmfile version v0.139.9
Jonathan avatar
Jonathan

Could you show how you’re using the parameter?

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk
- name: ghcr
  url: ghcr.io/.....
  username: ...
  password: ...
  oci: true
  passCredentials: true
Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

2021-07-10

Joaquin Menchaca avatar
Joaquin Menchaca

Is there any way to patch output of a helm chart in a helmfile?

Without helmfile, I would do something like this:

helm template ... | linkerd inject | kubectl apply -f -

So I am wondering how I could do the equivalent in helmfile? Make post hook with some patch.json or linkerd inject?

Joaquin Menchaca avatar
Joaquin Menchaca
feat: experimental integration with helm-x by mumoshu · Pull Request #673 · roboll/helmfileattachment image

This enhances helmfile so that it can: Treat K8s manifests directories and Kustomize projects as charts Add adhoc chart dependencies on sync/diff/template without forking or modifying chart(s) (#6…

Joaquin Menchaca avatar
Joaquin Menchaca

Is it possible to inline replacement, trying to do equivalent of --set-file without the set-file.

releases:
  - name: linkerd
    namespace: linkerd
    chart: linkerd/linkerd2
    version: 2.10.2
    values:
      - identityTrustAnchorsPEM: {{ readFile "certs/ca.crt" }}

I thought I would get this in-place, but then I think it is causing maybe an indentation problem. I am not sure.

I get this error

in ./helmfile.yaml: failed to read helmfile.yaml: reading document at index 1: yaml: line 13: could not find expected ':'
Joaquin Menchaca avatar
Joaquin Menchaca
Joaquin Menchaca avatar
Joaquin Menchaca

I tried using this

foo: |
{{ readFile "path/to/file" | indent 2 }}

But as it is a multi-line file, I need the whole block to be idented.

Joaquin Menchaca avatar
Joaquin Menchaca
readFile reads only first line · Issue #731 · roboll/helmfileattachment image
Hi, I have the following folder structure for openldap: in common.yaml.gotmpl I have the following lines: customLdifFiles: init.ldif:{{ readFile &quot;../files/ldif/init.ldif&quot; }} Content of…
Joaquin Menchaca avatar
Joaquin Menchaca
readfile is not identing every line · Issue #1911 · roboll/helmfileattachment image

When using a values list item, readFile + indent is only indenting the first line. Steps to Reproduce helmfile –debug template repositories: - name: linkerd url: https://helm.linkerd.io/stable rel…

Joaquin Menchaca avatar
Joaquin Menchaca

Is it possible to reference the {{ .Release.Namespace }} in the helmfile.yaml?

2021-07-11

Joaquin Menchaca avatar
Joaquin Menchaca

I was following docs from https://github.com/roboll/helmfile#labels-overview, where there’s an example of {{ .Release.Name }} , but I get this:

err: error during helmfile.yaml.part.0 parsing: template: stringTemplate:7:30: executing "stringTemplate" at <.Release.Name>: can't evaluate field Release in type state.EnvironmentTemplateData
in ./helmfile.yaml: error during helmfile.yaml.part.0 parsing: template: stringTemplate:7:30: executing "stringTemplate" at <.Release.Name>: can't evaluate field Release in type state.EnvironmentTemplateData
Joaquin Menchaca avatar
Joaquin Menchaca

I wrote this up, I cannot figure it out from the docs. https://github.com/roboll/helmfile/issues/1914

Cannot render .Release.Name · Issue #1914 · roboll/helmfileattachment image

In following documentation regarding .Release.Name in the helmfile.yaml, I found that I could not get this to resolve correctly. Based on the documentation, using either {{ .Release.Name }} or {{`{…

Joaquin Menchaca avatar
Joaquin Menchaca

After finding a more COMPLETE example online (ref. https://www.arthurkoziel.com/managing-helm-charts-with-helmfile/), I found few things that I was doing wrong, in case anyone interested in this:

• can reference {{{{ .Release.Name }}}} in templates.default values, but not values passed to the helm chart. • cannot seem to use {{ .Release.Name }} in helmfile under releases:

Joaquin Menchaca avatar
Joaquin Menchaca

I eventually settled on:

repositories:
  - name: linkerd
    url: <https://helm.linkerd.io/stable>

templates:
  default: &default
    chart: linkerd/{{`{{ .Release.Name }}`}}
    missingFileHandler: Warn
    values:
      - ./values/{{`{{ .Release.Name }}`}}.yaml.gotmpl
      - installNamespace: false

releases:
  - name: linkerd2
    namespace: linkerd
    <<: *default
  - name: linkerd-viz
    namespace: linkerd-viz
    <<: *default
  - name: linkerd-jaeger
    namespace: linkerd-jaeger
    <<: *default

I wish that I could use if blocks around the .Release.Name, but as these are rendered later, that is not possible. I wish I could also reference the .Release.Name in the helm chart values themselves, as that would be useful.

2021-07-13

 avatar
10:10:33 PM

How many of you received any help from this chat

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
SweetOps #helmfile for July, 2021

SweetOps Slack archive of #helmfile for July, 2021.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

A lot. Just a lot also falls on one mans shoulders :-)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
4

2021-07-17

Joaquin Menchaca avatar
Joaquin Menchaca

On my helmfile templates exploration, I wish I could use the .Release.Name, to branch, like if you are this chart, do this thingy. The best that can be done is to put the chart specific changes values file that is fetched using .Release.Name as part of the file name. There will undoubtedly be redundancy. In looking at github issues, it would look like that to change this current behavior would require some major refactoring, especially as .Release.Name is determine later in the process.

In the case of my linkerd use case, then extensions are all consistent, and because linkerd is more unique and requires deploying CRD plus other components that take a few minutes to come up, I may have these as separate stages, maybe through environments, and let the operator determine when to install the extensions. The other problem is that there might be some rate limit to access docker images from Azure (some anti-competitiveness), but cannot prove 100%, just noticing behavior from Azure to where Linkerd stores their images (AWS) with CrashBackOffs.

Lucho avatar

Hi folks, how are you? I’m new with helm and I want to know if helmfile can help me to deploy Application and Database at the same time. I want to deploy Database only If some flag is true inside my values(eg: mongo.enabled.true )… Someone here have experience doing it ? Or do you recommend another idea? Thanks a lot!

2021-07-18

vicken avatar

Hi all I’m trying to reconcile what the difference between these two (inline vs file) could be (v0.139.9):

I am able to do this, the value with the “env” gets picked up with FOO=bar helmfile -e myenv diff

environments.yaml

environments:
  myenv:
    values:
      - foo: {{ env "FOO" }}

However, it seems referencing the same “env” in a values file does not work. Getting an error in helmfile.d/environments.yaml: failed to read environments.yaml: failed to load environment values file "values/myenv.yaml": yaml: invalid map key: map[interface {}]interface {}{"env \"FOO\"":interface {}(nil)}

environments.yaml

environments:
  myenv:
    values:
      - values/myenv.yaml

values/myenv.yaml

foo: {{ env "FOO" }}

Not sure if this is also supposed to work or not

2021-07-19

z0rc3r avatar

Is there a sensible way install CRDs from urls prior to installing chart that depends on CRDs availability via just helmfile? For example this chart requires CRDs to be installed via kubectl https://github.com/piraeusdatastore/helm-charts/tree/main/charts/snapshot-validation-webhook

piraeusdatastore/helm-chartsattachment image

Collection of useful charts for Piraeus and similar projects - piraeusdatastore/helm-charts

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk
  1. Put CRD to your helmfile repo
  2. add a new release containing those CRD
  3. make your main chart depend (via needs on that new release)
piraeusdatastore/helm-chartsattachment image

Collection of useful charts for Piraeus and similar projects - piraeusdatastore/helm-charts

z0rc3r avatar

Ideally I’d like not to keep CRDs in my repo, hence the OP statement install CRDs from urls

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

OP is an original post ?

z0rc3r avatar

yeah

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

you may write a shell script that call kubectl -f URL and call it from of of your templates / values gotmpl / helmfile

z0rc3r avatar

hmm, interesting, I need to look whether I can template it in a way where helm-diff would work as expected

z0rc3r avatar

thanks!

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

just be sure you’re downloading / applying a fixed (aka pinned) version of remote CRD, not the latest ones

z0rc3r avatar

yeah, version number is stored in url

1
Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

For the templating and helm diff feature I may suggest following

  1. add your URL as a values
  2. create a template that will iterate over those values, download particular URL and outputs it
  3. so when rendering a chart - you be able to see downloaded CRD, thus helm diff may detect and show changes
1
z0rc3r avatar

here is my current solution that does crd’s installation via raw chart and templates with exec. it isn’t pretty, but works rather ok

- name: external-snapshotter-crds
  namespace: kube-system
  chart: incubator/raw
  version: 0.2.4
# trimPrefix-indent-trim pipes do proper allignment of CRD within values
  values:
  - resources:
    - {{ exec "curl" (list "<https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v4.1.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml>") | trimPrefix "\n---\n" | indent 6 | trim }}
    - {{ exec "curl" (list "<https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v4.1.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml>") | trimPrefix "\n---\n" | indent 6 | trim }}
    - {{ exec "curl" (list "<https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v4.1.1/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml>") | trimPrefix "\n---\n" | indent 6 | trim }}
Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

@z0rc3r and where resources are used ? In some template ? or elsewhere ?

z0rc3r avatar

it’s “feature” of incubator/raw chart. resources is a list of plain kube manifests that can be anything valid in kubectl terms

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

ahh, ok ok

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

thnx

Ievgenii Shepeliuk avatar
Ievgenii Shepeliuk

So, congrats

z0rc3r avatar

(on the side note: i wish raw was helmfile’s built-in feature, or at least maintained, as incubator repo is readonly and deprecated)

2021-07-20

2021-07-25

batistein avatar
batistein

Hello all, helmfile is really a great tool. However, there is a use-case that I have all the time and wondered if my cumbersome way is the right one.

I have to add for example istio virtualService and Gateway templates to several charts. Optimal would be a way where I can add templates. Currently I solve the problem by creating a new chart that contains these templates but this is not very good because I can’t reuse values from the original chart and also not the helper.tpl….

Is there any smarter way?

Andrew Nazarov avatar
Andrew Nazarov

Probably a monochart approach is the way to go. You can put everything you might need in a single chart and then enable or disable a certain functionality (for example virtualService) via flags.

We are doing something similar having a base/common chart with templates of all objects we might need and then for microservices we create “real” charts which have this common chart as a dependency.

charts/incubator/monochart at master · cloudposse/chartsattachment image

The “Cloud Posse” Distribution of Kubernetes Applications - charts/incubator/monochart at master · cloudposse/charts

batistein avatar
batistein

Hi thank you very much! This is definitely a good approach for inhouse charts! I forgot in my question the main aspect for my question. I have remote charts as source! So for example I am using pgadmin helm chart and need to add to it a virtualService…

z0rc3r avatar

Try incubator/raw chart. It allows to create CRs via raw chart values, maybe it would work for you

Andrew Nazarov avatar
Andrew Nazarov

You might find some of the kustomize features of helmfile useful as well

helmfile/advanced-features.md at master · roboll/helmfileattachment image

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

batistein avatar
batistein

thank you very much!

2021-07-26

2021-07-27

batistein avatar
batistein

Hello together,

I have several helmfiles which I want to bundle. So far I do this via helmfiles: in another helmfile. However I then lose the parallel processing of releases. Is there a way to wrap helmfiles and still specify that some of the subhelmfiles can run in parallel?

voron avatar

the only option I’m aware of is to use single helmfile with some other sort of including, like {{- tpl (readFile "../common/templates.yaml") . | nindent 0 }}

batistein avatar
batistein

Wow :tada: Awesome you saved my day! I tried before with readFile "../common/templates.yaml" but this was rendered as last. tpl did the trick Definitely something for the docs.

batistein avatar
batistein

@voron but how to you overcome this problem?

  line 5: field releases already set in type state.HelmState
  line 28: field releases already set in type state.HelmState
  line 68: field releases already set in type state.HelmState
voron avatar

we don’t overcome this. It should be single helmfile, we cannot include helmfile.yaml into helmfile.yaml Some gotmpl magic may be useful to build single helmfile, f. e. https://github.com/dysnix/helmfiles/blob/master/releases/cert-manager/helmfile.yaml but it’s not a helmfiles: replacement.

helmfiles/helmfile.yaml at master · dysnix/helmfilesattachment image

Dysnix Distribution of Helmfiles for Kubernetes. Contribute to dysnix/helmfiles development by creating an account on GitHub.

2021-07-28

Sean S avatar

Hello everyone, Is it possible to construct helmfile dynamically from a yaml config file or some other format? I wanted have a service specific config file where I can define all my dependencies and then on run time it constructs a helmfile

Andrew Nazarov avatar
Andrew Nazarov

Don’t think I got you right. However, we are kinda creating some of our helmfiles dynamically using, for example, range function.

{{- range $instance, $map := .Values.instances }}

It reads environment values and construct some parts out of it.

Sean S avatar

Thanks for your response @Andrew Nazarov . I have common repo which performs some action on all services. I want this common repo to have config file which construct helmfile and deploy on kube clusters

Sean S avatar

or can we pass a git repo file address as chart value in helmfile

muhaha avatar

Guys? I am trying to recreate map from values to CR, but getting null instead, any idea? Thanks

environments:
  default:
    values:
      - applications:
        - name: test-app
          spec:
            project: test-project
            source:
              repoURL: <https://git.example.com/foo/bar>
              targetRevision: HEAD
              path: john/doe
            destination:
              server: <https://example.com>
              namespace: argocd

repositories:
- name: incubator
  url: <https://charts.helm.sh/incubator>

releases:
- name: argocd-applications
  namespace: argocd
  createNamespace: true
  chart: incubator/raw
  version: 0.2.3
  values:
    - resources:
        {{- range $pIndex, $application := .Values.applications }}
        - apiVersion: argoproj.io/v1alpha1
          kind: Application
          metadata:
            name: {{ $application.name }}
            namespace: argocd
            finalizers:
              - resources-finalizer.argocd.argoproj.io
          spec: 
           {{- toYaml $application.spec | nindent 6 }}
        {{- end }}
Sean S avatar

hello everyone, can we’ve helmfile defined in some-other repo ? which point to chart that reside in some other github repo

voron avatar

use git sub-module to add external helmfiles

use chart repos, maybe helm-git, to point to chart in another git repo

Sean S avatar

thanks @voron

Sean S avatar

@voron: can you point me any example where we using / pointing to remote charts using helm-git in helmfile ?

Sean S avatar

ok got it

# helm-git powered repository: You can treat any Git repository as a charts repository
- name: polaris
  url: git+<https://github.com/reactiveops/polaris@deploy/helm?ref=master>
voron avatar

you may hit issues w/ helm-git and large charts repos, thus use it with small repos only.

Sean S avatar

ok.. my usecase is to just pass helm charts for a Service and it’s dependency in a different repo where i’m constructing helmfile dynamically

2021-07-29

tomaszjdul avatar
tomaszjdul

Hello, is there any way to include encoded secret.yaml inside helfile via gitlab url raw file? I’ve tried using url: as well ref+sops wo any result

voron avatar

is git submodule suitable for you ? Not a helmfile-specific way, but …

tomaszjdul avatar
tomaszjdul

I would prefer to avoid such tricks but i need it asap so @voron Could you send any examples?

2021-07-30

Sean S avatar

hello All, can we create a template / yaml file with app configurations and create helmfile dynamically using this template config file?

    keyboard_arrow_up