#helmfile (2023-03)

https://github.com/helmfile/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles

Archive: https://archive.sweetops.com/helmfile/

2023-03-02

William avatar
William

Hi everyone!

Is there anyway that helmfile apply will do an upgrade of a release even when helm diff does not detect changes in that release?

Currently what we see is that we have a release A and B.

  1. Helm diff detects changes in release A but not in B
  2. We get the information that release A is updated from helm diff.
  3. Afterwards, we get the output in terminal “Affected releases are: B Updated”
  4. Checking the cluster afterwards in helm list - we can see the version number has incremented for both releases
yxxhero avatar
yxxhero

@William please post a issue. and need more info.

1

2023-03-07

Phil avatar

Hey guys!

Is there anyway to “package” a helmfile? It’s already possible to point to a remote helmfile, but what I’d like to do is to add some custom manifests into it (like argocd application, config file, etc), but I’m not sure what’s the best way to achieve that. The simplest way I can think of would be to just write all those manifests into the helmfile itself, but looking at the full helmfile specification I don’t think it’s possible to do that.

yxxhero avatar
yxxhero

@Brandon you can post a feature request.

2023-03-09

Viacheslav avatar
Viacheslav

Hi everyone,

As a part of the automation pipeline, I generate stateValues.yaml , and my helmfiles located in helmfile.d/ . But when I run apply or template, I got this error:

helmfile --state-values-file stateValues.yaml template -e asa -l app=kubeflow-core -i --skip-deps
in helmfile.d/00-cluster-addons.yaml: environment values file matching "stateValues.yaml" does not exist in "."

As I understand, it happens because the working directory changed to helmfile.d/ when Helmfile proceeded multiple helmfiles, but my stateValues.yaml locates in the root folder, so Helmfile tries to find helmfile.d/satetValues.yaml and fails, because a path to this file is relative.

My question: is there any applicable solution for this issue, besides providing a state values file as ${PWD}/stateValues.yaml ? Maybe some change of folders structure (environments, bases, helmfile.d) may help?

I’ll be glad to hear any advice.

yxxhero avatar
yxxhero

please provide your folder structure now.

Viacheslav avatar
Viacheslav

@yxxhero

bases
  environments.yaml
  helmdefaults.yaml
  repositories.yaml
environments
  product
    asa
      values.yaml
      stateValues.yaml
helmfile.d
  00-cluster-addons.yaml
  01-kubeflow.yaml
templates
  3rd-party
    <chartname>
      values.gotmpl
bases.yaml

all references in files points to helmfile.d/*yaml location, for example ./bases.yaml references starts with double dots:

bases:
- ../bases/helmdefaults.yaml
- ../bases/environments.yaml
yxxhero avatar
yxxhero

ok. Thank you.

yxxhero avatar
yxxhero

@Viacheslav maybe you can use subhelmfile feature. do you mind showing the context of helmfile.d.

Viacheslav avatar
Viacheslav

Sure, @yxxhero. The one of files under helmfile.d looks like:

{{ readFile "../bases.yaml" }}
---
repositories:
- name: banzaicloud-stable
  url: <https://kubernetes-charts.banzaicloud.com>

templates:
  infra: &infra
    values:
    - ../templates/3rd-party/{{`{{ .Release.Name }}`}}/values.gotmpl

releases:
- name: mychart
  <<: *infra
  chart: mychart
  version: 0.0.1
  namespace: default
  labels:
    app: '{{`{{ .Release.Name }}`}}'

Others are similar but for different software sets. And base.yaml has a reference to base/environments.yaml, which contains:

environments:
  {{ .Environment.Name }}:
    values:
    - ../environments/project/{{ .Environment.Name }}/values.yaml
    secrets:
    - ../environments/project/{{ .Environment.Name }}/secrets.yaml

And helmfile accepts this structure when I pass absolute path to stateValues.yaml.

About subhelmfiles, does it mean to use a single helmfile.yaml, but specify all helmfiles that are currently located under helmfile.d as a list?

helmfiles:
  - path: helmfiles/00-cluster-addons.yaml
  - path: helmfiles/01-kubeflow.yaml
yxxhero avatar
yxxhero

yeah. you are right.

yxxhero avatar
yxxhero

give you a try.

yxxhero avatar
yxxhero
helmfiles:
  - path: 00-cluster-addons.yaml
  - path: 01-kubeflow.yaml
yxxhero avatar
yxxhero

or set the location of stateValues.yaml generated to helmfile.d every time?

Viacheslav avatar
Viacheslav

@yxxhero thanks a lot :slightly_smiling_face: , I will check it. Now I see some variables become missing (product name) when I did this change, so I’ll try to debug this.

could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read ../bases/environments.yaml.part.0: environment values file matching "../environments//asa/values.yaml" does not exist in "."
error in first-pass rendering: result of "../bases/environments.yaml.part.0":
 0: environments:
 1:   asa:
 2:     values:
 2:     - ../environments/<no value>/asa/values.yaml
 3:     secrets:
 5:     - ../environments/<no value>/asa/secrets.yaml
 6: 

Regarding forced location of stateValues - it’s possible, but undesirable. Automation tool is atmos and to do this, it requires to split helmfile apply command in two steps: generate file, and the apply with this file. The same approach can be used to make ${PWD}/stateValues.yaml which also solve this issue, but I wish to avoid using multiple commands to make the life of an Operator easier.

I will update a thread when check helmfiles: as you suggested. Have a good weekend!

yxxhero avatar
yxxhero

You too. if you have more issue. please post issues in helmfile/helmfile.

1
Andrew Nazarov avatar
Andrew Nazarov

Hi. A question about the helmfile’s get function. I’m wondering if it supports nesting. For example if I want to use as default a value of .Values.myAnotherValue , but if this doesn’t exist - some another default. Couldn’t figure out the syntax. The naive assumption would be something like:

myVal: {{ .Value | get "myValue" (get "myAnotherValue" "1" ".Value") }}

But it definitely won’t work. Or what would be a proper way to achieve something like this?

bradym avatar

Have you tried something like this?

myVal: {{ .Values.myValue | default .Values.myAnotherValue }}
Andrew Nazarov avatar
Andrew Nazarov

But here if I don’t have myValue and myAnotherValue keys defined I’ll get an error.

Andrew Nazarov avatar
Andrew Nazarov

It seems I’ve found out why my approach didn’t work. Will try out one thing.

bradym avatar

Ahh, yes. Good point.

Andrew Nazarov avatar
Andrew Nazarov

It seems

myVal: {{ .Values | get "myValue" (get "myAnotherValue" "1" .Values) }}

works. The problem was the type of the third argument - it shouldn’t be string (note the absence of quotes).

(the typo about plural .Values wasn’t the case, in the real code everything was correct regarding this)

bradym avatar

Seems like it’s always something simple like that.

1
yxxhero avatar
yxxhero

awesome works.

yxxhero avatar
yxxhero

good example.

2023-03-10

2023-03-13

Brandon avatar
Brandon

where can I find docs explaining all the ways to use ref+ strings? I want to try and use them to refer to a non-secret file.

1
z0rc3r avatar
helmfile/vals

Helm-like configuration values loader with support for various sources

1
Brandon avatar
Brandon

@z0rc3r perfect. Thank you

yxxhero avatar
yxxhero

@Brandon welcome to post doc PR for this.

2023-03-15

z0rc3r avatar

It here a way in helmfile.yaml to set --skip-crds for one specific deployment? I have one chart deployment that includes CRDs that won’t be needed in my case, but rest of deployments in the same helmfile must install crds.

yxxhero avatar
yxxhero

set –skip-crds in a release? right?

z0rc3r avatar

@yxxhero if I understood you correctly, then yes.

AFAIU --skip-crds can be passed via cli for the whole helmfile. But I want to have this option to be available per release in helmfile.

yxxhero avatar
yxxhero

@z0rc3r yeah. you got me. would you mind posting an issue in helmfile/helmfile. so we can discuss the feature together.

z0rc3r avatar

sure, will do

z0rc3r avatar
#747 Allow to set `--skip-crds` per release in helmfile

There are/was hacky ways to pass this arg in cli for whole helmfile, like #713

I have a use case, where I want to pass this argument to single release in helmfile, but let other releases in the same helmfile to allow installing CRDs as usual.

yxxhero avatar
yxxhero

yxxhero avatar
yxxhero

@z0rc3r do you have an interest in doing that?

yxxhero avatar
yxxhero

z0rc3r avatar

I do, but i’m bit time constrained now, my golang expriense is kinda low and i’m not versed in helmfile codebase. So at best it will be a drag.

If i can allocate some time, i’ll submit a PR. But for now please don’t expect this to happen

yxxhero avatar
yxxhero

@z0rc3r welcome. I will give suggestions as more as I can.

1

2023-03-16

Ivo avatar

I’m just getting started with Helmfile. I have this basic helmfile:

releases:
  - name: argo-cd
    namespace: argocd
    chart: ../../../charts/argo-cd/5.23.3
    values:
      - ./values.yaml

The ArgoCD charts exists locally on my filesystem. When I run helmfile template I get:

Building dependency release=argo-cd, chart=../../../charts/argo-cd/5.23.3
in ./helmfile.yaml: [building dependencies of local chart: command "helm" exited with non-zero status:

PATH:
  helm

ARGS:
  0: helm (4 bytes)
  1: dependency (10 bytes)
  2: build (5 bytes)
  3: ../../../charts/argo-cd/5.23.3 (30 bytes)

ERROR:
  exit status 1

EXIT STATUS
  1

STDERR:
  Error: no repository definition for <https://dandydeveloper.github.io/charts/>. Please add the missing repos via 'helm repo add'

COMBINED OUTPUT:
  Error: no repository definition for <https://dandydeveloper.github.io/charts/>. Please add the missing repos via 'helm repo add']

Is Helmfile not able to resolve transitive dependencies? Do I need to specify this manually?

Ivo avatar

dandydeveloper is a dependency of the ArgoCD Helm chart

Ivo avatar

Can the repo be added automatically by Helmfile?

voron avatar


Is Helmfile not able to resolve transitive dependencies? Do I need to specify this manually?
it’s helm, not helmfile.

2023-03-17

Gadi Dor avatar
Gadi Dor

Hi All

yxxhero avatar
yxxhero

welcome

Gadi Dor avatar
Gadi Dor

dose helmfile work with argocd?

yxxhero avatar
yxxhero

yeah.

Gadi Dor avatar
Gadi Dor

is there plugin ?

2023-03-22

Dominic Cardellino avatar
Dominic Cardellino

Hello,

I have the problem that when I use helmfile --environment k8s-dev diff --suppress-secrets I still see my provided secrets (via vault) in the output:

loki:
  storage_config:
    aws:
      bucketnames: k8s-{{ .Environment.Values.envPrefix }}-loki
      endpoint: s3.eu-central-1.amazonaws.com
      region: eu-central-1
      access_key_id: <ref+vault://secret/cluster-logging/{{> .Environment.Values.envPrefix }}/aws/s3?proto=https#/aws-access-key-id
      secret_access_key: <ref+vault://secret/cluster-logging/{{> .Environment.Values.envPrefix }}/aws/s3?proto=https#/aws-secret-access-key
      insecure: false
      sse_encryption: false
      s3forcepathstyle: true

I thought --suppress-secrets should hide them?

bradym avatar

Try secretref as the prefix, ie: <secretref+vault://secret/cluster-logging>

Dominic Cardellino avatar
Dominic Cardellino

Doesn‘t work either. But I think I know why, because Loki uses a configmap for this configuration and therefore the secrets are shown as plain text

1
svil avatar

I’ve used to reference vault secrets like this before in the helm files, but then after implementing vault-sidecar injector leaking secrets became a thing of the past

2023-03-23

2023-03-28

Nathanael avatar
Nathanael

Hi there, i try to get started with helmfile. is there a way to add individual yaml resources to a helmchart in helmfile? I tried to add a new (dummy)release and just referenced a normal subfolder with some yamls files. - in fact that worked.

repositories:
- name: cilium
  url: <https://helm.cilium.io/>
releases:
- name: cilium
  namespace: mgmt-cni-cilium
  chart: cilium/cilium
  version: 1.13.0
  values:
    - values.yaml
- name: resources
  chart: ./resources
  values:
    - values2.yaml  # <-- does not work :(

My next problem is: how can i use dynamic values with gotmpl on that subfolder? ( not realy a chart more like a bunch of gotmpl-yamls ) I noticed that {{ .Values.foo }} is not available in this subfolder release but something like {{ requiredEnv "NAME" }} dos work. thanks for the help

Nathanael avatar
Nathanael

i found that “chartify” is creating a dummy Chart in the tmp folder and is doing some wierd things. 1. gotmpl (my actual file) is put into the main tmp-dir. 2. it gets renderd to files/template (ignoring my values) and 3. then is Files.Get from the template folder…

how can i get my variable in to the file? the gotmpl, seams to ignore the values and regular helm-templating is ignored because of the Files.Get

Nathanael avatar
Nathanael
environments:
  default:
    values:
      - foo: baaaaaaaaaaaa

this does work. So it looks like environment vars can be used but regular releases.values: not. why is that?

2023-03-29

    keyboard_arrow_up