#helmfile (2020-06)

https://github.com/roboll/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles Archive: https://archive.sweetops.com/helmfile/

2020-06-29

Vasco avatar
Vasco

Hello all, shouldn’t I be able to propagate env values from parent helmfile to child-helmfiles?

parent

environments:
  dev:
    values:
    - myChartVer: 1.0.0-dev

helmfiles:
  - ./mysub1/helmfile.yaml

child: (“./mysub1/helmfile.yaml”)

releases:
    chart: mysubchart
    version: {{ .Values.myChartVer }}

yet trying to to sync this ends in error:

helmfile.yaml.part.0 parsing: template: stringTemplate:6:23: executing "stringTemplate" at <.Values.myChartVer>: map has no entry for key "myChartVer"
Andrey Nazarov avatar
Andrey Nazarov

No, they are not inherited. You should have an environments: block in subhelmfiles as well.

Vasco avatar
Vasco

ok, thanks for confirming that. I’m trying to convert an umbrella chart with 10+ into a helmfile setup, but it looks a bit challenging to keep things DRY

Andrey Nazarov avatar
Andrey Nazarov

I raised a discussion about this some time ago:

https://github.com/roboll/helmfile/issues/1010

Releases from `helmfiles` are ignored when the environment is specified · Issue #1010 · roboll/helmfile

I&#39;m facing the issue when releases defined in separate .yaml files and referenced via helmfiles: are ignored when I try to install the full stack for a certain environment. Example: #environmen…

:--1:2
Vasco avatar
Vasco

right, I’m running into multiple challenges with my conversion from the umbrella chart and I’m starting to doubt if it can be done elegantly. My current setup relies on having a common base values file and another per environment (=namespace) so that updating becomes like:

helm upgrade my-software --namespace env1 --values base.yaml --values env1.yaml
Vasco avatar
Vasco

Is there a way to use a part of value file for a release? I have an umbrella chart setup with value file (values.yaml) like:

subchart1:
  fullnameOverride: myapp-A
  version: 1.2.3
  image:
    pullPolicy: Always
  service:
    type: NodePort

subchart2:
  fullnameOverride: myapp-B
  image:
    pullPolicy: Always
  service:
    type: LoadBalancer

but how can I reference subchart1 values in helmfile without typing them out individually?

releases:
  - name:"my-app-A"
    version: {{ Environment.Values.version }} #works!
    values:
      - values.yaml #how do I limit to just subchart1?
voron avatar
voron

Did you tried any hacks in go templates ?

{{- tpl (readFile "values.yaml" | <REMOVE-ALL-UNNEEDED-STAFF>) . | nindent 0 }}

Imho it may be simpler and clealer to move values to separate files. All these tricks are hard to maintain.

Vasco avatar
Vasco

I considered it, but found it too cumbersome. I currently have one common value file for the entire project (umbrella chart) and one specific value file per environment. The specific one overriding common values as needed. This pair controls all of the subcharts and I would find it difficult to maintain if I would have a separate value file for each subchart

voron avatar
voron

you built helmfile using umbrella chart and now don’t wanna migrate to real helmfile

Vasco avatar
Vasco

we have had the umbrella chart for quite some time. I got interested in helmfile because I thought it would help me with my main problem with the current setup: As the Chart.yaml (or requirements.yaml in helm 2.x) cannot be templated, I’ve been looking for a way to control the version numbers of my subcharts. For development we use ranges and that is great, but when it’s time to make a prod release I’d like to “lock” the subcharts to specific versions - which would be easy if the Chart.yaml would accept templating. I’m still not sure if this can be done with helmfile, but I definitely haven’t yet got my head around it.

voron avatar
voron


but how can I reference subchart1 values in helmfile without typing them out individually?
But how do you do this with helm ? I mean - deploy of single subchart.

voron avatar
voron

I suppose you don’t deploy subcharts separately

Vasco avatar
Vasco

exactly - we don’t deploy them individually. They make up one product and are not really usable separately.

Vasco avatar
Vasco

Some subcharts are mandatory in the sense that the product cannot work if they are missing, while others are optional “features” and for those I have a condition setup that can be flipped on or off via the values file

voron avatar
voron

but now you wanna split umbrella chart to helmfile charts with dependencies, but don’t wanna split values files

Vasco avatar
Vasco

well, like I said, my experience with helmfile is so far just some hours and I’m probably doing a lot of things wrong. I figured out that I would have to split the umbrella chart in order to control the subchart versions, but I thought I could still feed the values from one file.

voron avatar
voron

https://github.com/roboll/helmfile#importing-values-from-any-source did you tried yq to get your subchart values like

{{ readFile "values.yaml"| exec "yq" ( list "r" "-" "subchart2" ) }}

?

roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

2020-06-28

2020-06-27

2020-06-26

Balazs Varga avatar
Balazs Varga

hello all. WE use kiali operator and few other helm chart that on cleanup leave some files in cluster. Can I somehow delete with helmfile? postdelete e.g ? Like events?

Andrey Nazarov avatar
Andrey Nazarov

Probably you’ll find the following useful:

https://github.com/roboll/helmfile/issues/1291

General info about hooks in helmfile: https://github.com/roboll/helmfile#hooks

Quick question about hooks · Issue #1291 · roboll/helmfile

I am just wondering exactly when presync and postsync hooks are run, specifically: Are they run if a release is being uninstalled? Are they run if a release is already uninstalled and installed: fa…

roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

Balazs Varga avatar
Balazs Varga

thanks will check them

Kenny Younger avatar
Kenny Younger

Has anyone been able to use the jsonPatches or strategicMergePatches successfully? It really seems to throw off the rendering of my chart (produces output that should be disabled).

jason800 avatar
jason800

Hey All, Does anyone know if its at all possible to push variables through an exec input list? For example something like:

{{ exec "./mycmd" (list "{{ .Values.region }}" "arg2" "--flag1") | indent 2 }}

I’ve tried a bunch of different methods to attempt and get it to interpolate, but none successful. also not finding any examples of anyone doing this

jason800 avatar
jason800

I see the cloudposse guys did

{{ exec "curl" (list "-fsSL" (print "<https://raw.githubusercontent.com/cloudposse/grafana-dashboards/>" (env "GRAFANA_DASHBOARDS_VERSION" | default "1.0") "/nginx-ingress/nginx.json")) | indent 12 }}

somewhere. Not sure if that can extend to interpolating helmfile vars though ?

Craig Dunford avatar
Craig Dunford

I believe you don’t need to interpolate there; you should be able to:

{{ exec "./mycmd" (list .Values.region "arg2" "--flag1") | indent 2 }}

2020-06-25

Vasco avatar
Vasco

Hi! Can I override the chart’s dependencies versions via helmfile? For example, say that my chart is dependencies are (Chart.yaml)

dependencies:
  - name: sub-chart-A
    version: ~1.1
  - name: sub-chart-B
    version: ~2.1

But now I want define a release that uses sub-chart-B version 2.0.0, is it possible?

Vasco avatar
Vasco

should I in fact have a separate helmfile for sub-chart-A and sub-chart-B and a top level helmfile and then have those version numbers templated?

voron avatar
voron

where do you specify chart dependencies ? release, templates, sub-helmfile, … ?

Vasco avatar
Vasco

I’m very new to helmfile (hours) and we currently have an umbrella chart that has 10+ dependencies comprising our entire product. The pain point has been setting the versions for the subcharts as the Chart.yaml (or requirements.yaml before) cannot be templated.

2020-06-22

2020-06-20

muhaha avatar
muhaha

Is possible to install helm release from git source ? ( https://github.com/opendistro-for-elasticsearch/opendistro-build/tree/master/helm/opendistro-es ), it has no public helm chart available btw

muhaha avatar
muhaha

oh, i need helm plugin install <https://github.com/aslafy-z/helm-git>

3
Tim Birkett avatar
Tim Birkett

With helm chart maintenance falling back into the realms of maintainers it would be nice if git sources were a first-class citizen in helm

Kenny Younger avatar
Kenny Younger

@ agreed!

2020-06-18

2020-06-15

TBeijen avatar
TBeijen

I assume there isn’t a toggle in helmDefaults that’s equivalent to the cli switch --skip-deps , is there?

voron avatar
voron

I don’t see any similar options in the code https://github.com/roboll/helmfile/blob/master/pkg/state/state.go#L104

roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

TBeijen avatar
TBeijen

Also couldn’t find it.

TBeijen avatar
TBeijen

The thing I’m running into (posted to Kubernetes #helm slack channel):

I’m running into an issue with a chart (linkerd) referencing it’s requirements as for example file://../add-ons/grafana.

What I’m trying to do is: helm pull to have the chart locally in codebase (easy developing, meaningful diffs when upgrading chart). I use helmfile which out of the box executes a helm dependency build for every local chart. This breaks for the linkerd chart because there’s no ‘add-ons’ locally. (Note that this step can be disabled via a flag, which allows me to move forward).

If I compare this to the prometheus-operator chart, that references it’s requiremens using external repos, which works fine in the above setup.

This makes me wonder: Is publishing charts having requirements to relative paths considered a good practice?

voron avatar
voron


Is publishing charts having requirements to relative paths considered a good practice?
I don’t think so

muhaha avatar
muhaha

Guys ? Is https://github.com/mumoshu/terraform-provider-helmfile masking Kind: Secret in TF diff ?

mumoshu/terraform-provider-helmfile

Deploy Helmfile releases from Terraform. Contribute to mumoshu/terraform-provider-helmfile development by creating an account on GitHub.

muhaha avatar
muhaha

@mumoshu ping

mumoshu/terraform-provider-helmfile

Deploy Helmfile releases from Terraform. Contribute to mumoshu/terraform-provider-helmfile development by creating an account on GitHub.

Andrey Nazarov avatar
Andrey Nazarov

I think no by default, but we wanted to implement this. I can check this out soon.

Andrey Nazarov avatar
Andrey Nazarov

You mean the --suppress-secrets functionality?

muhaha avatar
muhaha

oh,

muhaha avatar
muhaha
support config rules for masking fields · Issue #67 · k14s/kapp

when showing diffs (–diff-changes) some users want to be able to mask values of Secrets (v1/Secret). let&#39;s add diffFieldMaskRules to config: apiVersion: kapp.k14s.io/v1alpha1 kind: Config diff…

muhaha avatar
muhaha

Its working well with kapp binary, like masking secrets even in CRDs

muhaha avatar
muhaha

We are collecting pipeline logs like plan/apply/destroy phases and suppresing secrets in outputs is something that we like

mumoshu avatar
mumoshu

--suppress-secrets is enabled by default since v0.2.0 thanks to Andrey’s https://github.com/mumoshu/terraform-provider-helmfile/pull/13

Always suppress secrets by andrewnazarov · Pull Request #13 · mumoshu/terraform-provider-helmfile

This suppresses secrets by default and always. We can do it configurable as well if needed. Related to #12

2020-06-12

Mr.OO avatar
Mr.OO

Hi, I want the functionality of requiredEnv , but I want a default value if the os environment variable is missing. How could I acheive that?

Craig Dunford avatar
Craig Dunford

you should be able to use env instead of requiredEnv piped to default , eg:

scheme: {{ env "SCHEME" | default "https" }}
Mr.OO avatar
Mr.OO

version: {{.Environment.Values | getOrNil "salt_master.version" | default (env "SALT_MASTER_VERSION") | default ""}}

krunker.bot avatar
krunker.bot

hi all! had a question, i am separating in to sub helmfiles, i added selectors: to the main helmfile.yaml like so:

helmfiles:
- path: ./helmfiles/app1.yaml
  selectors:
  - name=app1
- path: ./helmfiles/app2.yaml
  selectors:
  - name=app2

but when i run helmfile –selector name=app1 diff it still runs through both subhelmfiles am I missing something? Im using latest helmfile with helm 3

:--1:1
Zachary Loeber avatar
Zachary Loeber

I think you misunderstand selectors a little bit

Zachary Loeber avatar
Zachary Loeber

just set labels in your subhelmfiles for the apps and whack the selectors from your main helmfile.yaml

Zachary Loeber avatar
Zachary Loeber

then it should work as you like I believe

Zachary Loeber avatar
Zachary Loeber

though, I honestly did not know I could set selectors in the main helmfiles block like that, nifty

Zachary Loeber avatar
Zachary Loeber

Anyways, otherwise when you apply your main helmfile it is saying to deploy both app1.yaml with a selector label of name: app1, and app2.yaml with a selector label of name: app2

Zachary Loeber avatar
Zachary Loeber

helmfiles: - ../helmfiles/helmfile.traefik.yaml - ../helmfiles/helmfile.certmanager.yaml - ../helmfiles/helmfile.rbacmanager.yaml - ../helmfiles/helmfile.consul.yaml - path: ../helmfiles/helmfile.vault.yaml selectors: - component=vault

:--1:2
Zachary Loeber avatar
Zachary Loeber

I just did a test with that helmfile.yaml and applied it all

krunker.bot avatar
krunker.bot

ohhhhhhh k, ty so much

s_slack avatar
s_slack

awesome. will try that too

Zachary Loeber avatar
Zachary Loeber

it deployed everything, including vault, but did not install the additional (separate) ingress deployments within the vault.yaml chart

Zachary Loeber avatar
Zachary Loeber

so, I could see using this logic too, just don’t apply it via cli and drop the filters in your main helmfile chart,

Zachary Loeber avatar
Zachary Loeber

your question learned me something new thanks

krunker.bot avatar
krunker.bot

sorry can i just ask what you put for the label in your sub helmfile?

krunker.bot avatar
krunker.bot

@Zachary Loeber

Zachary Loeber avatar
Zachary Loeber

no sorries, glad to share:

Zachary Loeber avatar
Zachary Loeber

labels: chart: vault component: vault namespace: {{ .Values | getOrNil "vault.namespace" | default "vault" }}

Zachary Loeber avatar
Zachary Loeber

for better or worse that is what I’ve personally been using as my standard go to labels for all releases

Zachary Loeber avatar
Zachary Loeber

(the namespace one is likely overkill and is sourced from the values.yaml file that is referenced in my environment.yaml

krunker.bot avatar
krunker.bot

ok ty so much, this is really helpful

Zachary Loeber avatar
Zachary Loeber
zloeber/CICDHelper

Kubernetes deployment stitcher. Contribute to zloeber/CICDHelper development by creating an account on GitHub.

1
Zachary Loeber avatar
Zachary Loeber

that’s all of them, I’m not using tpl templates or anything fancy (yet) so it may not be as dry as it should be

krunker.bot avatar
krunker.bot

weird, so did the helmfile --environment=dev --selector component=app1 diff but same result it runs through both even though under the release i have the labels set

Zachary Loeber avatar
Zachary Loeber

and you removed the selector in your main helmfile?

krunker.bot avatar
krunker.bot

ahhhhh nope, missed that ok

krunker.bot avatar
krunker.bot

so removed path and selectors from main helmfile, it diffs the one app now, tho for some reason still feels the need to add the repo of the other app

Zachary Loeber avatar
Zachary Loeber

That’s something I cannot help with (unfortunately), I’m still trying to figure out the best way to do repo filtering

krunker.bot avatar
krunker.bot

hahahaha ok, ya, for now i can just run helmfile -f path/to/subhelmfile.yaml i guess

Zachary Loeber avatar
Zachary Loeber

though if you know you have the latest you can use the –skip-deps flag I think

krunker.bot avatar
krunker.bot

ok, ill look into that ty

Zachary Loeber avatar
Zachary Loeber

lemme know if you find anything

krunker.bot avatar
krunker.bot

ya will do, thx for all the help!

peterloron avatar
peterloron

When running a command triggered by a hook, is there a variable I can use which holds the path to the temporary cached chart that helmfile downladed?

Zachary Loeber avatar
Zachary Loeber

I’d really like to know the answer to this one too actually, I can find the cached repository data easily enough but cannot seem to find the downloaded charts

Zachary Loeber avatar
Zachary Loeber

Because how cool would it be to have a plugin that stores the downloaded/versioned charts and rewrites the chart repos to the local cached version for offline deployments?

voron avatar
voron

WDYT helmfile downloads chart[s] w/o helm ? Did you tried to download chart directly via helm pull inside hook ?

Zachary Loeber avatar
Zachary Loeber

right, this is more of a helm question than helmfile one

Zachary Loeber avatar
Zachary Loeber

that helm pull in a hook option may be a good way to get the charts downloaded into one location though… I’ll have to look into that option…

voron avatar
voron

I’m curious about caching/go-offline mode in this case

Zachary Loeber avatar
Zachary Loeber

I’d look to pull down all the charts into a dockerfile image and would rewrite the chart urls as part of that process if possible

Zachary Loeber avatar
Zachary Loeber

Idea would be to have an immutable deployment artifact for the cluster state that will always apply the same (regardless if upstream chart sources update beneath our feet)

peterloron avatar
peterloron

That’s an interesting idea!

peterloron avatar
peterloron

My use case is a little simpler…I just want to dynamically add another template to an upstream chart. I could copy the file with a hook, but I need to know where to copy the file to…

voron avatar
voron

U will, in case of helm pull

peterloron avatar
peterloron

If I’m running helmfile, I don’t manually do a “helm pull”, so how would I get the path?

voron avatar
voron

Do you wanna patch the chart on the fly ? Specify local filesystem chart in helmfile and use (prepare ?) hook to pull upstream chart via helm and then modify it. Path is current directory, as you specify current directory via chart : ./my-local-chart

peterloron avatar
peterloron

Hmm, ok. I’ll see if I can swing that

voron avatar
voron

use shell script as a hook to debug and get a PoC

2020-06-11

Balazs Varga avatar
Balazs Varga

hello guys.

what does this (<<) mean in hemfile ?

Craig Dunford avatar
Craig Dunford

that’s part of the yaml spec: https://yaml.org/type/merge.html

1
Andrey Nazarov avatar
Andrey Nazarov
roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

peterloron avatar
peterloron

Hello, all. Is it possible to inject new yaml into a chart using helmfile? I have an upstream chart I’m trying to use, but I need to create a Traefik ingressroute, which the chart does not support. Is there a way for me to inject the helm template for the ingressroute into the chart dynamically? I’d rather not fork the upstream chart if I can avoid it. Thanks.

voron avatar
voron

Take a look at helmfile release hooks, for example https://github.com/roboll/helmfile#helmfile--kustomize

roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

peterloron avatar
peterloron

Interesting, I’ll check that out. I was looking at the hooks, and having it just run a straight “kubectl apply”, but that doesn’t cover situations like destroying a deployment.

peterloron avatar
peterloron

I was hoping for a “copy this yaml into the chart/template directory so helm will process it” type command

voron avatar
voron

use hooks to get upstream chart, patch in on the fly and then release patched local chart via helmfile

peterloron avatar
peterloron

also possible, yeah. Was trying to keep it simple since helmfile already grabs the upstream chart and caches it temporarily.

voron avatar
voron

tell helmfile to use local chart to deploy, and helmfile will not grab upstream chart then

peterloron avatar
peterloron

sure. I’ll fiddle with this. Thanks!

2020-06-10

voron avatar
voron

What’s the best way to share release templates between multiple helmfiles ? bases produces yaml: unknown anchor referenced , readFile produces cannot unmarshal !!map into string, the only working option I’ve found is {{- tpl (readFile "../common/templates.yaml") . | nindent 0 }} , but it doesn’t have access neither to environment variables nor to yaml multipart variables inside tempalte.

s_slack avatar
s_slack

I have a Git repo where all my charts live in separate folders. In Helmfile, I have all apps listed with their name/url for each application, for example,

repositories:  - name: app1
    url: <git+ssh://[email protected]/examplecom/[email protected]?ref={{> .Values | getOrNil "app1._version" | default "master" }}&sparse=0
  - name: app2
    url: <git+ssh://[email protected]/examplecom/[email protected]?ref=>{{ .Values | getOrNil "app2._version" | default "master" }}&sparse=0
  ...

Every time i do a helmfile diff or apply it adds each and every repo. It is slow.

$ helmfile --environment=dev --namespace=default --selector name=app1 apply
Adding repo app1 <git+ssh://[email protected]/examplecom/[email protected]?ref=v0.1.2&sparse=0>
"app1" has been added to your repositories

Adding repo app2 <git+ssh://[email protected]/examplecom/[email protected]?ref=v0.1.2&sparse=0>
"app2" has been added to your repositories
...

I was wondering if there is a quicker way to update only 1 app and at times all apps or even a list of apps. How are you using Helmfile + helm-git ?

voron avatar
voron

Did you tried some magic like

{{ if has (env "UPDATE_REPO" | default "no") (list "yes" "True" "true") }}
repositories:
 - name: app1
    url: <git+ssh://[email protected]/examplecom/[email protected]?ref={{> .Values | getOrNil "app1._version" | default "master" }}&sparse=0
 - name: app2
    url: <git+ssh://[email protected]/examplecom/[email protected]?ref=>{{ .Values | getOrNil "app2._version" | default "master" }}&sparse=0
{{ end }}

?

voron avatar
voron

the idea is to use [env] var as a condition to add all repositories to the helmfile.

s_slack avatar
s_slack

let me try that magic

Andrey Nazarov avatar
Andrey Nazarov

I would use get instead of getOrNil wherever possible

s_slack avatar
s_slack

I am still trying but still not sure how to update a single app, a list, or all.

voron avatar
voron

it looks like helm doesn’t support specifying repos you wanna update. All or nothing.

s_slack avatar
s_slack

I wonder how Cloud Posse might be using this. I believe I read in one of the comments that they might be using a combination of Helm-Helmfile-Helm-git, Is that right @Erik Osterman (Cloud Posse)?

voron avatar
voron

I think he’s at Office Hours on call

1
s_slack avatar
s_slack

@Erik Osterman (Cloud Posse) would you have any recommendation on this?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, we use the helm-git plugin all the time

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s the secret weapon. with helm-git, you can say goodbye to needing helm repos and instead get something more akin to what we have with NPM or go, where you can import basically any chart

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles for Kubernetes - cloudposse/helmfiles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Note the bug though:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Cannot use release tags, see https://github.com/aslafy-z/helm-git/issues/9

Failure when ref is an annotated tag · Issue #9 · aslafy-z/helm-git

The plugin fails when ref refers to an annotated tag, which is the usual case for GitHub release tags. For example: helm repo add istio git+https://github.com/istio/[email protected]/kubernetes/helm?re

s_slack avatar
s_slack

Thanks. Do you keep multiple apps in the same repo like I mentioned? Helm-git is great. But cloning all repos for a 1 app diff or apply makes the process much longer. I wonder if you have a better way to control it

s_slack avatar
s_slack

Looks like you have a helmfile per application.

2020-06-09

Zachary Loeber avatar
Zachary Loeber

There is a section in the documentation titled, ‘Running Helmfile without an Internet connection’. It references using the depreciated charts action as being a solution to pre-download the charts. Firstly, I think that this maybe should be updated to be sync --skip-deps , Before putting in the PR to reflect this change I’d like to be certain my understanding is correct. Secondly, wouldn’t this only download the chart index files and references and not actually download the charts themselves?

jason800 avatar
jason800

Hey guys, I’m having an issue getting helmfile to do anything with repositories unless I list the repositories way down the chain of helmfile includes

jason800 avatar
jason800
environments:
  preprod:


\# Chart repositories used from within this state file
repositories:
- name: stable
  url: <https://kubernetes-charts.storage.googleapis.com>
- name: incubator
  url: <https://kubernetes-charts-incubator.storage.googleapis.com>
- name: bitnami
  url: <https://charts.bitnami.com/bitnami>

helmfiles:
{{ if eq .Environment.Name "preprod" }}
  - path: helmfile-preprod.yaml
{{ end }}

This is my main helmfile.yaml. if I do helmfile repos, I get an error about no selector match. if i do helmfile -e preprod repos , nothing happens at all.

jason800 avatar
jason800

If I stick the repositories down into an actual helmfile containing releases, it works just fine

Zachary Loeber avatar
Zachary Loeber

I’ve always had to include the repositories block in my helmfiles. same with the helmdefaults block. I ended up floating those up to their own files and have something like the following at the top of my helmfiles:

Zachary Loeber avatar
Zachary Loeber

--- bases: - ../config/environments.yaml - ../config/helmdefaults.yaml - ../config/repositories.yaml ---

Zachary Loeber avatar
Zachary Loeber

It works but also means that for any helmfile that gets deployed ALL repos will get synced. Not ideal.

Zachary Loeber avatar
Zachary Loeber

I’ve yet to round back on how to optimize this particular bit

jason800 avatar
jason800

I have a helm chart that we use to manage namespaces, which I hate, but in this case it actually came in handy because its a pre-req that it runs first

jason800 avatar
jason800

so I shoved the repos in that helmfile for that chart, and all the other runs subsequently get it

Zachary Loeber avatar
Zachary Loeber

Funny, that’s why I kept namespace creation as a separate chart as well, it is a handy dependency anchor

Andrey Nazarov avatar
Andrey Nazarov

Afaik this is the expected behaviour. Nothing is inherited from the main file.

Andrey Nazarov avatar
Andrey Nazarov

As of managing namespaces with Helm chart - chances to destroy everything by accident scare me a bit). But we are managing namespaces with a chart too.

2020-06-08

jqjq avatar

Any chance I can get someone to look at my helmfile PR? https://github.com/roboll/helmfile/pull/1296 Thanks!

Add the ability to load a remote environment values file by kevinjqiu · Pull Request #1296 · roboll/helmfile

This PR enables the user to specify a remote path for an environment values file, e.g., environments: cluster-azure-us-west: values: - git://git.company.org/helmfiles/global/azur>…

jqjq avatar

@mumoshu would you be able to comment on whether this feature fits the overall goal of the project? I’m sorry I pinged you directly but I haven’t gotten any feedback on the PR for a week and the feature would be great for our workflow (having a single repo for all global values across different data centres). Thanks and sorry for the intrusive direct pinging.

Add the ability to load a remote environment values file by kevinjqiu · Pull Request #1296 · roboll/helmfile

This PR enables the user to specify a remote path for an environment values file, e.g., environments: cluster-azure-us-west: values: - git://git.company.org/helmfiles/global/azur>…

Craig Dunford avatar
Craig Dunford

anyone know if helmfile populates a value containing the full path of the current file during the first pass rendering? I am trying to use exec to call a script from an environment file that is included by multiple different helmfiles; any paths I use seem to be relative to the original helmfile

Zachary Loeber avatar
Zachary Loeber

I just put together a proof of concept for bundling up helmfile charts in a docker image which then gets run inside the cluster to bootstrap all the base services, https://github.com/zloeber/CICDHelper (see example 3). While I’m not 100% certain how useful this is, I’m still happy at how well it works Dang helmfile is so cool…

zloeber/CICDHelper

Kubernetes deployment stitcher. Contribute to zloeber/CICDHelper development by creating an account on GitHub.

:100:1
:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Very cool @Zachary Loeber!

zloeber/CICDHelper

Kubernetes deployment stitcher. Contribute to zloeber/CICDHelper development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Steve Boardwell could be an interesting strategy

Andrey Nazarov avatar
Andrey Nazarov

Great job!

bradym avatar
bradym

I’m still having a difficult time understanding what values are available where when using helmfile. Does anyone know of somewhere that is documented? (issue? code?)

An example of something stumping me today:


\# helmfile.yaml

releases:
  - name: foo
    values:
      - values-apps/foo.values.yaml.gotmpl
    valuesTemplate:
      - service:
          port: <ref+awsssm://foo/stage/CONTAINER_PORT?region=us-west-1>

---

\# values-apps/foo.values.yaml.gotmpl

deployment:
  readinessProbe:
    exec:
      command:
        - "/bin/bash"
        - "/app/health.sh"
        - "localhost"
        - "{{ .Values.service.port }}"
        - ".1s"

I’m getting an error that .service.port is not in .Values. When I echo out {{ . }} in foo.values.yaml.gotmpl, the only values I can get to show up are ones that are passed in via the --state-values-set flag. I’ve been unable to get anything in helmfile.yaml to be available there. Could someone point me in the right direction?

mumoshu avatar
mumoshu

releases[].values are for helm. it’s a different concept than helmfile values ({{.Values}}) in your helmfile.yaml

mumoshu avatar
mumoshu

helmfile values can be defined using the top-evel values:

values:
- service:
    port: <ref+awsssm://foo/stage/CONTAINER_PORT?region=us-west-1>


---

releases:
- name: foo
  values: 
  - values-apps/foo.values.yaml.gotmpl
bradym avatar
bradym

That’s helpful, but also a little confusing.

mumoshu avatar
mumoshu

yeah true! when we added this, everyone seemed to agree that this should better be called values so that it become familiar to any helm users

mumoshu avatar
mumoshu

which turned out not true :cry: maybe we can try renaming it to variables or anything else hoping it makes things a little less confusing

bradym avatar
bradym

I’ve got several releases in one helmfile that all use the same chart. Additionally, I’m supporting two environments, so with the example you posted I’m not sure how I would set service.port for a different release since I can’t use it in the releases section.

bradym avatar
bradym

lemme get a more accurate example of what i’m doing to make it more clear

mumoshu avatar
mumoshu
values:
- foo:
     service:
       port: 1234
  bar:
     service:
       port: 2345

---

releases:
- name: foo
  values:
  - {{ .Values.foo | nindent 4 }}
- name: bar
  values:
  - {{ .Values.bar | nindent 5 }}
bradym avatar
bradym

Or you could just go and give me a great example

1
mumoshu avatar
mumoshu

you can even do this. but i’m not a fan of abstracting things so much / building a yet another configuration language on top of yaml+gotmpl

values:
- foo:
     service:
       port: 1234
  bar:
     service:
       port: 2345

---

releases:
{{ range $name, $values := .Values }}
- name: $name
   values:
  - {{ $values | nindent 4 }}
}}
mumoshu avatar
mumoshu

accept a bit of repetition in releases like my first example and it can be reviewed easily

bradym avatar
bradym

Yeah, the first example definitely seems more readable.

:--1:1
mumoshu avatar
mumoshu

anyway, if you could share me a concrete example, i can probably review/suggest something on/confirm it

bradym avatar
bradym

So this is what I’ve currently got:

templates:
  default: &default
    chart: charts/commonApp
    labels:
      app: "{{`{{ .Release.Name }}`}}"
    values:
      - global.yaml.gotmpl
      - values-apps/{{ .Values.repo }}.values.yaml.gotmpl
      - envs/{{ .Environment.Name }}.yaml.gotmpl
    valuesTemplate:
      - appName: "{{`{{ .Release.Name }}`}}"
      - environment: {{ .Environment.Name }}
      - repo: {{ .Values.repo }}
      - service:
          port: secretref+awsssm://{{ .Values.repo }}/{{ .Environment.Name }}/CONTAINER_PORT?region=us-west-1

releases:
  - name: foo-{{ .Values.branchSlug }}
    version: 1.0
    <<: *default

  - name: bar-{{ .Values.branchSlug }}
    version: 1.0
    <<: *default
bradym avatar
bradym

I’m not entirely clear on values vs valuesTemplate and this is before you suggested using the top level values object.

bradym avatar
bradym

Based on what you said, I think what I need to do is move some stuff to environments, something like this:

environments:
  stage:
    foo:
      service:
        port: 1234
  prod:
    foo:
      service:
        port: 5678
mumoshu avatar
mumoshu

valuesTemplate is usually used for access to Release.Name and Release.Namespace(I thought it was possible

mumoshu avatar
mumoshu

yeah, your use of environments seems valid to me

mumoshu avatar
mumoshu

in addition to that, you may find the top-level values useful for defining default values for environment values

bradym avatar
bradym

ahh, that’s helpful

:--1:1
mumoshu avatar
mumoshu

do you need different service.port values for releases foo and bar?

mumoshu avatar
mumoshu

if not, your use of templates seems good

bradym avatar
bradym

yeah, foo and bar are be different apps, so they’ll have different service.port values. But I use the same name in SSM for the thing following the pattern /appName/envName/varName

bradym avatar
bradym

That’s why I was hoping I could use release vars - then I don’t have to set the same things for every app separately in the environments section.

mumoshu avatar
mumoshu

then i think it should probably be

secretref+awsssm://{{` {{.Release.Name}} `}}/{{ .Environment.Name }}/CONTAINER_PORT?region=us-west-1
mumoshu avatar
mumoshu

the helmfile template renders

secretref+awsssm://{{` {{.Release.Name}} `}}/{{ .Environment.Name }}/CONTAINER_PORT?region=us-west-1

into

secretref+awsssm://{{.Release.Name}}/prod/CONTAINER_PORT?region=us-west-1

and the release template renders

secretref+awsssm://{{.Release.Name}}/prod/CONTAINER_PORT?region=us-west-1

into e.g.

<secretref+awsssm://foo-BRANCH_SLUG/prod/CONTAINER_PORT?region=us-west-1>
bradym avatar
bradym

That actually shows why I’m using {{ .Values.repo }} instead of the release name – I don’t want to include the branch slug in the ssm path.

mumoshu avatar
mumoshu

I hear you. But it can’t evaluate {{.Values.repo}} to a different value per release

bradym avatar
bradym

(Feel free to tell me I’m crazy and need to change my approach :D)

mumoshu avatar
mumoshu

So maybe you’d better build on Release.Name

mumoshu avatar
mumoshu
{{ .Release.Name | split "-" | index 0 }}
mumoshu avatar
mumoshu

i think it would work

bradym avatar
bradym

I’ve lost track, where would you suggest I put the secretref line?

mumoshu avatar
mumoshu

Under valuesTemplate, as it was in your example

mumoshu avatar
mumoshu
templates:
  default: &default
    chart: charts/commonApp
    labels:
      app: "{{`{{ .Release.Name }}`}}"
    values:
      - global.yaml.gotmpl
      - values-apps/{{ .Values.repo }}.values.yaml.gotmpl
      - envs/{{ .Environment.Name }}.yaml.gotmpl
    valuesTemplate:
      - appName: "{{`{{ .Release.Name }}`}}"
      - environment: {{ .Environment.Name }}
      - repo: {{ .Values.repo }}
      - service:
          port: secretref+awsssm://{{` {{ .Release.Name | split "-" | index 0 }}
 `}}/{{ .Environment.Name }}/CONTAINER_PORT?region=us-west-1
bradym avatar
bradym

Ok, testing that

mumoshu avatar
mumoshu

(fingers crossed)

mumoshu avatar
mumoshu

you can debug it using helmfile build, btw.

bradym avatar
bradym

ahh, nice. I’ve been using helmfile template - haven’t tried build yet

mumoshu avatar
mumoshu

yeah both helps. it’s just that build is for unit testing your helmfile.yaml, and template can be used for integration testing with helm and helmfile

:--1:1
bradym avatar
bradym

It did not like that..

failed executing templates in release "apps.yaml"."foo-bar": failed executing template expressions in release "foo-bar".values[3] = "[115 101 114 118 105 99 101 58 10 32 32 112 111 114 116 58 32 115 101 99 114 101 116 114 101 102 43 97 119 115 115 115 109 58 47 47 32 123 123 32 46 82 101 108 101 97 115 101 46 78 97 109 101 32 124 32 115 112 108 105 116 32 34 45 34 32 124 32 105 110 100 101 120 32 48 32 125 125 32 47 115 116 97 103 101 47 67 79 78 84 65 73 78 69 82 95 80 79 82 84 63 114 101 103 105 111 110 61 117 115 45 119 101 115 116 45 49 10]": template: stringTemplate:2:59: executing "stringTemplate" at <index 0>: error calling index: can't index item of type int
mumoshu avatar
mumoshu

seems like i confused argument ordering for index

mumoshu avatar
mumoshu

0 should be the second argument to index

mumoshu avatar
mumoshu
{{ index (.Release.Name | split "-") 0 }}
bradym avatar
bradym

hmm, same thing

mumoshu avatar
mumoshu

oh, same? shouldn’t it emit another error at least

bradym avatar
bradym

oof, can’t read template: stringTemplate:2: unexpected "(" in operand

bradym avatar
bradym

ok, that was my bad when I have the space like I should I get template: stringTemplate:2:31: executing "stringTemplate" at <index (.Release.Name | split "-") 0>: error calling index: value has type int; should be string

mumoshu avatar
mumoshu

ahh okay, we should use splitList rather than split

http://masterminds.github.io/sprig/string_slice.html

String Slice Functions

Useful template functions for Go templates.

bradym avatar
bradym

Yep, that got rid of the error

bradym avatar
bradym

But.. I’m back where I started

failed to render values files "values-apps/foo.values.yaml.gotmpl": failed to render [values-apps/foo.values.yaml.gotmpl], because of template: stringTemplate:12:21: executing "stringTemplate" at <.Values.service.port>: map has no entry for key "service"
mumoshu avatar
mumoshu

Do you still have reference to Values.service.port in values-apps/{{ .Values.repo }}.values.yaml.gotmpl ?

bradym avatar
bradym

Yeah, not sure what to set it to?

mumoshu avatar
mumoshu

you shouldn’t need that, as you already set the same service.port in valuesTemplate now

bradym avatar
bradym

I’m trying to use the value in the .gotmpl file, not set it. I’m guessing that isn’t possible?

mumoshu avatar
mumoshu

ah gotcha! yes, that’s not possible

mumoshu avatar
mumoshu

you should do this in valuesTemplate instead

deployment:
  readinessProbe:
    exec:
      command:
        - "/bin/bash"
        - "/app/health.sh"
        - "localhost"
        - secretref+awsssm://{{` {{ .Release.Name | split "-" | index 0 }}
 `}}/{{ .Environment.Name }}/CONTAINER_PORT?region=us-west-1
        - ".1s"

``

bradym avatar
bradym

Yeah, that makes sense. I was attempting to avoid that as the the readinessProbe stuff won’t always be the same and that means more repetition, but it seems I should either make them all the same or be ok with more repetition

mumoshu avatar
mumoshu

gotcha

mumoshu avatar
mumoshu

you can use {{ define "somehelper" }} and {{template "somehelper" "someargs" }} template functions to reuse a templated yaml sturucture

mumoshu avatar
mumoshu

or even use a helper defined in a separate fiile using

{{ tpl (readFile "somehelper.tpl") "someargs" }}

mumoshu avatar
mumoshu

hope that helps to alleviate the repetition issue

bradym avatar
bradym

yeah, definitely. I hadn’t thought of using templates for this, but it makes perfect sense.

1
bradym avatar
bradym

Thanks for all your help with this @mumoshu, I really appreciate it.

bradym avatar
bradym

helmfile is a really great tool, and you’ve helped me wrap my head around a couple things that were missing for me.

mumoshu avatar
mumoshu

glad to help! and thanks for the feedback. that really motivates me

2020-06-05

Zachary Loeber avatar
Zachary Loeber

My helmfile stitching framework is becoming a bit of a monster but after doing some tinkering with it the other day it now is a monster that works on Macs (with docker) as well: https://github.com/zloeber/CICDHelper

zloeber/CICDHelper

Kubernetes deployment stitcher. Contribute to zloeber/CICDHelper development by creating an account on GitHub.

:--1:4
1

2020-06-03

2020-06-02

Joe Hohertz avatar
Joe Hohertz

Hello, I am trying to see if I can get helmfile to do some things for me… seem to be having some basic issues, perhaps my understanding of the tool is not correct? On the most basic level… if I pass a yaml file with foo: bar to –values flag on helmfile, I should be able to see .Values.foo in the helmfile context, yes? (I never see any of the data passed via –values)

voron avatar
voron

It looks like --values and --set arguments are passed “as is” to helm, and aren’t parsed by helmfile. You may use things like

{{ readFile "myvalues.yaml" }}

to include values into helmfile context

voron avatar
voron

but usually it’s enough to use values/valuesTemplate helmfile directives to use values from file

Joe Hohertz avatar
Joe Hohertz

Apparently --state-values-file works?

Joe Hohertz avatar
Joe Hohertz

I thought that was more like including a helmfile but apparently not.

voron avatar
voron

--state-values-file should affect helmfile context --state-values-set affects it in my tests

voron avatar
voron

similar to adding values on top of helmfile.yaml

values:
  - k0: v0
    k1: v1
---
...
the rest of helmfile here
Joe Hohertz avatar
Joe Hohertz

What I am going for is to build a tool I feed a single yaml file into to configure an environment. So I basically want to pass the “environment” values in almost entirely at runtime. My understanding was that --values would do that, and the other flag seems to.

Joe Hohertz avatar
Joe Hohertz

Ahhh for your example, if the file passed had just:

k0: v0
k1: v1
Joe Hohertz avatar
Joe Hohertz

?

Joe Hohertz avatar
Joe Hohertz

If that’s right, then this IS the flag I want. I thought it did something else, as I use the --state-values-set to sort of create an environment on the fly. (and it seems to work), so I thought it was merging at the root of the helmfile.

voron avatar
voron


Ahhh for your example, if the file passed had just:
yes.

Joe Hohertz avatar
Joe Hohertz

Thanks for clarifying.

voron avatar
voron
roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

Joe Hohertz avatar
Joe Hohertz

That sounds up my alley… I’ll try to re-read it… a bit confused on the difference between “bases” and “helmfiles”

voron avatar
voron

use bases to build helmfile.yaml use helmfiles to join multiple [autonomous] helmfile’s.yaml imho

Joe Hohertz avatar
Joe Hohertz

Thanks. I may have more annoying questions as I explore this. What I am hoping to achieve is via helmfile, and helm git, being able to place charts, a helmfile fragment, and a values.yaml.gotmpl w/ the code in the git-repo, and then compose things via a bunch of git references in the “top” helmfile.

2020-06-01

    keyboard_arrow_up