#helmfile (2019-05)

https://github.com/roboll/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles Archive: https://archive.sweetops.com/helmfile/

2019-05-31

Andrey Nazarov avatar
Andrey Nazarov

Why helm-tiller is not bundled in a helmfile’s Docker image?

mumoshu avatar
mumoshu

I was just lazy to do so! PR welcomed

2019-05-30

Andrey Nazarov avatar
Andrey Nazarov

Can I define several templates? Something like

templates:
  - first: &first
    snip1
  - second: &second
    snip2
mumoshu avatar
mumoshu

Yep. Define each as dict items. Not array items

Andrey Nazarov avatar
Andrey Nazarov

Got it. Thank you.

2019-05-29

Ufou avatar

good morning, is it possible for helmfile to merge a map coming from environment yaml files so that I can take 2 maps and merge them together? eg if my values file looks something like:

default_config:
  resources:
    limits:
      cpu: 0.1
      memory: 200Mi
    requests:
      cpu: 0.1
      memory: 200Mi

service_config_a:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70
  resources:
    limits:
      cpu: 0.3
      memory: 400Mi
    requests:
      cpu: 0.3
      memory: 400Mi

service_config_b:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70
Ufou avatar

I’d like service_config_b to inherit values from default_config

mumoshu avatar
mumoshu

Does using YAML anchor work?

default_config: &default_config
  resources:
    limits:
      cpu: 0.1
      memory: 200Mi
    requests:
      cpu: 0.1
      memory: 200Mi

service_config_a:
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70
  resources:
    limits:
      cpu: 0.3
      memory: 400Mi
    requests:
      cpu: 0.3
      memory: 400Mi

service_config_b:
  <<: *default_config
  autoscaling: true
  minReplicas: 5
  maxReplicas: 70
Ufou avatar

hmm, I did not consider that, it may well do!

1
:--1:1
Ufou avatar

@mumoshu lovely stuff, thanks works beautifully

2
Ufou avatar

one other question - is the work for bases released yet?

mumoshu avatar
mumoshu

@Ufou yep! https://github.com/roboll/helmfile/releases/tag/v0.68.0 is the latest release including the bases feature, which all the known issues related to bases fixed

roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

1
Ufou avatar

thanks!

Andrey Nazarov avatar
Andrey Nazarov

@mumoshu what is the release schedule for helmfile if there is one?

mumoshu avatar
mumoshu

I release it regurarly via github releases. or perhaps you meant v1.0?

Andrey Nazarov avatar
Andrey Nazarov

Yeah, I noticed. It seems there is no particular schedule for these releases (like once a week every Sunday or so). The point is I’m building my own docker image since I need some tools for hooks. I think I should automate the building process.

mumoshu avatar
mumoshu

Gotcha. I’d say there’s no particular schedule! Have you tried renovatebot? I thought it had an ability to track upstream docker image update and automatically submit PRs to update the base image in your Dockerfile.

https://renovatebot.com/

Renovate Bot

Automated Dependency Updates

:--1:1
mumoshu avatar
mumoshu

@Andrey Nazarov

Andrey Nazarov avatar
Andrey Nazarov

I’m gonna reorganise things related to helmfile, try new features and stuff. @mumoshu Should I avoid set in my helmfile.yaml? I noticed somewhere you stated, that it was added for backward compatibility.

Andrey Nazarov avatar
Andrey Nazarov

Actually, with the latest additions it’s somewhat difficult to pick the right solution. Like helmfiles vs bases vs release templates vs something else. Since we’ve got tens of releases multiplied by tens of environments, the good old plain helmfile.yaml is becoming difficult to maintain. And we’ve got umbrella charts there. So, we are going to get rid of these umbrella charts and manage every single thing by helmfile, but we want to make it as DRY as possible). Let’s see what we’ll get.

mumoshu avatar
mumoshu

@Andrey Nazarov Sry for replying an old thread!

Yeah you should generally avoid set because it has typing issues the same as helm’s --set. The are only two use-cases that you’d like to use set, (1) set an array item (2) set value from a file(not setting kvs from file=values.yaml, but --set-file key=FILENAME)

:--1:1
mumoshu avatar
mumoshu

Also, I wanted to note that you should avoid templating whenever possible.

Regarding state templates vs release templates, I recommend you to avoid using state templates because it’s harder to use, and you can now use bases and release templates to achieve almost the same goal.

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

anyone see how far they could get with helm3-alpha?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
helm/helm

The Kubernetes Package Manager. Contribute to helm/helm development by creating an account on GitHub.

mumoshu avatar
mumoshu

i saw somewhere that the team plans to release alpha.2 and alpha.3 until GA.

thought alpha.1 doesn’t add many fancy things, other than docker registry as a chart repository feature.

my favorite feature “Lua event hooks” are coming in alpha.3!

helm/helm

The Kubernetes Package Manager. Contribute to helm/helm development by creating an account on GitHub.

mumoshu avatar
mumoshu

at last time i saw it, the dev branch were 800+ commits behind the master branch. not sure how the team cherry-picks all those…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

though alpha.1 apparently has addressed the tiller issue

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s huge

mumoshu avatar
mumoshu

yeah that’s nice!

2019-05-27

2019-05-21

2019-05-20

Pierre Humberdroz avatar
Pierre Humberdroz

Hey,

How can I manage deployments of crd’s with helmfile? I am currently migrating parts of our infra to helmfile stored in a repo and I need to deploy cert-manager as well

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

btai avatar

does helmfile remove the need to manually bump chart versions etc in the repo?

mumoshu avatar
mumoshu

hey! perhaps you wanna try the brand-new helmfile deps command that recently introduced?

https://github.com/roboll/helmfile/pull/593

i’m assuming it is what you basically want for “automatic chart version bump”.

it works like this - helmfile deps fetches latest chart versions that satisfies the chart version constraints described in your helmfile.yaml, saves version numbers to helmfile.lock. on e.g. helmfile sync, it reads the lock file and uses the chart version numbers in it

cc @Andriy Knysh (Cloud Posse)

feat: Dependency locking by mumoshu · Pull Request #593 · roboll/helmfile

In order to maintain predictable deployments, as developer I want to generate and use &quot;lock files&quot; for all chart versions retrieved from a helmfile. This change solves it by (1)enhancing …

:--1:2
btai avatar

thanks @mumoshu! “automatic chart version bump” is exactly what i want

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you have to specify the chart version directly in the helmfile, either by hardcoding it (e.g. https://github.com/cloudposse/helmfiles/blob/master/releases/cert-manager.yaml#L30) , or using ENV var

cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

btai avatar

thanks @Andriy Knysh (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@btai here’s an example of using the raw chart

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. if you don’t have the chart for the CRDs, but still want to use helmfile

2019-05-17

Andrey Nazarov avatar
Andrey Nazarov

Btw, the latest version of helmfile available via homebrew is v0.54.2.

Collin avatar
Collin

@Andrey Nazarov @mumoshu Was just coming online to ping you about the homebrew version Are we going to see an update for it in the future? I’m asking before I manually install it from git and there are some breaking changes that make my life hell.

mumoshu avatar
mumoshu

Hey!

Nope, the homebrew formula is out of my control :) Would you mind sending PRs to the formula, so that it gets updates.

Collin avatar
Collin

100% will do.

mumoshu avatar
mumoshu

What kind of breaking changes are you talking about? I think I didnt make those for months(also we regurarly add features and bug fixes

Collin avatar
Collin

I was saying I wanted to check with you about the version on homebrew before I pull directly from git and my yaml files require updating to any new formats. Going from 0.54.2 to 0.63 is a big jump.

It’s likely there are no breaking changes. I’ll confirm myself before making the formula update PR.

mumoshu avatar
mumoshu

Okay! Thanks for your support

Collin avatar
Collin

And a side (noobish) question: We’re trying to update some of our config maps via helm upgrade commands using --set but we’re unable to target the chart as it’s part of our helmfile. I’d normally use sync but I want to try a few direct changes. Am I being an idiot or would we have to move to individual chart files to be able to do this?

mumoshu avatar
mumoshu

I’m trying to understand. Is your chart able to manage configmaps via chart values, and you’re trying to set some values without creating values.yaml files, right?

Collin avatar
Collin

Correct.

Collin avatar
Collin

We’ve had a rollout where kubernetes-dashboard is failing, which fails the release. This means configmap versions ARE pushed but when looking at the pod describe for it it’s defaulting to the old mount.

So the config exists but because one chart failed, the release failed.

This is fine, and expected, I should just fix the kubernetes-dashboard release and everything will be fine, but I want to confirm something within the configmap before I do that.

I’d normally just helm upgrade --set key=value but I’m unable to target that chart name because there is only the helmfile.

mumoshu avatar
mumoshu
releases:
- name: yourrelease
  chart: yourchart
  values:
  - anykey: anyvalue

This is the better version of ` –set anykey=anyvalue` in helmfile. Would it be what you are looking for?

mumoshu avatar
mumoshu

And every change you make must be firstly declared in helmfile

Collin avatar
Collin

Is that in the helmfile itself or a new yaml file used for a quick helm upgrade patch?

mumoshu avatar
mumoshu

In the helmfile

mumoshu avatar
mumoshu

You made an interesting point!

mumoshu avatar
mumoshu

i was thinking that one of biggest reasons you might want to use helmfile would be to avoid adhoc upgrades like that

mumoshu avatar
mumoshu

have you tried helmfile diff or helmfile --interactive apply?

Collin avatar
Collin

I normally 100% would, this is one of those really odd edge-cases where I’ve run into an issue and want to go manual on it.

:--1:1
mumoshu avatar
mumoshu

it allows you to see the changes before syncing

mumoshu avatar
mumoshu

so i thought you might prefer the workflow of (1)changing one of values in helmfile.yaml (2)helmfile diff (3)finally helmfie sync even in such case

mumoshu avatar
mumoshu

but i think i’ve understood what you wanted - avoid editing yaml files until you figure out the correct config, right?

Collin avatar
Collin

Ye thats the basic goal. I would assume I could use like helmfile.yaml/chartname or something.

Collin avatar
Collin

And it would pull the yaml config out for that release. But that’s very likely not ‘right’ as the whole point of helmfile is to avoid something like this.

mumoshu avatar
mumoshu

yep

mumoshu avatar
mumoshu

helmfile does’t have a specific feature that helps today. but let me think what we can do

Collin avatar
Collin

Thanks so much for the help* and input. We really love using your tooling and will continue to use it.

mumoshu avatar
mumoshu

maybe something like helmfile plan --set helmfile.yaml/yourrelease/values.yaml:anykey=anyvalue thats stages the change to the values.yaml, diff, sync, and asks you to repeat or commit.

mumoshu avatar
mumoshu

thanks a lot for using helmfile and the feedback!

Collin avatar
Collin

I like that format. It’s similar to the normal helm upgrade releasename . [email protected] -f values.yaml -f values.${STAGE}.yaml that everyone is used to.

mumoshu avatar
mumoshu

@Collin Wrote up a github issue for this thread. Would you mind leaving your comment, or giving me your github username so that I can notice you whenever necessary

https://github.com/roboll/helmfile/issues/619

Faster way to iterate on values files? · Issue #619 · roboll/helmfile

Extracted from our discussion initiated by Collin at our official Slack channel: https://sweetops.slack.com/archives/CE5NGCB9Q/p1558087195000900 The goal was to quickly iterate on values files, per…

2019-05-16

mumoshu avatar
mumoshu

While my trial on replicated/ship, I came up with how we can potentially leverage kustomize to patch helm releases, but still installing it as a helm chart.

https://github.com/mumoshu/helm-x/issues/2

I think we can extend this to helmfile, so that helmfile can be used for applying adhoc “patches” to a helm chart without forking. The patches can be used for injecting init/sidecar containers, modifying other pod spec, adding configmap/secret required but not included in the chart, etc…

feat: replicated/ship integration · Issue #2 · mumoshu/helm-x

https://github.com/replicatedhq/ship The integration would basically run ship so that ship runs helm template and kustomize to generate a set of kubernetes manifests, which is then consumed by helm…

2019-05-15

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
10:29:36 PM

@Erik Osterman (Cloud Posse) set the channel topic: https://github.com/roboll/helmfile

Shane avatar
Shane

@mumoshu seeing some side effects from the new lockfile addition - https://github.com/roboll/helmfile/issues/598

lockfile causes issue with the same chart of different versions · Issue #598 · roboll/helmfile

When running commands such as template on a helmfile that has the same chart, but at different versions. in ./helmfile.yaml: in .helmfiles[1]: in helm/account/dev/x/dev/helmfile.yaml: duplicate cha…

mumoshu avatar
mumoshu

Thanks, just replied. I think we need to loosen the restriction partially or fully, depends on our reqs!

lockfile causes issue with the same chart of different versions · Issue #598 · roboll/helmfile

When running commands such as template on a helmfile that has the same chart, but at different versions. in ./helmfile.yaml: in .helmfiles[1]: in helm/account/dev/x/dev/helmfile.yaml: duplicate cha…

2019-05-14

Andrey Nazarov avatar
Andrey Nazarov

@mumoshu are you gonna make helmfile leverage helm-x to work with plain manifests or kustomize’ed stuff? I’m joking, but have you ever had these thoughts - to manage not only helm via a tool like helmfile. Actually, once I had a request about how to provide management of objects with one tool while some services were helmified, but others were not migrated yet.

mumoshu avatar
mumoshu

@Andrey Nazarov Is this a coincidence?!

I wrote up two feature requests yesterday that would result in integrating helm-x to helmfile

https://github.com/roboll/helmfile/issues/588 https://github.com/roboll/helmfile/issues/589

I think I have purposed the former issue to cover your exact use-case.

That being said, I’d definitely like to integrate helm-x to helmfile.

Please leave your :–1: or comments on issues you are interested, so that it is more likely to realize!

feature request: auto-chartify kustomize apps · Issue #588 · roboll/helmfile

You as an infrastructure engineer wants to manage the whole environment with a single helmfile.yaml. But one of your apps managed by your app-engineer colleague, and they wants to write his own app…

feature request: declarative sidecar injection · Issue #589 · roboll/helmfile

Sidecar proxies are important components that powers service meshes. Injecting sidecar proxies to your application is achieved usually by: install a mutating webhook onto your cluster or modifying …

Shane avatar
Shane

@mumoshu keep up the good work

Shane avatar
Shane

@mumoshu intriguing

Running helmfile from helm/account/dev


in ./helmfile.yaml: in .helmfiles[0]: in helm/account/helmfile.yaml: failed to read helmfile.yaml: helm exited with status 1:
  error loading config: no matching creation rules found
  Error: plugin "secrets" exited with error
 less ./helmfile.yaml                                                                                                                             
 less helm/account/helmfile.yaml                                                                                                                    
 ls -larth helm/account/helmfile.yaml                                                                                                               
-rw-r--r--  1 sstarcher  staff   4.5K May  8 17:50 helm/account/helmfile.yaml
 ls -larth ./helmfile.yaml                                                                                                                          
-rw-r--r--  1 sstarcher  staff   422B May 13 15:51 ./helmfile.yaml
Shane avatar
Shane

so it’s the secrets plugin choking

Shane avatar
Shane

Has something changed with the directory structure. It is not longer finding my .sops.yaml for sub helm charts.

Shane avatar
Shane

nope… something on my end has had to of changed

Shane avatar
Shane

So ya, probably certainly seems to be with secret decryption, but I can’t find out what would of changed. My .sops.yaml has always been at the root. Did it by chance change to run the secret decryption rooted from the helmfile instead of the root helmfile?

Shane avatar
Shane

Ya, seems something has changed related to sops. I now require a .sops.yaml in each of my directories that have a helmfile.

Shane avatar
Shane

Sops had a release 26 days ago so I also changed to the older sops release 3.2.0 from 2018. Both seem to have the same issue with helmfile.

Shane avatar
Shane

Well… I can fix it by changing ../../_environment/secrets.yaml to ../../../dev/_environment/secrets.yaml, but that does not explain why it just started failing.

Shane avatar
Shane

well I think I find a viable workaround… but then I caught things on even worse fire

Shane avatar
Shane
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x13cb127]

goroutine 1 [running]:
[github.com/urfave/cli.HandleAction.func1(0xc000251000)](http://github\.com/urfave/cli\.HandleAction\.func1\(0xc000251000\))
	/go/pkg/mod/github.com/urfave/[email protected]/app.go:474 +0x287
panic(0x145a340, 0x186a7a0)
	/usr/local/go/src/runtime/panic.go:522 +0x1b5
[github.com/roboll/helmfile/pkg/app.(*Error).Error](http://github\.com/roboll/helmfile/pkg/app\.\(\*Error\)\.Error)(0xc000192000, 0xc000192060, 0x2)
	/home/circleci/workspace/helmfile/pkg/app/app.go:343 +0xb7
[github.com/roboll/helmfile/pkg/app.(*Error).Error](http://github\.com/roboll/helmfile/pkg/app\.\(\*Error\)\.Error)(0xc000192030, 0xc000194090, 0x10ce2ee)
	/home/circleci/workspace/helmfile/pkg/app/app.go:339 +0x494
[github.com/roboll/helmfile/pkg/app.(*Error).Error(0xc000192060](http://github\.com/roboll/helmfile/pkg/app\.\(\*Error\)\.Error\(0xc000192060), 0x0, 0x12)
	/home/circleci/workspace/helmfile/pkg/app/app.go:339 +0x494
[github.com/roboll/helmfile/pkg/app.(*Error).Error](http://github\.com/roboll/helmfile/pkg/app\.\(\*Error\)\.Error)(0xc000192090, 0xc00016c090, 0x0)
	/home/circleci/workspace/helmfile/pkg/app/app.go:339 +0x494
[github.com/roboll/helmfile/cmd.toCliError](http://github\.com/roboll/helmfile/cmd\.toCliError)(0x1567d20, 0xc000192090, 0x0, 0xc00000cc40)
	/home/circleci/workspace/helmfile/cmd/cmd.go:88 +0xa7
[github.com/roboll/helmfile/cmd.FindAndIterateOverDesiredStatesUsingFlagsWithReverse(0xc000184140](http://github\.com/roboll/helmfile/cmd\.FindAndIterateOverDesiredStatesUsingFlagsWithReverse\(0xc000184140), 0xc00005e500, 0xc00005e5c0, 0x8, 0xc00000cbe0)
	/home/circleci/workspace/helmfile/cmd/cmd.go:79 +0xfa
main.findAndIterateOverDesiredStatesUsingFlags(...)
	/home/circleci/workspace/helmfile/main.go:619
main.main.func4(0xc000184140, 0x0, 0x0)
	/home/circleci/workspace/helmfile/main.go:220 +0x66
reflect.Value.call(0x143e300, 0x14e5af0, 0x13, 0x14c82e4, 0x4, 0xc000132fa0, 0x1, 0x1, 0xc00002e080, 0x1011bc3, ...)
	/usr/local/go/src/reflect/value.go:447 +0x461
reflect.Value.Call(0x143e300, 0x14e5af0, 0x13, 0xc000132fa0, 0x1, 0x1, 0xc00008a820, 0xc00008a868, 0x140)
	/usr/local/go/src/reflect/value.go:308 +0xa4
[github.com/urfave/cli.HandleAction(0x143e300](http://github\.com/urfave/cli\.HandleAction\(0x143e300), 0x14e5af0, 0xc000184140, 0x0, 0x0)
	/go/pkg/mod/github.com/urfave/[email protected]/app.go:483 +0x1ff
[github.com/urfave/cli.Command.Run(0x14c994c](http://github\.com/urfave/cli\.Command\.Run\(0x14c994c), 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x14de166, 0x3d, 0x0, ...)
	/go/pkg/mod/github.com/urfave/[email protected]/command.go:186 +0x8d1
[github.com/urfave/cli.(*App).Run(0xc000000f00](http://github\.com/urfave/cli\.\(\*App\)\.Run\(0xc000000f00), 0xc0000a2000, 0x5, 0x5, 0x0, 0x0)
	/go/pkg/mod/github.com/urfave/[email protected]/app.go:237 +0x601
main.main()
	/home/circleci/workspace/helmfile/main.go:578 +0x1fef
Shane avatar
Shane

helmfile version v0.60.1

Shane avatar
Shane

well fixed the 2 errors i was getting - https://github.com/roboll/helmfile/pull/594

1
party_parrot1
mumoshu avatar
mumoshu

Thanks for fixing! Just merged it

Shane avatar
Shane

Thanks sir when you get a chance can you cut a release

mumoshu avatar
mumoshu

I’ll do it in a hour so once I get home

mumoshu avatar
mumoshu

fyi: 0.60.2 and 0.61.1 are the latest releases containing your fix

Shane avatar
Shane

thank you

Andrey Nazarov avatar
Andrey Nazarov

@mumoshu sounds great!

2019-05-13

Ufou avatar

@mumoshu thanks! Yeah, I worked out as much, I just didn’t get it working yet and wondered if others were successful with it…

:--1:1

2019-05-11

mumoshu avatar
mumoshu

@Ufou Hey! Unfortunately I have never tried that myself. And I think you’d need to enhance keelhq a bit to “fully” make it work with helmfile

mumoshu avatar
mumoshu

Keel updates your container images referenced by your apps by calling helm on its own.

I thought what you want would be keel calls helmfile to update container image tags of your apps. That’s not how keel works at the moment

mumoshu avatar
mumoshu

Anyway, in case you really want keel just for updating your container image tags out of helmfile, just add keel-related values to your values.yaml managed by helmfile.

https://keel.sh/v1/guide/documentation.html#Helm-example

Documentation — Keel

Advanced, in-depth configuration of Keel - Keel

2019-05-10

Ufou avatar

is anyone successfully using helmfile with keel?

2019-05-09

mumoshu avatar
mumoshu

Do you guys want the new config bases: ["mybase.yaml"] added to helmfile, even though it won’t support templating, and yaml anchors across the border? https://github.com/roboll/helmfile/issues/388#issuecomment-489633839

Using helmfiles, but unable to pass Environment down to helmfile.yaml · Issue #388 · roboll/helmfile

When using {{ readFile &quot;environments.yaml }}} — helmfiles: - ./*/helmfile.yaml Environment.Values are not passed to the globbed helmfiles. I would expect that to happen/work, in order to kee…

mumoshu avatar
mumoshu

Started to think we should require .gotpl extension to enable the whole-file templating of helmfile.yaml https://github.com/roboll/helmfile/pull/573#issuecomment-491142465

@SeB @Erik Osterman (Cloud Posse) I’m interested in your opinion!

Added AWS SSM Parameter Store Integration by rms1000watt · Pull Request #573 · roboll/helmfile

See discussion here: #569 @osterman @mumoshu @sgandon @lanmalkieri Here is working code for the YAMLs below. This allows ssm to be defined and used: ssm: - name: east1 prefix: /dev/us-east-1/…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Is there a similar convention enforced by other projects?

Added AWS SSM Parameter Store Integration by rms1000watt · Pull Request #573 · roboll/helmfile

See discussion here: #569 @osterman @mumoshu @sgandon @lanmalkieri Here is working code for the YAMLs below. This allows ssm to be defined and used: ssm: - name: east1 prefix: /dev/us-east-1/…

mumoshu avatar
mumoshu

maybe, but nothing im aware of

mumoshu avatar
mumoshu

i think it isn’t a popular problem. there’s no major tool that deals with two phases of templates with a single tool

mumoshu avatar
mumoshu

for example gomplate works by evaluating go template once for each file

mumoshu avatar
mumoshu

variant command is a valid yaml. script and arguments can be templated go template expressions. but the whole command is a valid YAML and no file-wide templating is possible

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i think i see some helm charts now and then using the .gotpl extension

mumoshu avatar
mumoshu

that’s good news

mumoshu avatar
mumoshu

Also cc/ @Shane

mumoshu avatar
mumoshu
breaking change? `.gotpl` should be required to enable templating of `helmfile.yaml` · Issue #584 · roboll/helmfile

Extracted from #573 (comment) I think this, and the upcoming bases feature #388, would allow the user to choose either to prefer, (1) readability of release template expressions #439, or (2)less nu…

Shane avatar
Shane

Wait I thought we currently required a extension

Shane avatar
Shane

Yaml files don’t get templates

Shane avatar
Shane

What am I missing

mumoshu avatar
mumoshu

@Shane Yep, for values.yaml that’s true

mumoshu avatar
mumoshu

I’m proposing the same for helmfile.yaml

Shane avatar
Shane

Ahh

mumoshu avatar
mumoshu

(I think we’ve discussed about that long before… but at that time we thought it’s better to just use .yaml as helm templates do the same

Shane avatar
Shane

I typically don’t put anything in my helmfile except for chart references

mumoshu avatar
mumoshu

Oh!

Shane avatar
Shane

Hence never thinking about the templating of helmfile

mumoshu avatar
mumoshu

Do you use another tools to make it work against various envs, repositories, and so on?

Shane avatar
Shane

I keep pushing our teams to leverage it as little as possible if we control the chart

mumoshu avatar
mumoshu

i.e. generating helmfile.yaml with a tool like jsonnet

Shane avatar
Shane

All values are kept in separate values files

Shane avatar
Shane

So helmfile is a list of charts and value references

mumoshu avatar
mumoshu

that’s a neat idea

mumoshu avatar
mumoshu

so you maintain one helmfile.yaml per environment

Shane avatar
Shane

Almost yes

Shane avatar
Shane

In dev we have a helmfile per team also

mumoshu avatar
mumoshu

makes sense!

mumoshu avatar
mumoshu

i think the helmfile.yaml template is used (1)when people want to create a single helmfile.yaml that is the entrypoint to all the flavors of environments or (2)make each helmfile.yaml DRY when one has to maintain many of them, without introducing another tools like jsonnet

mumoshu avatar
mumoshu

so use-case like yours completely make sense to me

Shane avatar
Shane

Ya tons of options

1
SeB avatar

Hello, @mumoshu 1) could you explain in detail why we can’t consider helmfile.yaml as a template file and why we need to have both extensions. 2) Also why not using the same extension as values (gotmpl vs gotpl) or vise-versa otherwise this will be confusing.

mumoshu avatar
mumoshu

For 1), it doesn’t work well with release templating.

mumoshu avatar
mumoshu

For 2), gotmpl and gotpl should be merged. I think I meant gotmpl

mumoshu avatar
mumoshu

An example related to 1) can be seen at https://github.com/roboll/helmfile/pull/573

Added AWS SSM Parameter Store Integration by rms1000watt · Pull Request #573 · roboll/helmfile

See discussion here: #569 @osterman @mumoshu @sgandon @lanmalkieri Here is working code for the YAMLs below. This allows ssm to be defined and used: ssm: - name: east1 prefix: /dev/us-east-1/…

mumoshu avatar
mumoshu
releases:
  - name: redis-test
    chart: stable/redis
    values:
    - fake_key:
        redis_pass_east: {{ ssm "east1:redis_password" }}

{{ ssm "east1:redis_password" }} is evaluated twice in the above example.

mumoshu avatar
mumoshu

Another example for release templates, is that you can’t write:

releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

It must be:

releases:
  - name: redis-test
    values:
    - releases/{{` {{ .Release.Name }} `}}/values.yaml

Due to that helmfile.yaml itself is a go template.

SeB avatar

thank @mumoshu for your quick answer, but change the extension to gotmpl whould change what in the above examples ?

mumoshu avatar
mumoshu

@SeB What I suggested was to “add” .gotmpl ext. to explicitly enable helmfile.yaml templating. And release templates are always enabled. That being said

SeB avatar

but why not consider always helmfile.yaml a templates ?

mumoshu avatar
mumoshu

Making helmfile require .gotmpl to enable helmfile.yaml templating allows you to write a regular YAML file named helmfile.yaml

mumoshu avatar
mumoshu

So that

releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

THis is a valid helmfile.yaml with just release templaets

mumoshu avatar
mumoshu

Note that it isn’t a valid helmfile.yaml if helmfile.yaml itself is a go template

mumoshu avatar
mumoshu

This happens because there’s two template contexts. When templating helmfile.yaml as a whole, you have no access to {{ .Release }}

mumoshu avatar
mumoshu

When templating helmfile.yaml as a whole, we have no {{ .Release }} in the template context available, because helmfile.yaml is basically a template that “produces YAML” . No way to know what releases are defined in the helmfile.yaml before rendering and parsing it as YAML…

SeB avatar

ok, I think I got it

mumoshu avatar
mumoshu

And no way to know which {{ .Release }} it is rendering while templating helmfile.yaml as a whole

mumoshu avatar
mumoshu

// {{ .Release }} becomes available only when helmfile iterates over each release defined in the YAML

SeB avatar

so what you propose is to have 2 extensions : 1) helmfile.yaml

releases:
  - name: redis-test
    values:
    - releases/{{ .Release.Name }}/values.yaml

2) helmfie.gotmpl

releases:
  - name: redis-test
    values:
    - releases/{{` {{ .Release.Name }} `}}/values.yaml
SeB avatar

to have the same result

mumoshu avatar
mumoshu

Exactly!

SeB avatar

I am afraid this is too complicated for the end-user.

mumoshu avatar
mumoshu

Yep it is

mumoshu avatar
mumoshu

I’m thinking that it is already complicated and we’d better migrate towards explicit file ext. towards helmfile v1

mumoshu avatar
mumoshu

so that it becomes simpler

SeB avatar

I beleive this could be just much better explained in the documentation, this is what it is lacking today. The of precise documentation make pleople to hack/try/fail to have their helmfile working

mumoshu avatar
mumoshu

For advanced users I’d still suggest using helmfile.gotmpl. But as I’ve explained in the issue, you can fully achieve the same result even with helmfile.yaml without gotmpl, when combined with bases

mumoshu avatar
mumoshu

Hm, maybe in that case I completely remove helmfile.yaml and only helmfile.gotmpl supported.

mumoshu avatar
mumoshu

so that there’s no confusion anyway.

mumoshu avatar
mumoshu

And the docs you’ve suggested would help people understand why one needs backticks sometimes

SeB avatar

yep

mumoshu avatar
mumoshu

Great! Thanks for all the feedbacks

mumoshu avatar
mumoshu

So we can proceed that way

mumoshu avatar
mumoshu

And discuss about re-introducing helmfile.yaml but as a regular YAML file this time separately in longer term, maybe after docs improvements, if it is really people want

mumoshu avatar
mumoshu
breaking change? `.gotpl` should be required to enable templating of `helmfile.yaml` · Issue #584 · roboll/helmfile

Extracted from #573 (comment) I think this, and the upcoming bases feature #388, would allow the user to choose either to prefer, (1) readability of release template expressions #439, or (2)less nu…

mumoshu avatar
mumoshu

It is interesting that helmfile.yaml, rather than helmfile.gotmpl, started to look like for advanced users

2019-05-05

2019-05-03

SeB avatar

hello there, I am launching an helmfile apply on this

releases: 
- name: redis
  chart: stable/redis
  hooks:
    - events: ["cleanup"]
      command: "echo"
      args: ["foo bar"]

And expect to see “foo bar” on my console but nothing appears ?

SeB avatar

what am I doing wrong ?

Issif avatar
Issif

can you check logs of your deployment? it should in stdout of container, not yours

SeB avatar

what ! it does not make sense at all. The hook is not executed is any of the pod of my deployment.

SeB avatar

why would it and which container ?

Issif avatar
Issif

didn’t noticed was a hook, sorry

Andrey Nazarov avatar
Andrey Nazarov

prepare hook behaves the same. Try with --log-level=debug and you will see the output. I don’t know though whether it is an expected behaviour or not.

    keyboard_arrow_up