#helmfile (2019-04)

https://github.com/roboll/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles Archive: https://archive.sweetops.com/helmfile/

2019-04-30

Kevin Gimbel avatar
Kevin Gimbel

Here’s the cleaned-up log I am getting when I run helmfile sync

 helmfile sync 
Adding repo my_private_repo <https://XXX>
"my_private_repo" has been added to your repositories

Updating repo
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "my_private_repo" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈ 

Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/cerebro.s3.product.image
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX
Upgrading my_private_repo/XXX

[...]

UPGRADE FAILED
ROLLING BACK
Error: "cerebro-s3-product-image" has no deployed releases
Error: UPGRADE FAILED: "cerebro-s3-product-image" has no deployed releases

[...]

err: release "cerebro-s3-product-image" in "helmfile.yaml" failed: exit status 1
exit status 1
Kevin Gimbel avatar
Kevin Gimbel

I’ve replaced the names of all non-related releases with XXX and removed all the output before and after the error

Kevin Gimbel avatar
Kevin Gimbel

If I run helmfile destroy and then helmfile apply everything installs fine

Andrey Nazarov avatar
Andrey Nazarov

It’s a known case like if your release fails being deployed for the first time and you don’t set atomic flag to true then yes, you cannot do anything except purging your failed release first.

Andrey Nazarov avatar
Andrey Nazarov
Workaround for Error: "release-name" has no deployed releases · Issue #471 · roboll/helmfile

Since Helm doesn&#39;t allow to install a new release if the old one with the same name is not purged, helm diff shows errors. Thus helmfile apply returns errors too. So, we end up with something l…

SeB avatar

yes the error here is missleading and should propose a solution.

SeB avatar

If you want to check the status of you helm releases, use helm list -a

SeB avatar

and you may see some in FAILED or PENDING_INSTALL that you need to delete

Kevin Gimbel avatar
Kevin Gimbel

There is indeed a Failed deployment, but this was from a helm install not from helmfile - can in theory every failed deployment fail my helmfile deployments? We’re not yet running in production which is why I installed some stuff with helm and some with helmfile as of now

Kevin Gimbel avatar
Kevin Gimbel

I’ll look into the atomic flag, thanks a lot!

Andrey Nazarov avatar
Andrey Nazarov

It should work ok with releases in FAILED state if it’s not their initial state. Then, atomic flag allows you to restore the previous state if something goes wrong, hence even if the very first deployment fails Helm will remove it automatically.

Kevin Gimbel avatar
Kevin Gimbel


It should work ok with releases in FAILED state if it’s not their initial state.
This is true. With atomic: true I am able to update / upgrade the releases as I’d expect to, but only if nothing failed with the initial deployment

Lee Skillen avatar
Lee Skillen

Nice, helmfile supports private repositories out of the box - We (Cloudsmith) recently added support for Helm repositories and we’re looking at how we can document integration with a few of the solutions that are out there. Looking at helmfile and landscaper atm. This is mostly for user reference, but everyone likes a worked example. We’ve only got a very minimal example at the moment.

Erik Osterman avatar
Erik Osterman

@zadkiel.aharonian I’m trying ot use helm-git but running into errors

Erik Osterman avatar
Erik Osterman
helm repo add t1 "git+<https://github.com/stakater/[email protected]/kubernetes/chart/forecastle?ref=master>"
Erik Osterman avatar
Erik Osterman

i’ll move to #helm

2019-04-29

Erik Osterman avatar
Erik Osterman
Added SSM integration by rms1000watt · Pull Request #569 · roboll/helmfile

There&#39;s been some interest for helmfile integration with SSM. Here is an example of what it can look like. For our current workflows, we have Bash scripts that export Env Vars via aws-env then …

Erik Osterman avatar
Erik Osterman

@rms1000watt

rms1000watt avatar
rms1000watt
10:30:25 PM

@rms1000watt has joined the channel

rms1000watt avatar
rms1000watt

lololol thx

rms1000watt avatar
rms1000watt

@stobiewankenobi

stobiewankenobi avatar
stobiewankenobi
10:30:43 PM

@stobiewankenobi has joined the channel

rms1000watt avatar
rms1000watt

lolol ^

Erik Osterman avatar
Erik Osterman

That’s a cool idea

Erik Osterman avatar
Erik Osterman

can you add an example invocation in gotemplate syntax?

rms1000watt avatar
rms1000watt
- autoDiscovery:
          clusterName: {{ requiredEnv "cluster_name" }}
AWS_REGION=us-east-1 SSM_PATH=/redacted/ ./helmfile -f $GITHUB/calm/helmfile/stable/init.yml template
rms1000watt avatar
rms1000watt

super duper small example

rms1000watt avatar
rms1000watt

but nothing fancy

Erik Osterman avatar
Erik Osterman

@Jeremy Grodberg you might dig this.

Jeremy Grodberg avatar
Jeremy Grodberg
10:32:08 PM

@Jeremy Grodberg has joined the channel

rms1000watt avatar
rms1000watt

just a replacement of aws-env

Erik Osterman avatar
Erik Osterman

why aws-env vs chamber?

Erik Osterman avatar
Erik Osterman
Droplr/aws-env

Secure way to handle environment variables in Docker and envfile with AWS Parameter Store - Droplr/aws-env

rms1000watt avatar
rms1000watt

yea

rms1000watt avatar
rms1000watt

i haven’t used chamber enough to have an opinion

rms1000watt avatar
rms1000watt

but regardless, it’s 2 commands

rms1000watt avatar
rms1000watt

just thinking about terraform -> terragrunt

rms1000watt avatar
rms1000watt

i was thinking.. helmfile -> helmgrunt

rms1000watt avatar
rms1000watt

but like.. naaa, just update helmfile directly

Erik Osterman avatar
Erik Osterman

this will make helmfile easier to automate for us since we use ENVs for so much.

Erik Osterman avatar
Erik Osterman

we’ve been calling chamber exec ... helmfile ...

rms1000watt avatar
rms1000watt

rms1000watt avatar
rms1000watt

same, basically here

Erik Osterman avatar
Erik Osterman

@mumoshu

rms1000watt avatar
rms1000watt

and giving credit where credit’s due–this was all your idea initially

Erik Osterman avatar
Erik Osterman

aha, so I see this just loads every parameter into the env, so everything else just continues to work seamlessly.

Kevin Gimbel avatar
Kevin Gimbel

Hey everyone, I’m having sort of a crisis with helmfile. Maybe I misunderstood how it should work, but whenever I deploy a state (helmfile apply) I can not add new releases to it. If I make a change to the helmfile.yaml the new releases cannot be deployed because both helmfile sync and helmfile apply fail with an error indicating they cannot find (and therefore upgrade) the new release. Can I never again add releases to my helmfile without destroying everything first?

Andrey Nazarov avatar
Andrey Nazarov

@Kevin Gimbel What is the actual error shown? Could you please provide an example? It works for me very well.

2019-04-26

erik-stephens avatar
erik-stephens

Anyone know how to get --args=--context=2 relayed to the diff command when I helmfile apply?

2019-04-18

Andrey Nazarov avatar
Andrey Nazarov

Experiencing the issue when my releases are not get updated via helmfile apply even though the version of a chart in helmfile.yaml is bumped. helm diff doesn’t show me a diff either. Cannot figure out what’s the problem…

Kevin Gimbel avatar
Kevin Gimbel

Have you tried helmfile sync?

Kevin Gimbel avatar
Kevin Gimbel

This usually works for me

Kevin Gimbel avatar
Kevin Gimbel

Not sure if it’s correct, tho

Andrey Nazarov avatar
Andrey Nazarov

Yeah, I was thinking about that, but want to get to know why things stopped working for some releases. I think this is related to some mess with versions introduced recently. We are using Gitlab Ci and I should say it works terribly with monorepos. So we had to enable overwriting charts in the Chartmuseum. Doing so we should be very careful with versioning. And probably this hit back).

Andrey Nazarov avatar
Andrey Nazarov

It seems like one of the workload ran out of Helm control and stopped being a part of the release. I purged this release, but some stuff still exists. It came from a subchart that was disable it the latter release. What a curious case.

Andrey Nazarov avatar
Andrey Nazarov

I ended up with removing all objects manually and reapplying the config.

Brian avatar
Brian

Hey! How would I be able to use the value for namespace in my chart?

Erik Osterman avatar
Erik Osterman

I think it was just “{{ .Namespace }}” but on my phone so hard to check

Erik Osterman avatar
Erik Osterman

You are talking about using the value of the —namespace arg, right?

Brian avatar
Brian

Here’s what I just tried:

  #
  # Testing Pod
  #
  - name: testing
    namespace: blue
    chart: ./chart
    values:
    - "./values/testing-values.yaml"
    - "./values/master.yaml"
    wait: true
    set:
    - name: image.tag
      value: 74.1.192
    - name: deployment.env 
      value: {{ .Environment.Values.environment }}
    - name: deployment.replicas 
      value: {{ .Environment.Values.replicas }}
    - name: deployment.namespace
      value: {{ .Namespace }}
Brian avatar
Brian

I’m leaving the namespace hardcoded, although I saw that’s not recommended, as we’re using two separate charts for blue green deployments.

Andrey Nazarov avatar
Andrey Nazarov

Why are you trying to set the namespace twice? namespace: blue is enough to deploy your workloads in a namespace called “blue”.

Brian avatar
Brian

I need to assign the blue pods to an instance groups for blue, and green pods to instance group for green. I have a toleration set to noschedule based off of the namespace name.

Andrey Nazarov avatar
Andrey Nazarov

I don’t know the logic behind your helm chart, but what about just using {{ Release.Namespace }} inside your chart? It should pick the namespace defined in your helmfile.

Brian avatar
Brian

That’s exactly what I was looking for. Thanks a lot!

Andrey Nazarov avatar
Andrey Nazarov

Never thought about how to do blue-green using helmfile though. That’s kinda interesting.

Erik Osterman avatar
Erik Osterman
cloudposse/example-app

Example application for CI/CD demonstrations of Codefresh - cloudposse/example-app

Erik Osterman avatar
Erik Osterman

Have an example of how we demo blue green with Helmfile

Erik Osterman avatar
Erik Osterman

We also use #variant by @mumoshu to define a deploy tool

Erik Osterman avatar
Erik Osterman

This demo uses istio gateway

Andrey Nazarov avatar
Andrey Nazarov

Btw, why have cloudposse decided to opt values embedded in helmfile.yaml?

Andrey Nazarov avatar
Andrey Nazarov

Thank you for the link.

Erik Osterman avatar
Erik Osterman

We use environment variables because they are interoperable with all tools

Erik Osterman avatar
Erik Osterman

We use direnv for configurations (non secret)

Erik Osterman avatar
Erik Osterman

We use chamber for secrets

Brian avatar
Brian

Yeah. I just started messing around with Helmfile the other day. So far, here’s how I’m structuring my blue/green deployments:

├── chart
│   ├── Chart.yaml
│   └── templates
│       ├── NOTES.txt
│       ├── _helpers.tpl
│       ├── autoscale.yml
│       ├── deployment.yml
│       ├── ingress.yml
│       ├── service.yml
│       └── tests
├── enterprise-blue.yaml
├── enterprise-green.yaml
├── ingress
│   ├── Chart.yaml
│   └── templates
│       ├── _helpers.tpl
│       ├── configmap.yml
│       ├── ingress-backend.yml
│       ├── ingress-nginx.yml
│       ├── rbac.yml
│       ├── service.yml
│       └── tests
├── ingress-helmfile.yaml
├── single-helmfile.yaml

I have a separate values file for each microservice. I still have played around with it enough to determine if this is the best solution. It’ll probably change later today.

2019-04-17

2019-04-15

SeB avatar

Just a question out of the open, What is the advantage of using helmfile versus ansible ?

mumoshu avatar
mumoshu

No need to have a Makefile as the entrypoint, and a bit less verbose than ansible because helmfile focus on helm

2019-04-13

2019-04-12

Kevin Gimbel avatar
Kevin Gimbel

Hello all! I’ve just started using helmfile and I was wondering if I can tell it which service account it should use for the deployments? If I run helmfile apply it automatically uses system:serviceaccount:staging:default but I’d rather have it use system:serviceaccount:staging:helmfile or some other dedicated user. Is there anyway to do this with helmfile?

SeB avatar

helmfile is just relying on the helm command, to if you want to have a specific service accoujnt used for your helm deployments then you need to install the helm tiller with the service account you want.

SeB avatar

using this command : helm init --service-account <your service account>

SeB avatar

I may be wrong but believe there is no other way.

Kevin Gimbel avatar
Kevin Gimbel

Thanks! I guess that’d be good enough for me for now; Our use case isn’t too complex as of now.

Jake avatar

Hey, just a quickie; is there any place I can see what values I have available to me in a template? Trying to use variables like .Release.Name within the template file and not having much success.

Jake avatar

It’d appear it’s related to https://github.com/roboll/helmfile/issues/351

Namespace template doesn't render any value · Issue #351 · roboll/helmfile

When I try to use {{ .Namespace }} in a values.yaml.gotmpl template, it just renders as an empty string. It doesn&#39;t throw an error. This happens if I let helmfile use the default namespace or s…

SeB avatar

I have no trouble using .Release.name when I am inside a helmfile release element, what is your issue ?

Jake avatar

I have a .gotmpl file with some values which I’m trying to access .Release.Name & .Namespace *assuming that the release namespace is that variable

Jake avatar
tringTemplate:46:33: executing "stringTemplate" at <.Release.Name>: can't evaluate field Release in type state.EnvironmentTemplateData
SeB avatar

I am accessing .Release.Name inside the helmfile not in a values file

SeB avatar

but try this with double double curly bracket

SeB avatar
{{`{{.Release.Name}}`}}
Jake avatar

I did try that. seems like the double pass rendering doesn’t happen on the gotmpl files.

SeB avatar

let me try in gotmpl values file

SeB avatar

indeed same problem here.

SeB avatar

it seems like a bug to me, it is very strange how helmfile scopes values

Jake avatar

I’m also trying it with set: to apply the variables with –set at the command line with helm and that’s a mixed bag

SeB avatar

what I would recommend as a workaround is to use inlined values inside you helmfile. There I am sure the Release.Name works

Jake avatar

I’ll give that a go

Jake avatar

Similar problems to be honest

Jake avatar
values:
...
        host: "{{`{{ .Release.Name }}`}}-elasticsearch-client.{{ .Namespace }}.svc.cluster.local:9200"
        
>>>

               - name: ELASTICSEARCH_HOST
                 value: "{{ .Release.Name }}-elasticsearch-client..svc.cluster.local:9200"
Jake avatar

top is the config, bottom is the output from helmfile diff

SeB avatar

hummm… checking on my helm files….

SeB avatar

indeed I can also reproduce it

SeB avatar

heu… not sure

SeB avatar

this values passed in the helmfile

SeB avatar
  values:
    - tags:
        talend-zookeeper: "{{`{{.Release.Name}}`}}"
SeB avatar

sorry actually this is not interpreted

SeB avatar

I have it working on the values file names, chart name and chart version.

SeB avatar

but not in values.

Jake avatar

aye that’s my mileage too.

Erik Osterman avatar
Erik Osterman

@mumoshu have you met @hairyhenderson?

mumoshu avatar
mumoshu

@hairyhenderson hey! glad to meet you

hairyhenderson avatar
hairyhenderson

oh hi

hairyhenderson avatar
hairyhenderson
02:45:57 AM

@hairyhenderson has joined the channel

Erik Osterman avatar
Erik Osterman

Just thought I’d make the intro since I saw your comment here: https://github.com/roboll/helmfile/issues/392#issuecomment-455061273

Feat: Allow simple Vault integration · Issue #392 · roboll/helmfile

Currently there are many users that want to integrate Vault with Kubernetes, but there are no high level tools for this. The current Kubernetes AuthMethod for Vault is too complex and coupled to th…

2019-04-10

Andrey Nazarov avatar
Andrey Nazarov

Wow, I’ve been just struck by the curious failure. Not related with the aforementioned, btw. I noticed that for one of my release helmfile apply triggered helm upgrade --install three times in a row(!). There were no diffs for that particular release. One upgrade succeeded, but for another two the following error was printed:

UPGRADE FAILED
ROLLING BACK
Error: release: "XXXX-pgdump.v4" already exists
Error: UPGRADE FAILED: release: "XXXX-pgdump.v4" already exists

And it all ended up with err: release "XXXX-pgdump" in "helmfile.yaml" failed: exit status 1 and err: remove /tmp/values636482145: no such file or directory error.

It had worked ok before I bumped helmfile version to the most recent.

Andrey Nazarov avatar
Andrey Nazarov

Ok, I’ve investigated it a little bit. It seems it happens with the last release in the list. I rearranged releases and this led to the same error, but for the different release. The situation is pretty the same: no diffs, three helm upgrade followed by two “UPGRADE FAILED” errors. All of that ends up with two “err: remove /tmp/values470019469: no such file or directory” message. And if I have helmfile apply embedded to my CD system, this behaviour leads to the failed build.

$ helm version
Client: &version.Version{SemVer:"v2.13.0", GitCommit:"79d07943b03aea2b76c12644b4b54733bc5958d6", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.13.1", GitCommit:"618447cbf203d147601b4b9bd7f8c37a5d39fbb4", GitTreeState:"clean"}
$ helmfile --version
helmfile version v0.54.0
$ helm diff version
2.11.0+3
Andrey Nazarov avatar
Andrey Nazarov

No luck with ‘helmfile sync’ as well. And I don’t see any mention of releases that are supposed to be uninstalled in helmfile output.

mumoshu avatar
mumoshu

@Andrey Nazarov Hey! Thanks a lot for the detailed report. Indeed this seems like a bug introduced recently..

I’m going to have a next bug fixing sprint for helmfile soon.

Until then.. Would you mind sharing the version number of your old helmfile that worked?

Andrey Nazarov avatar
Andrey Nazarov

The working one is really old: 0.45.1))

Andrey Nazarov avatar
Andrey Nazarov

Should I create a bug report?

mumoshu avatar
mumoshu

I’ll definitely appreciate it!

mumoshu avatar
mumoshu

Do you have a small helmfile.yaml to reproduce it, by any chance?

Andrey Nazarov avatar
Andrey Nazarov

No, I don’t have, only the huge one. But I’ll create one for testing purpose.

1
1
Andrey Nazarov avatar
Andrey Nazarov

I can’t reproduce the issue with a small helmfile. At the same time, I can do it using the initial one (it has aprox. 25 releases defined). I should figure it out…

mumoshu avatar
mumoshu

My understanding of the problem is that helmfile sync was unable to delete releases at all, regardless of how many releases you have in helmfile.yaml

mumoshu avatar
mumoshu

And apply was able to detect deletion of the last release only

mumoshu avatar
mumoshu

and I’ve released v0.52.2 with the fix

Andrey Nazarov avatar
Andrey Nazarov

Sorry, but we are talking about a different issue here). This issue is about helmfile somehow triggers three helm upgrade commands for the last release in helmfile resulting in err: remove /tmp/values636482145: no such file or directory error and the failed build.

Andrey Nazarov avatar
Andrey Nazarov

I’m trying to catch it…

Andrey Nazarov avatar
Andrey Nazarov

Or you meant PR555 would fix this issue too?

mumoshu avatar
mumoshu

Yes, that’s my thought

Andrey Nazarov avatar
Andrey Nazarov

I’ll check this out).

1
Andrey Nazarov avatar
Andrey Nazarov

@mumoshu, you rule! It works smoothly! Thank you very much.

Andrey Nazarov avatar
Andrey Nazarov

This one has been fixed with Helmfile v.0.54.2.

2

2019-04-09

SeB avatar

I’ll create 2

Andrey Nazarov avatar
Andrey Nazarov

Hi. I was wondering how installed: false works… For some reason I don’t see my release marked with installed: false among ones taken for diff (for ex. “Comparing shared-dev-elastic elastic/elasticsearch”). And my release doesn’t get uninstalled.

Erik Osterman avatar
Erik Osterman

Did you use helmfile apply? Or helmfile sync?

Andrey Nazarov avatar
Andrey Nazarov

helmfile apply

Erik Osterman avatar
Erik Osterman

Hrmmm not sure what the problem is…

Andrey Nazarov avatar
Andrey Nazarov

What is under the hood of installed: false? Can anybody explain what is going to happen when I set it to false within my release? So, I see helmfile runs helm list ^release name$ in that case, what’s next?

mumoshu avatar
mumoshu

If you’ve set installed: false to already installed release and run helmfile sync or apply, helmfile should uninstall the release.

If it doesn’t work like that, it might be a bug!

Andrey Nazarov avatar
Andrey Nazarov

Has been fixed with Helmfile v.0.54.2.

Andrey Nazarov avatar
Andrey Nazarov

But in helmfile’s log output I see helm list against the release I want to uninstall.

mumoshu avatar
mumoshu

That, itself, would be an expected behavior, as helmfile determines if it needs to run helm delete according to the result of helm list

Andrey Nazarov avatar
Andrey Nazarov

Yep, so it partially works somehow. But doesn’t unistall the release in my case.

mumoshu avatar
mumoshu

Thanks! Would this look similar to your issue? https://github.com/roboll/helmfile/issues/554

Helm apply not deleting releases · Issue #554 · roboll/helmfile

Hi, starting at #526, release deletion is broken in apply. Here&#39;s the output of helmfile –log-level debug apply: processing file &quot;helmfile.yaml&quot; in directory &quot;.&quot; second-pas…

Andrey Nazarov avatar
Andrey Nazarov

Ah, sorry I messed up things, too many tabs opened)). Yes, it’s definitely the same issue.

Andrey Nazarov avatar
Andrey Nazarov

I’ve removed my first answer.

mumoshu avatar
mumoshu
fix: helmfile apply and sync should properly delete releases by mumoshu · Pull Request #555 · roboll/helmfile

Since #526, helmfile apply have been really able to detect deletion of the last release only, and sync has been unable to mark releases with installed: false for removal. Fixes #554

mumoshu avatar
mumoshu

v0.54.1 is being released shortly

:--1:1
Andrey Nazarov avatar
Andrey Nazarov

It works!

2019-04-08

SeB avatar

Hello everyone, Is there a way to load helmfile wide values without failing when the file is not found. I am using right now the environment.default but if the values file is not found it is failing.

environments:
  default:
    values:
      - ../values/chart-versions.yaml
      - {{ env "LOCAL_VALUES_CACHE" }}/foo/helmfile/values/chart-versions.yaml
mumoshu avatar
mumoshu

hey! unfortunately there’s no way currently

SeB avatar

Also it seems that the values file for an environment cannot be abosolute path but only relative ones ?????

mumoshu avatar
mumoshu

hey! yeah i have never thought anyone would use absolute paths for env values files..

worth writing a feature request?

2019-04-07

2019-04-06

SeB avatar

heu no, not in a environment values file, I do that in my helm file.

SeB avatar

here is my helmfile

environments:
  default:
    values:
      - ../values/chart-versions.yaml

templates:
  chart: &chart
    {{ $versionKey := printf "%sVersion" `{{.Release.Name}}` }}
    chart: {{ .Environment.Values | get ($versionKey) "" | eq "" | ternary "../.." "talend" }}/{{`{{.Release.Name}}`}}
    version: {{.Environment.Values | get ($versionKey) ""}}

releases:
- name: tpsvc-crypto
  <<: *chart
...and many more
mumoshu avatar
mumoshu

Made sense, a lot!

mumoshu avatar
mumoshu

This makes me think, we should create an issue for this as you suggested https://sweetops.slack.com/archives/CE5NGCB9Q/p1554477724054200

I have this helmfile.yaml

environments:
  default:
    values:
      - helmfile.values.yaml.gotmpl

releases:
- name: infra
  chart: {{ .Environment.Values.chartName }}

and this simple helmfile.values.yaml.gotmpl

chartVersion: ~0.0.0 #may also be "local"
{{if eq .Environment.Values.chartVersion "local"}}
chartName: ../my-chart
{{else}}
chartName: foo/my-chart
{{end}}

And I get the following errors

>helmfile -f helmfile.yaml apply
could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml: failed to load environment values file "helmfile.values.yaml.gotmpl": failed to render [/Users/sgandon/git-repos/helm-charts/platform-services/helmfile/test/helmfile.values.yaml.gotmpl], because of template: stringTemplate:2:20: executing "stringTemplate" at <.Environment.Values....>: map has no entry for key "chartVersion"
error during helmfile.yaml parsing: template: stringTemplate:10:24: executing "stringTemplate" at <.Environment.Values....>: map has no entry for key "chartName"
mumoshu avatar
mumoshu

How about explicitly forbidding any usage of .Environment.Values in environment values template? By e.g. emitting a template error

SeB avatar

yes great idea.

SeB avatar

and here is my charts-versions.yaml


\##############################

\# Version of the charts to deploy

\# when the values are empty it will use the local chart directly

tpsvc-cryptoVersion: 
...and many more 
SeB avatar

I just wanted to apologize to create so many tickets, but helmfile is a little rough around the edges and we at Talend are going to use it heavily internaly for our development process. When I have finds some time I’d be happy to contribute cause I believe this a great tool.

mumoshu avatar
mumoshu

thx for various feedbacks, and kind words!

im honestly not sure if I can catch up all the issues, but will definitely do my best

SeB avatar

On question though, what is the cost of adding values file that may not exists in terms of speed ? Cause for each of my charts I have the following values file setup using a template

templates:
  sharedvalues: &values
    missingFileHandler: warn # prevent the install to stop if some files are not found
    values:
      - "../values/{{`{{.Release.Name}}`}}.yaml"
      - "../values/{{`{{.Release.Name}}`}}.yaml.gotmpl"
      - "values/{{`{{.Release.Name}}`}}.yaml"
      - "values/{{`{{.Release.Name}}`}}.yaml.gotmpl"
      - "{{ env "TALEND_LOCAL_PATH" }}/platform-services/helmfile/hybrid-remote/values/{{`{{.Release.Name}}`}}.yaml"
      - "{{ env "TALEND_LOCAL_PATH" }}/platform-services/helmfile/hybrid-remote/values/{{`{{.Release.Name}}`}}.yaml.gotmpl"

And most of them are not there but could be. (I have to duplicate yaml and gotmpl extensions cause yaml is much nicer and simpler but it is not interpreted )

mumoshu avatar
mumoshu

not so much i think. it just checks for the existence of the file and logs it, which shouldn’t be time-consuming

mumoshu avatar
mumoshu

also - i have an idea to reduce duplication in your example. that is, allow brace expansion in values entries:

values:
- {../,,{{ env "TALEND_LOCAL_PATH" }}}values/{{`{{.Release.Name}}`}}.yaml{,.gotmpl}
mumoshu avatar
mumoshu

with missingFileHandler: error, this should succeed as long as the pattern matches one or more files

mumoshu avatar
mumoshu

the pattern would be expanded in the order you might want:

$ echo {../,,TALEND_LOCAL_PATH}values/RELEASE_NAME.yaml{,.gotmpl}
../values/RELEASE_NAME.yaml ../values/RELEASE_NAME.yaml.gotmpl values/RELEASE_NAME.yaml values/RELEASE_NAME.yaml.gotmpl TALEND_LOCAL_PATHvalues/RELEASE_NAME.yaml TALEND_LOCAL_PATHvalues/RELEASE_NAME.yaml.gotmpl
mumoshu avatar
mumoshu
brace expansion in values file paths · Issue #545 · roboll/helmfile

For more context, please see the relevant discussion in our official slack channel https://sweetops.slack.com/archives/CE5NGCB9Q/p1554569444071500 This feature would allow you to dramatically reduc…

SeB avatar

excellent !

2019-04-05

mumoshu avatar
mumoshu

didn’t it work? (i suppose it would work

i believe you can even write it shorter

gatewayChart: {{if eq .Environment.Values.gatewayVersion "local" }}../..{{else}}talend{{end}}/tdc-gateway
mumoshu avatar
mumoshu

i’m dreaming of helmfile build, that would render all the helmfile.yaml and values.yaml.gotmpl files into plain YAML files to be commited to a git repo for auditing and gitops

:--1:1
mumoshu avatar
mumoshu
Using helmfiles, but unable to pass Environment down to helmfile.yaml · Issue #388 · roboll/helmfile

When using {{ readFile &quot;environments.yaml }}} — helmfiles: - ./*/helmfile.yaml Environment.Values are not passed to the globbed helmfiles. I would expect that to happen/work, in order to kee…

SeB avatar

@mumoshu no I cannot make it work even with your shorter proposal

SeB avatar

I’ll try to create a simple example to test it.

SeB avatar

I have this helmfile.yaml

environments:
  default:
    values:
      - helmfile.values.yaml.gotmpl

releases:
- name: infra
  chart: {{ .Environment.Values.chartName }}

and this simple helmfile.values.yaml.gotmpl

chartVersion: ~0.0.0 #may also be "local"
{{if eq .Environment.Values.chartVersion "local"}}
chartName: ../my-chart
{{else}}
chartName: foo/my-chart
{{end}}

And I get the following errors

>helmfile -f helmfile.yaml apply
could not deduce `environment:` block, configuring only .Environment.Name. error: failed to read helmfile.yaml: failed to load environment values file "helmfile.values.yaml.gotmpl": failed to render [/Users/sgandon/git-repos/helm-charts/platform-services/helmfile/test/helmfile.values.yaml.gotmpl], because of template: stringTemplate:2:20: executing "stringTemplate" at <.Environment.Values....>: map has no entry for key "chartVersion"
error during helmfile.yaml parsing: template: stringTemplate:10:24: executing "stringTemplate" at <.Environment.Values....>: map has no entry for key "chartName"
SeB avatar

shall I create an issue ?

mumoshu avatar
mumoshu

Ah gotcha. Unfortunately you can’t self-reference environment values from within environment values file template

mumoshu avatar
mumoshu

I’ve added it as requested but I’d admit that its very confusing

mumoshu avatar
mumoshu

maybe we can add something similar to Release Template(see the helmfile best practices guide under docs/) so that it can potentially be written like:

helmfile.values.yaml (not helmfile.values.yaml.gotmpl)

chartVersion: ~0.0.0 #may also be "local"
chartName: "{{if eq .Environment.Values.chartVersion "local"}}..{{else}foo{{end}}/my-chart"
mumoshu avatar
mumoshu

Back to your original work, it is really a text template. helmfile.values.yaml.gotmpl is firsly executed to generate a YAML file. At that time, .Environment.Values.chartVersion is not avaiable yet, as the YAML data isn’t rendered hence not loaded yet

mumoshu avatar
mumoshu

It’s hard(imho) and that’s why I’ve been saying I’d recommend using jsonnet to generate necessary files before running helmfile, or give up making it DRY too much.

But I had not received much traction about it so far!

SeB avatar

I am fine with this not working I just saw some examples in some github issues.

SeB avatar

Actually I do use the templates feature and it works fine.

SeB avatar

I do this

templates:
  chart: &chart
    {{ $versionKey := printf "%sVersion" `{{.Release.Name}}` }}
    chart: {{ .Environment.Values | get ($versionKey) "" | eq "" | ternary "../.." "talend" }}/{{`{{.Release.Name}}`}}
mumoshu avatar
mumoshu

Great!!

mumoshu avatar
mumoshu

Oh so are you saying you managed to use templates in an environment values file?

2019-04-04

SeB avatar

hello, Is it possible from the same values.yaml.gotmpl file used for loading some environment values to have template check for values in this very same file? here is an example

gatewayVersion: ~0.0.0 #may also be "local"
{{if eq .Environment.Values.gatewayVersion "local"}}
gatewayChart: ../../tdc-gateway
{{else}}
gatewayChart: talend/tdc-gateway
{{end}}
SeB avatar

here you can see I want to templatize the release charts name according to the version value to use a “locat” chart or the one from our registry with a given version.

Erik Osterman avatar
Erik Osterman

Yea makes sense

Erik Osterman avatar
Erik Osterman

I am not sure off the top of my head but seems reasonable.

Erik Osterman avatar
Erik Osterman

@Shane do you know? :point_up_2:

Shane avatar
Shane

Off the top of my head I’m not sure, but I think the environment is on the second render pass

2019-04-03

mumoshu avatar
mumoshu

im not very sure but perhaps thats just coincidence

mumoshu avatar
mumoshu

i do have suggested helmfile integration to skaffold before https://github.com/GoogleContainerTools/skaffold/issues/169

Suggestion: Helmfile integration · Issue #169 · GoogleContainerTools/skaffold

Hi, thanks for sharing the great project! Would you mind if I contribute a Helmfile integration as a deployment plugin to skaffold? Helmfile is basically a declarative spec for deploying a set of h…

mumoshu avatar
mumoshu

but the state of skaffold’s multi helm releases support isn’t something I’d suggested

mumoshu avatar
mumoshu

like it misses templating

SeB avatar

Hello guys, I am gad a new PR pointed me this slack repo, it is much easier to chat than creating github issues. I’d like to contribute when I find some time. I did try on the helmfile destroy but @mumoshu was faster than me.

SeB avatar

thanks a lot for your effort @mumoshu

:1000:1
Erik Osterman avatar
Erik Osterman

Good to have you here @SeB

Erik Osterman avatar
Erik Osterman

Yes, @mumoshu is absolutely remarkable. I don’t know how he can implement so much so quickly.

fast_parrot1
SeB avatar

I agree, this is impressive.

Erik Osterman avatar
Erik Osterman

Have you checked out some of his other projects?

Erik Osterman avatar
Erik Osterman

Some notable ones are #variant and kubeaws

1
SeB avatar

I’ll have look thanks.

2019-04-02

mumoshu avatar
mumoshu

@Erik Osterman managed to add helmfile destroy. also, both delete and destroy now tries to delete releases with installed: true only, as we’ve discussed https://github.com/roboll/helmfile/pull/530

feat: `helmfile destroy` deletes and purges releases by mumoshu · Pull Request #530 · roboll/helmfile

This adds helmfile destroy that is basically helmfile delete –purge. I&#39;ve also tweaked the behavior of delete and destroy for releases with installed: false, so that it becomes consistent with…

1
Erik Osterman avatar
Erik Osterman

Thanks @mumoshu !

Pablo Costa avatar
Pablo Costa

Hello, I was looking at some skaffold examples and I realized that it has similarities with helmfile sintaxe when deploying helm charts. Does anybody knows if this is just coincidence ?

Pablo Costa avatar
Pablo Costa
GoogleContainerTools/skaffold

Easy and Repeatable Kubernetes Development. Contribute to GoogleContainerTools/skaffold development by creating an account on GitHub.

    keyboard_arrow_up