#helmfile (2020-04)

https://github.com/roboll/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles Archive: https://archive.sweetops.com/helmfile/

2020-04-29

Alucas avatar
Alucas

Hmm, I’m assuming it’s not possible to append to a nested value that’s defaulted in a chart or another values file?

2020-04-28

2020-04-27

s_slack avatar
s_slack

Just ran into this issue https://github.com/roboll/helmfile/issues/760 … hopefully it will get implemented soon

Unable to use .Release.Name in values.gotmpl · Issue #760 · roboll/helmfile

I have a use case where I want the .Release.Name to be used as a variable in a values.gotmpl file. However, no matter what I do, the values.gotmpl file will not render .Release.Name. Is this expect…

mumoshu avatar
mumoshu

@ I’d apreciate it if you could submit a pr for that

I already provided a brief guidance to implement it. It should be a good-first-issue.

I don’t need this specific feature myself. But I’m eager to review/merge it if someone could contribute!

Unable to use .Release.Name in values.gotmpl · Issue #760 · roboll/helmfile

I have a use case where I want the .Release.Name to be used as a variable in a values.gotmpl file. However, no matter what I do, the values.gotmpl file will not render .Release.Name. Is this expect…

2020-04-22

yuri avatar

hey guys, is anyone used DAG aware install with multiple helmfiles? my helmfile:

helmfiles:
 - releases/docker-creds.yaml
 - releases/core-a.yaml
 - releases/core-b.yaml

how do i reference core-a to be depended on docker-creds release, referring by name does not work

yuri avatar
`needs:` Can't check for a release to be existent in another Helmfile · Issue #1181 · roboll/helmfile

I created a pod monitor that needs monitoring/prometheus-operator to be existant. However Helmfile tells that the release doesn't exist: in ./helmfile.yaml: "kube-system/prometheus-operato…

mumoshu avatar
mumoshu

Yes subhelmfiles doesn’t support DAG

mumoshu avatar
mumoshu

But why you need that?

Sub-helmfiles basically means that everything contained in the parent helmfile is dependent on sub-helmfiles

mumoshu avatar
mumoshu

which is basically an another basic form of DAG

yuri avatar

thanks, so i assume that the order of sub helmfiles does install them by this order?

mumoshu avatar
mumoshu

yes

yuri avatar

cool, thank u for the quick response!

mumoshu avatar
mumoshu

glad to help!

2020-04-21

rms1000watt avatar
rms1000watt

Is there a way to add files on the helmfile side, that are accessible from the helm side? https://github.com/helm/charts/blob/master/stable/locust/templates/worker-cm.yaml#L12

Like how can I add more files to tasks/*?

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think you want to flip this around.

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Just use the raw chart and use Helmfile to provision:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ...
data:
...
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Where do these files come from?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Do you want helmfile to stage these files in the directory for you? e.g. using hooks?

mumoshu avatar
mumoshu

As long as the chart uses .Files.Glob to grab the files, it’s impossible.

.Files.Glob by definition has access to files contained in the chart only

Helmfile by its nature has no ability to modify charts on the fly

mumoshu avatar
mumoshu

Perhaps the intended way for the chart author would be to fork the chart.

Then from the helmside you can point to the locally forked chart with chart: ./path/to/forked/chart for your use

rms1000watt avatar
rms1000watt

gotcha

rms1000watt avatar
rms1000watt

that makes sense

:--1:1
rms1000watt avatar
rms1000watt

since it’s impossible, as @mumoshu mentioned, I’m taking @Erik Osterman (Cloud Posse) approach to create the configmap and override

:--1:1
rms1000watt avatar
rms1000watt

my next battle is using exec with a git.sh script

rms1000watt avatar
rms1000watt

\#!/usr/bin/env bash

\# git.sh

set -e

outfile="../helmfile.log"

{
if [[ -z ${1} ]]; then
  echo "ERROR: no argument is provided"
  exit 1
fi

split=$(echo "${1//\/\//}")
git_full=$(echo "${split}" | cut -d'+' -f1)
repo=$(echo "${git_full}" | sed 's:.*/::' | cut -d'.' -f1)
file=$(echo "${split}" | cut -d'+' -f2)

echo "split: ${split}" >> "${outfile}"
echo "git_full: ${git_full}" >> "${outfile}"
echo "repo: ${repo}" >> "${outfile}"
echo "file: ${file}" >> "${outfile}"

echo git clone "${git_full}" "/tmp/${repo}" >> "${outfile}"
echo cat "/tmp/${repo}/${file}" >> "${outfile}"

git clone "${git_full}" "/tmp/${repo}" 1>&2
cat "/tmp/${repo}/${file}" >&1

\# head -999999999 "/tmp/${repo}/${file}"

} | tee -a "${outfile}"
rms1000watt avatar
rms1000watt
{{ exec "../git.sh" (list "[email protected]:calm/REDACTED.git//path/to/file.py") | indent 8 }}
rms1000watt avatar
rms1000watt

the cat at the end doesn’t get slurped into helmfile. but I think this is my problem to solve. don’t want to distract you guys with this

mumoshu avatar
mumoshu

Would you mind giving me a context in the values.yaml.gotmpl/helmfile.yaml around the exec line?

mumoshu avatar
mumoshu

Several lines around the exec would be enough

rms1000watt avatar
rms1000watt

yea

rms1000watt avatar
rms1000watt
- name: locust-cm-{{ .Namespace }}
  namespace: {{ .Namespace }}
  chart: calm/monochart
  version: !!string 0.0.23
  values:
  - name: locust-cm-{{ .Namespace }}
  - configmap:
      enabled: true
      file: tasks.py
      file_content: |
        {{ exec "../git.sh" (list "[email protected]:calm/locust.git//locustfile.py") | indent 8 }}
rms1000watt avatar
rms1000watt

the monochart concept is within calm lol

:--1:1
:100:1
mumoshu avatar
mumoshu

It shousld be nindent 8

mumoshu avatar
mumoshu

Not sure it’s the source of the whole problem, though

rms1000watt avatar
rms1000watt

oh, gotcha

rms1000watt avatar
rms1000watt

lemme mess with it more, it’s surely on my side

mumoshu avatar
mumoshu

Does the script alone work on your machine?

For me, it’s like

$ ./git.sh [email protected]:calm/locust.git//locustfile.py
Cloning into '/tmp/locust'...
fatal: remote error:
   is not a valid repository name
  Email [[email protected]](mailto:[email protected]) for help
rms1000watt avatar
rms1000watt

yeah, it’s private repo

rms1000watt avatar
rms1000watt

i found the problem

rms1000watt avatar
rms1000watt
set -e

git clone "${git_full}" "/tmp/${repo}" 1>&2
rms1000watt avatar
rms1000watt

the repo was already there, so it exit > 0

rms1000watt avatar
rms1000watt

so just needed

||: 

at the end of it

git clone "${git_full}" "/tmp/${repo}" 1>&2 ||:
:--1:1
rms1000watt avatar
rms1000watt

i need to harden it

rms1000watt avatar
rms1000watt

but at least it’s working as expected for the happy path

mumoshu avatar
mumoshu

Even after adding ||: it’s not working for my own repo:

$ ./git.sh [email protected]:mumoshu/variant//README.md
Cloning into '/tmp/variant\'...
fatal: remote error:
   is not a valid repository name
  Email [[email protected]](mailto:[email protected]) for help
cat: /tmp/variant\/README.md: No such file or directory

Maybe an escaping issue? But never mind if it’s working fine for you

rms1000watt avatar
rms1000watt

oh, it’s because it’s missing .git

rms1000watt avatar
rms1000watt
./git.sh [email protected]:mumoshu/variant.git//README.md
rms1000watt avatar
rms1000watt

yeah, that command worked locally for me

rms1000watt avatar
rms1000watt

for your repo’s README.md

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

What does ||: do? First time to see that notation.

mumoshu avatar
mumoshu

I was guessing if it’s || : i.e. || true?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

aha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

like :>file truncates a file. that : is funky

rms1000watt avatar
rms1000watt

heh, yea https://stackoverflow.com/a/3224910

Also it’s usually used as the function name in forkbombs

What is the purpose of the : (colon) GNU Bash builtin?

What is the purpose of a command that does nothing, being little more than a comment leader, but is actually a shell builtin in and of itself? It’s slower than inserting a comment into your script…

1
1
rms1000watt avatar
rms1000watt

:(){ :|: & };:

1
fast_parrot1
rms1000watt avatar
rms1000watt
Fork() Bomb - GeeksforGeeks

A Computer Science portal for geeks. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Looks like a hipster emoji to me.

oza_shu avatar
oza_shu

Hi. Could you someone help me? A problem I’ve been struggling with a problem for a while. I’m trying to install datadog with helmfile, I can decrypt api_key encrypted by sops. But I can’t pass it to template file. I wrote a VALUES file using this document as a reference, do you know what’s wrong with it? https://github.com/roboll/helmfile#environment-secrets

err: release "datadog" in "helmfile.yaml" failed: helm exited with status 1:
 Error: Failed to render chart: exit status 1: Error: failed to parse /tmp/values104369418: error converting YAML to JSON: yaml: line 2: did not find expected key

I can’t get a value with {{ .Values.apikey }}, is there something wrong with the way I wrote it? And I try {{ .Enviroment.Values.apikey }} in dev values yaml , but can’t deploy it.

bash-4.2# egrep apikey -n -B1 -A1 helm/datadog/datadog/values/dev.yaml
datadog:
 apiKey: {{ .Values.apikey }}
oza_shu avatar
oza_shu

The decrypted file is specified as follows.

helm/datadog/datadog/helmfile.yaml
environments:
 dev:
  secrets:
  - environment/dev/secrets.encrypted.yaml

I’m encrypting the API key with SOPS.

bash-4.2# head helm/datadog/datadog/environment/dev/secrets.encrypted.yaml
apikey: ENC[AES256GCM,data:hogehogehogehogehoge,type:str]
oza_shu avatar
oza_shu

I use helm3 & helmfile v0.108.0

bash-4.2# helm version
version.BuildInfo{Version:"v3.2.0-rc.1", GitCommit:"7bffac813db894e06d17bac91d14ea819b5c2310", GitTreeState:"clean", GoVersion:"go1.13.10"}
bash-4.2# helmfile --version
helmfile version v0.108.0
oza_shu avatar
oza_shu

I think this part is bad because if you give a raw API key, it will be deployed.

mumoshu avatar
mumoshu

@ Probably you’re confusing and mixing envrionment secrets with release secrets

mumoshu avatar
mumoshu

What you can refer from the chart template(any files contained in the template directory of your chart) must be release values and secrets.

mumoshu avatar
mumoshu

So give dev.yaml a place under the yaml array at releases[].secrets

mumoshu avatar
mumoshu

not environments.ENV_NAME.secrets.

mumoshu avatar
mumoshu

the latter is for templating the whole helmfile.yaml

mumoshu avatar
mumoshu

the former is for helm

oza_shu avatar
oza_shu

Thank you for your reply! I can pass api_key to template file! Thanks!

1

2020-04-19

Meerkat avatar
Meerkat

Anyone know how to deploy a helm chart that has a requirements.yaml for the dep charts? My goal is to deploy DAPR (https://github.com/dapr/dapr/tree/master/charts/dapr) using Tilt + Helmfile, but that isn’t working out for me. It will deploy the main components, but it doesn’t load in the requirements.yaml and load up the RBAC charts.

dapr/dapr

Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge. - dapr/dapr

mumoshu avatar
mumoshu

@ Could you try running helmfile template against your example to see it does include the RBAC resources from the dep charts?

It should just work. Helmfile fully delegates the install/update/deletion of the chart and its dependencies to Helm. So if Helm could handle the installation correctly, so should Helmfile.

dapr/dapr

Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge. - dapr/dapr

mumoshu avatar
mumoshu

Running helm template against the chart does produces RBAC-related resources.

$ helm3 template dapr dapr/dapr --namespace dapr-system | grep kind
kind: ServiceAccount
kind: Secret
kind: Secret
kind: ClusterRole
kind: ClusterRoleBinding
- kind: ServiceAccount
  kind: ClusterRole
kind: ClusterRoleBinding
- kind: ServiceAccount
  kind: ClusterRole
kind: Service
kind: Service
kind: Service
kind: Service
kind: Deployment
kind: Deployment
kind: Deployment
kind: Deployment
kind: Configuration
kind: MutatingWebhookConfiguration

helmfile template should also produce the exact result.

Meerkat avatar
Meerkat

When I do a helm template I can see the RBAC objects, but it doesn’t install with the rest. When I do a helm install it shows up just fine.

mumoshu avatar
mumoshu


but it doesn’t install with the rest.
What’s actually missing?

I see several services and deployments, clusterroles etc when i run helm template. I can see they do exist in the cluster after helm install

mumoshu avatar
mumoshu

So one of cluster roles managed by the chart includes secret-reader, which does exist in my cluster after installation

$ k get clusterrole | grep secret
secret-reader                                                          57s
mumoshu avatar
mumoshu

Perhaps I need to ask about how you confirmed “it doesn’t install with the rest”

mumoshu avatar
mumoshu

So that I can better understand your issue

Meerkat avatar
Meerkat

I’ll replicate my problem again here in a bit and give you a better description.

:--1:1

2020-04-18

2020-04-17

2020-04-16

Vadim Bauer avatar
Vadim Bauer

I want to create with incubator/raw a ConfigMap for grafana dashabord example. https://github.com/xiaoping378/k8s-monitor/blob/master/manifests/grafana-import-dashboards-configmap.yaml

However the dashboard .json content contains mustache syntax {{short_version}}" Now Helmfile is trying to interpret the content so I am not sure how to accomplish to import something as plain text. https://grafana.com/api/dashboards/9628/revisions/4/download

bradym avatar
bradym

I ran into that with rules for prometheus. You can escape it like this:

{{` {{short_version}} `}}
Andrey Nazarov avatar
Andrey Nazarov

Yes, we’ve fixed it basically the same way, but escaped the whole json

data:
  javamelody-dashboard.json: |-
    {{`{
    ....
    }`}}
1
bradym avatar
bradym

Good point. I’ve used both approaches.

Vadim Bauer avatar
Vadim Bauer

found an easier way:

- kind: ConfigMap
  apiVersion: v1
  metadata:
    name: postgres-grafana-dashboard
    labels:
      grafana_dashboard: '1'
    annotations:
      dashboardSource: "<https://grafana.com/api/dashboards/11376/revisions/2/download>"
  data:
    postgres-grafana-dashboard-11376.json: {{ readFile "postgres-grafana-dashboard-11376.json" | toJson }}

readfile and then toJson. toRawJson would be better, but helmfile doesn’t use an up to date sprig version

2020-04-15

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yep, we use this plugin all the time.

Vadim Bauer avatar
Vadim Bauer

I am trying to structure my helmfile according to https://github.com/roboll/helmfile/blob/master/docs/writing-helmfile.md#layering-state-files. However it doesn’t work

>cat base.yaml
environments:
  stage:
  prod:

helmDefaults:
  wait: true
  verify: true

>cat helmfile.yaml

bases:
- base.yaml


repositories:
 ...
 redacted

releases:
  - name: letsencrypt-cert-issuer
    namespace: cert-manager
    chart: incubator/raw
    values:
       - {{ .Environment.Name }}-values.yaml
       - email: {{ .Values.acme_email }}

The values form {{ .Environment.Name }}-values.yaml aren’t processed.

in stage-values.yaml I have a simple entry:

acme_email: "<mailto:[email protected]|[email protected]>"

The message is: “stringTemplate” at <.Values.acme_email>: map has no entry for key “acme_email”

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

environments did not work with bases for us

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ended up not using bases

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

maybe there is a solution for that (did not spend much time investigating)

mumoshu avatar
mumoshu

I think the doc is correct. Perhaps you’re missing --- between bases and other parts of helmfile.yaml?

mumoshu avatar
mumoshu

@Andriy Knysh (Cloud Posse) Did you use --- as documented when you tried to make bases + environment values work?

2020-04-14

Balazs Varga avatar
Balazs Varga

hello all. have you ever seen that helm /helmfile cannot delete all resource during destroy ?

Balazs Varga avatar
Balazs Varga

As I see it cannot deletes clusterrole and customresources sometimes

Andrey Nazarov avatar
Andrey Nazarov

If they are created via hooks or something it will be like that.

Also I’ve noticed leftovers after failed deletion (permission issues or other troubles). Thus a Helm release was removed, but objects were still in the cluster.

Paul Catinean avatar
Paul Catinean

Same here, especially if the release is not fully deployed before you call destroy

Balazs Varga avatar
Balazs Varga

thanks, will check them

Gourav avatar
Gourav

Hi All.. I have created a project using helmfile. Is there anyway to generate the application yaml files rather than directly applying ?I tried using helmfile template --file <<filename> --output-dir /tmp but some how not getting the mainfest files in yaml format.

Could someone please guide me on the same?

Gourav avatar
Gourav

Got it.. i think need to use template at the end.

Nelson Jeppesen avatar
Nelson Jeppesen

Heavy user of helmfile here. Question, are there any options to patch a helm chart? I’d rather not fork it. It seems like helm-x could be used for this but it’s not 100% clear to me from the docs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Only on a case-by-case basis.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. some let you disable ingress then we use the raw chart to define what we need.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but there’s no generic way that works for all situations.

Nelson Jeppesen avatar
Nelson Jeppesen

Damn Well thanks for letting me know!

Zachary Loeber avatar
Zachary Loeber

maybe, I always cloned/forked when I got to the point where it required modifications beyond what the original chart offered. Using the git plugin (https://github.com/aslafy-z/helm-git.git) means you don’t even have to technically build/publish the chart afterwards either

aslafy-z/helm-git

Helm plugin to fetch charts from Git repositories. Contribute to aslafy-z/helm-git development by creating an account on GitHub.

roth.andy avatar
roth.andy

This plugin has been a lifesaver.

aslafy-z/helm-git

Helm plugin to fetch charts from Git repositories. Contribute to aslafy-z/helm-git development by creating an account on GitHub.

:--1:3

2020-04-13

Paul Catinean avatar
Paul Catinean

Hi guys, I’m trying to setup a scenario where I can bootstrap multiple clusters with cert-manager which I don’t pass custom variables much

Paul Catinean avatar
Paul Catinean

The plan is to give a list of cluster contexts and have helmfile install/upgrade them on the respective clusters and also prep the cluster by adding the CRD’s

Paul Catinean avatar
Paul Catinean

To do this efficiently I would have to first check the kubernetes version and apply the standard or legacy crd’s depending on that

Paul Catinean avatar
Paul Catinean

Now I have a python script that does this check but I need to execute it in the hooks for each cluster separately

Paul Catinean avatar
Paul Catinean

My questions would be 1) How do i execute a script that’s relative to the helmfile (without absolute path) and 2) Do I have to manually switch the context with the script in order to communicate with the proper cluster?

Paul Catinean avatar
Paul Catinean

A similar requirement is on this issue I see: https://github.com/roboll/helmfile/issues/891

Helm3 doesn't automatically create namespace · Issue #891 · roboll/helmfile

Helm3 doesn&#39;t automatically create namespace - see https://v3.helm.sh/docs/faq/#automatically-creating-namespaces How can we solve this with helmfile, so that we don&#39;t have to manually crea…

2020-04-12

Andrey Nazarov avatar
Andrey Nazarov

I’m facing a stange issue using terraform-provider-helmfile. It seems that terraform plan triggers helmfile diff against the version stored in the tf state. Basically, we have a bunch of helmfiles refferencing a global helmfile like the following:


\#my_env_helmfile.yaml
helmDefaults:
  verify: false
  wait: false
  timeout: 600
  recreatePods: false
  force: false

environments:
  my_env:
    values:
      - environments/dev/my_env.yaml

helmfiles:
  - path: git::<https://my_user>:{{ requiredEnv "REPO_TOKEN" [}}@gitlab.my>_<http://domain.com/my_project/[email protected]/helmfile.yaml?ref={{|domain.com/my_project/[email protected]/helmfile.yaml?ref={{](mailto:}}@gitlab.my|}}@gitlab.my>_<http://domain.com/my_project/[email protected]/helmfile.yaml?ref={{) env  "INFRA_VERSION" }}
    values:
      - environments/dev/my_env.yaml

and in tf file we set INFRA_VERSION:

...
  environment_variables = {
    INFRA_VERSION = var.infra_version
    EXTERNAL_IP = data.google_compute_address.dev_ip_address.address
  }
...

We split up some release into many in a global helmfile(we made two out of one and renamed both) and start facing the issue with existed resources. But the cause is that terraform plan is run against the old INFRA_VERION . It’s obvious from logs. But as obvious as this that a new version of INFRA_VERSION is passed to the tf command. Say, here is the arguments to cli

2020/04/12 17:37:17 [INFO] CLI args: []string{"/bin/terraform", "plan", "-out", "planfile", "-var", "environment=my_env", "-var", "helmfile_path=my_env_helmfile.yaml", "-var", "infra_version=v3.42"}

see v3.42 and here is the diff output

in ./my_env_helmfile.yaml: in .helmfiles[0]: in /builds/my_project/my_repo_2/.helmfile/cache/https_gitlab_my_domain_com_my_repo_git.ref=v3.37/deployment/helmfile.yaml: 2 errors:

it grabs v3.37 Version 3.37 could be taken only from the tf state file. I can’t see any other option right now.

mumoshu avatar
mumoshu

@Andrey Nazarov Very odd. I take this as terraform has saved the old INFRA_VERSION value into the tfstate and used it to run terraform plan with it, even though var.infra_version has updated. Am I following you correctly?

mumoshu avatar
mumoshu

Could there be any bug in Terraform around dependency resolution among vars, envvars, etc?

If so, would it work if you avoid envvars as the way for communication between tf and helmfile?

mumoshu avatar
mumoshu

i mean, try using:

helmfiles:
  - path: git::<https://my_user>:{{ requiredEnv "REPO_TOKEN" [}}@gitlab.my>_<http://domain.com/my_project/[email protected]/helmfile.yaml?ref={{|domain.com/my_project/[email protected]/helmfile.yaml?ref={{](mailto:}}@gitlab.my|}}@gitlab.my>_<http://domain.com/my_project/[email protected]/helmfile.yaml?ref={{) .Values.infraVersion }}
mumoshu avatar
mumoshu

and

resource "helmfile_release_set" "yourapp" {
  values = {
    infra_version = "v3.42"
  }
}

or

resource "helmfile_release_set" "yourapp" {
   values = {
    infra_version = var.infra_version
  }
}
Andrey Nazarov avatar
Andrey Nazarov


Am I following you correctly?
Yes, that how it looks like.
resource “helmfile_release_set” “yourapp” {
values = {
infra_version = “v3.42”
}
}
I always get executing "stringTemplate" at <.Values.infraVersion>: map has no entry for key "infraVersion" no matter of how I pass variables to tf. I tried

resource "helmfile_release_set" "yourapp" {
  values = {
    infra_version = "v3.42"
  }
}

and

resource "helmfile_release_set" "yourapp" {
  values = {
    infraVersion = "v3.42"
  }
}

and

values_files = [
		<<EOF
{"infraVersion": "v3.37"}
EOF
  ]
Andrey Nazarov avatar
Andrey Nazarov

I’m using

terraform:0.12.24
helm:v3.1.2
helmfile:v0.106.3
helmdiff:v3.1.1
terraform-provider-helmfile:built from master
Andrey Nazarov avatar
Andrey Nazarov

Hm, it might be that my helmfile provider version is not that fresh. I’m rebuilding it.

mumoshu avatar
mumoshu

ah sry! it must be json under values, e.g.

	values = [
		<<EOF
{"infraVersion": "v3.37"}
EOF
	]
Andrey Nazarov avatar
Andrey Nazarov

Still executing "stringTemplate" at <.Values.infraVersion>: map has no entry for key "infraVersion" . Actually previous errors of that sort were due to the old version of terraform-provider-helmile. Now it’s rebuilt from sratch from the master branch.

Andrey Nazarov avatar
Andrey Nazarov

I’m wondering if helmfile diff is somehow triggered against resources stored in the tf state files first. This way it’s natural that I have this error, because this value is not in the state yet. And it corresponds well to the initial error where probably the old environment variable was used (taken from the state). And then it does the same for local resources in tf files. I don’t know the logic behind the code (how tf plan works under the hood), but it feels like something strange is happening here.

Andrey Nazarov avatar
Andrey Nazarov

Say, a config

resource "helmfile_release_set" "common_stack" {
  path = "custom_helmfile.yaml"

  working_directory = path.module
...

ends up with

specified state file helmfile.yaml is not found

but it works for the name hamefile.yaml

Ok, it seems one cannot just change the path or name for helmfile, as tf wants to make a diff using the old file as well:). So, at this time one need to have 2 helmfile files presented in the filesystem. Not that obvious at first)

Andrey Nazarov avatar
Andrey Nazarov

Should I file an issue on GH?

Andrey Nazarov avatar
Andrey Nazarov

Regarding our problem, we’ve decided to bypass TF and set environment variable outside of it. It seems it was overengineering initially.

mumoshu avatar
mumoshu

Ahh that makes sense. Terraform internally call underlying provider’s get operation, and seems to compare the results of gets for the old and the new desired state.

mumoshu avatar
mumoshu

terraform-helmfile-provider runs helmfile diff to respond get so yeah, it should require both the old and the new helmfile.yaml to work.

mumoshu avatar
mumoshu


Should I file an issue on GH?
Yes! That definitely helps

2020-04-10

DanP avatar

hello

DanP avatar

I’m trying to make the following scenario work. I have a helpers directory that contains a template file add_http_repos.tpl. When I try to import te template in my values.yaml.gotmpl file as follows:

{{ readFile "helpers/add_http_repos.tpl" | tpl . }}
DanP avatar

When I run helmfile I receive executing "stringTemplate" at <.>: wrong type for value; expected string; got state.EnvironmentTemplateData

Vadim Bauer avatar
Vadim Bauer

this is a gotemplate issue

Vadim Bauer avatar
Vadim Bauer

a string is expected but state.EnvironmentTemplateData is set

DanP avatar

ok, and how can I fix this?

DanP avatar

the error suggests the input of the tpl function which si set to . is not corrent. somehow it expects a string but don’t know why

Jim avatar

I’ve been using syntax like the following if that helps?

{{ tpl (readFile "helpers/add_http_repos.tpl") . }}
DanP avatar

Do you know how can I fix this?

2020-04-08

Vadim Bauer avatar
Vadim Bauer

I have a question, the sample helmfile reference on https://github.com/roboll/helmfile#configuration do the values for the keys represent defaults?

or where can I find all the defaults values? if they aren’t mentioned in the example? eg:

 verify: true
    # wait for k8s resources via --wait. Defaults to `false`
    wait: true
    # time in seconds to wait for any individual Kubernetes operation (like Jobs for hooks, and waits on pod/pvc/svc/deployment readiness) (default 300)
    timeout: 60
    # performs pods restart for the resource if applicable
    recreatePods: true
    # forces resource update through delete/recreate if needed
    force: true
    # set `false` to uninstall on sync
    installed: true
    # restores previous state in case of failed release
    atomic: true
    # when true, cleans up any new resources created during a failed release
    cleanupOnFail: false

its kind of confusing especially with the comments in

helmDefaults:

\# defaults for verify, wait, force, timeout and recreatePods under releases[]
  verify: true
  wait: true
  timeout: 600
  recreatePods: true
  force: true

@mumoshu do you have a reference?

mumoshu avatar
mumoshu

well sorry i dont fully understand your question.

yes, its basically default values for each entry under releases[]

Vadim Bauer avatar
Vadim Bauer

ok, the question is what are the defaults for each section?

for

helmDefaults

and for

releases
mumoshu avatar
mumoshu

ah okay i believe we don’t have a refence for that

Vadim Bauer avatar
Vadim Bauer

because I have the feeling the eg.

releases:
 atomic: true #default is actually false
mumoshu avatar
mumoshu

yeah, it is atomic: false by default

mumoshu avatar
mumoshu

it mostly correspond to helm flags with the similar names

Vadim Bauer avatar
Vadim Bauer

can I make a pull request and start adding defaults? because I was under the impression that the values are all defaults. especially for ( all booleans)

mumoshu avatar
mumoshu

so the default would be no flags passed to helm

Vadim Bauer avatar
Vadim Bauer

ok, so the reference here would be Helm and not Helmfile right?

mumoshu avatar
mumoshu

which means

helmDefaults:
  #tillerNamespace:  
  tillerless: false
  kubeContext: ""
  cleanupOnFail: false
  args:
  verify: false
  wait: false
  timeout: 300
  recreatePods: false
  force: false
  tls: false
  #tlsCACert: "$HELM_HOME/ca.pem"
  #tlsCert: "$HELM_HOME/cert.pem"
  #tlsKey: "$HELM_HOME/key.pem"
  historyMax: 10
mumoshu avatar
mumoshu


ok, so the reference here would be Helm and not Helmfile right?
correct. and translating helm defaults would produce something like https://sweetops.slack.com/archives/CE5NGCB9Q/p1586342938045600?thread_ts=1586342576.043500&cid=CE5NGCB9Q

which means

helmDefaults:
  tillerNamespace: tiller-namespace  #dedicated default key for tiller-namespace
  tillerless: false                  #dedicated default key for tillerless
  kubeContext: kube-context          #dedicated default key for kube-context (--kube-context)
  cleanupOnFail: false               #dedicated default key for helm flag --cleanup-on-fail
  # additional and global args passed to helm
  args:
    - "--set k=v"
  verify: false
  wait: false
  timeout: 300
  recreatePods: false
  force: true
  # enable TLS for request to Tiller
  tls: true
  # path to TLS CA certificate file (default "$HELM_HOME/ca.pem")
  tlsCACert: "path/to/ca.pem"
  # path to TLS certificate file (default "$HELM_HOME/cert.pem")
  tlsCert: "path/to/cert.pem"
  # path to TLS key file (default "$HELM_HOME/key.pem")
  tlsKey: ""
  # limit the maximum number of revisions saved per release. Use 0 for no limit (default 10)
  historyMax: 10
Vadim Bauer avatar
Vadim Bauer

could we update the docs with that values

mumoshu avatar
mumoshu

how would you update it? just replacing the whole helmDefaults in the example?

Vadim Bauer avatar
Vadim Bauer

it would make life easyier fo so many just by looking at your example. i take the example as a reference

mumoshu avatar
mumoshu

okay that makes sense

mumoshu avatar
mumoshu

but would you also leave the information about how you could customize it

Vadim Bauer avatar
Vadim Bauer

yes, or making comments eg-

 historyMax: 1 # dafaults 10

mumoshu avatar
mumoshu

like


\# When set to true, enable TLS for request to Tiller
  tls: false
mumoshu avatar
mumoshu

yeah that would work too

Vadim Bauer avatar
Vadim Bauer

for me and my team mates this file is the reference, i usually only look at this one: If I need more I look at the description If I am still clueless I look at helm.sh docs

mumoshu avatar
mumoshu

if you prefer the uncommented line to denote a default, im fine with

```

# Number of hostorical releases to be retained in the cluster. Defaults to 10. Set it to e.g. 1 if you want only 1 release to be retained historyMax: 10

Vadim Bauer avatar
Vadim Bauer

in this order, that why i call it the reference

Vadim Bauer avatar
Vadim Bauer

this is also fine

mumoshu avatar
mumoshu

got it. that makes sense

mumoshu avatar
mumoshu

would you mind giving it a shot?

Vadim Bauer avatar
Vadim Bauer

ok,

Vadim Bauer avatar
Vadim Bauer

i’ll make a PR wait

mumoshu avatar
mumoshu

i mean would you mind submitting a pr for that as the foundation

mumoshu avatar
mumoshu

thanks a lot!

Vadim Bauer avatar
Vadim Bauer
update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

Vadim Bauer avatar
Vadim Bauer

There are some inconsistencies in the docs. Some defaults are written on separate line some other behind the values. I added defaults as a comment behind the values.

However I am open to make adaptions.

An alternative might be.

  1. if a comment above KV exist add the default in there.
  2. If no comment exist append default in the same line as the value
Vadim Bauer avatar
Vadim Bauer

What do you think @mumoshu? or is it ok as is?

mumoshu avatar
mumoshu

I prefer the option 1 and making it consistent in all places(incl. helmDefaults and releases[]) if possible

mumoshu avatar
mumoshu

thanks anyway! just reviewed it

Vadim Bauer avatar
Vadim Bauer

@mumoshu so only 1. not the combination of 1+2?

mumoshu avatar
mumoshu

yes that’s my intention.

just curious, but do you have any motivation to mix 1 and 2? (i had no intention to do that myself - i just didn’t read that part of readme that much these days

Vadim Bauer avatar
Vadim Bauer


2. If no comment exist append default in the same line as the value
I would put the comments and explanation into helmDefaults and keep the release clean as much as possible

Vadim Bauer avatar
Vadim Bauer

also the defaults in releasedepend on helmDefaults if you dont set wait it will fallback to helmDefault

adding a default in there might be not correct as it depends on helmDefaults

mumoshu avatar
mumoshu

agreed. so how about only doing the option 1 for helmDefaults and removing comments for defaults under releases[]?

:--1:1
mumoshu avatar
mumoshu

I mean comments in this part of the example does look like confusing:

https://github.com/roboll/helmfile/pull/1189/files#diff-04c6e90faac2675aa89e2176d2eec7d8R156-R160

update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

mumoshu avatar
mumoshu
update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

mumoshu avatar
mumoshu
update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

mumoshu avatar
mumoshu

https://github.com/roboll/helmfile/pull/1189/files#diff-04c6e90faac2675aa89e2176d2eec7d8L151 should be just wait for k8s resources via --wait..

Defaults to false` in the end of the line looks confusing

update docs by Vad1mo · Pull Request #1189 · roboll/helmfile

Added default comments into helmfile.yaml example fixed some typos Signed-off-by: Vadim Bauer [email protected]

Vadim Bauer avatar
Vadim Bauer

ok.

Vadim Bauer avatar
Vadim Bauer

I guess I understand. let me give it another try

mumoshu avatar
mumoshu

thank you so much!

Vadim Bauer avatar
Vadim Bauer

regarding https://github.com/roboll/helmfile/pull/1189/files#diff-04c6e90faac2675aa89e2176d2eec7d8L149 I would prefer to have the comment in helmDefault section, as the values also references to helmDefault. This way there is we can keep it a bit DRY

Vadim Bauer avatar
Vadim Bauer

@mumoshu what do you think now?

mumoshu avatar
mumoshu

the reference is the readme section you linked

2020-04-07

2020-04-06

2020-04-05

Jim avatar

Hi. I’ve been struggling with a problem that I hope you guys may be able to help with. I need a way for a values file used by multiple releases to substitute different values, based on what the release is.

Jim avatar

E.g. I might have a helmfile.yaml that contains:

releases:
    - name: app-init
      namespace: app
      chart: ./helm/chart/app-init
      installed: true
      values:
        - helm/vars/app.yaml.gotmpl

    - name: app
      namespace: app
      chart: ./helm/chart/app
      installed: true
      values:
        - helm/vars/app.yaml.gotmpl
Jim avatar

And helm/vars/app.yaml.gotmpl may contain something like the following (that doesn’t work):

configFiles:
  environment_file: |
    {{- if eq .Release.Name "app-init" }
    DB_USER={{ requiredEnv "ADMIN_DB_USER" }}
    DB_PASS={{ requiredEnv "ADMIN_DB_PASS" }}
    {{- else }}
    DB_USER={{ requiredEnv "DB_USER" }}
    DB_PASS={{ requiredEnv "DB_PASS" }}
    {{- end }}
Jim avatar

The intention above being to populate some values using different environment variables depending on which release is going out. The values file is unable to access .Release.Name and so the above doesn’t work.

I’ve also tried changing the values: file in the release example above to something like:

      values:
        - HelmReleaseName: app-init
        - helm/vars/app.yaml.gotmpl

and:

      values:
        - HelmReleaseName: app
        - helm/vars/app.yaml.gotmpl

And changing the test in the values file to be:

{{- if eq .Values.HelmReleaseName "app-init" }}

But again this doesn’t work.

Jim avatar

Is there a way to have a test in the values file be able to see something unique based on the release being installed?

mumoshu avatar
mumoshu

@ Release.Name isn’t available within the gotmpl file. It’s only available in helmfile.yaml template. This is to avoid misuses(like “hiding” the release specific values)

And - you don’t need to complicate things with the if-else for your specific example.

I’d suggest this:

releases:
    - name: app-init
      namespace: app
      chart: ./helm/chart/app-init
      installed: true
      values:
        - helm/vars/app-init.yaml.gotmpl

    - name: app
      namespace: app
      chart: ./helm/chart/app
      installed: true
      values:
        - helm/vars/app.yaml.gotmpl

where app-init.yaml.gotmpl is:

configFiles:
  environment_file: |
    DB_USER={{ requiredEnv "ADMIN_DB_USER" }}
    DB_PASS={{ requiredEnv "ADMIN_DB_PASS" }

app.yaml.gotmpl:

configFiles:
  environment_file: |
    DB_USER={{ requiredEnv "DB_USER" }}
    DB_PASS={{ requiredEnv "DB_PASS" }}
Jim avatar

Thanks @mumoshu but that isn’t going to work for us too well :disappointed: I’ve simplified the configuration a lot for this example. The values files (of which there are multiple being passed) contain large configuration files that will end up in Kubernetes secrets and directly mounted into the file-system within the containers. I’m trying to avoid having to duplicate those configuration files multiple times(in app-init.yaml.gotmpl and again in app.yaml.gotmpl)

mumoshu avatar
mumoshu

So you have a large duplication within the single string values(configFiles.environment_file) for app and app-init?

Jim avatar

Yeah (for this example, again it doesn’t really match reality). We have a number of configuration files below configFiles: and each one is for a different component (PHP, Apache, the Application itself) and some of those configuration files are quite large. We have about 5 variables that are used (sometimes once, sometimes across multiple files) that need to differ for one release vs the other, where both releases use the same configuration files. So I was trying to solve that by using some helmfile conditional logic as part of templating out the config files.

mumoshu avatar
mumoshu

How about aggregating all the common env vars into a single yaml file common.env and reusing it?

app-init.yaml.gotmpl:

configFiles:
  environment_file: |
    DB_USER={{ requiredEnv "ADMIN_DB_USER" }}
    DB_PASS={{ requiredEnv "ADMIN_DB_PASS" }
    {{ readFile "common.env" | nindent 4 }}

app.yaml.gotmpl

configFiles:
  environment_file: |
    DB_USER={{ requiredEnv "DB_USER" }}
    DB_PASS={{ requiredEnv "DB_PASS" }}
    {{ readFile "common.env" | nindent 4 }}
mumoshu avatar
mumoshu

the more control syntax you introduce, the more your config template gets unreadable/buggy.

so i would prefer avoiding using if whenever possible

mumoshu avatar
mumoshu

otherwise - filing a feature request to add support for Release.Name within values.yaml.gotmpl might be needed

Jim avatar

Ok. I’m not sure I can easily do what you’ve suggested, but I’ll give it a shot. Thanks for your help.

But the take-away is that currently there is no way me to test anything defined in the release form within the values file right?

Jim avatar

This looks like a related ticket: https://github.com/roboll/helmfile/issues/760

Unable to use .Release.Name in values.gotmpl · Issue #760 · roboll/helmfile

I have a use case where I want the .Release.Name to be used as a variable in a values.gotmpl file. However, no matter what I do, the values.gotmpl file will not render .Release.Name. Is this expect…

mumoshu avatar
mumoshu

yes. i thought i intentionally decided to not implement that because it seemed to become a maintainance/user support nightmare

mumoshu avatar
mumoshu

awesome! I couldn’t recall that

Jim avatar

If I have trouble implementing the solution you’ve got above, would you consider allowing us to shoot ourselves in the foot?

mumoshu avatar
mumoshu

do you mean you still need .Release.Name within values.yaml.gotmpl?

mumoshu avatar
mumoshu

if so - yeah i would definitely consider implementing it. i think that feature is requested too many times to be ignored

Jim avatar

Yeah, if I can’t break up the configuration files like you’ve shown above, it would be nice if I could test for .Release.Name as an alternative.

mumoshu avatar
mumoshu

undestood. would you mind adding your comment on the issue saying “i need this too”?

Jim avatar

Yes, I’ll do that now. Thank you for your help.

mumoshu avatar
mumoshu

thanks for your support too!

Andrey Nazarov avatar
Andrey Nazarov

@mumoshu , btw, does double render work for values.yaml.gotmpl? Like {{ {{ .Release.Name }} }} . I always get confused with it.

mumoshu avatar
mumoshu

unfortunately no

Jim avatar

I attempted that.

mumoshu avatar
mumoshu

and double rendering only render helmfile.yaml twice

mumoshu avatar
mumoshu

it doesn’t happen (and unnecessary) for values.yaml.gotmpl

Andrey Nazarov avatar
Andrey Nazarov

Ok, good to know. Thanks!

mumoshu avatar
mumoshu
{{` {{ ... }} `}}

is for release templates

mumoshu avatar
mumoshu

np!

mumoshu avatar
mumoshu
roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

Andrey Nazarov avatar
Andrey Nazarov

We’ve never used gotmpl values files, so I don’t know any details. But we are using this double-render trick in our main helmfile.yaml

mumoshu avatar
mumoshu

yes, that makes sense. you should avoid gotmpl values whenever possible.

Andrey Nazarov avatar
Andrey Nazarov

Yes, thanks). I’m aware of how it works for releases. But thank you anyway)

:--1:1
mumoshu avatar
mumoshu

(i got to think that the complication from release templates vs values go tmpl vs double rendering all coming from the fact that we’re abusing yaml templates too much…

mumoshu avatar
mumoshu

regarding that i’ve recently talked with @Erik Osterman (Cloud Posse) about a dream: hcl2-based dsl for helmfile

https://sweetops.slack.com/archives/CFFQ9GFB5/p1585873245019000

@mumoshu this begs the question - have you considered HCL2 for Helmfile?

Andrey Nazarov avatar
Andrey Nazarov

On the other hand, sometimes you can do some crazy things with helmfile)). But the readability suffers a lot)

:--1:1
Jim avatar

FYI. I got this technique to work. Oh but the horrible hacks I had to do in order to get it to work makes it pretty unreadable. Unfortunately the order of the content of the files in question mattered (they’re not just simple env files) and the variables were spread across them, and so the files had to be broken up into multiple parts. Having to change content of the file means finding its multiple parts, plus some bits in multiple helmfile values files. Think of a values file like:

configFiles:
  config_file_1: |
    {{- readFile "start_of_file" | nindent 4 }}
    VAR1={{ requiredEnv "X" }}
    VAR2={{ requiredEnv "Y" }}
    {{- readFile "bit_more_of_file" | nindent 4 }}
    VAR3={{ requiredEnv "Z" }}
    {{- readFile "end_of_file" | nindent 4 }}

  # The same sort of butchery in the next few config files too...
  config_file_2: | 

Then having to have lots of values files because there are different combinations of the VARs that can happen for different releases. So I ended up with lots of file “parts” and 4x values files to drive different releases that all used the same config files, but with the values varying.

Jim avatar

If .Release.Name was available, then I would just have 1 file per configuration file, with some statements like the following within it:

{{- if eq .Release.Name "X" }}
VAR1=a
VAR2=b
{{- else if .Release.Name "Y" }}
VAR1=x
VAR2=y
{{- else }}
VAR1=m
VAR2=n
{{- end }}

Although messy… I still think it’s heaps cleaner than what I had to do.

2020-04-02

2020-04-01

hari avatar

Hello Everyone, How can we differentiate the configmaps with different environments (Dev,Int,Prod). We don’t use the helmfile what is the recommended approach ? should we allow helm to create the configmaps by keeping the configmaps under template directory? for example if i keep the configmap-dev.yaml, configmap-int.yaml, configmap-prod.yaml under template directory it will try to create the configmap object for all 3 config files with same name as the spec will be as below

kind: ConfigMap
metadata:
name: envconfig-configmap-{{ .Release.Name }}
hari avatar

may i know recommended approach to sort it out please

    keyboard_arrow_up