#helmfile (2019-11)

nope. what do you see though?

well I’m trying to loop over a dict that might not exist…

so I’m trying to do something like this:

{{ range $version, $semver := (getOrNil .Values.deploys "somerelease") | default {} }}

the | default {} doesn’t seem to work, so I left it out

but then I get: wrong type for value: expected string, got map[string]interface{}

on that line

core reason for this is that we need to deploy the same stack to quite a few namespaces, all different versions, so now we have a central base that includes a deploys map like this:

deploys:
  service1:
    v0: ">=0.0.1-alpha.1"
    v1: "1.0.2"
  service2:
    v2: "2.0.0"

The file containing this is loaded depending on the environment that’s being deployed

@Bart M. It should be getOrNil "somerelease" .Values.deploys

ow

right damn

and {} isnt a valid symbol in go template so probably you want (getOrNil "somerelease" .Values.deploys) | default (dict) or even shorter get "somerelease" .Values.deploys (dict)

hmm ic

allright - thanks, seems to work like that

awesome!

Like ‘tag: 123’ and then ‘image: tag: {{.Values.tag}}’

Hmm, and it looks like I can also you values from the environment. Interestingly, env variables override values passed in

Hey!
I can’t use .Values from a helmfile in .gotmpl files.

I thought this is actually opposite. You can access .Values from within .gomtpl files listed under releases[].values[]

But I’d say it’s the only exception

To pass .Values down to sub-helmfiles, you need to explicitly state so in helmfiles[].values

it looks like I can also you values from the environment. Interestingly, env variables override values passed in

I don’t understand this. Would you mind providing an example?

helmfile doesn’t override values with env variables except that you explicitly stated so

Nothing visual

we have our implementation of them here: https://github.com/cloudposse/helmfiles/tree/master/releases

kiam is to obtain IAM roles needed by cert-manager for Let’s Encrypt validation for wildcard domains

cert-manager is needed for the PKI certs that kiam uses between agents/servers

reloader is needed to restart the pods for kiam when the certificates change (or else you have a forced outage)

@Erik Osterman (Cloud Posse) Thank you very much

https://github.com/helm-notifier/Kubernetes-Infrastructure

Check the helmfile.d they get executed in alphabetical order

I work with references from one (master) helmfile to (child) helmfiles. like this I think the order is also preserved plus additionally I can use common values and so on. But what doesn’t work is this wait time…

e.g.

bases:
  - common.yaml
---
helmfiles:
  - path: "storageclass.yaml"
  - path: "reloader.yaml"
  - path: "certmanager.yaml"
#  - path: "appscode.yaml"
  - path: "others.yaml"
    values:
    - environment: {{ .Values.environment }}  
    - domain: {{ .Values.domain }} 

what is your definition of the chart is settled?

I thought wait: true i.e. helm’s --wait, more or less, wait for all the pods under deployments are ready in terms of pod readiness.

is that what you want? or perhaps you have an another criteria to be met before helmfile proceeds to the next release?

Hi @Frank, were u able to achieve the ordering of starting the helm chart? if so, can you pls clarify how the issue was solved?

just finished reporting those all

That’s an underlying helm issue

Happens when you fail the first release

indeed

atomic: true could help here

thanks for your responses

Which version of helmfile are u using?

Do you have tillerless: true set?

You need to turn it off

Thanks it worked with removing tiller plugin from dockerfile

Hi mumoshu…if i set tillerless: false using helm3 then i get the same error

bash-5.0# helmfile --environment=singlenode apply --concurrency=1 Building dependency ../../helm-installer/rook-operator Comparing rook-operator ../../helm-installer/rook-operator in ./helmfile.yaml: in .helmfiles[0]: in environments/singlenode/00-helmfile.yaml: failed processing release rook-operator: helm exited with status 1: Error: context deadline exceeded Error: plugin "diff" exited with error

hey, it might be this one:

https://github.com/roboll/helmfile/issues/980#issuecomment-555302340

tl;dr; pls try reinstalling helm-diff with –version flag

diff 3.0.0-rc.7 Preview helm upgrade changes as a diff

yeah but you may still have onld helm-diff binary!

I think diff is of correct version

nope

surprisingly.

helm plugin install` clones the repo. the version number is taken from the the plugin.yaml contained in the repo. but installation of helm-diff binary happens independently

https://github.com/databus23/helm-diff/blob/master/install-binary.sh

Thanks it worked…that was the only issue….

and this seems to have an issue(?) that, when --version is omitted, installs the latest “non-rc” release

glad it worked!

So do you mind to add this fix in the latest Dockerfile for v3.

did we miss that in dockerfile?

argh https://github.com/roboll/helmfile/blob/80dcdfb82188f613869fcb22e16a21d63a5a16d4/Dockerfile.helm3#L35

@Shikhar Goel would you mind submitting a pr for that?

Sure

When i make 300s then again i am facing issue.

could not deduce environment: block, configuring only .Environment.Name. error: failed to read 00-helmfile.yaml.part.0: reading document at index 1: yaml: unmarshal errors: line 14: cannot unmarshal !!str 300s into int in ./helmfile.yaml: in .helmfiles[0]: in environments/singlenode/00-helmfile.yaml: failed to read 00-helmfile.yaml: reading document at index 1: yaml: unmarshal errors: line 14: cannot unmarshal !!str 300s into int

https://github.com/roboll/helmfile/blob/990d31c667caaa969ff951c85603084c233cd33f/pkg/state/state.go#L1529-L1535

it should be a number so 300 should be okay

But it is giving error when used 300

@Shikhar Goel which helmfile command are you running?

i thought i’ve fixed it for helm upgrade

helmfile apply

Hmm

helm3 wants it in form of 300s and helmfile accepts it in form of 300(int)

yes and helmfile automatically append s in helm3 mode

but it is not in our case

interesting

could you provide a smallest possible example to reproduce it?

i have no idea how that would happen..

Sure one minute

`releases: #ROOK #ROOK-INIT {{ if .Environment.Values.isRook }}

• name: rook-operator chart: ../../helm-installer/rook-operator values:
  • /home/dmp/helmfile/{{ requiredEnv “env_file_name” }} wait: true timeout: 300 atomic: true {{ end }}

#PREREQUISITE-PROMETHEUS-GRAFANA {{ if .Environment.Values.isPrometheusGrafana }}

• name: prerequisite-prometheus chart: ../../helm-installer/prerequisite-prometheus values:
  • /home/dmp/helmfile/{{ requiredEnv “env_file_name” }} wait: true timeout: 300 atomic: true`

This is my helmfile and when i am running helmfile apply it is giving error Upgrading ../../helm-installer/rook-operator in ./helmfile.yaml: in .helmfiles[0]: in environments/singlenode/00-helmfile.yaml: failed processing release rook-operator: helm exited with status 1: Error: invalid argument "300" for "--timeout" flag: time: missing unit in duration 300

can we generalize it to this?

releases:
  - name: rook-operator
    chart: stable/envoy
    wait: true
    timeout: 300
    atomic: true
  - name: prerequisite-prometheus
    chart: stable/envoy
    wait: true
    timeout: 300
    atomic: true

`

so that i can run it locally

actually we have local helm charts that we are running…but yes we can regenrate like this also

thanks!

okay reproduced

HELMFILE_HELM3 helmfile apply still works

so maybe we have a bug in https://github.com/roboll/helmfile/pull/963

so i have to use HELMFILE_HELM3 helmfile apply explicitily in helmfile apply command

yeah for now

once the bug is fixed it isn’t needed

ok cool but do i have to add some plugin or anything to make it work

because i am getting bash-5.0# HELMFILE_HELM3 helmfile --environment=singlenode apply --concurrency=1 bash: HELMFILE_HELM3: command not found

i thought it’s something posix-ish

HELMFILE_HELM3=1 helmfile

try adding =1

that is already set in env

then you can omit it from the command

still got the same error

Upgrading ../../helm-installer/rook-operator in ./helmfile.yaml: in .helmfiles[0]: in environments/singlenode/00-helmfile.yaml: failed processing release rook-operator: helm exited with status 1: Error: invalid argument “300” for “–timeout” flag: time: missing unit in duration 300

what do you get from echo $HELMFILE_HELM3?

or export | grep HELMFILE

bash-5.0# env | grep -i HELMFILE_HELM HELMFILE_HELM3=1 bash-5.0# echo $HELMFILE_HELM3 1

hm then your helmfile must be outdated..?

i have just downloaded from your master branch today

but it seems to be working for me

could you double-check your path and timestamp on the helmfile binary being used?

yes i have done that….

i have build using docker build -t helmfile:0.1 .

is there anything else we should do…

..?

could you provide me a full procedure to reproduce it?

what do you run exactly after docker building it?

Actually we have dockerfile inside helmfile repo so when we run docker build command then a docker image is build using that docker file…

so for this i have removed your Dockerfile and renamed Dockerfile.v3 with Dockerfiule

and then ran the command

so how do you run helmfile after that?

i have made a shell script that clone the helmcharts inside the docker image and call helmfile apply method after that

Is there any other way to make docker image for helmfile

??

no it seems good

but i cant reproduce it easily

could you try just running make install to install helmfile locally and see if it works as intended?

from what i see, all i can say atm is you may have outdated helmfile binary (or source repo)…

also be sure to run go mod vendor before docker-build

if you’ve outdated vendor/ under the helmfile repo

Sorry i had previos cloned repo…but when i do docker build on your latest pulled master i get the following error

did you run go mod vendor?

Nope let me try that

Is this expected?

after building with latest image….still this issue persist

no

did you manually removed .git after cloning?

Nope…

no idea…

where the error is coming from?

Can you do one thing…can you please build helmfile docker image and make it pubic

you can just grab it from quay

ok…

but why were you buijding it manually? did you have to do so??

Actually i was adding sleep into it so that pod is not complete after execution

can you please send me the quay url with helm3 and latest helmfile

https://quay.io/repository/roboll/helmfile?tab=tags

use tags prefixed with helm3

ok thanks

after using your image also same issue persist

Upgrading ../../helm-installer/rook-operator in ./helmfile.yaml: in .helmfiles[0]: in environments/singlenode/00-helmfile.yaml: failed processing release rook-operator: helm exited with status 1: Error: invalid argument "300" for "--timeout" flag: time: missing unit in duration 300

@Shikhar Goel I’ve tried to reproduce it but had no luck so far

#
# On my macOS machine:
#

$ docker run -it --net host -v $HOME:$HOME --rm quay.io/roboll/helmfile:helm3-v0.93.1 sh

#
# In the docker container:
#

# helmfile -v
helmfile version v0.93.1
# helm version
version.BuildInfo{Version:"v3.0.0", GitCommit:"e29ce2a54e96cd02ccfce88bee4f58bb6e2a28b6", GitTreeState:"clean", GoVersion:"go1.13.4"}
# cat helmfile.timeout.yaml
releases:
- name: envoy
  chart: stable/envoy
  timeout: 10
# helm repo add stable <https://kubernetes-charts.storage.googleapis.com/>
# helmfile -f helmfile.timeout.yaml sync
<succeeds>

It just works as expected for me.

ok thanks mumoshu…i will try it once again

Thanks momushu..it worked with 0.93.1…earliar i was using 0.93.0

Thanks Alot!…

glad it worked!

Just one more thing how do you think of implementing namespace feature that helm3 has.

Like each helm chart will have its all info stored in its namespace

in the form of secrets

just specify namespace: yourns under releases[] and it should work

Yup…that it works but i was thinking that there can be some namespace variable in helmdefaults so that if the user wants to send all the info in one namespace then he or she can do that easily.

yeah can you just use helmDefaults.namespace?

ok cool..

Ah!

So you mean helmfile doesn’t have support for that yet, right?

seems so. i have never realized that we missed it under helmDefaults

This should be a good feature request

yup…actually i tested that…and i didn’t find that…thats why i was asking

got it! thx

would you mind writing a feature request?

Yup i will start it now only.

Much appreciated!

I want to run my charts sequentially is there any other way i can do that

how did you verify that?

i thought we had tests for that so need to investigate how and where it’s happening

I ran my helmfile and there was no execution order

is it actually running concurrently? or the ordering is just random?

order is just random

ah

maybe it’s alphabetically sorted, right?

or is it really random?

random, like each run results in different ordering?

i think it is alphabetically sorted

but i want to execute in the order of defination

okay i think that’s a regression since 0.90.0 or so

as the workaround, you can add needs to specify the order

release:
- name: foo
- name: bar
  needs:
  - foo

this makes bar processed after foo

regardless of concurrency.

yup i think that is the only option now…and also regarding timeout…it didn’t work i have to remove explicitly from the helmfile

probably i can try to bring back the old behavior in coming days but not today

Ya sure thanks…that will be really helpfull

will take a look into that as well. thx for your patience and support!

Do i file a bug for that…or you will be notify me once done

filing a bug would be super helpful, as i’m always overwhelmed by todos and the github issues are the only way to keep me reminded

Are you sure about alphabetical order? For me it seems random even with v0.87.0.

@Andrew Nazarov Does it seem random even when you’ve --concurrency -1 ?

If it’s not set or set to 2 or bigger, the ordering should be random, regardless of helmfile is pre v0.90 or not

I ran helmfile --log-level=debug --environment=alpha --selector type!=ingress apply --concurrency=1 . helm diff and helm upgrade calls looked pretty random. I used the helm-tiller plugin, btw.

oh… good to know. thx!

Haven’t checked the most recent version yet. And I’m in a way to switch to Helm 3.

based on how it’s implemented, i thought it should be the order of definition.

perhaps that’s my mlsconception, or perhaps it’s conditional.

anyways i’ve added some test case for that

And my examples were related to Helm 2. The initial question was about Helm 3.

might just helm render then kubectl apply

wow this looks like an interesting issue

do you have any chart or helmfile.yaml i can use to reproduce?

So the process that got me here (sort of impossible to reproduce)

1. kubectl apply -f generic-deployment-only.yaml
2. let developers do a bunch of kubectl commands against this deployment
3. Import this deployment to helm without destroying it first
4. Add/update env vars and notice they don’t get updated

Honestly, I doubt it’s a helmfile issue. Probably just helm being helm

Thanks! How did you import the deployment to helm?

Hmmm, no idea unfortunately. All Helmfile does is to shell out to helm-secrets expecting it would exit with a non-zero code when something went wrong

Reading Helmfile code to see if I’ve made silly mistake in it like ignoring exit code from helm-secrets at all..

Haven’t pinpointed it. We now know that something with co-workers AWS config was incorrect, causing decrypt to fail. However helmfile was silent about it. In another project decrypt errors were clearly stated.

Only difference between projects was that the latter secrets file had a fallback-pgp key configured as well, presenting an interactive screen for pgp password if KMS decryption failed.

So maybe that’s the difference: Interactive screen triggering a decrypt error in Helmfile, where simply failing to decrypt goes unnoticed (and then results in some missing var error).

Now need to focus on something else but will try to pinpoint if this difference indeed is what causes the different behaviour.

Does it fail as intended when you call helm-secrets dec $FILE directly?

Additional complexity: We run tiller-less. Need to look into that later as it ‘worked on my machine’.

Will later try to set-up scenario where it doesn’t decrypt. Then work my way up: sops, helm secrets, helm tiller, helmfile.

does look like a helm issue which is frustrating

aha! busted helm-diff install! https://github.com/databus23/helm-diff/issues/161

Yes, it was a similar issue in the helmfile repo.

https://github.com/roboll/helmfile/issues/980

And https://github.com/roboll/helmfile/issues/994

Omg @mumoshu you are the greatest

Thank you for this! @Andriy Knysh (Cloud Posse)

@Andriy Knysh (Cloud Posse) Let’s try this this week with terraform cloud

@Daren @Alex Siegman @Rhooker @btai heads up

@coreygale start using Helmfile yet? :)

@Shane think you mentioned some interest in this

will try, great work @mumoshu

nice nice

What’s cool about this “ value = helmfile_release_set.mystack.apply_output” is we can use a regex to extract parts of the output (e.g. NOTES.txt) which can contain metadata. Or for custom charts, output JSON in notes and use terraform interpolations to unpack

Is it possible to reference remote helmfile.yaml?

Good question! I assume so. @mumoshu ?

Yes it should work. path is basically -f in Helmfile and it should accept remote urls.

How does it work with environments? I mean environmnent values are taken from the file referenced in the main helmfile.yaml. Or I could somehow define them in tf config directly. I’m figuring out the best way to use the plugin.

We are using terraform to create clusters on GKE and we are using a repo per stage to restrict the access. At the same time we think it would be better to separate cluster creation and app deployment. Moreover we want to keep the existing helmfile.yaml and don’t repeat its configuration. Our helmfile.yaml is highly templatized and we are using environments.yaml as a simple interface for our developers. All they need to do is to change or add some values in this file and that’s it.

So if somebody had some thoughts to share I would really appreciate this.

it really depends on the use-case but generally speaking, i’d use terraform variables for a few parameters like the name of environment, region, availability zone.

and set it via helmfile_release_set values

values = {
      # Corresponds to --state-values-set region=whatever
      region = "whatever"
    }

btw it’ll likely to be changed to

values = [
  <<EOS
{
  "region": "whatever"
}
EOS
]

terraform native maps doesn’t really support arbitrary nested maps that we cant do this:

values = {
      # Corresponds to --state-values-set region=whatever
      foo = {
        bar = {
          region = "whatever"
        }
      }
    }

a good news is that terraform 0.12 is able to produce a structured diff against these json-object-looking strings

so it isn’t a huge u/x breaker

Moreover we want to keep the existing helmfile.yaml and don’t repeat its configuration

how do you provide helmfile environments.yaml file today?

if you’re passing it via helmfile --state-values-file environments.yaml, you can just point that file from values_file in tf helmfile_release_set resource

Now it’s just like that

bases:
- environments.yaml
---

got it. would you consider to specify the path to environments.yaml from tf-side?

or the whole contents of the environments.yaml, or only some part of it?

Yes, something like that

in any case, the general recommendation would be that u should do as much as possible in helmfile

Probably only a part of it, based on the environment passed to tf.

Ok)

i was considering to allow inlining the whole helmfile.yaml content in tf, it it helps

you could use tf string interpolation syntax to write whatever helmfile.yaml you like in it

Alright, thank you very much)

Why move to HEREDOC?

(string) vs structured data in HCL?

Interesting. Unfortunately its not compatible with Terraform Cloud as it relies on cli tools + config

Can’t those be installed just like custom terraform provider?

…pretty sure

@Erik Osterman (Cloud Posse) That’s due to the limitation in HCL/TF resource

https://sweetops.slack.com/archives/CE5NGCB9Q/p1574841839216000?thread_ts=1574585241.174500&cid=CE5NGCB9Q

I think that’s why the helm provider doesn’t support maps for setting values as well https://www.terraform.io/docs/providers/helm/release.html#example-usage

Should we move to a set syntax similar to the helm provider, something like:

set {
  name = "metrics.enabled"
  bool = true
}

set {
  name = "image.tag"
  string = "sometag"
}

set {
  name = "deployment.replicas"
  int = 3
}

I follow what you are suggesting, but don’t understand how if one is writing a custom provider how it would be limited to using certain data types, of which map is not supported. (Not speaking from experience)

Is that what you are saying?

Hrmm I follow a little bit more

Though I thought this is possible with locals

yeah i believe you’re following me correctly

from my experience, terraform resource spec’s is restricted to have

a static and predefined schema

So for consistency with helm provider maybe makes sense to do the same

Otherwise it could be a flat map with dot notation

i’ll try taking a deeper look but probably the set syntax would be better

a flat map wit dot nation seems not work as we need to give a specific type(string, int, bool, etc.) to the schema

values = {
  "foo.bar" = 1,
  "bar.baz = true
}

probably this can’t be expressed in today’s tf resource schema

I wonder if this syntax would work:

values = yamlencode({
      # Corresponds to --state-values-set region=whatever
      foo = {
        bar = {
          region = "whatever"
        }
      }
    })

o

looks good

then we don’t need to change anything

from the helmfile provider’s perspective, values is just a array of strings

oh cool

where you use yamlencode to convert the HCL map into string

I confirmed the syntax is acceptable

…with nested maps

cool!

actually, when i start working on the helmfile provider, i was blindly believing that it’s possible without yamlencode

…i would have assumed the same!

pretty disappointed when it turned out i had to change maps to strings. hopefully this will be fixed in tf in the future..

Out of curiosity, would would happen if one did this?

values = yamlencode({
      # Corresponds to --state-values-set region=whatever
      foo = {
        bar = {
          region = "{{ .Env.REGION }}"
        }
      }
    })

would helmfile render the gotmpl stuff?

i have not tried yet but it should

just a moment..

ah sry it won’t work. under the hood it is just that the provider produces a temporary values.yaml to be passed to --state-values-file

aha, ok - was just curious. don’t have a use-case yet.

you’d need to make it .gotmpl` to let gotmpl expressions evaluated

ok

maybe we can add values_templates for that

I was wondering if terraform apply works correctly with presync hooks installing crds… I mean in situations like this one: https://github.com/roboll/helmfile/issues/538#issuecomment-499237514

can i ask why you think so?

my theory was that it should work if vanilla helmfile apply works, as the tf provider just run helmfile apply under the hood

Haven’t checked it yet with the most recent versions of helmfile and tf provider (currently dealing with some different troubles with cert-manager), but considering the provided example I had to run helmfile sync first and only then apply started working.

thx. do you see any specifc error from the first helmfile apply?

It was some application specific error. I don’t remember exactly. I will try to check this out a bit later.

It seems now it works quite ok.

Another problem I have is that I always get the following error when running terraform apply, but helmfile apply works ok:

Error: could not get apiVersions from Kubernetes: unable to retrieve the complete list of server APIs: admission.stash.appscode.com/v1alpha1: the server could not find the requested resource, admission.stash.appscode.com/v1beta1: the server could not find the requested resource, repositories.stash.appscode.com/v1alpha1: the server could not find the requested resource

Probably it’s related with the installation order. So, since stash-operator always shows diff helmfile always tries to reinstall it, hence for some time the api are not available. When this happens another services cannot be installed due to the https://github.com/helm/helm/issues/6361

But somehow it always works for helmfile apply, but never for terraform apply. Any hints? The order is different? Concurrency is different?

hmm, could you share your tf file?

does any actual upgrades via terraform apply works after the successful helmfile apply?

It’s as simple as that

variable "project_id" {
  type = string
  description = "Google Cloud project ID"
}

variable "region" {
  type = string
  description = "Default Google Cloud region"
}

provider "google" {
  version = "~> 2.18.0"
  project = var.project_id
  region = var.region
  credentials = "${file("./credentials/account.json")}"
}

provider "helmfile" {}

resource "helmfile_release_set" "mystack" {
  path = "git::https://******/helmfile.yaml"

  working_directory = path.module

  environment = "staging"
}

data "google_compute_address" "dev_ip_address" {
  name = "***-dev"
}

output "external-ip" {
  description = "External static IP adress for dev"
  value = data.google_compute_address.dev_ip_address.address
}

output "mystack_diff" {
  value = helmfile_release_set.mystack.diff_output
}

output "mystack_apply" {
  value = helmfile_release_set.mystack.apply_output
}

ohh… it does look simple

Some updates happen initially as well, but since the whole terraform apply fails with the error, the state is not updated. So we cannot use terraform to manage that half-baked infra). Except for giving another try to apply.

i reread the relevant part of the providers code. unfortunately i have no clue so far. it is really just calling helmfile apply.

would it be easily reproducible for me if i tried to install stash-operatory?

I’m not sure actually). I had the same errors some time ago even with helmfile apply, but subsequent applying helped. So, I think this is just a coincidence.

I will play with it more. Don’t waste your time exploring this right now).

I’ve decided to move stash-operator to the dedicated helmfile and reference it via helmfiles: from the main one. But for some reason it’s ignored during installation. What I’m doing wrong?

bases:
- environments.yaml
---
helmfiles:
  - stash-operator/helmfile.yaml

repositories:
....

releases:
....

I can see the rendering though

first-pass rendering result of "helmfile.yaml.part.0": {staging map[] map[]}
vals:
map[]
defaultVals:[]
second-pass rendering result of "helmfile.yaml.part.0":
 0: repositories:
 1:  - name: incubator
 2:    url: <https://kubernetes-charts-incubator.storage.googleapis.com/>
 3:  - name: appscode
 4:    url: <https://charts.appscode.com/stable/>
 5: 
 6: releases:
 7:   - name: namespace-stash
 8:     namespace: default
 9:     chart: incubator/raw
10:     values:
11:       - resources:
12:         - apiVersion: v1
13:           kind: Namespace
14:           metadata:
15:             name: stash
16:           spec:
17: 
18:   - name: stash-operator
19:     namespace: stash
20:     chart: appscode/stash
21:     version: v0.9.0-rc.2
22:     needs:
23:       - default/namespace-stash
24:     labels:
25:       type: backup

helmfile --helm-binary helm3 -f helmfile.all.yaml apply worked for me. but on helmfile destroy it failed with the error we talked about:

FAILED RELEASES:
NAME
stash-operator
in ./helmfile.all.yaml: in .helmfiles[0]: in ./helmfile.stashop.yaml: release "stash-operator" failed: helm3 exited with status 1:
  Error: uninstallation completed with 1 error(s): could not get apiVersions from Kubernetes: could not get apiVersions from Kubernetes: unable to retrieve the complete list of server APIs: admission.stash.appscode.com/v1alpha1: the server is currently unable to handle the request, admission.stash.appscode.com/v1beta1: the server is currently unable to handle the request, repositories.stash.appscode.com/v1alpha1: the server is currently unable to handle the request

if helmfile apply doesn’t install stash-operator for you, probably something in your top-level helmfile.yaml relates

hmm, could it eb related to https://github.com/roboll/helmfile/issues/1008?

Hm, maybe. But my situation is a little bit different. In my case helmfile ignores releases defined in helmfiles:. I’ll open up a new issue with the example to reproduce.

Hm, or you’ve already fixed that)). Let me double check that we the latest release.

It seems no. I’ll file the issue. Probably I’m doing something wrong.

https://github.com/roboll/helmfile/issues/1010

@Andrew Nazarov Thx! I was flying to re:Invent. I’ll take a look soon. Curious but did it work before?

Haven’t used helmfiles: block before.

got it! thanks

Hm, it seems I need to define environments in subhelmfile for this to work. But shouldn’t they be taken from the main one?

@Andrew Nazarov Replied to you in https://github.com/roboll/helmfile/issues/1010#issuecomment-561364261

I’m gonna do two things - wdyt?

@Erik Osterman (Cloud Posse) Still getting our feet wet with Helmfile but yes it’s in our new k8s stack Thanks again for showing up the light

It’s interesting but assumes you can call it from Docker

But we run inside of Docker already with geodesic

So it’s a non starter. Docker in Docker would be overkill.

That said it was an interesting demonstration

This has huge potential

Also, is this ready for Helm 3.0 can I simply supply the newest helm binary and expect this to work?

Yes, it should work as helmfile detects helm version automatically

Okay I think I am behind a couple version so will slip in the helm 3.0 binary with the 0.93.2 or something else over the holidays.

Looking forward to it! JFYI, we have images for the helm3 setup in [quay.io/roboll/helmfile:helm3-](http://quay.io/roboll/helmfile:helm3-){ver} for reference

https://quay.io/repository/roboll/helmfile?tab=tags https://github.com/roboll/helmfile/blob/master/Dockerfile.helm3

Prefixing the release name with the namespace? I stumbled into this same problem and got around it by doing just that: prefixing release with Namespace and passing that a value with the --namespace option

Thx!

Hi! It sounds like what you might need is --state-values-set namespace=namespaceA and {{.Values.namespace}}

--namespace is for overriding namespaces for all the releases, regardless of what their “default” values are in helmfile.yaml

So you’d better use a more general template .Values and the corresponding flag to inject it at runtime --state-values-set to code your convention into helmfile.yaml

it may be super surprising but you actually have an outdated helm-diff binary installed

even you see diff 3.0.0-r.7 in helm plugin list`

https://github.com/roboll/helmfile/issues/980#issuecomment-555302340

try this