#helmfile (2019-03)

https://github.com/roboll/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles Archive: https://archive.sweetops.com/helmfile/

2019-03-31

2019-03-30

mumoshu avatar
mumoshu

thx! i’ll add helmfile destroy in my next sprint on helmfile

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

….which would operate only on (installed: true) ?

mumoshu avatar
mumoshu

no, i wasn’t considering to do so! im interested in what would be the benefit, though

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

was thinking that was maybe the closest to terraform destroy?

mumoshu avatar
mumoshu

trying to understand

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so “that with a state of installed should be destroyed”

mumoshu avatar
mumoshu

ah thats an interesting idea! and sounds feasible

mumoshu avatar
mumoshu

i’m thinking if we need ignored: true that is different than installed: true to express it

mumoshu avatar
mumoshu

hm, maybe an overkill…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

since the state in helmfile is not external (e.g. helmfile.state) but instead defined directly inside of the helmfile.yaml, that makes the installed state determined by installed: true

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but yea, it gets confusing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just for flavor….

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terraform plan has an argument for -destroy which generates a plan to destroy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s kind of like installed: false + helmfile apply

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or another way of thinking about it:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

helmfile apply + installed: false == helmfile destroy + installed: true

mumoshu avatar
mumoshu

that equation does help me get the idea

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

now, i don’t know if that’s what the community wants

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just logically that makes sense to me

mumoshu avatar
mumoshu

so am i. and i think theres no downside of making it as such

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the nice thing is then terraform destroy undoes exactly what was created from terraform apply

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

… which is a nice way to iterate

mumoshu avatar
mumoshu

agreed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you realize what you’re doing…. you’re making it so we can use something like https://github.com/gosuri/terraform-exec-provider

gosuri/terraform-exec-provider

Contribute to gosuri/terraform-exec-provider development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
samsung-cnct/terraform-provider-execute

execute arbitrary commands on Terraform create and destroy - samsung-cnct/terraform-provider-execute

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Shane ever look into something like this?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Shane avatar
Shane

I’m familiar with it, but never caught on in my designs.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we could have a generic provider in terraform that calls command with apply, or destroy and maybe we can use helmfile then with terraform

:--1:1
mumoshu avatar
mumoshu

thought if installed: true better be ignored: false or enabled: true but turned out either of them fits my mental model

2019-03-28

mumoshu avatar
mumoshu

shouldn’t helmfile diff and helm diff hide diffs of secrets by default? please :–1: on this issue if you agree https://github.com/databus23/helm-diff/issues/128

Suppressing secrets by default? · Issue #128 · databus23/helm-diff

helm-diff as of today show diffs of secrets, which results in exposing your secret values to stdout. This seems problematic as when used in CI, the user ends up unexpectedly leaves the secret value…

mumoshu avatar
mumoshu

considering to add helmfile destroy while deprecating helmfile delete. wdyt? https://github.com/roboll/helmfile/issues/511

helmfile delete shoud use --purge as default · Issue #511 · roboll/helmfile

I never understood why you would not want to purge everything upon delete. it seems that is may be used for troubleshooting but if you want to delete a release why would you want to troubleshoot an…

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

agree about --purge

2019-03-27

2019-03-26

2019-03-25

Abhsihek Jaisingh avatar
Abhsihek Jaisingh

Is there any performance difference between helmfile sync & helmfile apply, like if I am using helmfile in my CI running on every commit. Which should be more preferable?

mumoshu avatar
mumoshu

apply is preferable, as it syncs only when there are changes to be applied also see https://github.com/roboll/helmfile/issues/205#issuecomment-411811549

Only sync release if there is a difference · Issue #205 · roboll/helmfile

Any possibility of adding a feature to run the sync for releases in the helmfile that have differences. With the feature of helm diff I would imagine this would be something that would fit?

Abhsihek Jaisingh avatar
Abhsihek Jaisingh

Agreed, but is helmfile apply any different from just helmfile diff && helmfile sync?

mumoshu avatar
mumoshu

yep, the latter creates new helm releases even for unchanged ones, afaik

Abhsihek Jaisingh avatar
Abhsihek Jaisingh

@mumoshu got it, thanks github-check-mark

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in a CI/CD context, helmfile apply coupled with the intalled flag is the way to go

:--1:1

2019-03-24

2019-03-22

mumoshu avatar
mumoshu

@Erik Osterman (Cloud Posse) Can I add a link to this channel in the helmfile readme?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@mumoshu please do!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We also have an archive of the channel publicly available here: https://archive.sweetops.com/helmfile/

helmfile

SweetOps is a collaborative DevOps community. We welcome engineers from around the world of all skill levels, backgrounds, and experience to join us! This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build sweet infrastructure.

Shane avatar
Shane

@mumoshu or anyone else I’m getting more interested in this or something similar due to some requests from my teams - https://github.com/roboll/helmfile/issues/483

Lock file · Issue #483 · roboll/helmfile

Summary In order to maintain predictable deployments, as developer I want to generate and use lock file for all chart versions retrieved from a helmfile. Rationale We have an environment git repo f…

mumoshu avatar
mumoshu
Lock file · Issue #483 · roboll/helmfile

Summary In order to maintain predictable deployments, as developer I want to generate and use lock file for all chart versions retrieved from a helmfile. Rationale We have an environment git repo f…

mumoshu avatar
mumoshu

Perhaps we can import the relevant helm pkg from the upstream, so that we don’t need to shell-out to helm search..? But it may not worth efforts

Shane avatar
Shane

If we could decide on a design that also achieves what I need to do I can probably do the PR.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think a locking mechanism with helm is desperately needed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i think a locking mechanism in helmfile would help, but where we need it most is when we deploy our own apps, which can happen dozens of times a day

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(E.g. unlimited staging environments)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the problem is if helm is executed twice in short succession to “upgade” an environment, the outcome is unpredictable

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use helmfile for those deployments as well, however the pipeline is different

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and the pipeline isn’t always executed by atlantis; sometime codefresh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so I see the actual fix as being some kind of locking mechanism that is in k8s ,not on the filesystem

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Shane

Shane avatar
Shane

a helm upgrade is predictable if you don’t run repo update

Shane avatar
Shane

but ya in codefresh you would need to get a new index

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

to concurrent helm upgrades on the same release is predictable?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

where is the locking happening?

Shane avatar
Shane

Whenever we run helm standalone we always specify a version so we don’t have your issues.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. helm upgrade --wait, i see that affected

Shane avatar
Shane

interesting never had a usecase for a concurrent upgrade against the same helm release

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

well, we do not want concurrent upgrades

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just if we are deploying an app on every commit to a branch, those can happen very close to each other

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

one upgrade starts before the other one finishes and they both are in --wait state

Shane avatar
Shane

ahh ya, that’s an entirely different issue than what a lockfile would accomplish

Shane avatar
Shane

You want helm to block.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(or helmfile)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but in a distributed manner

Shane avatar
Shane

That will also be a issue with helm v3

1
Shane avatar
Shane

I do agree a change lock should be available.

Shane avatar
Shane

which would only make sense inside of k8s

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we do all our app deploys also with helmfile; we call it the “helm cartridge” strategy. basically with every app, we ship a helmfile.yaml inside of it that describes how to deploy the app.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

“inside of it” = “inside of the docker image”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that said, i don’t personally have any complaints about adding a lock file

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

…and we’ll probably use the flag if it exists

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just it doesn’t solve all our headaches

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think what would solve our problems is terraform-helmfile-provider

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then we can piggy back on all the capabilities of terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and all the power/flexibility of helmfile

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and we’d get locking to boot (since terraform already supports state locking)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@mumoshu first planted this seed in my head

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Shane how’s that for a weekend project?

Shane avatar
Shane

haha

Shane avatar
Shane

terraform solves your locking problem, but I personally don’t know if I see the value in other things terraform can do.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

soooooo what about creating that IAM role for your service so it can write to S3

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what about creating that DNS zone for your external-dns

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what about creating that ElasticSearch cluster for your kibana UI

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s interesting if we can provision the backing services needed to deploy things in k8s

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

k8s handles a very narrow slice of automation, the rest is handled by tf

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:52:41 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:52:46 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:52:53 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:53:05 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

there are a few examples of where we wrote the terraform modules for provisioning the backing services needed by apps deployed with `helmfile

Shane avatar
Shane

Sure, but no specific reason to couple terraform infrastructure and terraform for services. You certainly can, but does not mean they need to be in one location.

Shane avatar
Shane

Essentially we have a process that runs terraform and a second process that runs helmfile.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, we do something similar

Shane avatar
Shane

certainly coupling them is another option.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just orchestrating the relationships between them is not “solved”

Shane avatar
Shane

But from our design I don’t view it as preferred to couple them.

Shane avatar
Shane

If anything I would rather helmfile pull from a terraform state output.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what do you think about Terraform Operators?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
rancher/terraform-operator

Use K8s to Run Terraform. Contribute to rancher/terraform-operator development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(experimental)

Shane avatar
Shane

I think they are a massive ton of work to actually get functional

Shane avatar
Shane

due to all of the migration edge cases where it would destroy your infrastructure.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

lol

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes

Shane avatar
Shane

I essentially wrote one for provisioning kubernetes, rds, s3 and a few other resources at my last job.

Shane avatar
Shane

It was essentially a operator to provision cloud resources for our platform

Shane avatar
Shane

It was …..painful

Shane avatar
Shane

We essentially used terraform in the background, but we had to have specific migration plans when terraform was not backwards compatible between 2 versions.

Shane avatar
Shane

overall we decided that using terraform to do that type of work was a disaster for us.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

wow, interesting you already went down that path

Shane avatar
Shane

it works fine if you have well worn modules, but it’s not very good for adhoc oh I need to change X

Shane avatar
Shane

not like you can just rename resources and apply the terraform

Shane avatar
Shane

you will break everything at least for the time being

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i was concerned about those points. so basically, having purpose built operators in k8s might work, but a generalized operator for tf might be impractical for the reasons you mentioned

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

purpose built operator = etcd operator, mysql operator, etc

Shane avatar
Shane

I took a interview with these guys recently and it sounds like they are doing exactly what you are talking about just not in terraform - https://crossplane.io/

Crossplane

The open source multicloud control plane.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that looks interesting

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’m torn. I think what we want to achive is hard to achieve e2e in terraform

Shane avatar
Shane

ya, they are very very early. I hope they can accomplish what they are aiming for it would be great.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

a higher order language is probably required.

Shane avatar
Shane

ya, which is why they are not doing it in terraform as the evolution is to hard.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Stakater

Kubernetes Expert! Stakater offers companies a highway to Kubernetes adoption for their DevSecOps automation - [✩Star] if you’re using it! - Stakater

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

have you seen them?

Shane avatar
Shane

npe

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

they are also like cloudposse

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

more k8s focused

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

some interesting projects, all open source

2019-03-21

mumoshu avatar
mumoshu

FYI: Use raw/incubator to add adhoc templated resources to your helm release

https://github.com/roboll/helmfile/issues/494#issuecomment-474697430

[Feature Request] Add local Helm Chart templates to repository chart release · Issue #494 · roboll/helmfile

Let me start with describing the issue I am trying to solve. Many (official) Helm charts allow you to specify annotations on object descriptions (e.g. ingress descriptions). Ingress controllers sup…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes love that chart!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We use it too

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

mumoshu avatar
mumoshu

I think I’ve firstly discovered the chart reading your commits made somewhere in cloudposse-related repo thx about that

mumoshu avatar
mumoshu

i’ve added templates setting to the chart just now, which helps creating an adhoc chart thats creates a k8s secret with helm-secrets https://github.com/mumoshu/charts/blob/56d5b63d2998bcd519b52639b586d896b9016633/incubator/raw/README.md#templated-resources

mumoshu/charts

Curated applications for Kubernetes. Contribute to mumoshu/charts development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Brilliant!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Igor Rodionov

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So this came up again this week. We are now running Helmfile under Atlantis for a customer

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

For now, we are downloading a tarball from GitHub, but would love to do remote sources

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thinking maybe to have @Andriy Knysh (Cloud Posse) submit a PR

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Any suggestions for him?

mumoshu avatar
mumoshu

I think we have potential need to support a lot of sources

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Our objective is to version pin Helmfiles like we do terraform modules

mumoshu avatar
mumoshu

Sounds great :–1:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The risk is of course it is incompatible with some capabilities of Helmfile

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Is that acceptable?

mumoshu avatar
mumoshu

Do you have any concrete example(s) about the incompatibility? (I had none

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

No… not immediately

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This was what I was thinking of…

mumoshu avatar
mumoshu

Let me recall..

mumoshu avatar
mumoshu

Ah, so we usually have a companion values.yaml(.gotmpl) files that is intended to be stored and used along with the referring helmfile.yaml

mumoshu avatar
mumoshu

in certain cases you want to read a remote helmfile.yaml, you also want companion values.yaml files to be fetched and used altogether

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Could we generalize what it means to “open and read” a file?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So that a file could be really just anywhere

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

E.g. use curl

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Which supports every scheme imaginable

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

libcurl

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Package curl

libcurl go bingding

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If no scheme specified, we default to file://

mumoshu avatar
mumoshu

sounds good

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) project for you

mumoshu avatar
mumoshu

so if you specify <https://example.com/foo/bar/helmfile.yaml>, what might happen?

mumoshu avatar
mumoshu

downloading only helmfile.yaml results in you’re unable to read any dependent values.yaml(.gotmpl) from the remote helmfile.yaml

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Then then it is up to the remote Helmfile to use a well formed url if it should be a companion

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Or if not, then it would use the local file

mumoshu avatar
mumoshu

so do we need to download <https://example.com/foo/bar>, extract everything into a local dir, then reads helmfile.yaml in it?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Thanks Erik :)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So for example: referencing “values.yaml.gotmpl” maps to “file://./values.yaml.gotmpl”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(On my phone)

mumoshu avatar
mumoshu


Then then it is up to the remote Helmfile to use a well formed url if it should be a companion

does that mean you need to, in your remote helmfile.yaml, turn every path to values.yaml to something like <https://path/to/your/values.yaml> ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes, if the implication is that all deps are remote

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But for values, that’s a perfect example of what we would usually customize per customer

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We don’t want that to be remote

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That’s their flavor

mumoshu avatar
mumoshu

hmm.. i’m still unsure

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So here is how we use it

mumoshu avatar
mumoshu

would your customer customizes the remote helmfile.yaml fetched by helmfile?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We provide these sample overrides

mumoshu avatar
mumoshu

or just values.yaml?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But we would never want these defined by us

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

These are what end users ultimate set to customize

mumoshu avatar
mumoshu

if that’s the latter, how do your customer pass the custom values.yaml to the remote helmfile.yaml without forking it?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So the remote Helmfile loads a local values file

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(I don’t fully understand the code implications of what I am saying)

mumoshu avatar
mumoshu

helmfile finds values.yaml relative to the helmfile.yaml

mumoshu avatar
mumoshu

as of today.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes yes

mumoshu avatar
mumoshu

for reproducibility of helmfile deployment

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But the root Helmfile or each included Helmfile?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I see what you are getting at.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I just always assumed it was relative to the root Helmfile or pwd

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I look at the values.yaml like terraform.tfvars

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So I guess I see it all relative to my current working directory

mumoshu avatar
mumoshu


But the root Helmfile or each included Helmfile?

each included helmfile.yaml

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea not sure how to reconcile that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That gets messy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I mean, I get why it works that way today for consistency

mumoshu avatar
mumoshu

that way, you can even helmfile sync the sub-helmfile alone

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Maybe a new keyword

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Like base_url

mumoshu avatar
mumoshu

but i do understand it isn’t intuitive to some people, and it doesn’t work straightforward when you want a library helmfile.yaml

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(Thinking like the base tag in html)

mumoshu avatar
mumoshu

yep, a new keyword may work

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) sounds like we have some more thinking to do on this

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ok let me re-read all the above

1
mumoshu avatar
mumoshu

do we probably need a way to override helmfile env..?

helmfiles:
- source: <https://path/to/helmfile.yaml>
   overrides:
      environments:
         customprod:
         - myownprod.yaml

so that helmfile --environment customprod helmfile.yaml works against the remote helmfile.yaml, even the customprod isn’t defined in the remote helmfile.yaml?

mumoshu avatar
mumoshu

this defeats the reproducibility of sub-helmfiles. but does allow us to use the remote helmfile.yaml as a reusable, customizable package

mumoshu avatar
mumoshu

like a helm chart

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Igor Rodionov

mumoshu avatar
mumoshu

Created an issue to track it before I forget https://github.com/roboll/helmfile/issues/523

feat: Allow environment values to be overrode when referring sub-helmfile · Issue #523 · roboll/helmfile

This is similar to #361, but for helmfile.yaml. You can write helmfile.yaml containing one or more sub-helmfiles that is processed before releases, in helmfiles: environments: default: values: - en…

mumoshu avatar
mumoshu

Anyway, what I think recently is: We generally need abilities to (1)pin and (2)update any dependencies. a semver in the source spec, and a command to fetch the latest ver that accomodates the version constraint would be useful

mumoshu avatar
mumoshu

after that, we can add any datasource users want…

mumoshu avatar
mumoshu

as long as it supports version pinning and updating based on semver

mumoshu avatar
mumoshu

anydep was my attempt to achieve that https://github.com/mumoshu/anydep

mumoshu/anydep

General-purpose project/application dependency manager - mumoshu/anydep

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh yes, I recall you mentioning this to me

    keyboard_arrow_up