#helmfile (2020-1)

https://github.com/roboll/helmfile

Questions and discussion around helmfile https://github.com/roboll/helmfile and https://github.com/cloudposse/helmfiles Archive: https://archive.sweetops.com/helmfile/

2020-01-24

Cameron Boulton

It would be helpful if you would all add your commentary to that issue if you experienced it. Hopefully they will reopen or at least point to a new issue with resolution at some point.

Cameron Boulton

@mumoshu Any ideas how I might accomplish value key removals in Helmfile during the values merge? I’m looking for behavior similar to this: https://github.com/helm/helm/issues/1966 Using null however does not seem to make it past the Helmfile value merge operation (that is merging a discreet/defined value with null does not appear to be subtractive; the original value key remains)

Specifically, we have some global value keys like resources, probes, etc. that we want to remove for the default environment only (we use default for local development). Does that make sense?

Can't remove keys defined in values.yml · Issue #1966 · helm/helm

Since the introduction of deep merging (#1620), it's now not possible to remove keys from values.yml entirely. For example, the telegraf values has a default entry for single.config.inputs.infl…

2020-01-23

2020-01-22

Kashif Rahman

Guys anyone faced such issue that kubernetes job created with helm for db migrations always succeed although when we manually deploy job it shows up actual error

$ node_modules/node-pg-migrate/bin/pg-migrate -m ./migrations-app -v up
No migrations to run!
Migrations complete!
Done in 0.62s

if I manually deploy a job it will show up db migration error which is actual output what could be the case where job from helm is passing in every case ?

Erik Osterman

Sounds like maybe the exit code from node_modules/node-pg-migrate/bin/pg-migrate is not getting returned

Erik Osterman

can you share how you call it in your docker image? for example, if you’re running it in a bash script, you’ll want to have set -e to ensure you exit non-zero on all errors

Kashif Rahman

@Erik Osterman it is called by yarn run <command> in package,json

Erik Osterman

ok, but what then calls yarn run

Erik Osterman

also, can you share the snippet from package.json where it’s called?

Kashif Rahman

1 min

Kashif Rahman
Erik Osterman

ok, so the good news is your package.json looks good. That should pass through the exit codes.

Erik Osterman

So how do you call this? We need to ensure that everywhere exit codes are preserved.

Kashif Rahman

hmmm makes sense I am rebuilding images, seems I have found something but let me test it

2020-01-21

Mahesh

helm 3 is very buggy, we face a lot of issues, eg. https://github.com/helm/helm/issues/7426

Helm3 install --atomic hangs forever · Issue #7426 · helm/helm

Hi, I am trying to install a release using the –atomic flag but it seems that it hangs forever: helm3 install bar stable/mariadb -n default –atomic Error: release bar failed, and has been unins…

mumoshu

helm-diff w/ helm 3 is unaffected, as it still shows the diff between the release stored in the cluster(!= the current state of k8s resources originally created for the release) and the manifests rendered by helm template.

but yeah the diff and the install/upgrade result can be much more reliable than in helm 2, as helm3 tries its best not to accidentally “revert” manual changes

1
mumoshu

@Mahesh does it still hang when you fix your deployment? (at glance it can happen when the k8s resources created by the chart is stuck in error or not-ready states, which is not issues in helm

mumoshu

you can just rerun it without --atomic and see if it reveals the underlying issue = cause of the hang.

helm should do a better error reporting on user errors if it’s actually user error, though.

Mahesh

we just do helm delete and delete k8s objects created by helm package (its very picky even for secrets)

mumoshu

to fix the hang?

Mahesh

yeah, to redeploy

rms1000watt

Oh crap… good point @Dudi Cohen you’re right. Yeah.. I think I would have to reconcile with kubectl apply with the helmfile template output

Dudi Cohen

@rms1000watt then you won’t have a release in helm

Zachary Loeber

Random helm/helmfile tip: For what its worth, if you are upgrading to helm 3 ensure you sync more than once for each new chart you deploy for the first time. The three way merge in helm 3 means certain helm constructs will be problematic (such as the autogenerated ClusterIP: “” of a service for instance). Ran into this issue a few times now without realizing it until after the fact.

Andrey Nazarov

ClusterIP was a PITA. Had to set force to false to deal with it. So far so good.

:--1:1
Cameron Boulton

Disabling force still doesn’t solve some of the edge cases with Helm 3’s three-way merge unfortunately: https://github.com/helm/helm/issues/6378#issuecomment-556212320 They closed the issue but you can see many people are still reporting the issue.

We’re still blocked from upgrading util the Helm team 1) seriously acknowledges the issue and 2) resolves it

Helm 3 - upgrade nginx - spec.clusterIP: Invalid value: "": field is immutable · Issue #6378 · helm/helm

I use the following to install / upgrade a chart: ./helm upgrade –install –set rbac.create=false –set controller.replicaCount=2 –set controller.service.loadBalancerIP=$ip –wait main-ingress st…

2020-01-20

rms1000watt

@mumoshu It would be interesting for helmfile diff to have a “hardcore” mode that compares against the k8s state instead of the helm state. Embarrassed to say, i hit cases where there’s manual changes to k8s resources that aren’t reflected in helm state. (If i’m misunderstanding helmfile diff -> helm diff , my bad ) Anyways, been using something along the lines of:


\# Render all the k8s yaml
helmfile -f hello.yml template > ~/Desktop/hello.helmfile.yml


\# Diff the new yaml with what's actually deployed
tail -n +2 ~~/Desktop/hello.helmfile.yml | kubectl diff -f - > ~~Desktop/hello.helmfile.diff 


\# If diff is acceptable, run helmfile
helmfile -f hello.yml apply
Dudi Cohen

@rms1000watt I also share your concerns about helm diff not comparing to the actual state of the cluster. I suggested a workaround similar to yours in this issue: https://github.com/databus23/helm-diff/issues/176#issuecomment-576291610

Show diff for actual state vs desired state · Issue #176 · databus23/helm-diff

Hi, At the moment, if you make any manual changes to resources (not via helm) helm diff will not reflect these changes. I suggest that the output should reflect the desired vs actual state of the r…

Dudi Cohen

@rms1000watt btw, would your workaround actually work in a scenario where kubectl diff finds a diff but helmfile apply doesn’t find a diff? Wouldn’t it just exit in that case?

rms1000watt

@timduhenchanter for visibility. And kudos to your always helmfile template | kubectl apply -f- methodology

rms1000watt

@stobiewankenobi for visibility too. Rofl. Afterthought.

Erik Osterman

@rms1000watt out of curiosity, are you using helm3? was wondering if it would do a better job.

Erik Osterman
Erik Osterman
Frequently Asked Questions

What are the key differences between Helm 2 and Helm 3? Visit the FAQs for insights.

rms1000watt

Solid

rms1000watt

I need to upgrade

Erik Osterman

I’m not sure if this impacts the helm-diff plugin or not.

rms1000watt

but this is a great lead

Erik Osterman

let me know what you find out!

2020-01-19

can anyone give me pointers on how to use this: https://github.com/roboll/helmfile/pull/906 but for aws secret manager? I have this in my helmfile but it just uses the literal as is, debugging doesn’t indicate it tried to resolve the secret

trying to piece together info from the README and this repo: https://github.com/variantdev/vals

variantdev/vals

Helm-like configuration values loader with support for various sources - variantdev/vals

helmfile version v0.98.2

doc is wrong, should be ref+awssecrets://..., and depending on your secret name format it may not work at all: https://github.com/variantdev/vals/issues/18 @mumoshu

awssecrets does not work for Secret Ids that do not end in a '/' character · Issue #18 · variantdev/vals

This works: $ ~/.local/bin/aws secretsmanager get-secret-value –secret-id DanTest/ { &quot;Name&quot;: &quot;DanTest/&quot;, &quot;VersionId&quot;: &quot;4853e4d6-d7e8-4a30-9099-89cb8c522099&quot;…

2020-01-17

Zachary Loeber

so what rbac policies did you apply?

Zachary Loeber

may I recommend tillerless instead?

2
James Huffman

another vote for tillerless

2020-01-16

Erik Osterman
How to Set Up a Kubernetes Clusters with Helmfile

Code examples of how Adobe Experience Platform uses helmfile in Kubernetes to streamline large-scale application management.

:--1:2
vgdub

Hi All , apologies in advance if this is not the right place to ask this , but I yesterday faced an issue with the helmfile diff --suppress-secrets . I was simply applying some RBAC policies but tiller is now giving full of errors with this.

vgdub
vgdub

I can confirm that tiller has cluster-admin privileges in kube-system namespace so I am not sure why this diff is failing.

2020-01-15

Interested in hearing the communities thoughts on the above puzzles ^^

Zachary Loeber

@DanB wouldn’t you use an environment variable for that?

that is what i am trying to do

but set it once to reduce duplication

James Huffman

so what you want is for an environment variable to be assigned to a key in your helmfile but propagate up into the base chart?

Yes but multiple charts, not just one chart

James Huffman

so our solution to this, which may or may not work for you, is to make a values file section called global in each of our charts. within global are all of the common values of this nature we’d like to set, initially with placeholders (since they will get overwritten). then we have a macro file which handles setting all of the fields to their real values at run time, using env calls to collect them. in each helmfile, we include this macro file as another values file:

values:
      - macros/deploy.yaml.gotmpl
James Huffman

having to put the placeholders into each chart is the most annoying bit, but helm template helpers might let you avoid that

Ah hmm. So in our case the value is the same in each chart but each chart may use a different key in their values. I guess wed need the global section to duplicate the value for each possible key?

James Huffman

to be fair, you only have to put in each chart the specific keys you care about for that chart. any keys in the macro file which don’t exist in the underlying chart just won’t do anything.

To put what I want in another way: I want to keep all usage of env vars out of individual chart values. I want to centralize all use of env vars to one place (environment? Base? Whatever works). Pain point I am trying to avoid is if an env var name changes or we introduce a new override env var we only have to change it in one place instead of n where n = # of charts. In our case n is 50ish

James Huffman

you could make a single .yaml.gotmpl file containing a single parent key, under which are all of the keys you want and how to obtain their values (e.g. through env calls), then add that file to the values: array for each helmfile you’re using. to make it fully work you would then update your charts to pull in that whole section. this would be a one-time deal so any time you updated this master file, all charts would see it when they render.

James Huffman

imagine you called the section in your master values file global: and put all your keys below it. you can grab them in each of your charts with $root.Values.global i believe.

hmm, some of these charts I do not control

To me this seems like it’d be a common use case, I wonder if I am missing something or over complicating things

James Huffman

it’s mostly a consequence of how helm works. a particular value is only meaningful if it lands in the correct place within a chart. so with different charts from different sources, some of which you cannot easily modify, there’s no one-size-fits-all solution. everybody writes charts their own way

ignoring env vars, is there a way for me to set a static global value I can reuse in individual chart .yaml.gotmpl via mykey: {{ .Value.someStaticGlobal }}

to give a concrete example my charts want to know the “name” of the cluster in the values file. When I run helmfile I set an envvar CLUSTER_NAME=k8s-dev-01 now I want to set some variable once in one location and then use that value in various chart values, trick is chart A may expect cluster name in a variable named clusterName chart B might define it as cluster chart C might define it as cluster-name etc

i can sprinkle the env `CLUSTER_NAME through my chart .yaml.gotmpls, but I’d really like to avoid that

James Huffman

only way to handle that is chart by chart, unfortunately. you can’t do a generic solution since the charts themselves vary so much. we’ve run into the same thing

ah ok, really appreciate your thoughts and time!

:--1:1

Oh I think I figured it out: this was the key: https://github.com/roboll/helmfile/issues/640

Introduce State Values(Environment Values + Defaulting) · Issue #640 · roboll/helmfile

This is a copy-paste of #361 (comment) for visibility. We&#39;re going to introduce State Values, that should be the foundation for various useful features. (Note that this isn&#39;t a breaking cha…

In my helmfile I specify values: that set values based on envvars. I simply use {{ .Values.key }} in my chart value gotmpl file. I can override these as well in my environments.yaml.gotmpl which is set as bases: in my helmfile

2020-01-14

James Huffman

OK, that’s what i was wondering, if i needed to write a shell script to do it instead. thank you!!

:--1:1
vgdub

Hi folks, I am just wondering if anyone tried to add roles/clusterroles and their bindings via helm file , I am looking to apply some security polcies for multiple namespaces via helm ? is this possible ?

Zachary Loeber

try rbac-manager along with some incubator/raw charts

Zachary Loeber

that’s what I’ve been using and it works well enough

vgdub

@Zachary Loeber looks good , I am wondering in incubator/raw under resources and templates I can specify both role/clusterrole and bindings

vgdub

my goal is to be able to specify/create only get, list,watch policies in namespaces

Zachary Loeber
- name: inv-ingest-rbac
  chart: incubator/raw
  namespace: inv-ingest
{{- if eq (env "HELM_VERSION" | default "2") "3" }}
  needs:
  - kube-system/namespace-inv-ingest
{{- end }}
  values:
  - resources:
    - kind: ClusterRole
      apiVersion: [rbac.authorization.k8s.io/v1](http://rbac.authorization.k8s.io/v1)
      metadata:
        name: inv-ingest-cluster-role
        labels:
          [app.kubernetes.io/name](http://app.kubernetes.io/name): inv-ingest
      rules:
      - apiGroups: [""]
        resources: ["pods", "services", "configmaps"]
        verbs: ["get", "list", "watch", "create", "delete", "update", "patch"]
      - apiGroups: [""]
        resources: ["secrets"]
        verbs: ["get", "watch", "list"]
    - kind: ClusterRoleBinding
      apiVersion: [rbac.authorization.k8s.io/v1](http://rbac.authorization.k8s.io/v1)
      metadata:
        name: inv-ingest-role-binding
        labels:
          [app.kubernetes.io/name](http://app.kubernetes.io/name): inv-ingest
      roleRef:
        apiGroup: [rbac.authorization.k8s.io](http://rbac.authorization.k8s.io)
        kind: ClusterRole
        name: inv-ingest-cluster-role
      subjects:
      - name: default
        namespace: inv-ingest
        kind: ServiceAccount
Erik Osterman

(markdown plz )

1
Zachary Loeber

my bad (my personal social skills are just as awkward as my online ones unfortunately….)

vgdub

this is great , thanks @Zachary Loeber I am on the right path then. many thanks for telling me about this raw chart

Zachary Loeber

gladly!

Erik Osterman

The raw chart is great

Andrey Nazarov

We are doing kinda the same.

Zachary Loeber

That’s a fairly full example of a helmfile chart for a spark application that uses the default service account and requires more rights than I’m comfortable with

Zachary Loeber

you can pare it back to just what you’d need I’d think.

Erik Osterman

@Zachary Loeber you’re doing some interesting things over there. would love a demo sometime.

Zachary Loeber

It all feels like hacks layered on hacks to me

Erik Osterman

Haha that’s the reality though… it’s why I hate these demo videos that deploy hello word apps and proclaim that victory! “Deployments made easy”

Erik Osterman

The reality is that it’s difficult. Especially when you don’t control the tool chain from top to bottom. Integration is all about hacking bits and pieces together.

:--1:2

I’m digging in to helmfile for the first time and trying to do something I think should be straight forward. I want to set a value that can be used “globally” in chart value files (via gotmpl). I’m not sure where to put this. If I put it in my main helmfile.yaml I get an error ``line 4: field foo not found in type state.HelmState, if I put it in a file that is listed under bases, I get an error line 1: field foo not found in type state.HelmState` so where does it go?

basically I want to use {{ .Value.foo }} in several chart values.yaml.gotmpl files

This is what is in my defaults.yaml base file foo: '{{ coalesce (env "FOO") (env "LOCAL_FOO") }}'

2020-01-13

James Huffman

hello, i’ve been trying to do something that probably shouldn’t be difficult but i’m struggling. i want to do an {{ if }} block in my helmfile which checks for the existence of a file. ultimately, i only want a particular release to be deployed if a specific file exists locally. is there a standard way of doing that? (i have seen nothing of the sort in the docs/examples and have fought with my own approaches for a while now)

Erik Osterman

Hrmmmm that should be possible if Sprig supports a function for that

Erik Osterman
File Path Functions

Useful template functions for Go templates.

Erik Osterman

I don’t see a function for that

Erik Osterman

@James Huffman what underlying business logic are you trying to implement? maybe there’s an alternative way that doesn’t depend on the existence of files.

Erik Osterman
feat: add "fileExists" function · Issue #766 · roboll/helmfile

I&#39;m generating a list of additional values files for a release driven by some other dynamic configuration. I&#39;d like to be able to detect existence of those files prior to declaring them in …

Erik Osterman

the issue above has a workaround you can use for now

Erik Osterman
{{ if eq (exec "./fileexists.sh" (list $valueFile)) "true" }}

2020-01-10

Zachary Loeber

I could create one in about 3 minutes based on the default redis chart. Why don’t you give it a whirl first as you will almost always need to customize whatever anyone else precreated anyway.

Matt McLane

So what is the “industry standard” for pipelines to run helmfile? We are trying to move toward a gitops approach, which is why I have been looking at flux so much. We also like some of the functionality helmfile brings us. We could build something from scratch but I much rather be in line with what others are doing.

Zachary Loeber

@Matt McLane Good question, I’ve been trying to figure out the same. I’m looking into ArgoCD for this because it seems to have easier plugin capabilities to support helmfile. But it also seems that argo ‘apps’ are synonomous with helmfiles (generically) https://github.com/argoproj/argo-cd/issues/2143.

helmfile support · Issue #2143 · argoproj/argo-cd

Is your feature request related to a problem? Please describe. Similar to helm, helmfile support would be great. Describe the solution you’d like Support for helmfile.

1
Matt McLane

Have you found any documentation on how to plug helmfile into it? I am kinda figuring out that Flux isn’t going to work.

helmfile support · Issue #2143 · argoproj/argo-cd

Is your feature request related to a problem? Please describe. Similar to helm, helmfile support would be great. Describe the solution you’d like Support for helmfile.

Matt McLane

I am wondering if I can use a postsync hook within Argo CD.

Zachary Loeber

Nothing done with it yet, I’m still on the research/interest stage, sorry

Matt McLane

It’s all good

Erik Osterman

@Matt McLane we are using atlantis; i would not argue it’s an industry standard, but it works well enough.

Erik Osterman
Erik Osterman

atlantis lets one define custom workflows with a plan and apply phase which we map to diff and apply in helmfile

Matt McLane

Interesting. We run atlantis too for our terraform modules. In those cases we have created custom workflows to run terragrunt instead of terraform.

I can see where using it for helmfile will work.

Erik Osterman

Yup, very similar….

Matt McLane

How do you handle different environments?

helmfile -e dev apply vs helmfile -e qa apply?

Matt McLane

How do you promote things?

Erik Osterman

We have one repo per AWS account.

Erik Osterman

We use remote helmfiles pinned to a github release

Erik Osterman

so to promote, we open a PR for that account environment and pin it to a new release

Matt McLane

Gotcha

2020-01-09

Matt McLane

Morning helmfile folks.

Matt McLane

I have been struggling the past could days on a problem related to helm file and maybe someone in here can help. Basically I am trying to get weave flux to use helmfile instead of going straight to helm. Here is what I have done so far. I have setup flux to use manifest generation which allows me to run helmfile. Using that I can actually get helmfile to run and build things, but that isn’t a very good use of flux. Basically I am just using flux to clone git. What would be better is if I could get helmfile to write to stdout like kustomize build does. I am told by the folks that develop flux that it should work. It would allow us to benefit from the templating and secrets of helmfile while getting the gitops benefits of flux. Plus we could possibly use helmfile on our local systems if we wanted to.

So the question is who do I get helmfile to output to stdout like kustomize? I am currently running the following thinking it might do the trick, but it doesn’t seem to do anything in flux:

helmfile -e dev -q -f ./helmfile.d/helmfile.yaml build

Any help or suggestions would be great!

Erik Osterman

@Matt McLane have you seen the helmfile operator?

Erik Osterman
mumoshu/helmfile-operator

Kubernetes operator that continuously syncs any set of Chart/Kustomize/Manifest fetched from S3/Git/GCS to your cluster - mumoshu/helmfile-operator

Erik Osterman

I think that might make it a simpler integration with weave flux because you can just use CRDs

Matt McLane

I have seen it but it didn’t look all that functional and I didn’t know how to set it up. I was also concerned that there is a standing issue titled How to run helmfile-operator?

https://github.com/mumoshu/helmfile-operator/issues/4

How to run helmfile-operator? · Issue #4 · mumoshu/helmfile-operator

Hey, I had a look at this project and tried to set it up on our cluster. But I struggle with that. All the single pieces of this operator are described but there is on example or docs about how to …

Erik Osterman

haha “howto” docs are a nice-to-have on open source projects

Erik Osterman

but yea, it’s more in the incubator stage

Erik Osterman

@mumoshu is around, he can probably answer questions if they come up.

Zachary Loeber

Have you used that operator yet? I’ve not been able to get it to compile

Matt McLane

That is what I was worried about. it didn’t look complete to me.

Matt McLane

But I am willing to be wrong.

Erik Osterman

@mumoshu

Balaji J

hi , is there predefined helmfile for redis native cluster(not using sentinel)? or anyone worked on creating one. Please let me know

2020-01-07

Jonathan

This might not be the right place for this question, if not, sorry! I’m trying to deploy grafana, and import dashboards from a git repo. The issue is that the repository I want to import from is private, so even though the dashboards.default.local-dashboard.url is correct, I can’t reach it for obvious reasons, but I cannot find anywhere in the documentation how to pass secrets/username+pass etc. to authorize myself so I can read it. Does anyone have any pointers?

Chris Maxoutis

username:[email protected] where user and password you can pass them usually from env vars or even vault.

{{ requiredEnv PASSWORD" }}
Jonathan

I’ll try that out, thanks a bunch!

2020-01-06

TBeijen

2 questions:

• What are the intended use cases for helmfile apply vs helmfile sync. If I understand correctly syncwould also remove when installed: false. Otoh, apply has the diff output which is nice feedback.

• Shouldn’t helmfile have a cool logo? (Running into that every time I need to create presentations for team or business)

Zachary Loeber

Anyone out there setup the operator lifecycle manager as a helmfile?

Zachary Loeber

heck, anyone deploy/use it in any way?

deftunix

hi all, quick question about helmfile. I would like to apply a gotmpl kubernetes manifeest during the helmfile release apply phase. do you have any idea?

deftunix

I need to render the template and then apply it

Zachary Loeber

maybe use the raw chart with a values.yaml.gotmpl file?

Zachary Loeber

or delve into kustomize (something I’ve yet to do honestly)

deftunix

@Zachary Loeber I want just patch the ingress controller of a community helm chart without change it

deftunix

from the helmfile run

Zachary Loeber

I guess that’s what you use helmfile for in general if I’m reading your statement correctly.

Zachary Loeber

I don’t know about others, but I tend to completely disable ingress on all public charts then float ingress up to a custom chart of my own so I can more quickly make lateral ingress moves if so required.

1
Zachary Loeber

That way you aren’t trying to patch 20 different ingress chart implementations which may or may not be charted out the same.

Zachary Loeber

You could do such a thing without a custom chart as well I suppose. again, maybe use the incubator/raw chart with a values.yaml.gotmpl file

Zachary Loeber

other (smarter) people on this channel may know better ways though

deftunix

yes, I fix in exactly this way

1
deftunix

I create a custom chart

deftunix

for the ingress

scottcressi

I’m trying to use ref+vault integration for secrets and I’m getting vault: get string: key “foo” does not exist in secrets foo yet it does

scottcressi

Has anyone used the vault integration successfully? I assume people have

scottcressi

I figured it out :)

2020-01-05

Mahesh

Is there is any tool like kube-applier to manage deployments for helm?

Pierre Humberdroz

for helm? or Helm deployments?

Pierre Humberdroz

There is: https://keel.sh/

Keel

Kubernetes Operator to automate Helm, DaemonSet, StatefulSet & Deployment updates

2020-01-03

deftunix

I want just add some rule in the ingress controller without change the chart to support it

Erik Osterman

And by changing the rule, you hope to accomplish what? …what is the business objective

deftunix

Ssl redirect with alb ingress

deftunix

He needs a rule and an annotation

deftunix

It needs sorry

Erik Osterman

Does the chart support disabling the ingress?

deftunix

But the chart doesn’t support it

deftunix

Yes

deftunix

The chart support the ingress disabling

Erik Osterman

Perfect. Then you can use Helmfile

deftunix

I will disable and just add a kustomize or manifest?

Erik Osterman

Disable the ingress. Then define a new one using the raw chart and Helmfile

Erik Osterman

We have used this pattern in the past

deftunix

Do you have same repo?

Erik Osterman

You mean example?

deftunix

Yes

Erik Osterman

I couldn’t point you directly (on my phone), but you have seen our massive repo?

Erik Osterman
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles for Kubernetes - cloudposse/helmfiles

Erik Osterman

Go to the releases folder

deftunix

Thanks

Erik Osterman

Maybe search for the ingress keyword or raw

deftunix

Ok

2020-01-02

erik-stephens

Would like to automate our helmfile-centric workflow a bit more. Developing an Operator to handle the watching as well as the bits that helm/helmfile not able to perform. Would still like to leverage our helmfile effort, at least initially to quickly prototype. Anyone else go on a similar adventure? Recommendations on Operator sdk/framework (I’m currently looking at Metacontroller)?

mumoshu

In addition to helmfile-operator, I’ve built a POC of a GitOps + operator for helmfile deployments for that. It’s based on Brigade, Helmfile, and Flux and available at https://github.com/mumoshu/brigade-helmfile-demo

mumoshu/brigade-helmfile-demo

Demo for building an enhanced GitOps pipeline with Flux, Brigade and Helmfile - mumoshu/brigade-helmfile-demo

erik-stephens

Looks like operator-sdk new --type=helm ... is designed to address this use case.

Erik Osterman

@erik-stephens Have you seen the Helmfile operator by @mumoshu ?

Erik Osterman
mumoshu/helmfile-operator

Kubernetes operator that continuously syncs any set of Chart/Kustomize/Manifest fetched from S3/Git/GCS to your cluster - mumoshu/helmfile-operator

erik-stephens

I have not, but it’s on the short list of things to evaluate. Thanks!

Zachary Loeber

I’ve looked at it but was unable to get it to work at the time

deftunix

@Erik Osterman hello, do you have any example of usage helmfile to patch chart service spec without touch the chart?

Erik Osterman

You can’t “monkeypatch” with helmfile as it just wraps helm

Erik Osterman

So if helm provided someway to do that, then helmfile could.

Erik Osterman

@deftunix - describe instead what you want to accomplish, and perhaps we can think of a way to do it.

2020-01-01

Zachary Loeber

So, how are we all handling the helm 3 upgrades without automatic namespace creation anyway?

Zachary Loeber

updating all helmfiles to include a presync like this?

Zachary Loeber
  • events: [“presync”] showlogs: true command: “/bin/sh” args: - “-c” - >- kubectl get namespace “{{{{ .Release.Namespace }}}}” >/dev/null 2>&1 || kubectl create namespace “{{{{ .Release.Namespace }}}}”;
Erik Osterman
Helm3 doesn't automatically create namespace · Issue #891 · roboll/helmfile

Helm3 doesn&#39;t automatically create namespace - see https://v3.helm.sh/docs/faq/#automatically-creating-namespaces How can we solve this with helmfile, so that we don&#39;t have to manually crea…

Erik Osterman

Best option I think is to use raw chart

Roderik van der Veer

I’ve had trouble with the raw chart, multiple applies fail because the namespace already exists

Erik Osterman

Hrmm… but it works for other resource types?

Erik Osterman

I think if this were the case, then it would fail just as well for Deployments as it would for Namespace

Mahesh

Yes, we faced thi issues and we had to delete the whole deployment and also all k8s objects for the chart. atomic release option didn’t help much :(

Zachary Loeber

I decided to just whip up a point solution for the hell of it.

Zachary Loeber

Slack robocop told me not to swear. It took me a good long moment to realize I had done so… sorry I guess

1
:--1:1
Erik Osterman

lol, yes, it’s a little bit strict

Zachary Loeber

The helm chart I put together is so simple its not even worth publishing but hey, it does allow one to at least change the helm resource policy from ‘keep’ to whatever else it needs to be to allow for redeployments (if you need to do that)

Zachary Loeber

I was going to do v2 for helm3 but it should work for both helm 2 and 3 I think so I left it at v1

Zachary Loeber

@Erik Osterman, you ever not working/geeking out?

Erik Osterman

haha, not enough…

Erik Osterman
thomastaylor312/helm-namespace

Namespace auto-creation for Helm 3. Contribute to thomastaylor312/helm-namespace development by creating an account on GitHub.

Erik Osterman
Enable Release Namespace Creation in Helm3 · Issue #6794 · helm/helm

Overview Helm2 provided support for the Release namespace {{ .Release.Namespace }} via –namespace option if the release namespace did not exist. This functionality was considered rudimentary, and …

Zachary Loeber

I did, its in the notes for alternative options.

:--1:1
Zachary Loeber

It works fine if you are all helm 3 and willing to change your base helm commands. Honestly, its probably a better solution in general

Erik Osterman
zloeber/helm-namespace

A generic helm namespace chart. Contribute to zloeber/helm-namespace development by creating an account on GitHub.

Erik Osterman

Zachary Loeber

read somewhere that 3.1 will add namespace creation back anyway, so its likely a moot point

:--1:1
Erik Osterman

Yea, not worth investing in

Zachary Loeber

I’m looking at a pretty large stack of helm 2 charts all deployed with tillerless and helmfile that have gobs and gobs of secrets polluting the tiller namespace that makes me itchy to move to helm 3

Zachary Loeber

too many clusters for a single devops guy to look to migrate ATM so I’m using both helm3 and 2 in the same clusters like a fool

Zachary Loeber

anyone using helmfile in a gitops style deployment?

Zachary Loeber

with flux or argocd or something?

    keyboard_arrow_up