#kubernetes

Archive: https://archive.sweetops.com/kubernetes/

2019-04-30

2019-04-29

rms1000watt

https://github.com/roboll/helmfile/issues/392#issuecomment-455065039 @Erik Osterman I feel like I need to write some middleware for Helmfile so it can use SSM directly

Feat: Allow simple Vault integration · Issue #392 · roboll/helmfile

Currently there are many users that want to integrate Vault with Kubernetes, but there are no high level tools for this. The current Kubernetes AuthMethod for Vault is too complex and coupled to th…

rms1000watt
roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

rms1000watt

lol think just right here

mgrube

Has anyone found a good solution for user management on EKS? The best solution I can find is setting up roles that users can assume, but it doesn’t seem like an optimal solution.

Issif
Installation de aws-iam-authenticator - Amazon EKS

Amazon EKS utilise IAM pour fournir l’authentification pour votre cluster Kubernetes via l’ authentificateur AWS IAM pour Kubernetes . À partir de la version Kubernetes 1.10, vous pouvez configurer le client kubectl normal afin qu’il utilise Amazon EKS en installant l’authentificateur AWS IAM pour Kubernetes et en modifiant votre fichier de configuration

Issif

really usefull to manage by IAM

Erik Osterman

Gravitational Teleport

Erik Osterman

But you still map k8s roles to saml roles

Erik Osterman

have you looked at Gravitational Teleport?

Erik Osterman

it supports integration with SSO

Erik Osterman

and audited session logs with replay

mgrube

I haven’t, but I will take a look!

mgrube

Thanks @Erik Osterman

rms1000watt

https://github.com/roboll/helmfile/pull/569 SSM integration with Helmfile. Lets see the lash-back. lol

Added SSM integration by rms1000watt · Pull Request #569 · roboll/helmfile

There's been some interest for helmfile integration with SSM. Here is an example of what it can look like. For our current workflows, we have Bash scripts that export Env Vars via aws-env then …

2019-04-26

tolstikov

does someone have something to say about https://www.ovh.co.uk/kubernetes/ ?

Managed Kubernetes®: orchestration of containers in the cloud - OVH

Benefit from a free, managed and highly available Kubernetes® service to orchestrate your containerised applications in the OVH cloud Free hosted master nodes

Erik Osterman

Haven’t used that in particular… I used OVH back in the day with CoreOS. Loved the service for the value. Unlimited bandwidth and beefy bare metal instances.

1

2019-04-25

what are y’all thoughts on 3 clusters/3az vs 1 cluster/3az? does the 3 cluster approach give us much more reliable availability?

for the 3 cluster approach either using federation or dns round robin

mgrube

1 cluster, 3 az’s, multi-master setup

mgrube

if you want more clusters, set those up in different regions instead with dns failover and/or geo/round robin load balancing between them

1

Anyone run tillerless helm? https://rimusz.net/tillerless-helm/

Tillerless Helm v2

Helm really became a de-facto as Kubernetes Package Manager. Helm is the best way to find, share, and use software built for Kubernetes as it states on https://helm.sh. That’s true and sounds very cool. Since Helm v2, helm got a server part called The Tiller Server which is

Erik Osterman

that’s interesting!

2019-04-24

Erik Osterman

@rms1000watt I often use this example when talking to customers that out of the box, kubernetes supports a very basic form of canary+rolling updates. The reason for istio is to have full control over that process. Also, the “gold standard” for canary deployments is to tie it into your monitoring backplane so that you proceed to increase the level of traffic only so long as some KPIs are true. This level of controlled rollouts are more difficult to orchestrate with kubernetes primitives, which is why Istio is used. Also, it doesn’t have to be limited to blue/green. It can be a full rainbow of colors, where the traffic is spread across them.

1
rms1000watt

rms1000watt

Makes a ton of sense

rms1000watt

I think we’re all saying the same thing.. or at least on the same train of thought. It’s definitely a nice thing to have as business requirements expand (as they always do). Soon enough, it’ll become a required thing to have.

1

2019-04-23

rms1000watt

@Erik Osterman (or anyone) I got a fundamental question for ya..

What’s the difference between:

- a canary deployment with 5% increments

- a rolling update with maxUnavailable==0 && maxSurge==5% (with a RR Load Balancer in front)

rms1000watt

(no rush.. food for thought)

rms1000watt

What I’m thinking about is.. can a native rolling update be used in place of a canary deployment.. Assuming they can monitor the same metrics for health

rms1000watt

Because.. spinnaker / istio just for the sake of canary might not be reason enough

rms1000watt

midnight thoughts

2019-04-19

deftunix

hi all, anyone has experience with eks to assign a pool of static ip address/eni to pods based on the AWS high availability zone?

Erik Osterman

haven’t seen that done before; don’t know if it’s possible

2019-04-15

Erik Osterman

hahaha FTW!

Erik Osterman

2019-04-14

Issif

I use it, very useful : https://github.com/derailed/k9s

derailed/k9s

Kubernetes CLI To Manage Your Clusters In Style! - derailed/k9s

2
rms1000watt

@Erik Osterman You were able to convert @stobiewankenobi lolololol

Terraform -> SSM Then aws-ssm + helm + helmfile at deploy time

SSM is beastmode.. love using serverless.com pulling from there also

2019-04-13

Erik Osterman
kubernetes-sigs/krew

Package manager for “kubectl plugins”. Contribute to kubernetes-sigs/krew development by creating an account on GitHub.

Erik Osterman
ernoaapa/kubectl-warp

Kubernetes CLI plugin for syncing and executing local files in Pod on Kubernetes - ernoaapa/kubectl-warp

Erik Osterman
replicatedhq/ship

A better way to deploy Kubernetes Helm charts. Contribute to replicatedhq/ship development by creating an account on GitHub.

Erik Osterman
replicatedhq/k8s-secret-generator

Contribute to replicatedhq/k8s-secret-generator development by creating an account on GitHub.

Erik Osterman

awesome idea! anytime you need a shared secret, generate it

2019-04-11

@btai you can run helm with debug/dryrun enabled - this should show you how the values are being generated which may help you work out what/why a variable value is not as expected

2019-04-08

Humberto Oliveira
CNCF Formally Adopts CRI-O Runtime for Kubernetes - Container Journal

The Cloud Native Computing Foundation has formally accepted a container runtime designed specifically for Kubernetes as an incubation project.

how do you avoid merging maps in helm?

Erik Osterman

have a more concrete example? @btai

i think it might be the way the chart is written

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

that DRONE_DATABASE_DATASOURCE can be set as a postgres url i.e. <postgres://username:[email protected]/dbname>

but I set that as an envSecret value to pull from a k8s secret: https://github.com/helm/charts/blob/master/stable/drone/values.yaml#L159

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

but i guess when the values get merged for the deployment, the secret env vars get written first then the default env vars: https://github.com/helm/charts/blob/master/stable/drone/templates/deployment-server.yaml#L74

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

so the DRONE_DATABASE_DATASOURCE value gets overriden by the default sqlite value

possibly just a poorly written helm chart?

or is there a way of ignoring those default values

let me know if that makes any sense..

2019-04-07

Tim Malone

Re kops release schedule having slowed, anything to do with EKS perhaps? i.e. is pickup of kops slowing too?

Erik Osterman

I wonder… could be

Erik Osterman

Also the number of options available means there’s not as much support for anyone offering

2019-04-05

Alex Siegman

I know kops is somewhat intentionally behind Kubernetes in releases, but it looks like they are only “stable” on 1.11, which technically went EOL when 1.14 went GA if I’m not mistaken. That seems like a really slow release cadence to me. Is it still the go-to for doing home-spun K8S in AWS? EKS isn’t keeping up either.

Erik Osterman

Yea I am not sure why it’s slowed

Erik Osterman

kube-aws is also worth checking out. @mumoshu is a maintainer.

Alex Siegman

That’s the one you mentioned two weeks ago that I didn’t write down. Will take a look. Thanks!

casey

has anyone had any luck installing kiam on eks?

mgrube

yeah I am using kiam on eks

casey

how did you install it? I tried with with the helmfile in cloudposses repo, but no luck.

casey

Im pretty sure it’s because I couldn’t run the kiam server on a master node since eks doesnt let you do that

mgrube

I did a separate node-group for kiam-server that has the required credentials

mgrube

i used the stable helm chart to install it

casey

what do you mean node-group ?

mgrube

add --kubelet-extra-args --node-labels=${name_of_node_group} to your userdata

mgrube

that lets you use a nodeSelector when deploying your pods

casey

ah okay so do you have one node specifically for kiam? or do you run other pods on it as well?

Erik Osterman

In our case, we run kiam servers on masters

mgrube

i have a few daemonsets like node exporter, but not any other applications

Erik Osterman

And agents on all other nodes

mgrube

yeah you can’t run stuff on eks masters that I am aware of

1
Erik Osterman

Oh right

Erik Osterman

Yea we use kops predominantly

mgrube

The kiam-server node will have the ability to assume any role, so its best not to run anything else on it

2

2019-04-04

casey

hi all I have a quick question regarding dns zones which I am unsure of, if anyone could help it would be much appreciated.

casey
I want my domain name to be [example.com>. I have a hosted zone in aws route53 for [example.com>, which sits in my root account (I can not move it from the root account at this time, because its being used). I have another aws account called production which contains the hosted zone production.example.com, this account is also where my eks cluster is in. In my root account route53 zone *<http://example.com example.com](http://example.com)* i have an NS record *<http://production.example.com production.example.com](http://example.com)* so that the production account can handle those domains.
casey
If I use external-dns in my eks cluster, and allow it to create records in the [production.example.com> hosted zone, will my ssl cert hold? The SSL cert I have is a wildcard for *.example.com, and the records that get created from external-dns will look like **.<http://production.example.com production.example.com](http://production.example.com)*
casey

I believe that they wont, but I am not sure. Is there any common way to handle this kind of situtation?

aknysh

We request SSL certificates In each account separately

aknysh

The root certificate will not work in different accounts

aknysh

So prod.example.com will have its own certificate

aknysh

With wildcard

aknysh

and even if you created all environments in one account, a cert for *.[example.com> could be used for [prod.example.com](http://prod.example.com), but will not apply to *.<http://prod.example.com|prod.example.com](http://example.com)

oscarsullivan_old

How do you get ..example.com carts.. when I Google it they cost like $1200 a hear.

oscarsullivan_old

Is this by importing your domain example.com into ACM and issuing within

aknysh

You mean certs? :) They are free on AWS because they can be used only with other AWS resources for which you pay, e.g. load balancers . Not with servers external to AWS

oscarsullivan_old

Yeh!

oscarsullivan_old

I did take a look a month ago

oscarsullivan_old

But that was only 3 weeks into using AWS so I was occupied with transferring all my other provider knowledge to aws

oscarsullivan_old

Eill give another shot and.post in AWS channel

aknysh

and they are automatically renewed on AWS (which will save you a lot of headache because we always forget to renew them )

1

2019-04-03

Erik Osterman
Kubernetes basic glossary

Must-know terminology to understand Kubernetes concepts

2019-04-01

would it be bad practice to deploy my CI tool in the same k8s cluster as what it is deploying?

Erik Osterman

Depends on what you want to accomplish with the CI tool

Erik Osterman

For example if you want the CI tool to upgrade the cluster it operates in, that won’t work

Erik Osterman

However no reason to limit yourself to one CI service

no

k8s cluster would hold the app and the CI tool that deploys the app

Tim Malone

probably ok but maybe in a different namespace?

@Tim Malone that would be the plan yeah

    keyboard_arrow_up