#kubernetes (2019-04)

kubernetes

Archive: https://archive.sweetops.com/kubernetes/

2019-04-30

2019-04-29

rms1000watt avatar
rms1000watt

https://github.com/roboll/helmfile/issues/392#issuecomment-455065039 @Erik Osterman (Cloud Posse) I feel like I need to write some middleware for Helmfile so it can use SSM directly

Feat: Allow simple Vault integration · Issue #392 · roboll/helmfile

Currently there are many users that want to integrate Vault with Kubernetes, but there are no high level tools for this. The current Kubernetes AuthMethod for Vault is too complex and coupled to th…

rms1000watt avatar
rms1000watt
roboll/helmfile

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

rms1000watt avatar
rms1000watt

lol think just right here

mgrube avatar
mgrube

Has anyone found a good solution for user management on EKS? The best solution I can find is setting up roles that users can assume, but it doesn’t seem like an optimal solution.

Issif avatar
Issif
Installation de aws-iam-authenticator - Amazon EKS

Amazon EKS utilise IAM pour fournir l’authentification pour votre cluster Kubernetes via l’ authentificateur AWS IAM pour Kubernetes . À partir de la version Kubernetes 1.10, vous pouvez configurer le client kubectl normal afin qu’il utilise Amazon EKS en installant l’authentificateur AWS IAM pour Kubernetes et en modifiant votre fichier de configuration

Issif avatar
Issif

really usefull to manage by IAM

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Gravitational Teleport

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But you still map k8s roles to saml roles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

have you looked at Gravitational Teleport?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it supports integration with SSO

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and audited session logs with replay

mgrube avatar
mgrube

I haven’t, but I will take a look!

mgrube avatar
mgrube

Thanks @Erik Osterman (Cloud Posse)

rms1000watt avatar
rms1000watt

https://github.com/roboll/helmfile/pull/569 SSM integration with Helmfile. Lets see the lash-back. lol

Added SSM integration by rms1000watt · Pull Request #569 · roboll/helmfile

There's been some interest for helmfile integration with SSM. Here is an example of what it can look like. For our current workflows, we have Bash scripts that export Env Vars via aws-env then …

2019-04-26

tolstikov avatar
tolstikov

does someone have something to say about https://www.ovh.co.uk/kubernetes/ ?

Managed Kubernetes®: orchestration of containers in the cloud - OVH attachment image

Benefit from a free, managed and highly available Kubernetes® service to orchestrate your containerised applications in the OVH cloud Free hosted master nodes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Haven’t used that in particular… I used OVH back in the day with CoreOS. Loved the service for the value. Unlimited bandwidth and beefy bare metal instances.

:--1:1

2019-04-25

btai avatar

what are y’all thoughts on 3 clusters/3az vs 1 cluster/3az? does the 3 cluster approach give us much more reliable availability?

btai avatar

for the 3 cluster approach either using federation or dns round robin

mgrube avatar
mgrube

1 cluster, 3 az’s, multi-master setup

mgrube avatar
mgrube

if you want more clusters, set those up in different regions instead with dns failover and/or geo/round robin load balancing between them

:--1:1
btai avatar

Anyone run tillerless helm? https://rimusz.net/tillerless-helm/

Tillerless Helm v2

Helm really became a de-facto as Kubernetes Package Manager. Helm is the best way to find, share, and use software built for Kubernetes as it states on https://helm.sh. That’s true and sounds very cool. Since Helm v2, helm got a server part called The Tiller Server which is

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s interesting!

2019-04-24

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@rms1000watt I often use this example when talking to customers that out of the box, kubernetes supports a very basic form of canary+rolling updates. The reason for istio is to have full control over that process. Also, the “gold standard” for canary deployments is to tie it into your monitoring backplane so that you proceed to increase the level of traffic only so long as some KPIs are true. This level of controlled rollouts are more difficult to orchestrate with kubernetes primitives, which is why Istio is used. Also, it doesn’t have to be limited to blue/green. It can be a full rainbow of colors, where the traffic is spread across them.

:--1:1
rms1000watt avatar
rms1000watt

rms1000watt avatar
rms1000watt

Makes a ton of sense

rms1000watt avatar
rms1000watt

I think we’re all saying the same thing.. or at least on the same train of thought. It’s definitely a nice thing to have as business requirements expand (as they always do). Soon enough, it’ll become a required thing to have.

:--1:1

2019-04-23

rms1000watt avatar
rms1000watt

@Erik Osterman (Cloud Posse) (or anyone) I got a fundamental question for ya..

What’s the difference between:

  • a canary deployment with 5% increments
  • a rolling update with maxUnavailable==0 && maxSurge==5% (with a RR Load Balancer in front)
rms1000watt avatar
rms1000watt

(no rush.. food for thought)

rms1000watt avatar
rms1000watt

What I’m thinking about is.. can a native rolling update be used in place of a canary deployment.. Assuming they can monitor the same metrics for health

rms1000watt avatar
rms1000watt

Because.. spinnaker / istio just for the sake of canary might not be reason enough

rms1000watt avatar
rms1000watt

midnight thoughts

2019-04-19

deftunix avatar
deftunix

hi all, anyone has experience with eks to assign a pool of static ip address/eni to pods based on the AWS high availability zone?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haven’t seen that done before; don’t know if it’s possible

2019-04-15

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hahaha FTW!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

2019-04-14

Issif avatar
Issif

I use it, very useful : https://github.com/derailed/k9s

derailed/k9s

Kubernetes CLI To Manage Your Clusters In Style! - derailed/k9s

:--1:2
rms1000watt avatar
rms1000watt

@Erik Osterman (Cloud Posse) You were able to convert @stobiewankenobi lolololol

Terraform -> SSM Then aws-ssm + helm + helmfile at deploy time

SSM is beastmode.. love using serverless.com pulling from there also

2019-04-13

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
kubernetes-sigs/krew

Package manager for “kubectl plugins”. Contribute to kubernetes-sigs/krew development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
ernoaapa/kubectl-warp

Kubernetes CLI plugin for syncing and executing local files in Pod on Kubernetes - ernoaapa/kubectl-warp

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
replicatedhq/ship

A better way to deploy Kubernetes Helm charts. Contribute to replicatedhq/ship development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
replicatedhq/k8s-secret-generator

Contribute to replicatedhq/k8s-secret-generator development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

awesome idea! anytime you need a shared secret, generate it

2019-04-11

Ufou avatar

@btai you can run helm with debug/dryrun enabled - this should show you how the values are being generated which may help you work out what/why a variable value is not as expected

2019-04-08

Humberto Oliveira avatar
Humberto Oliveira
CNCF Formally Adopts CRI-O Runtime for Kubernetes - Container Journal attachment image

The Cloud Native Computing Foundation has formally accepted a container runtime designed specifically for Kubernetes as an incubation project.

btai avatar

how do you avoid merging maps in helm?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

have a more concrete example? @btai

btai avatar

i think it might be the way the chart is written

btai avatar

@Erik Osterman (Cloud Posse) basically for this chart: https://github.com/helm/charts/blob/master/stable/drone/values.yaml#L153

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

btai avatar

that DRONE_DATABASE_DATASOURCE can be set as a postgres url i.e. <postgres://username:[email protected]/dbname>

btai avatar

but I set that as an envSecret value to pull from a k8s secret: https://github.com/helm/charts/blob/master/stable/drone/values.yaml#L159

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

btai avatar

but i guess when the values get merged for the deployment, the secret env vars get written first then the default env vars: https://github.com/helm/charts/blob/master/stable/drone/templates/deployment-server.yaml#L74

helm/charts

Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

btai avatar

so the DRONE_DATABASE_DATASOURCE value gets overriden by the default sqlite value

btai avatar

possibly just a poorly written helm chart?

btai avatar

or is there a way of ignoring those default values

btai avatar

let me know if that makes any sense..

2019-04-07

Tim Malone avatar
Tim Malone

Re kops release schedule having slowed, anything to do with EKS perhaps? i.e. is pickup of kops slowing too?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I wonder… could be

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also the number of options available means there’s not as much support for anyone offering

2019-04-05

Alex Siegman avatar
Alex Siegman

I know kops is somewhat intentionally behind Kubernetes in releases, but it looks like they are only “stable” on 1.11, which technically went EOL when 1.14 went GA if I’m not mistaken. That seems like a really slow release cadence to me. Is it still the go-to for doing home-spun K8S in AWS? EKS isn’t keeping up either.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea I am not sure why it’s slowed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

kube-aws is also worth checking out. @mumoshu is a maintainer.

Alex Siegman avatar
Alex Siegman

That’s the one you mentioned two weeks ago that I didn’t write down. Will take a look. Thanks!

casey avatar
casey

has anyone had any luck installing kiam on eks?

mgrube avatar
mgrube

yeah I am using kiam on eks

casey avatar
casey

how did you install it? I tried with with the helmfile in cloudposses repo, but no luck.

casey avatar
casey

Im pretty sure it’s because I couldn’t run the kiam server on a master node since eks doesnt let you do that

mgrube avatar
mgrube

I did a separate node-group for kiam-server that has the required credentials

mgrube avatar
mgrube

i used the stable helm chart to install it

casey avatar
casey

what do you mean node-group ?

mgrube avatar
mgrube

add --kubelet-extra-args --node-labels=${name_of_node_group} to your userdata

mgrube avatar
mgrube

that lets you use a nodeSelector when deploying your pods

casey avatar
casey

ah okay so do you have one node specifically for kiam? or do you run other pods on it as well?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

In our case, we run kiam servers on masters

mgrube avatar
mgrube

i have a few daemonsets like node exporter, but not any other applications

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

And agents on all other nodes

mgrube avatar
mgrube

yeah you can’t run stuff on eks masters that I am aware of

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh right

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea we use kops predominantly

mgrube avatar
mgrube

The kiam-server node will have the ability to assume any role, so its best not to run anything else on it

:--1:2

2019-04-04

casey avatar
casey

hi all I have a quick question regarding dns zones which I am unsure of, if anyone could help it would be much appreciated.

casey avatar
casey
I want my domain name to be [example.com>. I have a hosted zone in aws route53 for [example.com>, which sits in my root account (I can not move it from the root account at this time, because its being used). I have another aws account called production which contains the hosted zone production.example.com, this account is also where my eks cluster is in. In my root account route53 zone *<http://example.com example.com](http://example.com)* i have an NS record *<http://production.example.com production.example.com](http://example.com)* so that the production account can handle those domains.
casey avatar
casey
If I use external-dns in my eks cluster, and allow it to create records in the [production.example.com> hosted zone, will my ssl cert hold? The SSL cert I have is a wildcard for *.example.com, and the records that get created from external-dns will look like **.<http://production.example.com production.example.com](http://production.example.com)*
casey avatar
casey

I believe that they wont, but I am not sure. Is there any common way to handle this kind of situtation?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

We request SSL certificates In each account separately

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

The root certificate will not work in different accounts

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

So prod.example.com will have its own certificate

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

With wildcard

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and even if you created all environments in one account, a cert for *.[example.com> could be used for [prod.example.com](http://prod\.example\.com), but will not apply to *.<http://prod.example.com|prod.example.com](http://example\.com)

oscarsullivan_old avatar
oscarsullivan_old

How do you get ..example.com carts.. when I Google it they cost like $1200 a hear.

oscarsullivan_old avatar
oscarsullivan_old

Is this by importing your domain example.com into ACM and issuing within

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

You mean certs? :) They are free on AWS because they can be used only with other AWS resources for which you pay, e.g. load balancers . Not with servers external to AWS

oscarsullivan_old avatar
oscarsullivan_old

Yeh!

oscarsullivan_old avatar
oscarsullivan_old

I did take a look a month ago

oscarsullivan_old avatar
oscarsullivan_old

But that was only 3 weeks into using AWS so I was occupied with transferring all my other provider knowledge to aws

oscarsullivan_old avatar
oscarsullivan_old

Eill give another shot and.post in AWS channel

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and they are automatically renewed on AWS (which will save you a lot of headache because we always forget to renew them )

:point_up_2:1

2019-04-03

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Kubernetes basic glossary attachment image

Must-know terminology to understand Kubernetes concepts

2019-04-01

btai avatar

would it be bad practice to deploy my CI tool in the same k8s cluster as what it is deploying?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Depends on what you want to accomplish with the CI tool

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

For example if you want the CI tool to upgrade the cluster it operates in, that won’t work

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

However no reason to limit yourself to one CI service

btai avatar

no

btai avatar

k8s cluster would hold the app and the CI tool that deploys the app

Tim Malone avatar
Tim Malone

probably ok but maybe in a different namespace?

btai avatar

@Tim Malone that would be the plan yeah

    keyboard_arrow_up