#kubernetes (2019-04)
Archive: https://archive.sweetops.com/kubernetes/
2019-04-01

would it be bad practice to deploy my CI tool in the same k8s cluster as what it is deploying?

Depends on what you want to accomplish with the CI tool

For example if you want the CI tool to upgrade the cluster it operates in, that won’t work

However no reason to limit yourself to one CI service

no

k8s cluster would hold the app and the CI tool that deploys the app

probably ok but maybe in a different namespace?

@Tim Malone that would be the plan yeah
2019-04-03

2019-04-04

hi all I have a quick question regarding dns zones which I am unsure of, if anyone could help it would be much appreciated.

I want my domain name to be example.com. I have a hosted zone in aws route53 for example.com, which sits in my root account (I can not move it from the root account at this time, because its being used). I have another aws account called production which contains the hosted zone production.example.com, this account is also where my eks cluster is in. In my root account route53 zone example.com i have an NS record production.example.com so that the production account can handle those domains.

If I use external-dns in my eks cluster, and allow it to create records in the production.example.com hosted zone, will my ssl cert hold? The SSL cert I have is a wildcard for *.example.com, and the records that get created from external-dns will look like *.production.example.com

I believe that they wont, but I am not sure. Is there any common way to handle this kind of situtation?

We request SSL certificates In each account separately

The root certificate will not work in different accounts

So prod.example.com will have its own certificate

With wildcard

and even if you created all environments in one account, a cert for *.[example.com](http://example.com)
could be used for [prod.example.com](http://prod.example.com)
, but will not apply to *.[prod.example.com](http://prod.example.com)

How do you get ..example.com carts.. when I Google it they cost like $1200 a hear.

Is this by importing your domain example.com into ACM and issuing within

You mean certs? :) They are free on AWS because they can be used only with other AWS resources for which you pay, e.g. load balancers . Not with servers external to AWS

Yeh!

I did take a look a month ago

But that was only 3 weeks into using AWS so I was occupied with transferring all my other provider knowledge to aws

Eill give another shot and.post in AWS channel

and they are automatically renewed on AWS (which will save you a lot of headache because we always forget to renew them )
2019-04-05

I know kops
is somewhat intentionally behind Kubernetes in releases, but it looks like they are only “stable” on 1.11, which technically went EOL when 1.14 went GA if I’m not mistaken. That seems like a really slow release cadence to me. Is it still the go-to for doing home-spun K8S in AWS? EKS isn’t keeping up either.

Yea I am not sure why it’s slowed

kube-aws is also worth checking out. @mumoshu is a maintainer.

That’s the one you mentioned two weeks ago that I didn’t write down. Will take a look. Thanks!

has anyone had any luck installing kiam on eks?

yeah I am using kiam on eks

how did you install it? I tried with with the helmfile in cloudposses repo, but no luck.

Im pretty sure it’s because I couldn’t run the kiam server on a master node since eks doesnt let you do that

I did a separate node-group for kiam-server that has the required credentials

i used the stable helm chart to install it

what do you mean node-group ?

add --kubelet-extra-args --node-labels=${name_of_node_group}
to your userdata

that lets you use a nodeSelector when deploying your pods


ah okay so do you have one node specifically for kiam? or do you run other pods on it as well?

In our case, we run kiam servers on masters

i have a few daemonsets like node exporter, but not any other applications

And agents on all other nodes


Oh right

Yea we use kops predominantly

The kiam-server node will have the ability to assume any role, so its best not to run anything else on it
2019-04-07

Re kops release schedule having slowed, anything to do with EKS perhaps? i.e. is pickup of kops slowing too?

I wonder… could be

Also the number of options available means there’s not as much support for anyone offering
2019-04-08


The Cloud Native Computing Foundation has formally accepted a container runtime designed specifically for Kubernetes as an incubation project.

how do you avoid merging maps in helm?

have a more concrete example? @btai

i think it might be the way the chart is written

@Erik Osterman (Cloud Posse) basically for this chart: https://github.com/helm/charts/blob/master/stable/drone/values.yaml#L153
Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

that DRONE_DATABASE_DATASOURCE
can be set as a postgres url i.e. <postgres://username:pw@hostname/dbname>

but I set that as an envSecret value to pull from a k8s secret: https://github.com/helm/charts/blob/master/stable/drone/values.yaml#L159
Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

but i guess when the values get merged for the deployment, the secret env vars get written first then the default env vars: https://github.com/helm/charts/blob/master/stable/drone/templates/deployment-server.yaml#L74
Curated applications for Kubernetes. Contribute to helm/charts development by creating an account on GitHub.

so the DRONE_DATABASE_DATASOURCE
value gets overriden by the default sqlite value

possibly just a poorly written helm chart?

or is there a way of ignoring those default values

let me know if that makes any sense..
2019-04-11

@btai you can run helm with debug/dryrun enabled - this should show you how the values are being generated which may help you work out what/why a variable value is not as expected
2019-04-13

Package manager for “kubectl plugins”. Contribute to kubernetes-sigs/krew development by creating an account on GitHub.

Kubernetes CLI plugin for syncing and executing local files in Pod on Kubernetes - ernoaapa/kubectl-warp

A better way to deploy Kubernetes Helm charts. Contribute to replicatedhq/ship development by creating an account on GitHub.


Contribute to replicatedhq/k8s-secret-generator development by creating an account on GitHub.

awesome idea! anytime you need a shared secret, generate it
2019-04-14

I use it, very useful : https://github.com/derailed/k9s
Kubernetes CLI To Manage Your Clusters In Style! - derailed/k9s

@Erik Osterman (Cloud Posse) You were able to convert @stobiewankenobi lolololol
Terraform -> SSM Then aws-ssm + helm + helmfile at deploy time
SSM is beastmode.. love using serverless.com pulling from there also
2019-04-19

hi all, anyone has experience with eks to assign a pool of static ip address/eni to pods based on the AWS high availability zone?

haven’t seen that done before; don’t know if it’s possible
2019-04-23

@Erik Osterman (Cloud Posse) (or anyone) I got a fundamental question for ya..
What’s the difference between:
- a canary deployment with 5% increments
- a rolling update with maxUnavailable==0 && maxSurge==5% (with a RR Load Balancer in front)

(no rush.. food for thought)

What I’m thinking about is.. can a native rolling update be used in place of a canary deployment.. Assuming they can monitor the same metrics for health

Because.. spinnaker / istio just for the sake of canary might not be reason enough

midnight thoughts
2019-04-24

@rms1000watt I often use this example when talking to customers that out of the box, kubernetes supports a very basic form of canary+rolling updates. The reason for istio is to have full control over that process. Also, the “gold standard” for canary deployments is to tie it into your monitoring backplane so that you proceed to increase the level of traffic only so long as some KPIs are true. This level of controlled rollouts are more difficult to orchestrate with kubernetes primitives, which is why Istio is used. Also, it doesn’t have to be limited to blue/green. It can be a full rainbow of colors, where the traffic is spread across them.


Makes a ton of sense

I think we’re all saying the same thing.. or at least on the same train of thought. It’s definitely a nice thing to have as business requirements expand (as they always do). Soon enough, it’ll become a required thing to have.
2019-04-25

what are y’all thoughts on 3 clusters/3az vs 1 cluster/3az? does the 3 cluster approach give us much more reliable availability?

for the 3 cluster approach either using federation or dns round robin

1 cluster, 3 az’s, multi-master setup

if you want more clusters, set those up in different regions instead with dns failover and/or geo/round robin load balancing between them

Today we launched a new open source project called k3OS. K3OS is a Linux distro built for the sole purpose of running Kubernetes clusters. Read more.

Anyone run tillerless helm? https://rimusz.net/tillerless-helm/
Helm really became a de-facto as Kubernetes Package Manager. Helm is the best way to find, share, and use software built for Kubernetes as it states on https://helm.sh. That’s true and sounds very cool. Since Helm v2, helm got a server part called The Tiller Server which is

that’s interesting!
2019-04-26

does someone have something to say about https://www.ovh.co.uk/kubernetes/ ?

Benefit from a free, managed and highly available Kubernetes® service to orchestrate your containerised applications in the OVH cloud Free hosted master nodes

Haven’t used that in particular… I used OVH back in the day with CoreOS. Loved the service for the value. Unlimited bandwidth and beefy bare metal instances.
2019-04-29

https://github.com/roboll/helmfile/issues/392#issuecomment-455065039 @Erik Osterman (Cloud Posse) I feel like I need to write some middleware for Helmfile so it can use SSM directly
Currently there are many users that want to integrate Vault with Kubernetes, but there are no high level tools for this. The current Kubernetes AuthMethod for Vault is too complex and coupled to th…

Deploy Kubernetes Helm Charts. Contribute to roboll/helmfile development by creating an account on GitHub.

lol think just right here

Has anyone found a good solution for user management on EKS? The best solution I can find is setting up roles that users can assume, but it doesn’t seem like an optimal solution.

Amazon EKS utilise IAM pour fournir l’authentification pour votre cluster Kubernetes via l’ authentificateur AWS IAM pour Kubernetes . À partir de la version Kubernetes 1.10, vous pouvez configurer le client kubectl normal afin qu’il utilise Amazon EKS en installant l’authentificateur AWS IAM pour Kubernetes et en modifiant votre fichier de configuration

really usefull to manage by IAM

Gravitational Teleport

But you still map k8s roles to saml roles

have you looked at Gravitational Teleport?

it supports integration with SSO

and audited session logs with replay

I haven’t, but I will take a look!

Thanks @Erik Osterman (Cloud Posse)

https://github.com/roboll/helmfile/pull/569 SSM integration with Helmfile. Lets see the lash-back. lol
There's been some interest for helmfile integration with SSM. Here is an example of what it can look like. For our current workflows, we have Bash scripts that export Env Vars via aws-env then …