#kubernetes (2018-12)
Archive: https://archive.sweetops.com/kubernetes/
2018-12-03
K8s API server cve https://groups.google.com/forum/m/#!topic/kubernetes-announce/GVllWCg6L88
2
More details
@here upgrade Kubernetes ASAP. remote root capabilities are believed possible.
2018-12-05
https://news.ycombinator.com/item?id=18612571 gravitational guys have reproduced the kubernetes exploit
Any feels on EKS yet?
We’re still not doing much with it
hey @patrickleet! just connected you with @davidvasandani in #terraform
2018-12-06
2018-12-07
what a nightmare
I’m sure theres people out there not realizing their clusters exposed
Created a #kubecon channel for those going. Would be create if we could have some members connect.
2018-12-11
yea, that’s neat that it works together with helm
looks like it introduces a new primitive to manage the process, so it should work well with other CI/CD systems (e.g. #codefresh)
@dustinvb
anyone read this?
The article? Yeah it looks great. Can’t wait to try it out.
llol sorry i meant the book i posted below!
the article looks awesome too tho!
haha
i haven’t read this one but i the quality of the “in action” series… go in action taught me everything i know about go
2018-12-12
ksonnet looks really interesting, trying to uncover what advantages this has over go templates
what is the standard people are using to deploy kubernetes across the board? KOPS or somthing?
we use mostly kops since it provides a complete solution with in place cluster updates/upgrades
but we also have EKS modules
EKS currently lacks some features and requires blue/green deployment with ASGs to perform updates/upgrades
yea just trying to get some more hands on experience with it!
Yea, so there’s no one tool that I would recommend
using one tool for all clouds, means you only get the lowest-common-denominator
many of the clouds offer some features to differentiate themselves
so for google, I’d recommend GKE all the way and using gcloud on the command line
vs, on AWS I think kops is still the best option
wow
And using github to publish the roadmap
that’s forward
I created a #kops channel for specific chatter around that.
I was all… YAAAAAAY
and then AWWWWWW
Oops, swing and a miss!
2018-12-13
@Daren how are you deploying Argo?
Yes
I saw it looked like it was 3 charts
Do you deploy all of them?
Lukasz manages it, invite him in and I’m sure he will share
He has worked closely with them and contributed back
@wookasz ^
@wookasz has joined the channel
hey @wookasz! i’m curious about deploying argo for one of our clients
We do deploy it with helm using codefresh. It’s not helmfile (yet) so we have a separate overrides file and separate codefresh step for each environment
Are you using the official chart(s)?
almost, we actually have a fork which adds ability to forgo deploying the UI container, we weren’t using the UI and the image had some critical vulnerabilities so we decided to get rid of it for the time being, just haven’t gotten around to contributing that change back
It seems like it’s a few pieces now… a CI service and CD services and something else (forget what… maybe UI?)
each one has their own chart
are you installing all 3?
we only use the argo one
though argo-events is something i want to look into the future, it combines argo workflows with the ability to trigger them based on kafka messages, http webhooks, and cron schedules
we currently deploy cronjobs to trigger argo workflows so argo-events would be a way to simplify that a bit
ok, fancy
ohhhhhhh
interesting. so out of the box argo doesn’t have the scheduling
it does not
and argo-events didn’t exist when we stared using argo
this is where you use argo-events or CronJob resources that curl a webhook or something?
the cronjobs schedule a simple container which has the argo cli installed and just call argo submit [parameters]
the biggest pain with that approach is that we have to build a container for each workflow since the workflow spec needs to be available to argo submit
hrmmmm yea, I can see that being a bit of a pain.
@daveyu shared this with me: https://brigade.sh/
Was this available when you guys did your research?
not sure if it’s a comparable solution.
for some reason brigade sounds familiar, but i can’t remember to a year and a half ago
the other issue we came across is that workflow resources are never deleted
which caused issues with etcd and kube state metrics
so we run a job to clean them up periodically
the argo team is working on adding a ttl to workflows though
out of curiosity, what’s the use case you’re looking at argo for? if you can share
yes - so one of our clients needs to do some scraping and ETL
would like a tool to manage and visualize the process
(without cough jenkins)
Cool seems like it would be a pretty good fit, especially if they’re scraping from multiple sources and hoping to re-use pieces of the pipeline. If they’re loading into S3 or a warehouse that has the ability to copy from files in S3 then they have the added benefit that argo does a really nice job of making writing to S3 as simple as configuring which output files you want written there.
was wondering if you could point me in the right direction
(are you using a helmfile by anychance?)
Yes
This makes me really worried to touch EKS for anything close to mission critical https://github.com/awsdocs/amazon-eks-user-guide/issues/17
they are working on that https://github.com/aws/containers-roadmap/issues/57
Yeah this was the post specifically got me . We’ve been talking about attaching an EKS cluster/vpc to prod here and moving things like Jenkins slaves and Prometheus as a first step but that is just scary.
yea + NLBs are unstable
always fun knowing in the back of your head the cluster might completely stop working at any time@
This all got me thinking kops still the way to go for AWS at this time :(. Maybe give it a year or so and revisit eks
doh - missed the part about “users reported on kops as well” thanks for sharing that @patrickleet and great find..
didn’t find it on purpose - my NLB stopped working! luckily someone in kubernetes slack had gone through the effort of figuring out why
2018-12-14
2018-12-16
Interesting.
@mumoshu does a great write up of all the tradeoffs with both helm-secrets and chamber
while it sucks to jump ship to aws secrets manager, it might be worth it
I’m not comfortable with secrets at present because they are in the clear. I attended a kubecon session on security, they talked about the encryption providers: https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/
Just need an AWS kms implementation to pop up
I’m less worried about the ci ability to decrypt than he is
i like the separation of concerns
but agree that the KMS provider you mention looks great
hadn’t heard of it
I could only find a poc by awslabs for kms
@Daren
this looks like the best solution so far
@mumoshu has joined the channel
that’s pretty sweet
like hightower said a couple weeks ago, “kubectl is the new ssh”
2018-12-17
2018-12-19
anyone take the k8 exam on here?
Which one?
cka
I had a call with one of the guys at KubeCost - they are looking for beta testers
If you missed the post, check this out
With kubecost you can see how much things running on kubernetes are costing you backed out to your AWS spend
know the cost of any labels
know how much a namespace is costing
etc.
let me know if your interested
