#random (2019-02)
Non-work banter and water cooler conversation
A place for non-work-related flimflam, faffing, hodge-podge or jibber-jabber you’d prefer to keep out of more focused work-related channels.
Archive: https://archive.sweetops.com/random/
2019-02-01
ummm wat? please keep us posted
2019-02-02
2019-02-03
whatever happened in kubernetes.slack.com slack channel(s) right now…. wtf is wrong with some people
what happened?
spam on couple channels with NSFW content
started by someone doing @channel
wow, that’s horrible
there’s no good mechanism to prevent that in slack.
welcome to the woes of IRC moderation. hope we can stave that off for a while.
2019-02-04
2019-02-05
Handy in a day to day sort of way
2019-02-06
any body knows a method for get picture object contour (cars ,vehicles) to get somethig like this?
Vectorization?
yeah, like this https://github.com/Raj-08/tensorflow-object-contour-detection
do you need to do this programmatically or just a one-off
?
i do this all the time for one-offs
there are (a) some good ios apps (b) vectorizer for osx
programmatically, i try to make a car photo assistant app for help to the people get awesome pictures for the stock,
2019-02-07
2019-02-08
bumping into this stud muffin on the net
lol
it’s amazing how often this is happening to me now too
most recently in chamber
i think we have 3-4 members here opening PRs there
2019-02-09
2019-02-11
We love metabase
oh yeah, it’s one hell of a software
2019-02-12
It almost sounds like a bad joke.. But this resonates quite a bit with (fortunately short lived) past experiences.
fuuuuuuuck
Big reddit0r, will chime in on this one
2019-02-13
I would appreciate some feedback https://github.com/segmentio/chamber/pull/187
@pecigonzalo Nice!
Looks great, only think I could suggest is maybe some tests as some of the other commands have
Yeah, its on the todo, im waiting first for confirmation they want that change
Makes sense
I dont want to build the tests and S3 support and get a rejection
true dat
If you liked that, you should like this even better https://github.com/segmentio/chamber/pull/188
Nice
Have you seen https://github.com/cloudposse/tfenv ?
Yeah, i quite like it and actually it was out of the same frustration
I was going to use tfenv but was worth PRing chamber direcly as we use it in our workflow
just be cautious about writing secrets to disk
Yeah, that is why we do both, but you can also do
<(chamber export this secret -f dotenv)
to pass it as input
-f dotenv does not work
if your envs have new lines \n (e.g. TLS certs)
so I’ve had to use the sh -c 'export -p' pattern
they tend to linger around
which defeats the purposes/advantages of chamber
Wow “We do know they ask your browser to draw an invisible image”
yea, pretty insane
virtual machine uses their own language, which they encrypt twice.
language is decoded with a key that is changed by the process of reading the language, and the language also changes as it is read.
I was suspecting that they check the mouse movements like trying to get to the box is not a movement done in one go, but this…
yea, seems like they sample everything they can, encode it in an image, then use ML+AI to identify anomalies.
Almost everyone on the Internet uses something owned by Google – search, mail, ads, maps – and as you know Google Tracks All Of Your Things. When you click that checkbox, Google reviews your browser history to see if it looks convincingly human.
freggin crazy
so basically, clicking on the box is just a gimmick.
the second you land on that page, they basically know who you are, and if you do normal “human” stuff.
2019-02-14
and yet with all that, I still have to click boxes of fire hydrants, traffic lights and cross walks all day
Yeah I was thinking that. If it’s so smart then why do I keep having to prove myself
Unless I give off robotic tendencies
haha, yea, that’s true - forgot about that and how often that happens
now i’ll just take offense when I see it next time
rather than thinking I was just helping the greater good disambiguate images
https://medium.com/@stobiewankenobi/why-i-use-terraform-for-templating-kubernetes-a137f10bb98a
@stobiewankenobi LOLOL, let the drama unfold
haha
Haha biting my tounge
Screw that, let’s hear some criticism/feedback.
I’m curious how you avoid hard coding iam role arns in helm charts for kube2iam, or ACM arns for elb’s?
Sure thing! As soon as I get back to keyboard :-)
For example this: https://github.com/cloudposse/charts/blob/6e36a5bf1814838f6f52184851a655a82a59e136/incubator/kube2iam-kops/values.yaml#L6 expects you to just hard code a value. My thought is, use terraform to render values.yaml and pass it dynamically the arns.
Here’s why we need a tool like helm and why I don’t back any of the dozens of “kubernetes templating” approaches for deploying apps.
TL;DW: helm is not about templating
helm is about package management and defining an interface for that which is “configuratino management tool agnostic”
Yeah I agree with it’s model for those purposes
1000s of ways exist to templatize and deploy resources on k8s
but (1) thing exists today for keeping a registry of apps installed: helm
Yes
In my blog I talk about how you can use helm and my model together
terraform for k8s installation will work well within an organization
Because I see the value of helm for versioning/history
but not translate well across organizatinos
and terraform for writing values.yaml is insufficient b/c the lack of conditinoals
Can you share an example of what you’re talking about with that?
dozens of examples: https://github.com/cloudposse/helmfiles/tree/master/releases
That’s not a values.yaml file?
that’s helm templating is it not?
yep
I see
Terraform 0.12 will do all of this
But sure, I get the value today of the complex templating if you have need for it
Question for you, the biggest thing I was trying to solve/annoyance I have, is how everything is hardcoded
IAM role arns for kube2iam, acm arns, etc…
how do you get around hard coding values?
so terraform 0.12 + conditional templating of values will make it much more appealing as a way to install apps using helm
that said, helmfile is still better
helmfile diff -> like terraform plan
so the “terraform interface” for installing helm releases in 0.12 will look nice. then there are some implementation things I’d like to see before considering it.
there’s also the argument to be had that helm is like dpkg or rpm; we need a tool like apt-gt or yum that sits on top of it. terraform is not that tool IMO.
I agree with that statement
But I come back to my biggest gripe (not jsut with helm but with all templating tools not terraform)
Question for you, the biggest thing I was trying to solve/annoyance I have, is how everything is hardcoded
IAM role arns for kube2iam, acm arns, etc…
how do you get around hard coding values?
ENVs
so i think your gripe is with the stastic nature of values.yaml
(btw, recommend not using kube2iam and instead moving to kiam)
kube2iam will DoS AWS APIs and get them to block your account
Ah I will look at kiam
good to know, I haven’t had the issue with kube2iam yet, what scale do you hit that? We have around 10,000 pods atm
here’s how we parameterize the iam roles
Even with external-iam-role, where are you getting that value?
if you have one pod that specifies the wrong role, kube2iam will keep hitting that
if you have lots of machines, it will keep retrying since every machine keeps it’s own cache
Ah..but I never have that, since it’s always templated with remote state from tf
Still great to know about the DOS
I will look at kiam
we’ve encountered it. very nasty. encountered it with multiple customers across accounts.
this explains all the problems quite well
also, the security architecture of kube2iam is much less desireable
basically, every node that needs to assume roles has at the host level admin permissions
Awesome, thanks for the refs. I will read that and checkout kiam
with kiam it’s a agent/server model
and then check out our helmfiles for installing it
Back again to the hard coding, even with env vars, where are you getting the env vars from?
chamber
SSM
so you can write those parameters to SSM via terraform
(don’t get me wrong, I love the tight interconnectedness inside of the terraform ecosystem)
So you’re taking terraform -> ssm -> env var -> helm values.yaml
just we don’t have the privileges
Why not just terraform -> values.yaml?
because of conditionals
Ah OK
Got it
(and because we started before the helm provider!)
For me, I haven’t had issues with complex conditionals with k8s configs, but you guys build more than I do for more people in a more unique per case basis.
But I see the value there
terraform+helm w/ (future) better values templating will eventually be more tempting
Awesome
Really excellent to hear more about this
Interesting how you guys do terraform -> ssm -> env var
That’s an interesting model.
hehe, sorry, I preach a lot - part of the business
I had wondered how others were avoiding hard coding
and have a lot of strong opinions required to guide a ship through stormy waters.
Yeah I get it dude, I have to do the same thing.
I love the conversation and debate!
yep! that was a good session.
Thanks dude!
I like the pattern with SSM, it’s kind of like terraform remote state, but feels more granular
i like that it’s interoperable with other tools
via chamber
Bonus!
so we’re using terraform to write a lot of settings for kops to SSM
then we call chamber exec kops -- kops ....
I started using s3 as a key/value store for similar reasons a while back
with chamber?
…we’re about to try that
No, just where ssm was not an option… A way to store values and make them available easily while controlling access tightly
it’s nice they added support for S3
People think of s3 as a place for storing files, but it’s all blobs all the way down. Can write most anything to a key
I’d not be surprised if ssm were just a specialized frontend for s3
That’s interesting @loren
I have also done that use case
I thought I was weird for doing that
You just made my day
PRE SSM Param Store/Secret Manager
hehe, i think many of us did
we did that, and then used goofys to mount S3 as a filesystem
(actually still have support for that in geodesic)
OH man I never mounted s3 as a fs
too slow the 1 time I tried it
it’s come a long way
though this is not for databases
it’s great for simple configs
Sure.
Huh, good to know.
Nice
love the name too
at one point, i created a module for storing an arbitrary map of keys/values in s3 but never published it… would use it to store both inputs and outputs so they’re queryable outside terraform
variable "create_keystore" {
description = "Controls whether to create the keystore"
default = true
}
variable "bucket_name" {
description = "Name of the keystore S3 bucket, must already exist"
type = "string"
default = ""
}
variable "key_value_map" {
description = "Map of S3 keys and values"
type = "map"
default = {}
}
variable "tags" {
description = "A map of tags to add to the S3 objects"
type = "map"
default = {}
}
locals {
keys = "${keys(var.key_value_map)}"
}
resource "aws_s3_bucket_object" "this" {
count = "${var.create_keystore ? length(local.keys) : 0}"
bucket = "${var.bucket_name}"
key = "${local.keys[count.index]}"
content = "${jsonencode(var.key_value_map[local.keys[count.index]])}"
content_type = "application/json"
tags = "${var.tags}"
}
of course limitations of tf <0.12 results in fun resource cycles on the keys, which is why i never published it
Will 0.12 really fix the count of problems? Guess I am not getting my hopes up
I know they’ve said it will make improvements at least, where it should know the number of elements in the count when the plan is generated
set the channel description: A place for non-work-related flimflam, faffing, hodge-podge or jibber-jabber you’d prefer to keep out of more focused work-related channels. Archive: https://archive.sweetops.com/random/
2019-02-15
hmm, using goofys to mount an s3 bucket to store configs is a good idea.
currently we’re using a persistent ebs volume. this works well for a single instance you want to scale up or down. but breaks as soon as you want to run > 1 instance reading from the config.
EFS is also a good candidate for this.
yay
But MAciej this is not funny man, random should be at least a bit funny
Anyone have any recommendations for a mac system-wide microphone mute in the status bar and/or touch bar? I see shush on the app store but it hasnt been updated in years
@Nikola Velkovski
@wannafly37 maybe it just works
@wannafly37 you can create a simple apple script which does what you need (at least for a user that’s executing it):
tell application "System Events" to set volume input volume 0
volume is an int between 0-100
Yea I thought about that but would really like some sort of visual indicator as to what that volume is currently set to
have you tried https://www.rogueamoeba.com/soundsource/
That looks useful - I’ll try it thanks. Just tried https://github.com/pixel-point/mute-me and it actually seems to work OK - but 2.0 is still RC and I use zoom a lot - I’ll give it a week or so
Anyone using Travis for deploying microservices ? I can’t seem to find global context environment variables. Would be interested in best practices! Cheers.
Yea, you can’t (we’ve been using travis for our terraform modules for years and have not seen this feature implemented)
#codefresh has “Shared Configurations” that achieve this
I’ve never used codefresh but I’ve been hearing really good things recently
Definitely worth checking out if you’re looking for alternatives
HA Jenkins (open source) no fun
Working with Jenkins feels like traveling back in time at least 10 years
I’m hoping to start looking for an alternative soon-ish (probably 2019 Q2)
Jenkins X is a tad nicer, but yes, it still lacks the UX improvements of other CI/CD platforms
@mrhen what options are you considering?
Haven’t looked at options in depth. Our pipeline is currently GitHub -> Jenkins -> ECR / ECS. Promoting between ECS clusters is managed by an internal tool.
What we need is a clean way to run tests through docker-compose after pulling from a GitHub PR.
But keeping the Jenkins instance up and happy with Docker is getting old. I don’t like having a non-managed instance out there.
@mrhen you can leverage jenkins git config plugin and spawn a new instance everytime a PR is being processed
so you don’t have to run jenkins all the time
Yeah, we could probably fix / improve our Jenkins setup but if we go through that effort I’m going to also look at alternatives. Frankly, nothing about Jenkins feels like it’s “helping” our pipeline. The only thing we really use it for is running docker-compose on a PR. It seems like there should be a simpler way to do manage that piece of the puzzle.
since you’re using ECR/ECS, have you tried codebuild?
No, but it’s on our radar. Have you used it?
Quite a bit, though mostly with CodeCommit. They do, now (not when we started), have a native integration with GitHub to build on PRs
Are you happy with it?
Well, it gets the job done, without needing to run/operate any servers. The integration with CodeCommit is dramatically lacking
We wrote our own integration/module to get CI for CodeCommit, https://github.com/plus3it/terraform-aws-codecommit-flow-ci
Ah, interesting
Is CodeCommit more or less a replacement for GitHub?
Yeah, but just the super basic source code repository bits. No status checks, no search, no real integrations, no forks(!)
Ah, boo
I wouldn’t use it if this customer had the ability to shell out for private GitHub repos in an org
Yeah, we are happy enough with GitHub
We used to use Jenkins for more stuff but at this point we’ve farmed off everything non-test related to other services
And the docker containers build and run tests themselves
I’d say give CodeBuild a try, at least. I think of it as a way to automatically run commands in a shell on some remote host, triggered by some kind of event.
Cool; thx for the recommendation
You get to define the commands to run in your buildspec, which you can keep in a file in your repo, or just in the definition of the CodeBuild project (not unlike jenkins)
2019-02-16
Thanks, but after 28th of February
2019-02-18
Gotta love Google eh. Made it sound like they landgrabbed .dev to protect their own and everyone elses use of .dev as an internal domain for testing. Let the dust settle Grab for cash
hehe
Wonder how much will break because of people’s /etc/hosts having .dev in them
Hmm do people still update their hosts file ? I really don’t think it’s a good practice.
Yes, and agreed
2019-02-19
Plus $11000 to make sure someonw doesn’t nab your domains seems like extortion to me
yeah that sucks big time.
2019-02-20
damn
I paid 30
Learn DevOps in 7 days
if only i had known
32019-02-21
Crazy pile up
I saw this some time ago, really scary
Anyone listening https://hashicorpalldayhashitalks.splashthat.com/ ?
@Nikola Velkovski I do follow this closely, yes.
cool
so what happened with the last talk I guess they had one device pointed at another
Hard to explain but there are so many audio sources you can adjust to make it to work well. As we can hear @andrey.a.devyatkin figured out the way to make it to work and rocking right now!
yup pretty good stuff
https://docs.google.com/forms/d/e/1FAIpQLSdnEEo0o2JgnIt8VOGffhkcYj-C2h9m5_NFzM0Q1AU-P8d0zA/viewform - enjoy My result was 4/20
Oh no, a colleague of mine went to work for them before the acquisition
just skip the clickbitty headline
2019-02-22
Hi Everyone, I’m looking for a type of Daemon, which can collect shadowed http traffic, pulls a unique set of URI’s from what’s collected every hour or so, and replays it to a different webserver ( for cache population ).
https://github.com/buger/goreplay looks alright
Yes, this is the one I’ve used a while ago for similar reason (it was a-la canary-deployments)
Goreplay is awesome, have used before
Go replay is nice; or nginx has a mirror config now
Its a bit of a weird mission. I need to replay regular http traffic to ‘Rendertron’, a tool which converts dynamic html to static html with a headless chrome, for google search. As this is quite slow, I have caching in front of it, and I’m looking for a way to dynamically populate the cache by shadowing regular traffic ( Envoy ) to something what collects the requests, creates a unique set of URI’s, and triggers the same ‘Rendertron’.
If delay is not an issue you can get GET urls from access-logs and process them async.
Yep came to that thought It’s in ElasticSearch so, let’s use it
Yea, that’s also a good trick
Just set my status with a Mexican flag - random indeed, but hey I am using the #random channel
2019-02-23
hey guys is any of you at the kubeday meetup today in Sunnyvale ? would love to meet you guys IRL ! raise your hand if so
That would be great!
(I am based in #lax so can’t make it… trying to think who here is based near there)
@OScar is up in NorCal
San Francisco Bay Area, east bay. Yeah was partly raised here :)
Unfortunately I’m skiing up in Tahoe, or…fortunately!
Jealous
Must be amazing up there
Damn. I haven’t been skiing for years now I’m jealous too :)
I’ve never been skiing. I’m not too sure my knees could handle it lol
It took me about 2 years to learn but once you got a handle on it, it’s amazing! Yeah I’m 48 now so gotta stay healthy in order to do this so I have a strict diet and fitness regimen. I’m trying to do this until I’m 80 at least :)
If any of you are ever planning a Tahoe trip, I welcome you on my place and give you a tour of the mountain and make sure you get the best experience!
Love Tahoe
if you make it up here this Winter, holler! I promise to ensure you get the best experience up here!
Not in the cards this winter, unfortunately but thanks!
@OScar Sac here ( well El Dorado HIlls ). Go up to Tahoe often.
@ryangolfs: I’m up every weekend since my daughter is in the race team Tahoe league so I use it as an excuse to ski lots. Let me know when you’re planning on being up, our mountain is Alpine Meadows!
2019-02-24
wow…it’s been a good winter season so far. This is coming from a vermonter who cant code whilst there’s an inch of the good fluffy stuff outside
@drexler are you also in the bay area?
nope. Over in VT.
Vermont, is on my bucket list @drexler!
ditto
2019-02-25
Click
neat
no commits since 2017 because it’s “done”, or because it’s “DONE”…
Hrmmm good point
AT&T offers “fiber” in Pasadena, CA, but they cap it to 25mbit, which is like flashback to Y2K. Now, almost 20 years later I can’t believe that my LTE phone has 4x faster internet than “fiber”. It’s frankly embarrassing how slow our internet is here, but not because of technological limits; it’s due to greed.
wut
I just called AT&T to upgrade my internet. Was so excited to get fiber in my area.
Just was flabbergasted when they said that the max speed for our building is 25mbit. Currently we have cable and get 150mbit.
Where are you?
washdc metro, verizon fios
ya
I had Fios before too. that was ncie
It just irks me when they advertise “fiber” and then sell you only 25mbit.
yeah, that’s robbery
saw something about project fi getting 5G shortly, via the sprint provider
i’m in the process of moving everything over to Fi
very happy with it
had for years now, got my SO and her brother on it
blocker for me was iOS support
tablets, phones, etc
highly recommend the FiSwitch app… at least around here it likes to pin to Sprint, even when the connection is awful. force it to tmobile and bam
oh, iphone… not sure the dual-radio works there yet?
yea, we don’t get max advantage
but we do get tmo
which sounds like that’s what I want anyways
just depends on coverage for each network, some places sprint is fine/better, others tmobile
being able to switch is really killer
overseas coverage is great, too. no extra costs, no extra phone/sim, just use your phone as usual
Should come to Australia - I can get 2mbit ADSL which drops out all the time, or 10mbit LTE that usually drops down to 1mbit in peak hours
haha - hopefully have 50mbit FTTN in the next few months
but it’s been “coming soon” for years
Funny how in New Zealand, which is insanely far from just about everything, I can easily get 700-900 Mbit fibre (about 110 NZD). Never needed that much though.
1Gbps for “free” (included in the rent in our building). Come to Sweden!
a colleague of mine who just moved out of Pasadena to Downtown LA had a cable connection (I don’t know from which provider) which was giving him a fiber 100Mbps, because he moved cities the same ISP offers him 400Mbps for the price of the internet he was paying in Pasadena.
2019-02-26
2019-02-27
2019-02-28
They talked about the main rancher guy working on this mid last year; sounded like a real labour of love. Pretty neat
selling domains broke google domains
haha