#general (2020-11)
General conversations related to DevOps/Automation
General Discussions
2020-11-01
Hey everyone, give a warm welcome to our newest members!
- @Mark Williams
Good to have you here =)
Interesting to see this. I haven’t thought about unikernels since doing a show on them with the IncludeOS folks. https://packetpushers.net/podcast/datanauts-114-unikernels-includeos/
Adding a 2014 post I wrote on the same topic that has some additional links and thoughts https://articles.microservices.com/after-docker-unikernels-and-immutable-infrastructure-93d5a91c849e
2020-11-02
the same tech that existed before docker still exists
open source cunts don’t know know how to market. things
(mind rephrasing?)
apologies
Hey everyone, give a warm welcome to our newest members!
- @Teja Goud Talla
Good to have you here =)
Hi everyone, I’m doing a master thesis on cloud computing adoption and I’d be grateful if anyone will answer this survey//managementism.eu.qualtrics.com/jfe/form/SV_9uXPZCLvOPWM1xP> It should take 10-15 minutes and there’s a chance to win amazon gift card Thanks
2020-11-03
Hey everyone, give a warm welcome to our newest members!
- @Esteban Guerrero
- @Hilal
- @Borys Borysenko
- @uselessuseofcat
- @bhavin vyas
- @Syn Romana
- @Sebastian Borrajo
- @Curtis Ruck
Good to have you here =)
1
2020-11-04
@Erik Osterman (Cloud Posse) had some great results in my initial tests with go-task… here’s a link to my 101 walk through experience. https://bit.ly/363K7AC. thanks for the recommendation!
Cool - nice write up
gotask is very similar to variant (v1 not v2)
I thought it served a different purpose?
I did try variant 2. What I didn’t find was it being as intuitive. When we discussed I think you said the goal wasn’t necessarily to replace Make/InvokeBuild but instead to provide a common grounds for cli interfaces?
If so would like to hear more sometime
Nice post. Would be helpful if you linked to the tools mentioned, assuming I’m not just blind and missed the links.
i noticed it too. The link was there but it wasn’t highlighted with the theme very clearly. I’ll update that soon. appreciate it!
Variant serves exactly the same purpose as gotask :-)
Starting with gotask is great though, then once that clicks variant and variant2 are more advanced tools that do the same thing
Variant1 is yaml based like gotask and variant2 is a complete rewrite that uses HCL
Okay I guess I got confused. In our last thread on this it seemed that the conclusion was that variant to wasn’t meant to replace things like make files…. Only be a common interface for launching those. I did actually try variant too and experiment with it but it didn’t fit my needs. Are you saying that in the future you may replace make files and local scripts with variant files?
Gotask can call makefiles too :-) that was just an example. Let’s say you have a bunch of invoke build scripts and makefiles and ansible scripts and terraform commands and helm commands and Helmfiles… you can stick all those in gotask
Just the same way you could stick them in variant
Why wouldn’t I just put those commands in go task itself?
Exactly - you can or you cannot - up to you
It’s a task runner
You can rewrite all those commands natively in gotask or you can just call those scripts or make targets - up to you
More duct tape!
Variant is identical in purpose to gotask. The difference is that variant is on steroids. It might not become evident until you try both
Here is an example in variant 1 yaml
This helps call the commands to setup or upgrade tiller
I mentioned I did try variant2. I’ve had exposure to both of them. I don’t have the use case of combining multiple task runners like make files plus wanting to wrap those up. I just want one interface. Sounds like a misunderstood what you were saying. I’ll definitely keep my eye on variant2 and revaluate for more complex scenarios in the future.
Variant can be invoked by running the “variant” command just like the “gotask” command, but it also supports #! style - which is where it becomes so cool.
Ya we want one interface - exactly
So I totally missed the mark then in explanation I gave you the first time month ago :-)
I think the makefile example Erik makes is that you could wrap an existing makefile and add additional stuff and then end up with a single way to interact with all of it.
The benefit there would be getting to a single interface more quickly. Then when you have time you can refactor to just do what the makefile does in your variant yaml/hcl.
1
Have anybody dealt with BigSwitchFabric?
Hey everyone, give a warm welcome to our newest members!
- @shahab-t
- @Wesley Chiang
- @Mario Dagrada
- @Cody Halovich
- @tristan
- @Marcos
Good to have you here =)
22020-11-05
Thanks guys, I’m jumping into a geodesic setup that already exists, I have no previous knowledge of this project, I may have questions in the future.
Cool! Happy to help. See #geodesic
Cheers, thanks @Erik Osterman (Cloud Posse), and Happy Birthday
Thanks Cody!
Hey everyone, give a warm welcome to our newest members!
- @Thomas Bergmann
Good to have you here =)
2020-11-06
Hey everyone, give a warm welcome to our newest members!
- @Strahinja Tasic
- @V M
Good to have you here =)
2020-11-07
Thanks
Hey everyone, give a warm welcome to our newest members!
- @Pankaj Pratap Singh
- @Ashish Agrawal
Good to have you here =)
Thanks @sweetops556
2020-11-08
Hey everyone, give a warm welcome to our newest members!
- @Victor Barros
Good to have you here =)
2020-11-09
Good Morning All!. Happy Monday.. COVID-19 is raging and the world is on the brink of Historic Firsts.. Which is good.. The Creeps running the world , for the moment. .will soon be kicked to the curb. We have proven, that foundational change is possible, if we stick together.
This message was deleted.
Hey everyone, give a warm welcome to our newest members!
- @svpetd
- @Eliran Wolff
- @labardinim
- @Abhi
- @CK
- @Mykola
Good to have you here =)
Nice to be here
Thanks @sweetops556 ! Glad to be part of the community!
2020-11-10
Hey everyone, give a warm welcome to our newest members!
- @melissa Jenner
- @Anthony Voutas
- @Dan
- @Flávio Moringa
- @Perry Luo
Good to have you here =)
22020-11-11
I have searched through the archives and google is not helping either… so hopefully this isn’t described somewhere I have just missed. I am working on setting up a new infra utilizing cloudposse OSS but have bumped into a few issued. I got all the accounts setup and that is working great, also got EKS and other application backing services up (eg RDS) all using terraform… but I am having troubles finding out how to make the next steps through IaC.
In particular I am trying to get Codefresh setup for GitOps but it is not clear how to do this as IaC. I got my Codefresh account setup (using Github to auth) but to add the K8s EKS integration for example I used the helm chart (cloudposse-incubator/codefresh-service-account ). But I did this by hand which seem wrong. Then to setup the project and pipelines I can only find how to do this by hand.
I am working through the cloudposse/codefresh repo but this seems to be make based, not terraform. So it is not clear how this would be applied using gitops. So I come into the same problem as with the helm chart above.
Does anyone have information about how to do these programmatically? Is just doesn’t seem correct to have these be done by hand.
Hey everyone, give a warm welcome to our newest members!
- @aravind.kandalam498
- @Dmitri Kaminin
- @savadev
- @EvanG
- @SC - SAIC
Good to have you here =)
Benvenuto Tutti!
2020-11-12
Hey everyone, give a warm welcome to our newest members!
- @Jeremy Branham
- @Derek Nordgren
- @Calman Steynberg
Good to have you here =)
52020-11-13
Hey! I’m working on preview environments implementation with helmfile and found a very useful conversation on https://archive.sweetops.com/. I’ve never seen this approach before (to put Slack’s archive to the web). Just want to say thank you to people who made this. This is brilliant and super cool.
Hey everyone, give a warm welcome to our newest members!
- @kaziqta
- @RICHU SHAJI
- @Andrew Cockerill
- @Francesco Ciocchetti
- @hashan perera
- @Gus Narea
- @Andrey Bronin
- @Tristan Juricek
Good to have you here =)
Benvenuto stay safe!
2020-11-14
Azure Terraform, anyone else who supports that Hashicorp needs to NERD up it’s support for Azure TF.
Hey everyone, give a warm welcome to our newest members!
- @Arjun Venkatesh
Good to have you here =)
2020-11-15
Besides the Phoenix project and the Google sre book, does anyone have any good sre/devops book recommendations?
Can you suggest Phoenix project to an experience DevOps engineer? Is it useful or more for fun?
I’m pretty experienced. Been doing sre stuff for a few years. Everytime i read a book on this stuff, i personally think differently
There’s a lot to understand in that book. It starts off easy and then gets pretty dense! I read The Goal too, which is what The Phoenix Project is based on - that’s even more dense Definitely recommended though.
I haven’t made time to read lately but I am planning to pick up books from Jez Humble, Nicole Forsgren among others
a number of folks previously recommended “scrum: the art of doing twice the work in half the time”
Unicorn Project
@Joe Niland I agree, “The Goal” is great. Note: It’s not related with SRE directly nor with IT.
Following on from the above question – very few SRE resources talk about how disaster recovery work. Mostly they assume you are Google scale, and you don’t need explicit disaster recovery because you are big enough for all your applications to run in a globally-distributed, fault-tolerant manner. But I work for smaller businesses and need to think about “what options do we have for cross-region DR?” and “how much do we want to spend?” and “how do we maintain this DR process?” Any suggestions on how to look at this from an SRE perspective?
i like to start with breaking it down into dependencies and state. dependencies are all the services and software you need to deploy/run your system. state is all the data you need to backup/restore to run your system. that’s generally the easy part, really, but it helps frame the problem. i.e. are all the dependencies available in primary and DR regions? do you have backups of state? do you test restores from state? can you deploy using a restored state? etc…
for the how much you want to spend - see if you can get that back to a “cost of loss of service” metric - that way you can look at the risk vs the cost and determine how much you want to spend to mitigate that risk
@Alex Jurkiewicz: you can start with some high level estimates. double the cost of your current environment, but scale down the remote DR side (run two instead of 12 web servers for example, run on slightly smaller instances to maintain replication). Do this for production AND staging. Double the cost of file & DB storage, snapshot storage, S3, etc.. That’ll give you a rough idea of infrastructure costs. More or less depending on how thing you want to provision the remote side. You can look at things like EFS (filesystem replication), S3 replication, or Aurora cross-region. Those might increase your costs a bit if not already running them.
Work with your business side of the house to calculate risk/cost and decide on a target RPO / RTO, then figure in the engineering resources to implement your targets.
The least data loss & service downtime will be the most expensive to implement (requires live replication vs. 24-hour snapshots for persisted data). Additionally, will your application or services need to communicate across regions (failover testing, planned failover, etc.)? If so, you will need to test and potentially modify the application to avoid stacking latency (multiple round-trip API calls in serial). You might do this if you have lots of applications / microservices and need to test / migrate them independently. It is simpler if you have a 24 hour RPO, just start new services on remote DR side and flip DNS.
If you need lower RPO, or seamless DR, be sure to practice it on a regular basis. Infrastructure changes, configurations change, whereas your documentation might not. Create run books and checklists, and verify they work. Schedule regular downtime for it if possible (annual, twice a year, or quarterly, depending on how low your RTO/RPO is).
Is there helm repo for https://github.com/cloudposse/prometheus-to-cloudwatch ? Looks like there isn’t. and I need to clone this repo by myself to be able to use from terraform helm provider?
Until the chart is available in helm repo, I do something like this to put the chart in my own git repo, so that terraform can read its content… git remote add prometheus-to-cloudwatch <https://github.com/cloudposse/prometheus-to-cloudwatch.git> git subtree add --prefix=prometheus-to-cloudwatch prometheus-to-cloudwatch 0.14.0 --squash
2020-11-16
I needed to follow some extra steps to make this work…
Mostly same as https://github.com/cloudposse/prometheus-to-cloudwatch/issues/10#issuecomment-446738814 , but since https://kubernetes-charts-incubator.storage.googleapis.com/ is no longer available, I changed repo url to new one.
cd chart
helm repo add common <https://charts.helm.sh/incubator>
helm dependency update
Well, this was still not enough. the chart uses outdated api version, I needed to change it to apps/v1
https://github.com/cloudposse/prometheus-to-cloudwatch/blob/14ca4c036f4908aff8490af2e9ed0a1de0e497db/chart/templates/deployment.yaml#L1
This message was deleted.
2020-11-17
Hey everyone, give a warm welcome to our newest members!
- @Zach H
- @Chris Kaminski
- @David
Good to have you here =)
33
Wilkommen!
2020-11-18
anyone have recommendations on how to find a good mentor for the whole SRE and devops world ? or maybe even tech in general ?
we just published a segment from yesterday’s office hours about your question, not sure if you were able to listen in but hope this helps!
Hi Everyone, I’m doing research on Cloud Computing adoption if you could spend maximum 10 minutes to fill this survey that would be awesome! https://managementism.eu.qualtrics.com/jfe/form/SV_9uXPZCLvOPWM1xP You have a chance to win Amazon Gift card too !
Hey everyone, give a warm welcome to our newest members!
- @Nlesh Deo
- @cflowe
- @V
- @nbrys
- @froch
- @Guy Eisenkot
Good to have you here =)
22020-11-19
Hey everyone, give a warm welcome to our newest members!
- @Hao Ke
- @Emily Melhuish
- @John K
- @Lee Van Steerthem
- @Antoine Taillefer
- @Tyler Riti
Good to have you here =)
52020-11-20
Hey everyone, give a warm welcome to our newest members!
- @Ray Smets
- @nicholas
- @Carlos Vences
Good to have you here =)
thank you!
2020-11-21
Hey everyone, give a warm welcome to our newest members!
- @igor.zecevic.89
- @luis192
- @Sergey Lysenko
- @wwoll14
Good to have you here =)
2020-11-22
Hey everyone, give a warm welcome to our newest members!
- @Stevo Slavić
- @pwatts
Good to have you here =)
2020-11-23
Hey everyone, give a warm welcome to our newest members!
- @Dominic Martino
- @ohad
Good to have you here =)
2020-11-24
Hiya! do you have some comparison about using GitHubActions or GiLab pipeline as CICD?
Hey everyone, give a warm welcome to our newest members!
- @Christopher Soto
- @joelleigh
- @Joan Porta
- @Juha Patrikainen
- @Tom Dugan
- @Bartosz Zawadzki
- @matthew wuenschel
- @David de Lucca
- @omry
Good to have you here =)
need to decommission 600 servers, running an app , need to generate a ticket after each server decommission, send ticket to service ServiceNow
Write a program
2020-11-25
Hey everyone, give a warm welcome to our newest members!
- @Cristian (cloudutil.io)
- @Tahir Hafiz
- @tim.davis.instinct
- @Josh Jaffe
- @JohnVal
Good to have you here =)
2
wave
12020-11-26
Hey everyone, give a warm welcome to our newest members!
- @Matt Simonsen
- @NJ
- @michaelssingh
Good to have you here =)
1
Thanks
This message was deleted.
2020-11-27
Hey everyone, give a warm welcome to our newest members!
- @Alex Chandra
- @Greg Rynkowski
- @JJ
Good to have you here =)
22020-11-28
Hey everyone, give a warm welcome to our newest members!
- @James Kinsman
- @Adam Klein
Good to have you here =)
12020-11-29
Hey everyone, give a warm welcome to our newest members!
- @Justin D
- @Zbigniew Żołnierowicz
- @Gordon Clark
Good to have you here =)
12020-11-30
hi, what’s the best channel to ask about kubergrunt, please?
Hey @Andrea - you’re right, no specific channel for it right now.
Can maybe try #terragrunt or #kubernetes
thanks @Erik Osterman (Cloud Posse)!
there does not seem to be a channel specific to that.
I have some continuing education money to burn any recommend training or books out there ?
And the unicorn project (I only have the digital edition)
Those books are great. O’Reilly has most of them via subscription at work for me. We also have a Udemy business subscription to do training for AWS, Cloud, Python, etc…
I recommend: “War - Peace - IT by Mark Schwarz”, “How to Win Friends and Influence People by Dale Carnegie”, and “How To Lead When You’re Not In Charge by Clay Scoggins”.
I’ve recently bought an O’Reilly book: “Technology Strategy Patterns by Eben Hewitt” and have just started to listen to “How To Lead In Product Management by Roman Pichler”.
I know, I know… not very tech focussed but I believe it’s never a bad thing to understand people, behaviour and some of the “softer skills” aka how to not be a d*ck.
These are great thank you for the suggestions!!!
Hey Everyone! I Hope you had a lovely Thanksgiving!
I’d love to invite you to Women in DevOps this coming Thursday!
If you haven’t heard of Women in DevOps Before then here is the super quick overview:-
Our aim is to help close the DevOps gender gap and empower the DevOps leaders of the future. Our unique platform has become a global movement and is used to not only amplify the voices of women, but of all minority groups within DevOps, to break down the barriers and drive positive change.
We’d like to set the perfect scene to hibernate at home and celebrate our achievements of 2020, whilst uncovering the highs and lows and biggest moments in the DevOps space! What a year!
As we welcome 2021, Women in DevOps and JFrog will be coming together for a panel discussion like no other.
Join us on 3rd December at 12PM PST / 3PM EST for our final networking initiative of the year; expect an insightful discussion with a unique insight led by JFrog’s expert panel to reflect on 2020 and discuss all things DevOps related.
Our Panel: Stephen Chin - Senior Director, Developer Relations, JFrog Melissa McKay - Developer Advocate, JFrog Kat Cosgrove - Partner, JFrog Batel Zahor Tova - Enterprise Solution Lead
You can RSVP here:- https://www.meetup.com/Women-in-DevOps/events/274819642/
Please note – our community is open to everyone and encourages everyone to attend!
If you haven’t been to a women in DevOps event before then check out a previous virtual event here - https://www.youtube.com/watch?v=OGLl_EUjH6Q We can’t wait to see you there!
Hey everyone, give a warm welcome to our newest members!
- @Abisoye
- @James Wade
- @Maxim Kostrikin
- @Rob Williams
- @Dipack Panjabi
Good to have you here =)
3