#general (2020-03)

https://sweetops.slack.com/archives/CBW0HJDS8/p1582212929118300

@Zachary Loeber

Also, check out our web archives https://archive.sweetops.com/search?query=jenkins%20x

I think that jx has the right idea for what needs to happen for fully cloud native projects. Beware that 90% of the jx guides out there are already outdated (specifically for creating new clusters), that jx is heavily opinionated towards trunk-based development, jx practically demands that your git repos be cloud hosted, jx is a gitops driven design that will force you to rethink how you do deployments, and is actually a tekton pipeline orchestrator.

That sums up the core things I’ve learned at least

(right or wrong, I’ll let you determine that though…)

Thanks @Zachary Loeber. Appreciate your input

my main question is around whether you use the open or enterprise version, and if open then how do you manage identities without the SSO feature???

We typically use the the enterprise version with GSuite

but long ago used it with the github provider - which is free

Btw, gravitational has their own public slack

i see, ok. buying things is hard for our customer

they might have startup pricing

(but without support)

heh. startup. unfortunately, just lots and lots and lots of red tape.

have to reinvent the wheel for so many things that would be faster, easier, and cheaper to just buy

yea - it would be often

and man, teleport is no easy task to replicate

are you strictly on AWS? then maybe the new SSM shell might be good?

this customer is eyeballing azure pretty hard, and will want an abstraction/translation layer on authentication and access. ssm and ec2 instance connect may buy some time. perhaps they’ll eventually get session recording. i also follow the aws github projects and there are a lot of complaints around the latency and performance when accessing through these proxies. so, not great in my book. plus need to figure out similar for azure

This tool looks awesome

Teleport seems pretty cool. What about their other product for packaging apps on kube? Gravity? https://github.com/gravitational/gravity

seems like the SCCM for kubernetes but without the stigma and migraines?

the project has a Makefile larger than most of mine so it must be good right?

Yea that’s a legit Makefile :-)

@tamsky is this in your wheelhouse

$20 says meowwacky is way ahead of me on that subject

SAD

did you mean for your example to contain an infinite loop?

I’ll take that as a no

well, maybe

you can merge dictionaries in yaml, but if you try to merge itself i don’t know what will happen because it would be infinite recursion

python handles it fine

Generally storage cost is inexpensive, additionally it can be tiered, making it even cheaper. First you have hot logs, then you can move it to cold storage. Don’t forget about retention.

Logs need to be meaningful. 56GB/5m is an overkill but I guess some companies may have such flow. If no one is looking at those logs they are either useless and can be disabled or you can split outputs and some logs send to ELK and majority of them to other “slow” and inexpensive storage.

What’s a reasonable amount of logs?

It depends on application, internal and external requirements. You should have enough logs to see that application works correctly and when it’s not you have enough to investigate.

https://archive.sweetops.com/search?query=pulumi

Thanks

What about gyro? I don’t see it in archive. Maybe you tried it? https://github.com/perfectsense/gyro

Never tried it. Terraform is by far the most popular tool for IaC. Pulumi is a distant second. Gyro isn’t even on the radar as far as I can tell.

There will of course be lots of other opinions. That one’s mine. Other big names are Ansible, Chef, Puppet, though I see those as more configuration management. They can do infrastructure, but I don’t see IaC as their primary use case

I’m waiting on https://github.com/pulumi/pulumi/pull/2697 before considering it seriously.

Pulumi relies on terraform providers via its own terraform bridge (https://github.com/pulumi/pulumi-terraform) for several of its own providers

that being said, if I were dropped into a python or typscript heavy environment or one that had an admin with a hardon for salt-stack (python) I’d likely dive into Pulumi as that means the ops people would be able to support the code I’d create for their infrastructure.

Terraform has a benefit of not being a programming language (and boy does it suffer for it in some ways)

it can ben supported by a larger number of infra folk than a repo of python code from my experiences.

though that is slowly starting to change so who knows

(shameless self promotion that seems relevant)

i’m making a terraform wrapper that lets you use python to generate terraform config. you put .tf.py files next to your existing .tf files and run the wrapper instead of terraform, then you’re able to define resources with a programming language.

https://github.com/raymondbutcher/pretf/blob/master/examples/aws/security-groups.tf.py

That’s really cool wrapper. Added to my backlog to test it.

I like the workflow feature in gyro. I’m currently trying to build blue-green deployments for AWS ElasticSearch using variant and Terraform. It more or less works but it feels like a big hack.

If you don’t get that feeling when working with terraform you aren’t doing it right I think….

What is Variant?

Hah, didn’t mean to send that to the channel

@randomy https://github.com/mumoshu/variant

@Erik Osterman (Cloud Posse) mentioned it a couple of times so I tried it out. It’s really useful to make cli tools quickly.

I’ll have to dig into that one more, my makefiles, while functional, almost require me to grow a long gray beard….

@Zachary Loeber i’m using make but I feel like I understand about 5% of what it’s doing

the alternatives point to one that is even more used

https://taskfile.dev/#/

github: https://github.com/go-task/task

2.3k stars

My team uses go-task heavily. We are very happy with it

Thanks guys - going to compare that one too

Ah right I have seen variant before but forgot about it. Thanks

@roth.andy does go-task support interactive tasks? It doesn’t right? https://github.com/go-task/task/pull/236

No, but I can do required variables by combining a var with a precondition. I do that a lot.

ok cool. Do you wrap terraform with it, or use it for other things?

All kinds of stuff, sure we have definitely used it with terraform.

The biggest use case we use it for is to externalize the scripting out of our Jenkinsfiles so we can run the same thing locally as Jenkins does in the pipeline. The test stage in Jenkins is just task test

ah I see - that’s cool

thanks for the info!

there’s a channel #variant …

For CLI I used python + click . Wrapping around tf , terragrunt, boto, k8s , etc.
Click is a Python package for creating beautiful command line interfaces in a composable way with as little code as necessary. It’s the “Command Line Interface Creation Kit”. It’s highly configurable but comes with sensible defaults out of the box. Bash is cool for simple tasks but for complex tasks and test coverage programming language will be more suitable.

@Marcin Brański click looks nice. Agreed that Bash has its limits. The issue in my case is Terraform’s limitations though.

what limitations? (i ask because pretf can solve a lot of problems)

click is freaking awesome

I’ve put entire comprehensive python apps into pipelines using it as the interface as a thin layer over declarative yaml configuration. As I dove into go recently it was the first thing I sought a replacement for

I will check out pretf as I hit the barriers of tf all the time

pretf supplies a masterfully crafted .envrc file in the example usage which blends two of my fav tools (direnv and asdf-vm) to an almost sublime level of sweetness -> https://pretf.readthedocs.io/en/latest/tutorial/direnv-and-asdf-vm/

Thanks, that’s very high praise!

I use direnv and asdf-vm in practically every project.

I’m only calling it as it is good sir

You met @btai and @Corey Gale of WLA DevOps right?

I have indeed and very excited to speak at their next meet up when social gatherings get the all-clear again! Can’t wait to get stuck in!

We’re already friends :)

I don’t know that I’m a leader in the DevOps space, but happy to meetup. HashiCorp is very dedicated to diversity in our workforce so I’d love to pick your brain on any successful campaigns to find/place women in tech positions.

@Jake Lundberg (HashiCorp) u def are!

Breweries:

1. Monkish - Possibly the best brewery in LA. Specializes in Belgian-ish beers, tart/sours and Hazy IPAs
2. Smog City - Mainly because it’s walking distance from Monkish, their Coffee Porter is one of the best in the style and they have some other greats like Infinite Wishes.
3. Highland Park - I’ve only had their beer on tap at Select Beer Store in Redondo Beach (my local pub), but have liked pretty much all I’ve had.
4. Mumford - Another I’ve not been to, but I like the beer I’ve had of theirs
5. El Segundo Brewing Company - Great if you like west coast IPAs. Not a favorite, but their beer is decent
6. Three Weavers - Decent craft, but not usually much outstanding
7. Firestone/Walker - Lower on the list because it’s not really a “local” brewery per se, but usually some great brews. Their barrel program may be one of my favorites. Food is decent but can be pricey
8. Brouwerij West - Because you haven’t lived until you’ve been to San Pedro (pronounced PeeDrow).

Food is a tough one, it’s a big city with a lot of choices. Anything in particular you want style wise?

Do you guys have any plans on improving the docs you generate with more terraform best practices? I’ve run across the https://www.terraform-best-practices.com/naming for example, and wanted to look into your documentation. A lot is just headers right now.

this is a labor of love by @antonbabenko

he has quite a lot of irons in the fire, so think it’s fallen behind a little bit.

btw, terraform has also been improving docs

i like this one: https://www.terraform.io/docs/cloud/workspaces/repo-structure.html

What do you guys use to generate your website? And @antonbabenko has done a really great job. I’d say empty sections mixed in make it a bit confusing like this terraform naming and all. Keep up the great work!

Hi there! Yes, it is rather behind, but “update terraform&terragrunt best practices” is climbing higher and higher in my to-do list due to virus and reduced travels… Today I wrote this post ( https://www.antonbabenko.com/terraform-aws-modules-as-a-code/ ) as a small report from trenches…

It solves so many issues. I want to basically do a build-harness project with invokebuild. PowershellBuild is a wrapper for this but not confident yet it does the simplicity i’m looking for. The setup has to be super easy yet secure

Welcome @Joe Bagdon!

Thanks @Erik Osterman (Cloud Posse) glad to be here. Ryan has told me great things about you guys. The bar is high!

Have you used it ? Can it (Riot) be a good enough replacement for Slack ?

looks cool @Erik Osterman (Cloud Posse) https://matrix.org/docs/projects/bridge/matrix-appservice-slack

Ah cool, missed that integration

(not using matrix though. are you?)

have you seen the online archive? https://archive.sweetops.com/search

SWEET OPS GALORE! So excited The other community I’m part of just piped everything through to discord. This means threads don’t exist, and as a result it’s so hard to keep track of stuff. I find this community very organized!

Will look for sure

What tooling is used to generate this archive. It’s one of the better archive solutions I’ve seen out there. Very nice

actually, we have one - just zero posts

haha

https://ask.sweetops.com

I haven’t been pushing it, which is why there’s zero activity. I’d like to talk to with anyone interested in this topic and how we can launch it.

Got you. Nice. I’m mixed on my philosophy on this. I’m seeing that Slack’s lack of making asynchronous a first class communication approach means I find it fragmenting at times. I’ve been leaning more and more towards asynchronous format like discourse, but it’s hard to find a good community as active as this

yea, my thinking is there are 2 types of users. people into chatting are perhaps not the same that hangout on forums.

The key is that both communities existing to me is problematic, you have to choose one or no one can depend on where to talk for sure. Add me as a beta feedback user if you look at using it more later on and need some active users

there’ll always be some intersection of course…

yeah, ok well I’ll mention one last thing then

One thing that might meet in the middle.

another idea I’ve thought about is adding summary answers in discourse. then that provides a way to keep the knowledge and away to ask questions on it.

but that would require some active moderators - and could only realistically do that with support from community.

What I’ve started doing in slack communities (dev/powershell/this), is if it’s a more complicated question, is to scope the question and start posting in stack overflow or devops.stackexchange.com so that the community benefits.

I think if someone asks a question and they have to narrow the scope of the question then posting in a format like that gameify’s, adds other community, and allows everyone to benefit.

I was thinking of that since “ask” means you might be looking at QA knowledge then making this part of devops.stackexchange could be great for others to know about you guys more + make knowledge more transparent and cream to the top and all.

My last 10 cents. I wouldn’t try to have a slack community and discourse personally. Too fragmented. I’d do slack + questions get put into devops.stackexchange and open ended discussions occur here. Either way, thanks for listening!

Here’s what the format for me looked like on a problem with AMI pipeline. This benefits everyone https://devops.stackexchange.com/questions/11050/how-do-fail-a-packer-pipeline-upon-powershell-provisioner-pester-test-failure/11054#11054

cheers!

what was that bot that was trying to post answers from prior slack threads?

Yup - something like that.

Oh foqal - it’s a startup. @vlad runs that.

maybe that bot could be extended to post questions and accepted answers to an external service, like discourse or stackexchange

yea, that would be rad.

That’s an interesting idea though where we focus on threaded converstations.

And clone those.

I find that being in .au most of the devops chat groups hangops, sweetops, etc are very US focused. So i kind of just deal with being late to he party and treat it half forum half chat

@vlad runs foqal? That’s pretty interesting from your description but I couldn’t find much. I’d love to be volunteer as a beta tester if it helps with slack communication searching and you need some beta testers I can’t subscribe to anything right now so I haven’t looked into the other cool machine learning based bots that do knowledge consolidation, but super interested in seeing it done right

@sheldonh check this out… what do you think about this discourse plugin?

This would allow us to curate and send specific questions/answers to discourse. This would allow some converstations to “live on”

Woooo. Fancy!!!

Did you build that? I want to up my chatops Fu!

this was more of an experiment

I don’t think anyone on our team is using it. The idea was to provide a “monorepo” like experience for our hundreds of repos (polyrepo)

That local-dev-scaffolding repo has not been sanitized for Open Source

I’m experimenting with something similiar, and actually settled on https://github.com/mateodelnorte/meta/ as the tool for bundling many repos into one, rather than git submodules.

@Abel Luck did you finally chose to leverage meta for consolidated view of multiple repos instead of using git submodules? https://sweetops.slack.com/archives/CQCDCLA1M/p1585297121019500?thread_ts=1585230544.012500&cid=CQCDCLA1M

(sorry for late reply), but yea we uses meta for consolidating multiple repos

I think that local-dev-scaffolding repo is private

(didn’t get a chance to open source it)

@mumoshu

I’m new to go and just starting. Should I leverage one of these to start with or just work with the built in flags for a while? I primarily want to build dev tooling, so not sure if I’d be over complicating my newbie stage with it or if it’s just a much easier way to start things up

I initially did that with Python and found that later on I really needed the power of Click to do what I wanted without re-inventing the wheel. Personally, I’m shortcutting lower level coding in places where it makes sense to do so

I tend to use cobra for anything that involves multiple levels of sub-commands. For simple commands without any sub-commands I’m happy with go’s standard flag pkg

are you using the viper package for config as well then?

exactly.

Every company will be different. Here’s Google’s guidelines: https://opensource.google/docs/releasing/approval/

My advice: Start really small. Get a toe in the door, and move on from there.

VERY helpful. This is good

So the Personal Projects (IARC) section is particularly relevant. The key is because our business interests are so narrow comparitevly and we don’t publish anything publicly, that the google process for the IARC is far more than I’d likely want to propose. I’ll take some hints from this, but any other resources are helpful thanks!

Oropharyngeal Airway?

Online Pool Analyzer?

not sure what that means as you can probably tell

open policy agent, here is a recent article on how one project uses it to find deprecated kubernetes api versions in deployment manifest code: https://opensource.com/article/20/3/deprek8

(as a prime example of a very good use case)

I think you might be misundestanding me.

I blog

I write code. Technically it’s all NDA, simply because they have had no experience with devops/powershell/scripting/tooling stuff.

I want to share that. I need to help guide them on a policy for how our company will allow this and log / flag any issues so it’s smooth and not a bunch of needless red tape, while still providing security to the company.

ah, sorry I misunderstood

I’m totally going to learn from this example. I want to do more slack apps not just bots and this is a great implementation

I want to stop emailed excel sheets being the change request approval process. I also don’t want to do it in our internal TFS tracking as I’m trying to get less reliance on that.

I can do a slack workflow button, but that’s barebones, maybe the best minimum viable improvement right now.

I’d like to use github issue with approval request noted in there, but the ideal case is to have contents in slack with an approve button, yet have it in something like Github issues so it’s really easy to audit compared to exporting slack workflows in a csv.

This repo in github would be basically just change requests, not actual code, so not seeing any way to have an “approve/decline” button or anything from github to slack in this.

Any ideas? It’s gotta be super intuitive, and was hoping to gain more traction on github issues over the complexity of TFS/Azure DevOps boards.

sounds like a use-case for a BPM tool

or possibly, something like stack-storm?

https://stackstorm.com/

glue a bunch of systems together to get a desired workflow?

I need bare minimum. If you are saying that StackStorm is bare minimum i’m up for evaluating.

I basically want to simplify the approval/review by using slack, but on the backend begin tracking this much more cleanly in github issues so it’s not just chat messages and later could use github issues more without the main system being different.

If you want a minimally functional solution to your companies custom change management system, I don’t think you’re going to find that. What I’m saying is you can use a project like StackStorm to glue together the pieces in your organization to accomplish the workflow you’re looking for.

but no, its not a zero effort, push-button thing

Ahh, I have to host my own instance and manage. I was hoping for saas or something free tier would be easy to quickly pipe together. It sounds pretty awesome overall if I had more time

there is a saas thing that might be helpful, trying to recall

cloudpipes or something like that

I remember that. Didn’t think of it. I used a little in the past for routing chat messages

yeah, cloudpipes, maybe give that a look - https://www.cloudpipes.com/channels

super detailed options. I’ll check this out for sure!

They got bought. I can’t find any details on the pricing anymore to find out if my free tier is good. It’s going to be part of a new platform and pretty sure focused on enterprise. Not sure I can use this right now

Tier 2 requires startup plan not free, and they don’t seem to have any current plan pricing details

Update. Switching over to azure devops. We’ll probably just just workitem form but I might try zapier +typeforms if I can sell that as way for people to ask for help in a more beautiful experience.