#gitops (2020-06)

Discuss continuous delivery of infrastructure Archive: https://archive.sweetops.com/gitops/

2020-06-29

2020-06-28

2020-06-27

2020-06-26

RB avatar

Working with multiple pull requests in .github/PULL_REQUEST_TEMPLATE/ with 2 files general.md and kms_secrets.md . When I create a new PR, I expected to see a button to select which template like we see with issue templates. What could the issue be ?

RB avatar

The less than ideal way… I tried the ?template= method as well and I still cannot select my templates

sheldonh avatar
sheldonh

Ok…. I want a cool win

Let’s say I want to add a whitelisting entry to a security group in QA. Right now it’s manual. Second option is terraform cloud cicd …. That said I am thinking of chatops or azure Pipelines or ssm automation runbook.

I want it to be super easy for team members to update their entry even if no GitHub knowledge so thinking maybe a messaging extension in teams to call an AWS lambda ? Easy to to do or learning curve steep. Also considered poshbot if I can do messaging extension but really wanted to use adaptive cards/form which makes sense to use lambda rest API for. Lastly I have mix of languages on the team and lambda would allow go, Powershell, and python contributions from team members vs poshbot probably making me run solo.

Messaging extension a good option or should I stick with GitHub runner and azure Pipelines concept, or gitops?

When I was talking through this with someone my preference would be pull request driven workflow. However I’m considering chat ops for this because a portion of those people don’t even have experience with GitHub.

sheldonh avatar
sheldonh

Struggling for the right fit without overcomplicating it so that later on more people could contribute. Lambda seems promising because somebody could contribute python Powershell go or whatever language they want. At the same time it feels like there’s a lot of plumbing to put together to have AWS talk to microsoft teams securely, so that’s why I was considering gitops workflow. It just limits the enthusiasm I’ll probably get on implementation.

bradym avatar
bradym

I’m confused by what you’re trying to do. The goal is to allow whitelisting, but you’re mentioning people writing code for a lambda function? Why would custom code be needed for this? Also, if people are comfortable writing code for this, why would a pull request workflow not work?

sheldonh avatar
sheldonh

Sorry my mobile diction wasn’t great

bradym avatar
bradym

Unless I’m missing something, a script hooked up to a pipeline/chatbot/something that takes a couple parameters should be able to handle it?

sheldonh avatar
sheldonh

I have two main audiences

sheldonh avatar
sheldonh

The first is people submitting tickets to whitelist an IP address in a QA environment. I want to eliminate if possible the need to even submit a ticket and instead allow them to submit a request or submit a request through a chatbot that is approved. Possibly this could be azure Pipelines, etc. This audience while primarily developers can’t be guaranteed to know how to use pull requests properly. Some of them are on team foundation server and don’t even have access to github and others are QA folks with limited knowledge of git.

sheldonh avatar
sheldonh

The second audience is the people performing the work. In this case it’s the devops team. There is a range of development related skills on this team. It’s heavily ops focused with very few writing code consistently and I’m pretty sure very few that if ever tackled writing a lambda from scratch. Not all of them have expertise in Powershell. Python is a common denominator that’s approachable for most.

Whatever I set up I want to be sustainable to get more contributions in the future. If somebody can write a simple python script I might be able to guide them through in the converting it into a lambda. In that case these isolated functions become more usable and allow them to choose the language of their choice. the catch to that is finding a nice way to operate those. Relay is a very promising solution yet still in beta.

sheldonh avatar
sheldonh

If I make the development workflow too complicated instead of saving them time over logging into the console they won’t use it. that’s why I’m considering compromising I’m doing chat apps for this work instead of a traditional terraform CI CD. Trying to eliminate Shadow IT changes

bradym avatar
bradym

Ok, I’m far less confused now.

bradym avatar
bradym

The first thing that comes to mind (since I use it) is setting up a rundeck instance. With rundeck you setup projects, and each project has jobs. Jobs can take parameters and then run code. It can be bash scripts, python, anything really.

You can give people access based on what they should be allowed to do.

For just one project, it would be massively over-engineering. But it sounds like you may want some sort of centralized thing you can build on top of going forward.

bradym avatar
bradym

You can use webhooks/api calls to trigger jobs, so it’d be easy to integrate into a chatbot.

bradym avatar
bradym

It also gives you a central place for logging all of the jobs that have run and who has run them, so some nice auditing built in.

bradym avatar
bradym
rundeck/rundeck

Enable Self-Service Operations: Give specific users access to your existing tools, services, and scripts - rundeck/rundeck

sheldonh avatar
sheldonh

This is good! I’ve thought about rundeck but since we have ssm automation docs as a possible solution, it would be hard to sell.

If rundeck has Microsoft teams integratation to launch and all automatically then maybe. It’s just if it doesn’t offer a freaking quick win and more infra to manage I see it flopping here. Whatever I do has to make a simple task like submitting new IP entry a faster and easier thing than doing manually and ideally allows anyone in teams to submit with our team being notified to click approve.

bradym avatar
bradym

I’ve never used microsoft teams myself, but apparently there’s something

Zach avatar

I have

sheldonh avatar
sheldonh

I looked at that but it’s just notifications.

I guess I really like the idea of rundeck but in this case it adds more duplicate functionality over what SSM automation offers. I’ve also consider octopus and bolt. Just feels like a big jump. Was hoping for something a bit smaller in scale and something with basic app integratation for Microsoft dialogues and all in teams. I can stick with gitops but it means still more toil for the second team

Zach avatar

yes the plugin is only notificatins

Zach avatar

its also awful and you’ll have to edit the template

Zach avatar

If you want ‘interaction’ from slack/teams you’re gonna have to write a ‘bot’ of some flavor

Zach avatar

Teams does have a pretty well documented bot framework though

bradym avatar
bradym

Yeah, rundeck is definitely only worth it if you decide to use it for lots of stuff. We’ve centralized all of our deployment and cron jobs onto rundeck, which makes it nice for keeping logs in one place and manually triggering jobs to run when needed.

Zach avatar

Im also using rundeck right now, but I don’t like it. Unfortunately there’s not much really competing with it other than Jenkins

bradym avatar
bradym

Yeah, it’s a shame there’s no real competition.

Zach avatar

We did finally crack the code on dynamic variables which is going to help us a bit

Zach avatar

their documentation didn’t make it clear at all that you could cascade the dynamic job variables

bradym avatar
bradym

Yeah, the documentation needs some work for sure.

sheldonh avatar
sheldonh

So I think from this either gitops, poshbot, or probably the lowest common denominator is to just use my azure Pipelines tooling as it can require approval steps and just use azure Pipelines. I think that will have to be it.

sheldonh avatar
sheldonh

It would help me promote azure Pipelines + would let me run in containers if I don’t need machine aceess. + Has an app in teams.

sheldonh avatar
sheldonh

I like using tooling that’s designed for specific purpose but in this case those, familiarity with them, easy integration with approvals and teams might just be the way I need to go for now. That also means it could be just a script instead of needing people to submit pull requests

sheldonh avatar
sheldonh

I think I’ll give chatops and ssm docs a test run then. Have the ssm doc for stuff on machines and general aws calls could be the bot Go ahead see if I can just stick with bot messages and not the buttons. Still would be an improvement

Zach avatar

what do you mean by ‘ssm docs’?

sheldonh avatar
sheldonh

AWS systems manager docs for running remote commands. It provides the ability to run on 1 or many machines a set of actions , or bypass ssh/winrm and connect directly to an instance in the terminal and more

Zach avatar

ah yah ok, just didn’t recognize the name

sheldonh avatar
sheldonh

Ao I’m at final stage but still looking for one more bit of feedback.

sheldonh avatar
sheldonh

I have used poshbot (Powershell chatops) before and big fan. I think that this option would let me build out a lot of things pretty quickly because I’m pretty good at Powershell and AWS tools.

sheldonh avatar
sheldonh

However I’m trying to think of what I can do to get people excited and contributing beyond me. So for automation purposes using chat ops as an interface is appealing as it doesn’t necessarily have tons of complicated code in it.

sheldonh avatar
sheldonh

But interacting with it can be challenging. !ec2 -enc llv prod -tag foo for example. With a lot of my consumers being uncomfortable with command line syntax for everything this becomes difficult to sell

sheldonh avatar
sheldonh

For example I solved the problem of needing to log into the ec2 console to get environment information and IP addresses. I barely got anybody executed even though it provided a wonderfully formatted response and bastion information

sheldonh avatar
sheldonh

So now I’m thinking of the messaging extensions in teams. If it’s not too complicated then it seems like you end up having dialogues and better interactivity for non-technical users… And then it executes something from API gateway and lambda?

Taking this monologue into account, would you feel messaging extensions for a bot requiring command syntax would have a better future adoption?

sheldonh avatar
sheldonh

There’s just so much plumbing going into this stuff kills me. Doing it as a side project can be challenging. What I really want is to be able to publish a lambda function and easily register that in a Microsoft teams room as a button allowing input and approvals. There’s nothing that I found that lets you do this easily. No easy button. If a chatbot like poshbot could provide actionable buttons easily then my problem might be solved but very few chatbots seem to offer that.

sheldonh avatar
sheldonh

Any other input welcome. Microsoft teams complicates this a bit from what I’ve gathered.

sheldonh avatar
sheldonh

Just found yellowant. That is a promising solution to help with some of this and has a free tier

Zach avatar

Did you look into the MS Teams bot framework?

Zach avatar

It seems very well organized

    keyboard_arrow_up