#general (2020-06)

Based on what Slackbot wrote: Your workspace is getting a free trial of Slack’s Standard Plan through August 25th! On the Standard plan, your team can now see all your past messages and files, work with external organizations in shared channels, make group calls, and more.

Oh, missed this, my bad. Thanks!

#ansible

THNX

@roth.andy I thought you had this working with the admission controller and IPA

I haven’t attempted something like this yet, but a Validating Admission Controller is how I would do the enforcement.

https://www.openpolicyagent.org/docs/v0.12.2/guides-kubernetes-admission-control/#example-policy-image-registry-safety

@Erik Osterman (Cloud Posse) @roth.andy I’m planning to do the enforcement at various points

• On commit using GitHub Actions and OPA to check Dockerfiles for whitelisted base images

• Configure EKS to only use AWS ECR as the Docker registry

• OPA Gatekeeper in Kubernetes

The other part to my problem is using ECR and updating it as a mirror to Docker hub only for whitelisted docker images.

So, engineers would submit a PR for whitelisting a Docker base image which would then be mirrors to ECS where their vulnerability scanner will scan the image. Then our in-house tool will provide us with the reports from ECR (Clair)

hash can’t be changed but the tag can

right. Looking for a medium article or something I can share with my team

you’d have to lock down tags to have the same immutability as the hash with the readability of tags

its still a feature request for github https://github.community/t/feature-request-protected-tags/1742

https://github.com/isaacs/github/issues/1091

basically, you cannot protect tags. using the hash is the securest way at the moment.

i cannot seem to find any blog post on this but if you find one, id be interested

seems like there is one way to do this using a git commit hook thats deployed everywhere. https://stackoverflow.com/a/40860947/2965993

@RB really good point about the fact that tags cannot be protected on GitHub

Hello! I’m a DevOps/Platform Engineer at Ibotta. I’m new to terraform, aws, k8s, helm/helmfile, and atlantis, but eager to learn. Nice to meet y’all!

Hi! I’m the co-founder of taloflow.ai - we offer turnkey AWS cost optimization for dev teams. Great to be here with all of you.

that was an old version of their draft. the latest version is 01.html

I spent a lot of time working with pulumi last year on a client project. (I also have had a heck of a lot of other things going on, so this is a high level summary based on 6/2019 knowledge)

in general, it wouldn’t recommend pulumi. basically, it’s terraform at its core. it behaves the same as terraform - the same plan/apply model, the same stored state, etc.

it doesnt really integrate into any existing code project in any usable way - you just build out your config in code. sort of. because it’s still terraform, and there’s still stored state and plan/apply steps, that causes a lot of atypical behavior in how the code executes.

also, the pulumi dev teams introduce a lot of their own specific desires into the libraries. you can’t functionally use javascript for example - you must use typescript. (you -can- use javascript, but they strictly and specifically depend on the type enforcement in TS…and if you dont have it, bad shit happens).

the python support was virtually nonexistent - just enough to call it a checkbox. all of the real core work was being done around typescript.

you’ll find yourself pulling in a ton of framework code to try to support anything….and a lot of that is still heavily in redevelopment. it was common to run into “oops, this piece doesnt work with that piece”.

and there were a remarkable amount of cases where you just can’t pass data from one place to another.

…and I was just trying to do “simple” stuff like lambda and fargate management.

Oh, thanks for expressing your thoughts! I find it quite valuable.

I tried Pulumi out once (more than a year ago) for a small demo project. Just wanted to play a bit because I really liked the ideas behind it. Nevertheless back then it didn’t convince me to start a transition from TF.

We are still keeping an eye on Pulumi project and I try to read as much thoughts on it as I can, but mostly there were about - it’s TF under the hood + gp language instead of clunky hcl, so consider for yourself.

The feedback is just amazing!

Appreciate your thoughts.

I don’t like the dependency it creates on app.pulumi.com.

This hints us at the intensions of the Company.

Unlike other open source communities which don’t force things on to users.

Say like Debian, Kubernetes, Prometheus etc.

Also Terraform.

Another challenge is that Terraform is good for System Admins who come with Bash Super Skills, but may not have and other programming skills.

In the contrast I thought my guys who are node js guys getting on to do Infra as Code might find it relatively quick to on boar.

Things have changed now.

Pulumi Python is now better. I started with python.

But I haven’t pushed it yet.

My intension is to democratization of Infrastructure as Code and have every Dev own thier Infra.

In the past when I was consulting for an Indonesian startup, which was using Elixir heavy found it hard to wrap their head around HCL.

I think it was more like a mental barrier.

I thought a Typescript/Python would help.

It’s like AWS CDK but without the Cloudformation baggage.

++++

Also one intresting thing was Terraform has no support for AWS Athena, but Pulumi has.

In the past I use CDK for such stop gap problems.

I think a lot has changed for it. It’s worth a shot in 2020.

I’ve spent a fair amount of time testing out different frameworks and languages for imperative creation of cloud infrastructure with different tools including PoCing my own framework.

The general conclusion I’ve arrived at is that infrastructure in the real world maps much more cleanly to a declarative model.

The amount of wrangling that any kind of imperative language or dsl requires you to do is way more overhead and on going technical burden than just learning and educating how to use a declarative framework like terraform directly.

Terraforms lack of imperative syntax is not a weakness it’s a strength, all the logic of the “how” is abstracted from the end user and handled by subject matter experts rather than end users. The end user has only to focus on modelling the actual end-state they’re after.

As much as I whine about Terraform I have to agree. Imposing imperative constructs on top of a fundamentally declarative product is what coders do to work around having to undergo a fundamental paradigm shift that has to occur to become powerful with declarative modeling.

That being said, I cannot believe it took this long to get depends_on into modules….

geesh

haha - the depends_on thing was tricky based on the way that modules actually worked.

they were kind of more like “import this stuff into the plan” rather than “treat this thing as a single entity”

Agree 100%!

I am glad I brought this topic for discussion.

As I have already gain some great opinions!

Thanks guys.

I will continue finishing my current experimental journey. And will share my experiences too.

Happy Friday.

Cheers @Hari Krishna Ganji - looking forward to hearing more.

Regarding the initial question, probably best to ping @Erik Osterman (Cloud Posse) about a channel.

Sure thanks for the suggestion.

@Erik Osterman (Cloud Posse),

It’s it possible to start a #Pulumi channel?

Thanks.

By the one more thing that just occured to my mind is a similar shift from declarative to imperative in the past.

Example:

• Maven to Gradle

• Grunt to Gulp

Android adopted Gradle.

Many other adopted Gulp over Grunt.

This transition occures when devs realize that they are fighting against the tool, and may need some freedom.

I also agree that Infrastructure is different.

Also on the other hand I guess Pulumi is also Declarative, in a sense the final apply, patch is still declarative. However the definition of that Infra is done using a Imperative Language.

By the way today I hit the first significant difficulty.

The CrossWalk for Pulumi that has many macro level modules only supports TypeScript.

And not Python.

Does Pulumi actually offer a way to create “modules” that work among various languages ?

I am not really sure.

The current engine takes care of the Diff/Patch and other provider related things.

However the offering called Pulumi Crosswalk offers a high level modules that are built on top of the base Language Api.

Given this approach, I arrive at the conclusion that Pulumi cannot offer language agnostic modules.

The unavailability of high level modules for Typescript but not Python also validates this.

Are you actually using crosswalk in production? If so I’d really be interested in how well it works for you

I originally was turned off to Pulumi as it was strongly Typscript driven with few Python modules. They apparently fixed that and have added far more languages. I cannot imagine it is easy to do language agnostic modules

I call it production, but I am risking with Pulumi because the App is not mission critical. It’s a young startup and we can afford a little downtime. Soon it may grow to acquire more complexity.

I am unable to use Crosswalk because the Python support is I think non-existent.

I watch the TypeSript code and transcode manually. So basically it’s a pain at the moment.

I was tempted to contribute to pulumi/pulumi-awsx repo. But I am sure I need to put in the time for multiple iterations and learning curve.

Yup. Language Agnostic is kind of painful.

Also when I saw @keen mention it basicall Terraform. I imagined he was paraphrasing and using the Term loosely.

But I just realized Pulumi actually uses the Terraform Providers underneath. This is very interesting. (Please correct me if my understand is wrong).

I guess, first thats how they are able to catchup with support across so many platforms.

Second, this makes Pulumi just a Developer Experience enhancing wrapper. In a sense.

So I got an email from Joe the founder for feedback after I signed up. Could be an automated email. However, I noticed that his past role was “Partner Director of Technical Strategy & Developer Tools”. at Microsoft.

In another Slack community I am part of I was mentioning that:

This explains Pulumi’s affinity towards TypeScript. Since TypeScript and VSCode if I remember correctly originated at MicroSoft.

Also since the rise of JAMStack and other Node Full Stack Devs (Note that I am hiring 3 of them) also kind of points us at why JavaScript ecosystem is kind of underserved by IaC and DevOps tooling.

That does connect some dots I suppose

I thought they use terraform providers via a bridge, but not TF directly

it would make sense to tap into that vast provider space if possible

However, I am still figuring out the nut and bolts.

So please take my words with some grain of salt.

Example, here: https://www.pulumi.com/docs/reference/pkg/nodejs/pulumi/aws/ssm/#Parameter

SSM API has the following note:
This provider is a derived work of the Terraform Provider distributed under MPL 2.0. If you encounter a bug or missing feature, first check the pulumi/pulumi-aws repo; however, if that doesn’t turn up anything, please consult the source terraform-providers/terraform-provider-aws repo.

Thanks!

Thanks

Too bad to get WSL2 you have to be on a version of windows that is so bleeding edge you get updates literally every day….

has that changed at all?

I’ve not really used Windows in about a year or so..

The new version is GA now

Guys I setup my Windows machine recently…. I am never going back to dual boot. WSL 2 is just awesome!

Having best of both Linux and Windows is great and works seamlessly so far

Thank you

Openldap?

https://github.com/glauth/glauth

@Matt Gowie I used it for a project a couple of years ago. We deployed serverless framework projects with it.

We provisioned it on EC2 with Terraform. Now they have docker images so I’m guessing you’d probably do well to deploy on ECS (if you’re using AWS.)

Overall I liked it. It is really streamlined and very flexible. Docker-based builds were a nice change at the time. Now everyone does it.

I remember secret management wasn’t great but now it looks pretty good.

The great thing was heaps of plugins for stuff like Slack notifications.

I don’t really see any specific cons other than it’s not a managed service if you’re self hosting. Security and HA would be your problem to an extent.

Happy to try to answer any questions you have.

Awesome — Solid review, Thanks @Joe Niland!

Keep us posted!

(btw there’s a #release-engineering channel)

Yeah, will do. Should be interesting. The client is running all services on ECS right now, but is changing over to K8s within the coming weeks/months, so I imagine it’ll be good to be on drone from the portability perspective.

Ah I see. Drone and k8s seem like a good match these days but no specific experience. Looks like you could migrate to using their k8s runner if you had builds running on Docker (EC2, ECS, etc)

Have been doing a lot of CodePipeline lately - this makes me want to try Drone again!

@Matt Gowie is this a good question for #office-hours ?

@Erik Osterman (Cloud Posse) if you don’t have many questions and you need one then sure. I was just trying to get a pulse from the community on this product / solution.

thanks for posting your resolution! that will help others.

sentry, elasticsearch

prometheus + actuator

Interesting! Thanks @muhaha

sorry, didn’t see this! please post it in #office-hours next time, otherwise we won’t see it

Okay!

Hey everyone!

try going to https://archive.sweetops.com/

then after the captcha, you should be able to search

That URL is hit after you search, with your query

it 404s

idk it seems to work for me

https://archive.sweetops.com/general/2019/12/#335dd344-9a41-4bad-932e-f155b500d8a5

ah I see it now https://archive.sweetops.com/search?query=test

Ah crap! thanks for bringing that to my attention.

Will get that fixed.

thanks erik!

also, on google, you can use site:[archive.sweetops.com](http://archive.sweetops.com) test

(for now)

oh cool very nice!

hey erik - I know I asked this before but failed to retain it - what are you guys using for the archive software?

We rolled our own

its’ one of the few things we havne’t open sourced

Yes! it’s still in use, however, there were a couple recent releases that had broken CI/CD so the artifacts were not pushed.

Oh, I only looked at the one for our elasticsearch cleanup module.

Ok, should be fixed when this PR merges: https://github.com/cloudposse/terraform-aws-ses-lambda-forwarder/pull/15

(and using a new tag)

Should be fixed now in 0.4.0

Be sure to see this: https://gist.github.com/joe-niland/b96150bfc13828c2a58751dfca7ffe7e

Helped me getting set up

This is really very helpful, Joe. Thanks very much!

@Francis just FYI, there is talk of deprecating the reference architecture in place of a new repo/module structure.

Is this still accurate @Erik Osterman (Cloud Posse)?

That said, I am still using it successfully on some projects, but I’ve been using terragrunt for smaller projects.

Yep! I’ve started updating it - but got pulled away

I would not recommend starting with the previous reference-architecture and have already opened PRs to remove it.

Cool, thank you for confirming

we still use of course all our modules, and geodesic

just organizing the project into a monorepo

And, correct me if wrong, using variant2 instead of make?

a lot of variant2, still some make - but want to provide remove most make

Big issue for me was the from-module limitations. Sometimes I need Terraform outputs/taint.

yea, it’s a PIA.

honestly, you can use terragrunt very easily to replace that functionality.

while still not going all-in with terragrunt. basically, just using it for the init-from pattern, but nothing else.

Thanks I will look into that

@Erik Osterman (Cloud Posse) any idea when a preview of the new setup may drop?

Thanks for that @Joe Niland. And glad to hear from you too, @Erik Osterman (Cloud Posse) I think until the new reference-architecture repo is out, I’ll keep looking at the current one, since you mentioned Joe that you’re still using it successfully in some projects.

Let me add that this is an area I’d be willing to help out on in reviews and PRs, as we have a couple of bigger projects about to start (and one we’d probably like to port to the planned forward path if it isn’t too onerous to do so)

@Joe Hosteny - not really, it’s not something we’ve allocated resources to. If you want to take a stab at this, I can share what’s involved. https://calendly.com/cloudposse

Thanks! That would be great.

@Francis let me know if you have questions

Will do @Joe Niland! Thanks again!

Hi @Joe Niland , @Erik Osterman (Cloud Posse) and fellow sweetops guys :) hope it’s been well. I’m really impressed with how you guys have designed the geodesic+root-terraform+per-account solution. This really makes sense - thanks again for sharing this to the community! I’ve been trying to get these different components to work the past few days by building them (geodesic+tf root module) individually and referencing its local docker versions per aws account repo build (e.g. root and production) and it’s worked so far. I had to marry up the right git tags between them to resolve some docker build errors, but I got there in the end. I’m no longer using the reference architecture repo (as it was mentioned here that it’s going to be deprecated) - presumably the approach I mentioned above is the right one (?) Just dropping a note to say thanks again for this awesome ecosystem of tools you’re providing here. I’m very excited to try the rest of them once I successfully setup the rest of my aws sub-accounts.

Thanks @Francis! So happy to hear you got your head around all this. It’s an achievement to say the least.

You’re right we’re deprecating the “original” reference-architecture, but we will be revising it in the coming months. It’s a sort of backburner project for us when client work slows down.

Thanks very much for that @Erik Osterman (Cloud Posse)