#office-hours (2020-08)

Meeting password: sweetops

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours

Meeting password: sweetops

2020-08-27

Vlad Ionescu avatar
Vlad Ionescu

For the DockerHub new limit thingie: https://twitter.com/pgarbe/status/1298280715575087108

Worried about the upcoming DockerHub rate limits? I build an CDK Construct that you an use to sync important images to ECR. https://github.com/pgarbe/cdk-ecr-sync

bbhupati avatar
bbhupati

Hello guys, I’m trying to install lens in centos 7.6(64-bit) using snap (https://snapcraft.io/install/kontena-lens/centos) and installation is successful, but when i run kontena-lens it gives below /snap/kontena-lens/110/kontena-lens: error while loading shared libraries: libgtk-3.so.0: cannot open shared object file: No such file or directory sudo yum provides libgtk-3.so.0 Last metadata expiration check: 0:40:31 ago on Fri 28 Aug 2020 06:00:51 AM UTC. gtk3-3.22.30-3.el8.i686 : GTK+ graphical user interface library Repo : @System Matched from: Provide : libgtk-3.so.0

gtk3-3.22.30-3.el8.i686 : GTK+ graphical user interface library Repo : rhel-8-appstream-rhui-rpms Matched from: Provide : libgtk-3.so.0

sudo yum install gtk3-3.22.30-3.el8.i686 -y

after installing all dependency packages still getting same error any suggestion on this ?

Install Lens on CentOS using the Snap Store | Snapcraft

Get the latest version of Lens for on CentOS - Lens - The Kubernetes IDE

2020-08-26

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:08 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Zoom avatar
Zoom
06:28:50 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:52 PM

Kareem Shahin has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:06 PM

Michael Holt has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:18 PM

Eddie Wizelman has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@here our devops #office-hours are starting now! join us on zoom to talk shop url//cloudposse.zoom.us/j/508587304) *password: sweetops

Zoom avatar
Zoom
06:31:10 PM

Sheldon Hull has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:11 PM

Vicken Simonian has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:28 PM

Alex Vorona has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:43 PM

Andrey Nazarov has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:35 PM

Ian Bartholomew has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:50 PM
Zoom avatar
Zoom
06:33:09 PM

Scott Rogers has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:15 PM

Perumal Varadharajulu has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:19 PM

Robert Horrox has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:21 PM

Nick James has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:52 PM

Drew Davies has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:39 PM

nata lie has joined Public “Office Hours”

Andrey Nazarov avatar
Andrey Nazarov

Another GUI for K8s https://kui.tools/

Kui

Kui is an open-source, graphical terminal designed for developers.

Andrey Nazarov avatar
Andrey Nazarov

We’ve used Lens a bit. It was ok, but sloooow

Zoom avatar
Zoom
06:35:48 PM

Anton Shakh has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:54 PM

Victor Avila has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:10 PM

Andrew Elkins has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:27 PM

Alex Siegman has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:53 PM

Victor Ma has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:51 PM

Jawwad Yunus has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:50 PM

Wanderley Teixeira has joined Public “Office Hours”

Zoom avatar
Zoom
06:42:43 PM

Vicken Simonian has joined Public “Office Hours”

Zoom avatar
Zoom
06:45:03 PM

Robert Horrox has joined Public “Office Hours”

Zoom avatar
Zoom
06:46:50 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
06:52:41 PM

Adedayo Akinpelu has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

https://blog.intive-fdv.com/a-comparison-between-packer-and-ecs-image-builder/#<i class="em em-~"</i>text=EC2%20Image%20Builder%20is%20relatively,them%20when%20creating%20different%20images>.

A Comparison Between Packer and ECS Image Builder - Intive Blog attachment image

Let’s consider the process of making a new server available to run our applications on: We have to follow several steps to be able to do that in production. We could divide this process into three main stages:  Stage 1: Starting an Instance  This phase covers all the necessary steps to run the application. It could also include updating security patches, …

Zoom avatar
Zoom
06:58:32 PM

Adam Crown has joined Public “Office Hours”

sheldonh avatar
sheldonh

If we have time I want a 101 of your future go based task project framework you are working on. a teaser?

Andrey Nazarov avatar
Andrey Nazarov

Actually it’s sad to see that there is no future for Kontena. There were cool.

Zoom avatar
Zoom
07:03:07 PM

Eric Berg has joined Public “Office Hours”

Andrey Nazarov avatar
Andrey Nazarov

Afaik Kontena tools were written in Ruby;)

sheldonh avatar
sheldonh

Definitely. The catch is i think the tooling we are talking about is more a “make file” type of framework that is a replacement.

Potentially could run a “build” that runs on windows, mac, linux, etc and full cross platform compatibility with a single binary for example. It’s a different approach rather than lots of other dependencies

sheldonh avatar
sheldonh

For me, the hard part for me to evaluate this will be ensuring all my powershell based tasks don’t require massive reworking. I’m looking forward to learning more on this.

Andrey Nazarov avatar
Andrey Nazarov

Not Lens though)

Andrey Nazarov avatar
Andrey Nazarov

Somebody might find this useful https://cuelang.org/

CUE

Validate and define text-based and dynamic configuration

muhaha avatar
muhaha

Nice ! Thanks, can I use json schema with cuelang ?

CUE

Validate and define text-based and dynamic configuration

sheldonh avatar
sheldonh

Oh, so maybe go-releaser for your tasks and you’d have a single binary easily for all project. Super interesting.

Zoom avatar
Zoom
07:08:34 PM

raphael francis has joined Public “Office Hours”

Zoom avatar
Zoom
07:09:21 PM
sheldonh avatar
sheldonh

Great teaser. excited. As long as I can figure out how not to start from scratch and use perhaps some of my powershell 7 stuff, i’m going to try it!

Zoom avatar
Zoom
07:16:11 PM

raphael francis has joined Public “Office Hours”

Andrey Nazarov avatar
Andrey Nazarov

Cool! That’s what I wanted to ask. How do you bootstrap things at Cloud Posse))

sheldonh avatar
sheldonh

look at their build-harness repo. super cool concept

kareem.shahin avatar
kareem.shahin

Had to hop off. Couldn’t find the ECS thread about setting env vars but outside of pulling them down from SSM programmatically in the app’s entrypoint you can directly reference them in SSM in the task definition

:--1:1
sheldonh avatar
sheldonh

also checkout cloudposses ecs task module as you can use terraform module inputs to generate the json. I think that’s super cool!

:--1:1
1
sheldonh avatar
sheldonh

I use InvokeBuild (Cross platform, and like Make on steroids), but the caveat is that you need apt install powershell for example. A precompiled binary provides consistency in multiple environments + also be able to run the same task in a build with no rework.

if this is too early to adopt and you have flexibility for considering powershell (object based vs text based is the main difference) then InvokeBuild is also great

sheldonh avatar
sheldonh
07:30:17 PM

general info on the Visual Studio Code “Task Explorer” i mentioned. Find it super helpful to ease of use. Wasn’t talking about a language server, but making dev experience super smooth to execute the same thing you run in build for example.

This is just a json “runner” built into vscode that calls a command. Would love to see a variant2 runner added in future. Play button to run, grouping of tasks and more.

sheldonh avatar
sheldonh

Keep up the great work

Zoom avatar
Zoom
09:34:02 PM

New Zoom Recording from our Office Hours session on 2020-08-26 is now available.

2020-08-20

loren avatar
loren
05:00:41 PM

@loren has joined the channel

2020-08-19

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:08 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

PePe avatar

Question : Setting = No IAM authentication, Group limit on IAM is a problem, No SSO and all manually setup in a separated aws account Question: How do you go about managing all users, groups and adding SSO and MFA with assume role using TF or other tool ( AWS Organizations, Control Tower etc)( we have Keycloak and google)

OliverS avatar
OliverS

Question for today although I’m not sure I can attend live (are these recorded?): is there a way of allowing an AWS lambda to http a service running in same VPC behind an AWS classic LB that filters on IP addresses? In other words I have a classic LB that I want to configure to allow incoming traffic only from corp network (I have done that part), or from the Lambdas. I’m thinking that it cannot be done robustly (I would have to find WAN IP of the lambdas), instead I need to create internal LB that the Lambda will target. Any insight would be much appreciated!

Zoom avatar
Zoom
06:32:15 PM

M Hunter has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:33 PM

Jose Netto has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:35 PM

Andrew Roth has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:40 PM

venkata has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:51 PM

Rob Horrox has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:11 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:13 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:44 PM

Alex Vorona has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:10 PM

Anton Shakh has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:30 PM

Dan Meyers has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:52 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:54 PM

Anton Shakh has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:55 PM

Andrey Nazarov has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:55 PM

Anton Shakh has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:16 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:33 PM

Gabriel Tam has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:45 PM

Vicken Simonian has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:52 PM

Igor Bronovskyi has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:55 PM

Pedro Torres has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:59 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:59 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:06 PM

Kareem Shahin has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:34 PM

pepe amengual has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:49 PM

Babajide Hassan has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:09 PM

Eddie Wizelman has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-opsgenie-incident-management

Contribute to cloudposse/terraform-opsgenie-incident-management development by creating an account on GitHub.

Zoom avatar
Zoom
06:37:20 PM

Nigel Kirby has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:57 PM

Blaisep has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:02 PM

Robert Jackson has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:09 PM

Adam Watson has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:15 PM

Rahul has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:39 PM

Drew Davies has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:50 PM
Zoom avatar
Zoom
06:41:18 PM
Zoom avatar
Zoom
06:46:49 PM
Vlad Ionescu avatar
Vlad Ionescu

Shoutout to https://github.com/haya14busa/action-bumpr which also does automatic releases I really like that action

haya14busa/action-bumpr

Bump semantic version tag on merging Pull Requests with specific lables. - haya14busa/action-bumpr

Zoom avatar
Zoom
06:48:19 PM

Mike Drummond has joined Public “Office Hours”

Vlad Ionescu avatar
Vlad Ionescu

For automatic comment /test all : https://github.com/peter-evans/create-or-update-comment or Mergify / Pullapprove ( they both have options to post comments when something happens IIRC)

peter-evans/create-or-update-comment

A GitHub action to create or update an issue or pull request comment - peter-evans/create-or-update-comment

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ah yes @loren just tipped me off to mergify

peter-evans/create-or-update-comment

A GitHub action to create or update an issue or pull request comment - peter-evans/create-or-update-comment

:--1:1
Zoom avatar
Zoom
06:56:34 PM

Ayrton Araújo has joined Public “Office Hours”

Vlad Ionescu avatar
Vlad Ionescu
Introducing the AWS Controllers for Kubernetes (ACK) | Amazon Web Services attachment image

AWS Controllers for Kubernetes (ACK) is a new tool that lets you directly manage AWS services from Kubernetes. ACK makes it simple to build scalable and highly-available Kubernetes applications that utilize AWS services. Today, ACK is available as a developer preview on GitHub. In this post we will give you a brief introduction to the […]

Andrey Nazarov avatar
Andrey Nazarov

I’ve heard Google has something similar (or will have). Will try to search for it.

Introducing the AWS Controllers for Kubernetes (ACK) | Amazon Web Services attachment image

AWS Controllers for Kubernetes (ACK) is a new tool that lets you directly manage AWS services from Kubernetes. ACK makes it simple to build scalable and highly-available Kubernetes applications that utilize AWS services. Today, ACK is available as a developer preview on GitHub. In this post we will give you a brief introduction to the […]

Vlad Ionescu avatar
Vlad Ionescu

Everybody does it It’s the new hotness. Pulumi and Terraform also have identical projects

roth.andy avatar
roth.andy
What’s new in Kubernetes 1.19? | Sysdig attachment image

Kubernetes 1.19 is about to be released! And it comes packed with novelties. Here is the detailed list of what’s new in Kubernetes 1.19.

Andrey Nazarov avatar
Andrey Nazarov
kubernetes-sigs/multi-tenancy

A working place for multi-tenancy related proposals and prototypes. - kubernetes-sigs/multi-tenancy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-ssm-tls-ssh-key-pair

Terraform module that provisions an SSH TLS Key pair and writes it to SSM Parameter Store - cloudposse/terraform-aws-ssm-tls-ssh-key-pair

roth.andy avatar
roth.andy
ndmckinley/terraform-provider-dominos

The Terraform plugin for the Dominos Pizza provider. - ndmckinley/terraform-provider-dominos

RB avatar

i should have asked this during office hours. doh.

https://sweetops.slack.com/archives/CCT1E7JJY/p1597864789200900

We use an office security group to allow ingress into our vpc. We’re approaching the 60 security group rule limit. What’s a good way to scale past this limit ?

Gowiem avatar
Gowiem

Have you asked AWS to increase the limit? They do that for a lot of limits… not sure about the SG one.

We use an office security group to allow ingress into our vpc. We’re approaching the 60 security group rule limit. What’s a good way to scale past this limit ?

RB avatar

nope, 60 is a hard limit

RB avatar

i think we may just be using the wrong tool for the job here. i think there might be a better solution for this than using security groups.

Gowiem avatar
Gowiem

Yeah that or continue using SGs but get a VPN for the office / team.

Gowiem avatar
Gowiem

I just learned about AWS VPN, but it seems insanely expensive if used at scale.

AWS VPN - Cloud VPN - Amazon Web Services

AWS Virtual Private Network (AWS VPN) lets you establish a secure and private encrypted tunnel from your network or device to the AWS global network. AWS VPN is comprised of two services: AWS Site-to-Site VPN and AWS Client VPN.

RB avatar

ah i havent looked into that

RB avatar

we have all these office ips that we cannot consolidate cause they are all from our external vpn provider

RB avatar

it would be ideal if they provided us with a single ipcidr block but unfortunately, it’s 19 different ones

Gowiem avatar
Gowiem

You could also have an office SG 1 and an office SG 2 couldn’t you? And then attach both to the resource that you’re looking to provide access?

RB avatar

ya that might be the easiest option

RB avatar

thanks Gowiem for your input!

voron avatar
voron

Maybe reference SG2 from SG1, as SG may be nested

PePe avatar

you could do that but when nested SGs you can’t use a SG id to allow connection in the Nested SG

PePe avatar

only on the first level SG you can do that

RB avatar

hmm i dont think i have done that before. Let’s say that our office security group is split into 2 security groups. office (same name) and sg2.

then we reference sg2 in sg1 and sg1 will then absorb all of sg2’s rules ?

PePe avatar

yes

PePe avatar

sg1 <– sg2 but sg2 is port and address only

PePe avatar

the limitation I think is on when

PePe avatar

sg1 <– sg2 and sg2 have SG ids as sources

PePe avatar

that will not work

PePe avatar

is basically a single level reference, you can’t cascade to multiple level sgs

RB avatar

ah I see so basically the SG1 that references a SG2 will only get SG2’s rules but they will not include SG2’s reference to SG3’s rules

RB avatar

oh ok that makes sense

PePe avatar

exactly

RB avatar

so ok relevant too, AWS just increased my limit from 60 to 100 rules per sg

RB avatar

but but but, now I want to split my rules up into multiple security groups, and keep my current sg that contains all my rules as the parent sg

RB avatar

then this parent sg can reference sg1, sg2, and sg3

PePe avatar

SGmain

• SG1

• SG2

• SG3

• etc

RB avatar

sigh…

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security-groups.html#security-group-rules
Another security group. This allows instances that are associated with the specified security group to access instances associated with this security group. Choosing this option does not add rules from the source security group to this security group.

PePe avatar

it does no add them up but it does allow the connection

PePe avatar

how the heck we did this back then????? we had like 300 rules

PePe avatar

or we attached multiple SGs to the resource?????

RB avatar

ya you probably added multiple security groups

PePe avatar

I’m pretty sure this works but you can’t combine sg-ids with ports

RB avatar

i checked with aws support and they confirmed that it won’t work. they pointed me to the docs

RB avatar

but i haven’t tested it.

PePe avatar

so SG-pepebullshit can’t have ports and sg-ids?

PePe avatar

ahh what they are saying is that if sg-pepebullshit could have a sg-id as source but that source can’t have other sg-ids as source

RB avatar

lol

RB avatar

unfortunately i cant attend today but food for thought for the next one

Zoom avatar
Zoom
08:51:30 PM

New Zoom Recording from our Office Hours session on 2020-08-19 is now available.

Andrey Nazarov avatar
Andrey Nazarov

Kubecon sessions should be available on YouTube in early September they say.

:--1:1
Andrey Nazarov avatar
Andrey Nazarov
KubeCon + CloudNativeCon Europe 2020 - Virtual

Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.

:--1:2

2020-08-17

omerfsen avatar
omerfsen
[EKS] Managed Node Groups Launch Template Support · Issue #585 · aws/containers-roadmap

Launch template support ability to launch managed nodes using a provided EC2 launch template. This will support multiple customization options for managed nodes including providing custom AMIs and …

:--1:1
omerfsen avatar
omerfsen
omerfsen avatar
omerfsen

Update your nodes just like you update your deployments(rolling update) + custom eks ami using launch templates also now can change instance type within worker node group specs (ie no need to create new node group)

omerfsen avatar
omerfsen

Tag ec2 instances just like you tag worker node groups (finally we will have eks nodes with Name tag on AWS Console)

Vlad Ionescu avatar
Vlad Ionescu

Still no Spot support But it’s on the way!

omerfsen avatar
omerfsen

Aws wont want us to use spot ;)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Does anybody know of a useful NOTES.txt file out there? The scaffold NOTES.txt is just noise to me, but I could see others coming up with some possibly useful information to stick in there. Quick google search didn’t turn up anything however.

2020-08-15

2020-08-14

2020-08-13

Briet Sparks avatar
Briet Sparks

it’s insecure to set secrets via environment variables, so I’ve read, because they can be accessed by any user on the OS. So, it would make sense to inject secrets at image build time (assuming the use of containers) by the CI runner. Any thoughts on this?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the point you bring up is correct regarding environment variables, but different companies will have different tolerances for this.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

secrets should absolutely not be injected at build time as then it’s on the image itself.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we can discuss next office hours

:--1:1
vFondevilla avatar
vFondevilla

We’re going via EnvVars when the code is a legacy one. If not, I’ve teached the devs to use the SDK for retrieving the passwords

dalekurt avatar
dalekurt

I’ve done secret management at runtime, never at build time.

dalekurt avatar
dalekurt

The issue I see with build time secrets other than what Erik has highlighted is being able to make changes to the env var when needed without rebuilding the image.

s_slack avatar
s_slack

I’ve been thinking some of this and I’m wondering if secrets held in Kubernetes secrets are really more secure since it’s just encoded. You could also encrypt the secret and put it in k8s secrets but that is cumbersome to make changes. I’m still not sure what best practices are in k8s

2020-08-12

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:15 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Zoom avatar
Zoom
06:25:47 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:00 PM

Eddie Wizelman has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:35 PM

venkata has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:17 PM

Vlad Ionescu has joined Public “Office Hours”

1
Zoom avatar
Zoom
06:27:47 PM

Andrew Roth has joined Public “Office Hours”

Gowiem avatar
Gowiem

There was a great discussion last time without too many questions, but if we’re looking for topics: it’d be great to chat through the solutions folks are using to solve the problem of disjointed terraform workflows.

For example, I’ve got a project where we’re using RDS with a bunch of databases. The RDS instance is of course in private subnets and access is only grants to particular application SGs and a Bastion SG. We want to use the Postgres Terraform provider to provide bootstrapping of the databases and its extensions, roles, etc. Now the problem is that the Postgres provider can’t connect to RDS without an SSH tunnel through our Bastion instance. So my solution was to carve up our project into multiple terraform projects / directories and then when creating a new workspace / environment the flow is to:

  1. terraform apply the AWS infra in a particular directory — Creates RDS, Bastion, etc etc etc
  2. Create the ssh tunnel to the RDS instance through the Bastion instance now that it’s up
  3. Go to the postgres terraform directory and then terraform apply there. This works, but I of course wonder if there is a better way and it’d be great to hear how others are tackling this type of thing!
Aleksandr Fofanov avatar
Aleksandr Fofanov

@Gowiem I will take a shame to propose 2 hacky solutions to described problem

  1. If you are a terragrunt user and would like to use terraform’s native postgres provider, you can use before/after hooks to port-forward ssh port on bastion host to localhost using SSM Session Manager. Obviously this requires your bastion host to be registered with SSM Session Manager (SSM agent running on bastion host + certain IAM permissions attached to instance profile). So before hook starts port-forwarding, tf postgres provider in your module connects to RDS via ssh tunnel and provisions required resources. After hook stops port-forwarding and that’s it.
  2. You can use Lambda function deployed to your VPC/subnets to provision required resources. I have tf module with such lambda opensources on github. It doesn’t have support for SSL connections and pg extensions yet, but is able to provision databases and roles in RDS instances with pg and mysql engines.
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This is a good topic. Will discuss today!

Gowiem avatar
Gowiem

Aha ya’ll didn’t discuss during the last one — Cool. I had to drop early.

Zoom avatar
Zoom
06:29:57 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:08 PM

Victor Fondevilla has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:10 PM

Nathaniel Alconcel has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:35 PM

Michael Holt has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:39 PM

Torsten Trzeciak has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:41 PM

Michael Martin has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:40 PM

Marcos Soutullo Rodriguez has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:40 PM

Robert Jackson has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:36 PM

Brian Tai has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:44 PM

Gabriel Tam has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:51 PM

Geoff Weinhold has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:08 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:16 PM

Dan Meyers has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:25 PM

Anton Shakh has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:45 PM

Adam Blackwell has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:52 PM

Adam Blackwell has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:54 PM

Adam Watson has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
What it feels like to be an open-source maintainer

Outside your door stands a line of a few hundred people. They are patiently waiting for you to answer their questions, complaints, pull requests, and feature requests. You want to help all of them, but for now you’re putting it off. Maybe you had a hard day at work, or you’re tired, or you’re just trying to enjoy a weekend with your family and friends. But if you go to , there’s a constant reminder of how many people are waiting:

When you manage to find some spare time, you open the door to the first person. They’re well-meaning enough; they tried to use your project but ran into some confusion over the API. They’ve pasted their code into a GitHub comment, but they forgot or didn’t know how to format it, so their code is a big unreadable mess. Helpfully, you edit their comment to add a code block, so that it’s nicely formatted. But it’s still a lot of code to read. Also, their description of the problem is a bit hard to understand. Maybe this person doesn’t speak English as a first language, or maybe they have a disability that makes it difficult for them to communicate via writing. You’re not sure. Either way, you struggle to understand the paragraphs of text they’ve posted. Wearily, you glance at the hundreds of other folks waiting in line behind them. You could spend a half-hour trying to understand this person’s code, or you could just skim through it and offer some links to tutorials and documentation, on the off-chance that it will help solve their problem. You also cheerfully suggest that they try Stack Overflow or the Slack channel instead. The next person in line has a frown on their face. They spew out complaints about how your project wasted 2 hours of their life because a certain API didn’t work as advertised. Their vitriol gives you a bad feeling in the pit of your stomach. You don’t waste a lot of time on this person. You simply say, “This is an open-source project, and it’s maintained by volunteers. If there’s a bug in the code, please submit a reproducible test case or a PR.” The next person has run into a very common error, with an easy workaround. You know you’ve seen this error a few times before, but can’t quite recall where the solution was posted. Stack Overflow? The wiki? The mailing list? After a few minutes of Googling, you paste a link and close the issue. The next person is a regular contributor. You recognize their name from various community forums and sibling projects. They’ve run into a very esoteric issue and have proposed a pull request to fix it. Unfortunately the issue is complicated, and so their PR contains many paragraphs of prose explaining it. Again, your eye darts to the hundreds of people still waiting in line. You know that this person put a lot of work into their solution, and it’s probably a reasonable one. The Travis tests passed, and so you’re tempted to just say “LGTM” and merge the pull request. However, you’ve been burned by that before. In the past, you’ve merged a PR without fully evaluating it, and in the end it led to new headaches because of problems you failed to foresee. Maybe the tests passed, but the performance degraded by a factor of ten. Or maybe it introduced a memory leak. Or maybe the PR made the project too confusing for new users, because it excessively complicated the API surface. If you merge this PR now, you might wind up with even more issues tomorrow, because you broke someone else’s workflow by solving this one person’s (very edge-casey) problem. So you put it on the back burner. You’ll get to it later when you have more time. The next person in line has found a new bug, but you know that it’s actually a bug in a sibling project. They’re saying that this is blocking them from shipping their app. You know it’s a big problem, but it’s one of many, and so you don’t have time to fix it right now. You respond that this looks like a genuine issue, but it’s more appropriate to open in another repo. So you close their issue and copy it into the other repo, then add a comment suggesting where they might look in the code to start fixing it. You doubt they’ll actually do so, though. Very few do. The next person just says “What’s the status on this?” You’re not sure what they’re talking about, so you look at the context. They’ve commented on a lengthy GitHub thread about a long-standing bug in the project. Many people disagreed on the proper solution to the problem, so it generated a lot of discussion. There are more than 20 comments on this particular issue, and it would take you a long time to read through them all to jog your memory. So you merely respond, “Sorry, this issue has been open for a while, but nobody has tackled it yet. We’re still trying to understand the scope of the problem; a pull request could be a good start!” The next person is just a GreenKeeper bot. These are easy. Except that this particular repo has fairly flaky tests, and the tests failed for what looks like a spurious reason, so you have to restart them to pass. You restart the tests and try to remind yourself to look into it later after Travis has had a chance to run. The next person has opened a pull request, but it’s on a repo that’s fairly active, and so another maintainer is already providing feedback. You glance through the thread; you trust the other maintainer to handle this one. So you mark it as read and move on. The next person has run into what appears to be a bug, and it’s not one you’ve ever seen before. But unfortunately they’ve provided scant details on how the problem actually occurred. What browser was it? What version of Node? What version of the project? What code did they use to reproduce it? You ask them for clarification and close the tab. The constant stream After a while, you’ve gone through ten or twenty people like this. There are still more than a hundred waiting in line. But by now you’re feeling exhausted; each person has either had a complaint, a question, or a request for enhancement. In a sense, these GitHub notifications are a constant stream of negativity about your projects. Nobody opens an issue or a pull request when they’re satisfied with your work. They only do so when they’ve found something lacking. Even if you only spend a little bit of time reading through these notifications, it can be mentally and emotionally exhausting. Your partner has observed that you’re always grumpy after going through this ritual. Maybe you found yourself snapping at her for no reason, just because you were put in a sour mood. “If doing open source makes you so angry, why do you even do it?” she asks. You don’t have a good answer. You could take a break; in fact you’ve probably earned it by now. In the past, you’ve even taken vacations of a week or two from GitHub, just for your own mental health. But you know that that’s exactly how you ended up in this situation, with hundreds of people patiently waiting. If you had just kept on top of your GitHub notifications, you’d probably have a more manageable 20-30 to deal with per day. Instead you let them pile up, so now there are hundreds. You feel guilty. In the past, for one reason or another, you’ve really let issues pile up. You might have seen an issue that was left unanswered for months. Usually, when you go back to address such an issue, the person who opened it never responds. Or they respond by saying, “I fixed my problem by abandoning your project and using another one instead.” That makes you feel bad, but you understand their frustration. You’ve learned from experience that the most pragmatic response to these stale issues is often just to say, &…

Zoom avatar
Zoom
06:45:41 PM

Anton Shakh has joined Public “Office Hours”

Vlad Ionescu avatar
Vlad Ionescu
name: Dependabot-hack

on:
  schedule:
    # run everyday at 11:00
    - cron:  '0 11 * * *'

jobs:
  Dependabot:
    runs-on: ubuntu-latest

    steps:
      - name: Clone repo
        uses: actions/[email protected]
        with:
          token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}

      - name: Update Terraform modules
        uses: patrickjahns/[email protected]
        with:
          github_dependency_token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
          token: ${{ secrets.PERSONAL_ACCESS_TOKEN }}
          target_branch: 'master'
Adam Blackwell avatar
Adam Blackwell
patrickjahns/dependabot-terraform-action

Github action for running dependabot on terraform repositories with HCL 2.0 - patrickjahns/dependabot-terraform-action

Vlad Ionescu avatar
Vlad Ionescu
Adds terraform 0.12 support by userhas404d · Pull Request #1388 · dependabot/dependabot-core

Fixes #1176 I opted for both hcl2json and terraform-config-inspect. hcl2json for terragrunt and terraform-config-inspect for tf 0.12 I wanted to go with terraform-config-inspect for both, but it di…

Zoom avatar
Zoom
06:50:16 PM

pepe amengual has joined Public “Office Hours”

Vlad Ionescu avatar
Vlad Ionescu
06:51:41 PM

How a PR looks

Vlad Ionescu avatar
Vlad Ionescu

^from some experiments

Vlad Ionescu avatar
Vlad Ionescu

https://octobox.io for GitHub notification hell

Octobox

Untangle your GitHub Notifications

Zoom avatar
Zoom
06:56:36 PM

Gabriel Tam has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Trailer - Accelerate your GitHub workflow | Product Hunt attachment image

Never miss a comment again. Track pull requests and issues across repositories, directly in your Notification Center or on any device.

Vlad Ionescu avatar
Vlad Ionescu
Trailer.app

Accelerate your GitHub workflow. Never miss a comment again. Track pull requests and issues across repositories, directly in your Notification Center or on any device.

Vlad Ionescu avatar
Vlad Ionescu
aws/aws-controllers-k8s

AWS Controllers for Kubernetes (ACK) is a project enabling you to manage AWS services from Kubernetes - aws/aws-controllers-k8s

venkata.mutyala avatar
venkata.mutyala

This is reallly cool. I’ve been looking for something like this. Is this well endorsed or being phased out?

aws/aws-controllers-k8s

AWS Controllers for Kubernetes (ACK) is a project enabling you to manage AWS services from Kubernetes - aws/aws-controllers-k8s

Vlad Ionescu avatar
Vlad Ionescu

IIRC Service Broker is being phased out, and aws-controllers-k8s is in RFC for v2

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
hashicorp/terraform-k8s

Terraform Operator for Kubernetes. Contribute to hashicorp/terraform-k8s development by creating an account on GitHub.

Mike Martin avatar
Mike Martin

Hey all - I’m trying to put together a career journey for SRE’s in our company. Does anyone have any examples? Whether public or you’d be willing to share?

Mike Martin avatar
Mike Martin

Almost like an internal job description…

Gowiem avatar
Gowiem
07:26:36 PM

Might be useful to throw into the mix:

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Kubernetes Retired

Kubernetes projects that are no longer actively maintained - Kubernetes Retired

Zoom avatar
Zoom
07:22:06 PM

Vicken Simonian has joined Public “Office Hours”

Zoom avatar
Zoom
08:43:50 PM

New Zoom Recording from our Office Hours session on 2020-08-12 is now available.

2020-08-05

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:14 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

1
sahil kamboj avatar
sahil kamboj

need to know about future with k3s

Zoom avatar
Zoom
06:27:00 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:03 PM

sahil kamboj has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:23 PM

Andrew Roth has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:30 PM

Nathaniel Alconcel has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:43 PM

Vicken Simonian has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:54 PM

Evgenii Prokofev has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:10 PM

Eddie Wizelman has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:41 PM

Vicken Simonian has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:58 PM

Tyler Stilwagen has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:27 PM

Robert Horrox has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:43 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:33 PM

Brian Tai has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:42 PM

Marcin Branski has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:10 PM

Nigel Kirby has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:46 PM

Gabriel Tam has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:04 PM

Adam Crown has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:34 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:18 PM

rb rb has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:01 PM

rb rb has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:23 PM

Adam Watson has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:30 PM

Neil Gealy has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:40 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:42:30 PM

Scott Rogers has joined Public “Office Hours”

Zoom avatar
Zoom
06:45:57 PM

Andrey Nazarov has joined Public “Office Hours”

Zoom avatar
Zoom
06:47:42 PM

Scott Rogers has joined Public “Office Hours”

Zoom avatar
Zoom
06:48:37 PM

Adam Blackwell has joined Public “Office Hours”

Zoom avatar
Zoom
06:50:44 PM

Babajide Hassan has joined Public “Office Hours”

Zoom avatar
Zoom
06:55:23 PM

PePe Amengual has joined Public “Office Hours”

Zoom avatar
Zoom
07:07:13 PM

Andrew Roth has joined Public “Office Hours”

Zoom avatar
Zoom
07:17:21 PM

Adam Blackwell has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Initial PR by 3h4x · Pull Request #1 · cloudposse/terraform-opsgenie-incident-management

what notes superlinter terrascan error is related to not supporting HCL2 yet. Issue already exist and feature should be soonish released accurics/terrascan#233

Vlad Ionescu avatar
Vlad Ionescu
reviewdog/action-tflint

Run tflint with reviewdog on pull requests to enforce best practices - reviewdog/action-tflint

Vlad Ionescu avatar
Vlad Ionescu
Terraform Code Reviews: Supercharged with Conftest

Learn how Doordash automated away some mundane code review tasks for infrastructure code.

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
open-policy-agent/conftest

Write tests against structured configuration data using the Open Policy Agent Rego query language - open-policy-agent/conftest

Andrey Nazarov avatar
Andrey Nazarov

Btw, @Erik Osterman (Cloud Posse) am I getting you right that you stopped using Atlantis?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ya, not really using it in new engagements. pushing towards terraform cloud / enterprise.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We still support atlantis for current customers and have many deployments of atlantis.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Just it doesn’t fit in a nice CI/CD workflow that promotes changes across multiple changes automatically in a pipeline

Andrey Nazarov avatar
Andrey Nazarov

We made some PoC couple of days ago, but we are leaning towards TF Cloud

:--1:1
Adam Blackwell avatar
Adam Blackwell

Do you know how much it might cost if a 200 person mostly opensource software organization were to onboard Terraform Enterprise.

Adam Blackwell avatar
Adam Blackwell

Cool looking OPA solution: https://github.com/fugue/regula

fugue/regula

Regula checks Terraform for AWS, Azure and GCP security and CIS compliance using Open Policy Agent/Rego - fugue/regula

Marcin Brański avatar
Marcin Brański

Maybe worth a try to test terraform with python?

https://github.com/GoogleCloudPlatform/terraform-python-testing-helper#example-usage This example looks really readable comparing to terratest

GoogleCloudPlatform/terraform-python-testing-helper

Simple Python test helper for Terraform. Contribute to GoogleCloudPlatform/terraform-python-testing-helper development by creating an account on GitHub.

1
Vlad Ionescu avatar
Vlad Ionescu
hashicorp/terraform-k8s

Terraform Operator for Kubernetes. Contribute to hashicorp/terraform-k8s development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Zoom chat

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(sorry everyone - I tend to not look at the zoom chat during the call)

Zoom avatar
Zoom
09:30:12 PM

New Zoom Recording from our Office Hours session on 2020-08-05 is now available.

2020-08-04

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Quick question - does anyone know if liveness probes continue to execute once a pod enters the Terminating state? If they do, and if they fail, will the pod be forcibly terminated and/or rescheduled? (https://github.com/kubernetes/kubernetes/issues/52817 looks somewhat related to my question)

    keyboard_arrow_up