#office-hours (2020-09)

Meeting password: sweetops

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours

Meeting password: sweetops

2020-09-23

Jay Zalowitz avatar
Jay Zalowitz

hey, apparently all the docs arent letting me do any of this due to the provider if im reading it correct

Jay Zalowitz avatar
Jay Zalowitz

action { name = “${var.application_name}-ecs-worker” category = “Deploy” owner = “AWS” provider = “ECS” input_artifacts = [“task”] version = “1” configuration = { ClusterName = aws_ecs_cluster.ecs_cluster.name ServiceName = “${var.application_name}-worker” # ActionMode = “REPLACE_ON_FAILURE” # OutputFileName = “CreateStackOutput.json” # StackName = “MyStack” # ImageDefinitionsFile = “worker-imagedefinitions.json” # TaskDefinitionTemplateArtifact = “task” # TaskDefinitionTemplatePath = “worker-imagedefinitions.json” } }

Jay Zalowitz avatar
Jay Zalowitz
Support AWS Provider 3 · Issue #65 · cloudposse/terraform-aws-codebuild

Found a bug? Maybe our Slack Community can help. Describe the Bug The version of the AWS Provider is pinned to 2.x in versions.tf. Since an installed version of AWS provider must satisfy therequire…

Jay Zalowitz avatar
Jay Zalowitz

id love to talk about this if you are open to it today

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:34 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

anybody have any experience with or recommendations for AWS WAF alternatives like signal science or anything.

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Simplest static site hosting in aws that I can use security groups with to keep internal?

Thinking a fargate task that cicd builds with static site and hosts with something like “ran” and done. S3 buckets don’t seem to have anything with groups and ec2 while ok wouldn’t allow me to set target tasks at 1 for it to autoheal itself.

Any better way?

Zoom avatar
Zoom
06:27:41 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:55 PM

Adam Crown has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:10 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:43 PM

Jeremy CloudPosse has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:57 PM

pepe amengual has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:03 PM

Fernando Castillo has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:06 PM

Anere Faithful has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:29 PM

Marcin Brański has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:44 PM

Patrick Joyce has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:59 PM

Michael Londeen has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:09 PM

Vitali Bystritski has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:34 PM

Christian Roy has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:42 PM

Justin Ober has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:59 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:06 PM
Zoom avatar
Zoom
06:32:11 PM

Kareem Shahin has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:24 PM

vicken has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:45 PM

Brian Tai has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:50 PM

Oliver Schoenborn has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:03 PM

Andrey Nazarov has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:13 PM

Nigel Kirby has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:42 PM

David Lundgren has joined Public “Office Hours”

btai avatar

Topic I’m interested in if we have time: Grafana users, have you found any useful community dashboards that you would recommend / what is the general opinion about community dashboards. Alternatively, how do you manage your Grafana dashboards? Should it only be codified + read only in the UI

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It's the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Zoom avatar
Zoom
06:35:13 PM

Anere Faithful has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:50 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:58 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:04 PM

Michael Holt has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:23 PM

Zadkiel AHARONIAN has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:31 PM

Nick James has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:34 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:52 PM

Marc Tamsky has joined Public “Office Hours”

Vlad Ionescu avatar
Vlad Ionescu

For downscaling k8s deployments on a schedule: https://github.com/hjacobs/kube-downscaler

hjacobs/kube-downscaler

Scale down Kubernetes deployments after work hours - hjacobs/kube-downscaler

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Neat! will check this out

hjacobs/kube-downscaler

Scale down Kubernetes deployments after work hours - hjacobs/kube-downscaler

Zoom avatar
Zoom
06:43:35 PM

Neil Gealy has joined Public “Office Hours”

Zoom avatar
Zoom
06:44:45 PM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
06:48:49 PM

Jim Park has joined Public “Office Hours”

Zoom avatar
Zoom
06:51:33 PM

Juan Soto has joined Public “Office Hours”

Zoom avatar
Zoom
06:54:19 PM

Laurence Giglio has joined Public “Office Hours”

Zoom avatar
Zoom
07:01:11 PM

Andrew Roth has joined Public “Office Hours”

Zoom avatar
Zoom
07:02:09 PM
roth.andy avatar
roth.andy
Custom Variable Validation in Terraform 0.13 attachment image

We’re excited to announce that custom variable validation is being released as a production-ready feature in Terraform 0.13. Custom Variable Validation was introduced as a language experiment in Terraform 0.12.20 and builds upon the type system introduced in Terraform 0.12 by allowing configurations to contain validation conditions for a given variable.

pjaudiomv avatar
pjaudiomv

waf?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ahhhh!! sorry, we ran out of time today @

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ll get to WAF next wednesday

:--1:1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

question for today: what is proper way of ensuring that kubectl command called in terraform (via local_exec) will succeed? I often (not all the time) find the command runs before the EKS cluster API server is ready so terraform aborts. If I re-run it again, that 10-20 seconds is sufficient for the server to be ready so terraform then completes the apply. I tried a few things, without success. Any docs on this would be awesome.

Zoom avatar
Zoom
08:50:53 PM

New Zoom Recording from our Office Hours session on 2020-09-23 is now available.

2020-09-22

2020-09-21

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

is there some way I can get tf to load a directory of variable files?

2020-09-18

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

:wave: Hi guys this is Nitin here and I have just came across this slack channel. If this is not the right channel then please do let me know. As part of provisioning EKS cluster on AWS we are exploring terraform-aws-eks-cluster https://github.com/cloudposse/terraform-aws-eks-cluster What is the advantage of using cloud posse terraform module over the community published terraform module to provision EKS cluster on AWS Thanks a lot

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Do you pin the version of TF and/or your providers/plugins? :one: No, I always use the latest Terraform and latest version of all plugins/providers

:two: I pin my Terraform (like 0.12.28) but don’t pin the providers (always use latest version of “aws” etc) 2 @, @ :three: I pin Terraform AND the providers (like aws 3.5.0) 3 @roth.andy, @, @PePe Created by @ with /poll

View original message

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:40:20 PM
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@roth.andy what a bummer

roth.andy avatar
roth.andy

yeah

roth.andy avatar
roth.andy

if you are running shell script I guess you could script it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ya

2020-09-17

2020-09-16

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:33 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

1
Zoom avatar
Zoom
06:26:16 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:24 PM

Taras Dyshkant has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:41 PM

Vicken Simonian has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:45 PM

Giles Billenness has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:51 PM

Adam Crown has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:59 PM

Neil Gealy has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:17 PM

Andrew Roth has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Incorrect "Provider produced inconsistent final plan" error when changing count or for_each of resources with create_before_destroy · Issue #25631 · hashicorp/terraform

When referencing multiple instances of a resource with create_before_destroy, reducing the number of instances will not be correctly updated on the first apply. For example: locals { things = { fir…

Zoom avatar
Zoom
06:30:19 PM

Paul Obalonye has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy (Cloud Posse) what’s the link to the cycle issue you reported?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@here starting now

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)
Cycle error when removing a resource along with create_before_destroy · Issue #26226 · hashicorp/terraform

Terraform fails to apply a plan, citing a dependency cycle, but I think that is wrong. I am not positive, because I do not quite understand how to parse the error message I am getting; maybe if I c…

Zoom avatar
Zoom
06:31:54 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:00 PM

Alex Siegman has joined Public “Office Hours”

Chris Picht avatar
Chris Picht

Anyone use https://github.com/jckuester/awsweeper ? Is there a better tool out there for blanking out an AWS account? When I am trying to make sure that my Code creates all of the Infrastructure I have, I find destroying to be nearly as important as creating.

jckuester/awsweeper

A tool for cleaning your AWS account. Contribute to jckuester/awsweeper development by creating an account on GitHub.

roth.andy avatar
roth.andy

No idea which is better but some people have been using https://github.com/rebuy-de/aws-nuke

rebuy-de/aws-nuke

Nuke a whole AWS account and delete all its resources. - rebuy-de/aws-nuke

:--1:1
Zoom avatar
Zoom
06:32:18 PM

Brian Tai has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:45 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:41 PM

Jeremy CloudPosse has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:54 PM

Ian Bartholomew has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:37 PM

Paul Obalonye has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:53 PM

David Lundgren has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:17 PM
Zoom avatar
Zoom
06:35:21 PM

Kareem Shahin has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:22 PM

Oludahun Bade-Ajidahun has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:35 PM

Robert Horrox has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:48 PM

Andrew Elkins has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:13 PM

Jim Park has joined Public “Office Hours”

Zoom avatar
Zoom
06:41:01 PM
Zoom avatar
Zoom
06:41:59 PM

azam has joined Public “Office Hours”

Zoom avatar
Zoom
06:43:59 PM

Anton Shakh has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Is anyone relying on the undefined behaviour of Helmfile that a multiple negated conditions in a single selector like helmfile -l foo!=foo,bar!=bar is unexpectedly treated as an OR sometimes?

I’m redefining it to be always AND, so that the behavior is consistent:

https://github.com/roboll/helmfile/pull/1478

This might be just a bug but I wanted inform you all for clarity because this seems like a long-standing bug anyway. Thanks!

Gowiem avatar
Gowiem
Add an alternative AWS provider for the DNS validation by dboesswetter · Pull Request #25 · cloudposse/terraform-aws-acm-request-certificate

In my current project I need to request certificates for a zone which lives in a different account. To let this module do the validation with this zone, I needed to use an alternative AWS provider …

Zoom avatar
Zoom
06:46:02 PM
roth.andy avatar
roth.andy
06:46:40 PM

Anybody use a bot to merge code? I’m wondering what that looks like under the hood

1
loren avatar
loren

we use dependabot and mergify for this, yes

:--1:1
loren avatar
loren

the mergify config uses a series of rules with conditions and actions. when the condition matches, it applies the action

loren avatar
loren

dependabot has its own config. it monitors the various package ecosystems and CVEs, and opens pull requests to update dependencies that match the conditions in its config

loren avatar
loren

dependabot is a github service now, so enabling it with permissions is managed in the repo settings

loren avatar
loren

mergify is a external service that has a github integration, and it needs to be approved for write permissions to the repo

loren avatar
loren

and if you have branch protection enabled with the setting “Restrict who can push to matching branches” then you need to add the mergify bot-user there

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’re likely circling back to mergify after many failed attempts doing it with GitHub actions

Zoom avatar
Zoom
06:46:53 PM

Marc Tamsky has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Convert to GitHub actions by osterman · Pull Request #392 · cloudposse/packages

what Drop codefresh pipeline for building docker image why Use github action instead for easier open source adoption

Zoom avatar
Zoom
06:48:11 PM

Omer Sen has joined Public “Office Hours”

Zoom avatar
Zoom
06:49:24 PM

alejandro chacon has joined Public “Office Hours”

Zoom avatar
Zoom
06:50:38 PM

Zadkiel AHARONIAN has joined Public “Office Hours”

Zoom avatar
Zoom
06:51:48 PM

pepe amengual has joined Public “Office Hours”

Zoom avatar
Zoom
06:52:03 PM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
06:54:25 PM

Adam Blackwell has joined Public “Office Hours”

Zoom avatar
Zoom
06:57:22 PM

ivan pedro has joined Public “Office Hours”

Adam Blackwell avatar
Adam Blackwell
cloudposse/reference-architectures

[WIP] Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Zoom avatar
Zoom
07:03:39 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
07:05:56 PM

Juan Soto has joined Public “Office Hours”

Zoom avatar
Zoom
07:10:42 PM

Blaise pabon has joined Public “Office Hours”

Zoom avatar
Zoom
07:12:33 PM
Zoom avatar
Zoom
07:16:21 PM

Ian Bartholomew has joined Public “Office Hours”

Adam Blackwell avatar
Adam Blackwell
07:26:05 PM

Side question about IAM access, we no longer use cross account assumptions for console access and instead use OneLogin. I’m curious if this diverges from the CloudPosse reference architecture and if others do something similar with Okta or OneLogin. If there are downsides that I’m not aware of, I’d love to know about them.

(in these sample screenshots I, as an SRE, only have admin and readonly for each account, but developers often have various other roles)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy (Cloud Posse)

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Yes, it diverges from the Cloud Posse reference architecture, which uses cross-account assume role. This provides a logistical advantage in that a single set of AWS credentials will support working on any environment. We used to generate a separate Geodesic shell and Git repo for each account, but we found that it created far too much work to keep accounts (dev/staging/prod) in sync. When we consolidated the configuration for all accounts into a single repo, the advantage of being able to assume a role in any account became much more pronounced.

This also includes having CI/CD tools that get a single set of credentials and operate on multiple accounts.

Adam Blackwell avatar
Adam Blackwell

Second side question: aws-nuke was mentioned in the beginning of office hours, which I know is used in this reference architecture:

https://github.com/cloudposse/testing.cloudposse.co/blob/master/.github/workflows/aws-nuke.yml https://github.com/cloudposse/testing.cloudposse.co/blob/4d02425da9a97bb8e7cbe61987d511f0ed6d1e4c/.github/workflows/aws-nuke.yml

I’m curious if others use this, but chose to run the workflow on private runners and use an IAM role to avoid needing to give AWS credentials to Github and if there are cons to the second approach.

cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

Gowiem avatar
Gowiem

@Adam Blackwell I created a TF module to spin this up as a scheduled task in ECS: https://github.com/masterpointio/terraform-aws-nuke-bomber

It supports what you’re targeting. I’m using it in my own testing account.

masterpointio/terraform-aws-nuke-bomber

A Terraform module to create a bomber which nukes your cloud environment on a schedule - masterpointio/terraform-aws-nuke-bomber

:--1:1
Gowiem avatar
Gowiem

A lot, lot more code than @Erik Osterman (Cloud Posse)’s aws-nuke GH action config, but does have some advantages.

Adam Blackwell avatar
Adam Blackwell

Cool, which advantages did you have in mind when writing it?

Gowiem avatar
Gowiem

@Adam Blackwell I think I was probably just looking for another Terraform / ECS project to open source. It’s a bit heavy weight for what it does honestly, but for your 2 mentioned requirements it does fit well:

  1. Can use IAM role via ECS metadata endpoint
  2. Private workers It’s self contained to the account too, so just closing the entire account would be all the cleanup you’d need to do.
Adam Blackwell avatar
Adam Blackwell

Ha, that’s reasonable motivation :-).

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ya, I empathize with the “heavy weight” part. We just wanted to deploy a single container for atlantis with ECS fargate and we ended up with https://github.com/cloudposse/terraform-aws-ecs-atlantis (a massive module)

cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

Zoom avatar
Zoom
08:44:45 PM

New Zoom Recording from our Office Hours session on 2020-09-16 is now available.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Can someone chime in on the pros and cons of using terraform “workspace”? I’m trying to see how to structure TF for multiple environments and most of the “advanced” gurus prefer to avoid it. This is the one im following and I’m so confused as a beginner newb 

https://www.oreilly.com/library/view/terraform-up-and/9781491977071/ch04.html

:100:3

2020-09-11

2020-09-10

Andrey Nazarov avatar
Andrey Nazarov

I’m watching the latest episode. Regarding of version-checker , Lens had the same functionality, there were some bugs in it, but it was more or less usable. Don’t know its current state though.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ya, lens has some nice stuff for that too.

Andrey Nazarov avatar
Andrey Nazarov

As of fat module vs decomposition I would join a @Vlad Ionescu’s camp. In the past we struggled a lot managing everything via just one tf apply. It looked cool at first that you theoretically could fire up all the things from the ground up. But then came the pain. Mostly it came firstly, as Vlad pointed out, from fundamental changes in modules and secondly - from unstable third-party or home-grown tf providers. And we’d encountered spoiled state quite often until we decomposed things in a way similar to CloudPosse’s 4-layered approach.

But, yes, it’s a matter of your use cases. For some fat modules might work perfectly.

Just my two cents on this.

1
Andrew Red avatar
Andrew Red

Hey, do you have a reference on CloudPosse’s 4-layered approach?

Andrey Nazarov avatar
Andrey Nazarov
07:50:43 AM

It was just a screen shared by @Erik Osterman (Cloud Posse) during one of the office-hours sessions. That’s all I know. Probably Eric could shed some light on it. I made a screenshot. I hope I didn’t violate anything and sorry for the quality)

:100:1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks @Andrey Nazarov for sharing the screenshot

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Haven’t yet published anywhere, but definitely something we need to do because it helps make it a lot easier to explain things

1

2020-09-09

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:19 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

David J. M. Karlsen avatar
David J. M. Karlsen

waiting to get in

gugaiz avatar
gugaiz

Hi, I am trying to create a security group with

module "app_db_sg" {
  source = "terraform-aws-modules/security-group/aws//modules/postgresql"

  name   = "${local.environment}-db-sg"
  vpc_id = module.vpc.vpc_id

  description = "Security group that controls access to DB"
  use_name_prefix = false

  computed_ingress_with_source_security_group_id = [
    {
      rule                     = "postgresql-tcp"
      source_security_group_id = module.app_beanstalk_environment[0].security_group_id
    }
  ]
  number_of_computed_ingress_with_source_security_group_id = 1

}

but I am getting One of ['cidr_blocks', 'ipv6_cidr_blocks', 'self', 'source_security_group_id', 'prefix_list_ids'] must be set to create an AWS Security Group Rule. I just want to know how I can check that the value returned by module.app_beanstalk_environment[0].security_group_id is right. I am using tfctl so terraform console does not work for me (or I am not sure how to use it).

kareem.shahin avatar
kareem.shahin

not sure of an easy way outside of querying the output from state using terraform output

gugaiz avatar
gugaiz

Sorry, I am new to terraform, but I am trying with $ terraform output module.app_beanstalk_environment[0].security_group_id and getting

Warning: No outputs found
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@here our devops #office-hours starting now! join us to talk shop zoom https://cloudposse.zoom.us/j/508587304

sheldonh avatar
sheldonh

Gitpod did this recently, a full setup of their EKS environment and export of the terraform plan and more with a single docker run. I was pretty impressed, esp as never having used helm it was amazing to see it all pretty much just work

Zoom avatar
Zoom
06:36:49 PM

David Karlsen has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:00 PM

raphael francis has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:09 PM

Andrew Roth has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:37 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:41 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:01 PM

Sheldon Hull has joined Public “Office Hours”

sheldonh avatar
sheldonh

Question:

• GitHub Actions —> Any easy way to trigger an action on demand?

• GitHub Actions –> Any update on any dashboard/centralized reporting for actions that have been run in an organization?

Zoom avatar
Zoom
06:39:10 PM

Anton Shakh has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:10 PM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:15 PM

Adam Crown has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:23 PM

Ian Bartholomew has joined Public “Office Hours”

Zoom avatar
Zoom
06:40:16 PM

PePe Amengual has joined Public “Office Hours”

Zoom avatar
Zoom
06:42:03 PM

Kareem Shahin has joined Public “Office Hours”

Vlad Ionescu avatar
Vlad Ionescu


GitHub Actions —
Any easy way to trigger an action on demand?
Yup, you can triger them manually now. They have a button! https://github.blog/changelog/2020-07-06-github-actions-manual-triggers-with-workflow_dispatch/

GitHub Actions: Manual triggers with workflow_dispatch - GitHub Changelog attachment image

GitHub Actions: Manual triggers with workflow_dispatch

:100:2
Zoom avatar
Zoom
06:42:59 PM

Victor Ma has joined Public “Office Hours”

Zoom avatar
Zoom
06:43:18 PM

Babajide Hassan has joined Public “Office Hours”

Zoom avatar
Zoom
06:43:42 PM

Taras Dyshkant has joined Public “Office Hours”

Zoom avatar
Zoom
06:44:35 PM

Andrew Elkins has joined Public “Office Hours”

Zoom avatar
Zoom
06:44:59 PM

Robert Horrox has joined Public “Office Hours”

Zoom avatar
Zoom
06:46:12 PM

Christopher Picht has joined Public “Office Hours”

sheldonh avatar
sheldonh

Chef got acquired. I think that’s a big change

Zoom avatar
Zoom
06:47:00 PM

eeic berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:49:14 PM
Zoom avatar
Zoom
06:50:01 PM

Zadkiel AHARONIAN has joined Public “Office Hours”

Zoom avatar
Zoom
06:51:47 PM

Pedro Torres has joined Public “Office Hours”

David J. M. Karlsen avatar
David J. M. Karlsen
evryfs/github-actions-runner-operator

K8S operator for scheduling github actions runner pods - evryfs/github-actions-runner-operator

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@mumoshu heads up

evryfs/github-actions-runner-operator

K8S operator for scheduling github actions runner pods - evryfs/github-actions-runner-operator

:--1:2
mumoshu avatar
mumoshu

@ nice to meet you! fyi, i’m co-maintaining a similar operator https://github.com/summerwind/actions-runner-controller#runnerdeployments. i’m looking forward to any form of collaboration with you :smiley:

at glance yours seems to support podTemplate for customizing the runner pod flexibly? that sounds great. mine has only limited support for customizing pod specs currently, although there has not been much complaint due to that yet.

summerwind/actions-runner-controller

Kubernetes controller for GitHub Actions self-hosted runnners - summerwind/actions-runner-controller

David J. M. Karlsen avatar
David J. M. Karlsen

hi! I think we crossed paths in some github repo earlier!

David J. M. Karlsen avatar
David J. M. Karlsen

I actually had a look at your operator in the beginning, but had a need for org-wide runners and was in contact with GH when they beta’ed it

David J. M. Karlsen avatar
David J. M. Karlsen

to be fair, I was on hunt for a project which required go (and k8s), so that’s how it ended there

David J. M. Karlsen avatar
David J. M. Karlsen

it’s a bit tricky to run it containerized due to docker under docker - and runners not really being designed for that as a start, but for most cases it works fine

David J. M. Karlsen avatar
David J. M. Karlsen

the next thing I’m looking into is improved security (and api quotas) by solving https://github.com/evryfs/github-actions-runner-operator/issues/75

David J. M. Karlsen avatar
David J. M. Karlsen
evryfs/github-actions-runner-operator

K8S operator for scheduling github actions runner pods - evryfs/github-actions-runner-operator

Vlad Ionescu avatar
Vlad Ionescu
philips-labs/terraform-aws-github-runner

Terraform module for scalable GitHub action runners on AWS - philips-labs/terraform-aws-github-runner

Zoom avatar
Zoom
06:56:49 PM

Michael Martin has joined Public “Office Hours”

Zoom avatar
Zoom
06:58:21 PM
Zoom avatar
Zoom
07:05:49 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
07:07:10 PM

Maged Abdelmoeti has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
AWS SaaS Factory Program

AWS SaaS Factory provides partners with direct access to technical and business content, best practices, and architects that can guide and accelerate their delivery of SaaS solutions on AWS.

sheldonh avatar
sheldonh

@Erik Osterman (Cloud Posse) can you share the parsing logic of the yaml? I’ve not found many good “flatten” examples. That part would be useful in my own work if possible

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s an example for opsgenie//github.com/cloudposse/terraform-opsgenie-incident-management/tree/master/examples/config>

cloudposse/terraform-opsgenie-incident-management

Contribute to cloudposse/terraform-opsgenie-incident-management development by creating an account on GitHub.

cool-doge1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-opsgenie-incident-management

Contribute to cloudposse/terraform-opsgenie-incident-management development by creating an account on GitHub.

sheldonh avatar
sheldonh

Such a great community. Always enjoy chatting with all of you

2
Vlad Ionescu avatar
Vlad Ionescu

^AWS SaaS Factory presentation on programatic Control Planes.

Vlad Ionescu avatar
Vlad Ionescu
07:21:34 PM

Screenshot from the above video

Andrey Nazarov avatar
Andrey Nazarov

Haven’t got a chance to participate today. Looking forward to watching the recorded version

:100:1
Vlad Ionescu avatar
Vlad Ionescu

Full disclosure: I’ll miss office-hours next week as I have a conflict

1
Zoom avatar
Zoom
09:00:35 PM

New Zoom Recording from our Office Hours session on 2020-09-09 is now available.

2020-09-04

2020-09-03

2020-09-02

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:22 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Jeff Wozniak avatar
Jeff Wozniak

i’m curious to know what the overall strategy is for handling the new version of the aws provider.

Zoom avatar
Zoom
06:25:00 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:13 PM

Jeff Wozniak has joined Public “Office Hours”

Zoom avatar
Zoom
06:26:30 PM

Anton Shakh has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:00 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:16 PM

Soham Jadiya has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:37 PM

Sheldon Hull has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:21 PM

17133029948 has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:33 PM

Ian Bartholomew has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@here our devops #office-hours starting now! join us to talk shop zoom https://cloudposse.zoom.us/j/508587304

Zoom avatar
Zoom
06:31:44 PM

Andrey Nazarov has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:50 PM

Michael Holt has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:02 PM

Kareem Shahin has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:04 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:06 PM

Neil Gealy has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:12 PM

nat lie has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:30 PM

Adam Crown has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:47 PM

Hugo Samayoa has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:54 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:23 PM

Jawwad Yunus has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:25 PM

Isa Aguilar has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Cloud Posse Explains

Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.

:--1:1
Zoom avatar
Zoom
06:33:38 PM

James Connolly has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:58 PM

Babajide Hassan has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:24 PM

Sean Conley has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:23 PM

Marc Tamsky has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:20 PM

Nick James has joined Public “Office Hours”

Zoom avatar
Zoom
06:43:40 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:46:13 PM

John D has joined Public “Office Hours”

sheldonh avatar
sheldonh

For versioning this is nice.

I have this running right now in a similar manner. I use gitversion that calculates the semver based on branching. If you do a breaking change you manually set the tag to bump otherwise all the patch versions generate pre-release draft releases on branch and normal minor patch.

Zoom avatar
Zoom
06:47:22 PM

Sheldon Hull has joined Public “Office Hours”

Zoom avatar
Zoom
06:48:26 PM

Andrew Roth has joined Public “Office Hours”

Zoom avatar
Zoom
06:49:14 PM

Olivier Chaine has joined Public “Office Hours”

Zoom avatar
Zoom
06:50:13 PM

Zadkiel AHARONIAN has joined Public “Office Hours”

sheldonh avatar
sheldonh

I adopted the null label stuff and love it. All my resources have randomized pet names with standard prefix. I have wanted to figure out the null label stuff so I’m excited to try this. The submodules having null label has confused me but this looks like it will help with this problem

sheldonh avatar
sheldonh

Nothing like provisioning a bunch of servers and my coworkers seeing “snarky-puppy-rds-foobar”

Zoom avatar
Zoom
06:59:17 PM

Sheldon Hull has joined Public “Office Hours”

sheldonh avatar
sheldonh
07:08:04 PM

Rube goldberg lol. Totally.

sheldonh avatar
sheldonh

Interesting. You are saying on IAM Service accounts that you wouldn’t manage this user provisioning through a master terraform security repo for example? How do you setup the user provisioning to be IAC at that point?

sheldonh avatar
sheldonh

Can we get a picture of this diagram and mind addressing sometime why terraform is only on foundational, when i’d guess that it has impact in all of the tiers

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
aws/aws-controllers-k8s

AWS Controllers for Kubernetes (ACK) is a project enabling you to manage AWS services from Kubernetes - aws/aws-controllers-k8s

Zoom avatar
Zoom
07:20:30 PM

Sheldon Hull has joined Public “Office Hours”

sheldonh avatar
sheldonh

If we have time at the end, I want to know what others are doing to provision their IAM user and defined role/groups across accounts via code. Are you using terraform pull request driven workflow, lambda with json in s3 buckets, etc?

1
Vlad Ionescu avatar
Vlad Ionescu

https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs :
To use these credentials with the Kubernetes provider, they can be interpolated into the respective attributes of the Kubernetes provider configuration block.
IMPORTANT WARNING _When using interpolation to pass credentials to the Kubernetes provider from other resources, these resources SHOULD NOT be created in the same apply operation where Kubernetes provider resources are also used. This will lead to intermittent and unpredictable errors which are hard to debug and diagnose. The root issue lies with the order in which Terraform itself evaluates the provider blocks vs. actual resources. Please refer to this section of Terraform docs> for further explanation._</span

The best-practice in this case is to ensure that the cluster itself and the Kubernetes provider resources are managed with separate apply operations. Data-sources can be used to convey values between the two stages as needed.

Provider Configuration - Configuration Language - Terraform by HashiCorp

Providers are responsible in Terraform for managing the lifecycle of a resource: create, read, update, delete.

Andrey Nazarov avatar
Andrey Nazarov
terraform-google-modules/terraform-google-kubernetes-engine

A Terraform module for configuring GKE clusters. Contribute to terraform-google-modules/terraform-google-kubernetes-engine development by creating an account on GitHub.

Andrey Nazarov avatar
Andrey Nazarov
mumoshu/terraform-provider-helmfile

Deploy Helmfile releases from Terraform. Contribute to mumoshu/terraform-provider-helmfile development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Announcing HashiCorp Terraform Cloud Business Tier attachment image

Today we’re announcing availability of the new Business tier offering for Terraform Cloud which includes enterprise features for advanced security, compliance and governance, the ability to execute multiple runs concurrently, and flexible support options.

sheldonh avatar
sheldonh

Managing terraform workspaces with terraformenterprise provider (import from yaml perhaps) is the only scalable way to do this

1
sheldonh avatar
sheldonh

You have to manage terraform workspaces via code at that point

sheldonh avatar
sheldonh

The challenge with managing terraform with terraform is pretty much that on free tier there is no additional levels of permission for folks. You can’t have readers, just admins. So you have to bump up the pay and then ensure that workspaces are NOT allowed to be created by any other method than code, or I feel it’s a lost cause to ensure this is managed consistently.

sheldonh avatar
sheldonh

Kinda frustrating but i don’t see how you can effectively manage manual + automated workspaces in a solid way if you don’t just have it all managed by a service account instead.

Vlad Ionescu avatar
Vlad Ionescu

^ this sounds like an awesome topic for next #office-hours . What changes come from managing 2-3 workspaces, 10s of workspaces, 100s, 1000s

Zoom avatar
Zoom
09:51:59 PM

New Zoom Recording from our Office Hours session on 2020-09-02 is now available.

Andrey Nazarov avatar
Andrey Nazarov

Following up this multi-level or multi-tier structure you showed. Having this stuff decoupled means that you define different pipelines for them. Is this like a pipeline per level? Separate repo for each? Or it might be several pipelines within the same level? By the pipeline I essentially mean terraform apply command which applies a set of modules. What is the CloudPosse approach?

How do you deal with different chicken-and-egg scenarios? Like you deploy Gitlab and its runners as level 3, but you need runners to run terraform commands on level 1 or even to deploy this Gitlab:)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Will answer next wednesday

    keyboard_arrow_up