#office-hours (2022-08)

Meeting password: sweetops

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cpco.io/slack-office-hours

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers!

https://cpco.io/slack-office-hours

Meeting password: sweetops

2022-08-02

david.gregory_slack avatar
david.gregory_slack

I’m just embarking on ‘decomposing’ a small number of shared/’monolithic’ AWS accounts into a larger number of focussed AWS accounts. Naming is hard. Feels like every account/workload/stage needs an expressive string to embed in its resource names, so when you’re referencing those resources across accounts, it’s easy to see what’s what without needing to parse account IDs. That string needs to be short (because of the many resource name length restrictions), expressive (so it actually helps) and unique (for facepalm avoidance). Am I thinking about this right? Any advice?

david.gregory_slack avatar
david.gregory_slack

That certainly looks useful, thanks!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’ll discuss on our call today

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I have an unusual situation with a client. They manage many remote sites and have physical devices (up to 20) at each location. Each device needs to send metrics to cloudwatch and upload files to S3 and they currently use static aws credentials (~~~/.aws/credentials). I would like to move them to IAM anywhere to use temporary credentials. The ask is if a device gets compromised how can we disable access to AWS from that particular device. I was thinking to use an IAM Role per device however they are expecting to have ~~~k devices online by the end of the year. I’d use Terraform to manage the roles and AWS organizations to use multiple accounts since there’s a 5k IAM role quota per account. Does this sound manageable? or is there a better approach?

2022-08-03

Mohammed Yahya avatar
Mohammed Yahya

Have a question? Please check out our Slack Community or visit our Slack Archive.

Slack Community

Describe the Feature

Adding missing EFS Terraform resources:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_file_system_policyhttps://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_replication_configuration

Goals

• EFS Policy will make sure TLS connection only to EFS and enable encryption in transit • more here https://aws.amazon.com/blogs/aws/new-for-amazon-efs-iam-authorization-and-access-points/ • EFS replication will help greatly with DR scenarios

Use Case

resource "aws_efs_file_system" "fs" {
  creation_token = "my-product"
}

resource "aws_efs_file_system_policy" "policy" {
  file_system_id = aws_efs_file_system.fs.id

  bypass_policy_lockout_safety_check = true

  policy = <<POLICY
{
    "Version": "2012-10-17",
    "Id": "ExamplePolicy01",
    "Statement": [
        {
            "Sid": "ExampleStatement01",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Resource": "${aws_efs_file_system.test.arn}",
            "Action": [
                "elasticfilesystem:ClientMount",
                "elasticfilesystem:ClientWrite"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "true"
                }
            }
        }
    ]
}
POLICY
}

resource "aws_efs_replication_configuration" "example" {
  source_file_system_id = aws_efs_file_system.fs.id

  destination {
    region = "us-west-2"
    kms_key_id = "xxx"
  }
}

also for the KMS key, if we used aws_kms_replica_key will allows to use the same key in DR regions:

provider "aws" {
  alias  = "primary"
  region = "us-east-1"
}

provider "aws" {
  region = "us-west-2"
}

resource "aws_kms_key" "primary" {
  provider = aws.primary

  description             = "Multi-Region primary key"
  deletion_window_in_days = 30
  multi_region            = true
}

resource "aws_kms_replica_key" "replica" {
  description             = "Multi-Region replica key"
  deletion_window_in_days = 7
  primary_key_arn         = aws_kms_key.primary.arn
}
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy G (Cloud Posse) will you be on the call today?

Have a question? Please check out our Slack Community or visit our Slack Archive.

Slack Community

Describe the Feature

Adding missing EFS Terraform resources:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_file_system_policyhttps://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_replication_configuration

Goals

• EFS Policy will make sure TLS connection only to EFS and enable encryption in transit • more here https://aws.amazon.com/blogs/aws/new-for-amazon-efs-iam-authorization-and-access-points/ • EFS replication will help greatly with DR scenarios

Use Case

resource "aws_efs_file_system" "fs" {
  creation_token = "my-product"
}

resource "aws_efs_file_system_policy" "policy" {
  file_system_id = aws_efs_file_system.fs.id

  bypass_policy_lockout_safety_check = true

  policy = <<POLICY
{
    "Version": "2012-10-17",
    "Id": "ExamplePolicy01",
    "Statement": [
        {
            "Sid": "ExampleStatement01",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Resource": "${aws_efs_file_system.test.arn}",
            "Action": [
                "elasticfilesystem:ClientMount",
                "elasticfilesystem:ClientWrite"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "true"
                }
            }
        }
    ]
}
POLICY
}

resource "aws_efs_replication_configuration" "example" {
  source_file_system_id = aws_efs_file_system.fs.id

  destination {
    region = "us-west-2"
    kms_key_id = "xxx"
  }
}

also for the KMS key, if we used aws_kms_replica_key will allows to use the same key in DR regions:

provider "aws" {
  alias  = "primary"
  region = "us-east-1"
}

provider "aws" {
  region = "us-west-2"
}

resource "aws_kms_key" "primary" {
  provider = aws.primary

  description             = "Multi-Region primary key"
  deletion_window_in_days = 30
  multi_region            = true
}

resource "aws_kms_replica_key" "replica" {
  description             = "Multi-Region replica key"
  deletion_window_in_days = 7
  primary_key_arn         = aws_kms_key.primary.arn
}
Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@Erik Osterman (Cloud Posse) I wasn’t planning to be on the call, but I can join if you want. LMK.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Mohammed Yahya this question is probably best sorted directly with @Jeremy G (Cloud Posse)

1
Mohammed Yahya avatar
Mohammed Yahya

and also there is a nice number of replication supported with Terraform now, which will help set up a DR easily:

• KMS

• ECR

• SecretManager

• EFS

• RDS

Gabriel avatar
Gabriel

Regarding GitOps …

Without going into too much detail, I use Github actions and a structure of application and env repositories. Applications publish charts and terraform modules. Env repositories use those in addition to a env specific configuration. Everything is automated, the infrastructure and Kubernetes resources state is located as code in the env repositories. There is a lot of reusability with GH actions, there are a few cool custom features like automatic conversion of terraform output to helm values, automatic blue/green deployments for k8s version upgrades and others. Because of GH actions I don’t have to deploy/manage/troubleshoot/fix anything myself, I don’t have to worry about scalability. Multi-tenancy is achieved by creating additional env/tenant repositories and updating configuration.

Questions are:

Are solutions like Flux/Argo worth the self managing/scaling/troubleshooting/migration efforts? Can those solutions handle terraform? Or would I need another service like Spacelift in addition to Flux/Argo? Anybody using Flux/Argo? Pros/Cons? Anybody migrated from GH to Flux/Argo?

Sorry for the general open ended questions, I know that in the end I will have to decide for myself but I was curious about your thoughts and experience. Maybe this is something for the Office Hours. I will also go through older episodes try to find GitOps content.

2
Eamon Keane avatar
Eamon Keane

I think a portion of your question was actually answered in the previous oh.

https://youtu.be/gWbBF-bflPw?t=2046

1
Eamon Keane avatar
Eamon Keane

just as an additional point, GCP are going on all in on the Kubernetes Resource Model (KRM) and have comprehensive coverage to let you manage your resources in yaml with the config managed by GCP (e.g. no need to centrally manage/backup the config in a management cluster).

https://cloud.google.com/config-connector/docs/reference/overview

Redhat is investing some resources in this KRM direction also with kcp (https://github.com/kcp-dev/kcp).

AWS is hedging its bets but defaulting to cloud formation it seems. Azure is going with flux and bicep. On AWS the two approaches that support the KRM model are Pulumi and Crossplane but to get the managed config cluster you need to get the hosted version and they may not have full coverage/good components yet.

1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Great - lets discuss today

Gabriel avatar
Gabriel

Listened to the episode, great input and points regarding flux/argo/etc. Will definitely help with decision making.

Gabriel avatar
Gabriel

Thanks a lot

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks @Gabriel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:01:37 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Zoom avatar
Zoom
06:27:25 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:40 PM

Ralf Pieper has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:51 PM

Chris Barnes has joined Public “Office Hours”

Zoom avatar
Zoom
06:27:56 PM

Dave Gregory has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:37 PM

Linda Pham (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:42 PM

dag viggo lokoeen has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:02 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:07 PM

Christopher Pieper has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:27 PM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:39 PM

Roy Sprague has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:49 PM

Zadkiel has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:02 PM

Ian Bartholomew has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:04 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:12 PM

Bradley Peterson has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:14 PM

Isa Aguilar has joined Public “Office Hours”

Ralf Pieper avatar
Ralf Pieper

I am interested in testimonials about Oracle Cloud.

1
Zoom avatar
Zoom
06:31:40 PM

Martin Palastanga has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:46 PM

Adam Buggia has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:51 PM
Zoom avatar
Zoom
06:32:01 PM

Michael Jenkins has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:32 PM

Charles Smith has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:28 PM

Zadkiel has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:35 PM

Andrew Vitko has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:01 PM

Ashwin Jacob has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:11 PM

Antarr Byrd has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:12 PM

Ben Smith (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:16 PM

Guilherme Borges has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:19 PM
Zoom avatar
Zoom
06:36:25 PM

Vincent Werner has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:34 PM

tamsky has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:44 PM

Michael Williams has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:25 PM

Yusuf has joined Public “Office Hours”

Zoom avatar
Zoom
06:40:57 PM

Naiman Daniels has joined Public “Office Hours”

Zoom avatar
Zoom
06:44:27 PM

Sherif Abdel-Naby has joined Public “Office Hours”

Zoom avatar
Zoom
06:54:14 PM

Azar AKB has joined Public “Office Hours”

Zoom avatar
Zoom
06:54:29 PM

Marc Tamsky has joined Public “Office Hours”

Zoom avatar
Zoom
07:00:51 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
07:01:18 PM

Paul Bullock has joined Public “Office Hours”

Zoom avatar
Zoom
07:08:42 PM

Isaac M has joined Public “Office Hours”

Zoom avatar
Zoom
07:09:33 PM

13153275398 has joined Public “Office Hours”

Zoom avatar
Zoom
07:09:35 PM

Isaac M has joined Public “Office Hours”

Zoom avatar
Zoom
07:10:24 PM

13153275398 has joined Public “Office Hours”

Zoom avatar
Zoom
07:10:48 PM

Isaac M has joined Public “Office Hours”

Zoom avatar
Zoom
07:14:10 PM

Sherif Abdel-Naby has joined Public “Office Hours”

2022-08-04

2022-08-05

2022-08-07

venkata.mutyala avatar
venkata.mutyala
CDK for Terraform Is Now Generally Availableattachment image

Cloud Development Kit for Terraform (CDKTF) has reached its first GA release, adding full support for Go and providing a GitHub action to use with Terraform Cloud.

2022-08-09

Isaac avatar

Discussion topic : I have the privilege of designing the VPC’s for my org and i’m looking for insights. What would you do differently if you had this luxury (e.g use IPAM e.t.c)? Should I go multi-account with multi-vpc or embrace the touted simplicity of Shared VPC?

Intro to Production-grade Design | Gruntwork Docs

With all the core concepts out of the way, let’s now discuss how to configure a production-grade VPC that looks

VPC sharing: A new approach to multiple accounts and VPC management | Amazon Web Servicesattachment image

My first interaction with AWS was immediately after the launch of the Asia Pacific (Sydney) AWS Region, just a bit over 6 years ago. Back then, the AWS Management Console had fewer services, and I quickly found the Amazon Virtual Private Cloud (VPC). In under 10 minutes, I could define a new VPC, with subnets, […]

    keyboard_arrow_up