#office-hours (2022-04)

“Office Hours” are every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers! https://cloudposse.com/office-hours

Public “Office Hours” are held every Wednesday at 11:30 PST via Zoom. It’s open to everyone. Ask questions related to DevOps & Cloud and get answers!

https://cpco.io/slack-office-hours

Meeting password: sweetops

2022-04-06

managedkaos avatar
managedkaos

Wanted to share this for office hours today. Its a pretty nifty tool that I’ve been putting to use lately. Its already helped me shave a couple dozen $$/mo off of a AWS account which will add up to hundreds/thousands by year end. https://www.infracost.io/ https://github.com/infracost/infracost

Cloud cost estimates for Terraform in pull requests | Infracostattachment image

Infracost helps engineers see cloud costs before launching resources. Map costs to code directly in pull requests. Take action directly in your workflow.

infracost/infracost

Cloud cost estimates for Terraform in pull requests Love your cloud bill!

Mazin Ahmed avatar
Mazin Ahmed
Monocle: How Chime creates a proactive security & engineering culture (Part 1)attachment image

Hear from David Trejo, a member of Chime’s Security Engineering Team, how he and the team created a proactive security culture at Chime

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Looks interesting

Monocle: How Chime creates a proactive security & engineering culture (Part 1)attachment image

Hear from David Trejo, a member of Chime’s Security Engineering Team, how he and the team created a proactive security culture at Chime

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Is monocle open source?

Mazin Ahmed avatar
Mazin Ahmed

Invited David to join us hopefully today!

wave2
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:11 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

deniz gökçin avatar
deniz gökçin

new member here wave . what is an office hour?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s a live session that we do - open mic, Q&A. usually about 30 people on the call.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

happens every wednesday

Chris Picht avatar
Chris Picht

easily the best hour you can spend to expand your thinking on DevOps.

1
deniz gökçin avatar
deniz gökçin

see you there!!

1
Zoom avatar
Zoom
06:30:46 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:50 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:59 PM

Emile Fugulin has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:02 PM

dario erregue has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:03 PM

David Hawthorne has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:09 PM

Michael Jenkins has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:12 PM

Mazin Ahmed has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:13 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:18 PM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:45 PM

deniz gokcin has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:51 PM

Mazin Ahmed has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:52 PM

Mohammed Yahya has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:32 PM

dag viggo lokoeen has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:56 PM

Jim Park has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:59 PM

Yusuf has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:21 PM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:24 PM

Connor High has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:38 PM

Connor High has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:40 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:46 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:00 PM

Ben Smith (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:56 PM

Jim C has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:06 PM

Jeremy (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:08 PM

Kris Musard has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:16 PM

Sherif Abdel-Naby has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:40 PM

stelios L has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:55 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:24 PM

Vicken Simonian has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:50 PM

Abraham Quintero has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:15 PM

Allen Lyons has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:24 PM

Ralf Pieper has joined Public “Office Hours”

Zoom avatar
Zoom
06:41:32 PM

Ross Rfd has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
infracost/infracost-gh-action

GitHub Action for Infracost. Shows cloud cost estimates for Terraform in pull requests.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
infracost/infracost-gh-action

GitHub Action for Infracost. Shows cloud cost estimates for Terraform in pull requests.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Terraform Cloud/Enterprise | Infracostattachment image

Infracost is an official HashiCorp partner. We work together to ensure that Infracost can be used alongside Terraform Cloud and integrated into your workflow.

Zoom avatar
Zoom
06:47:10 PM

Amer Zec has joined Public “Office Hours”

Zoom avatar
Zoom
06:47:56 PM

wasim k has joined Public “Office Hours”

Zoom avatar
Zoom
06:51:10 PM

dag viggo lokoeen has joined Public “Office Hours”

Mazin Ahmed avatar
Mazin Ahmed
trufflesecurity/trufflehog

Find credentials all over the place

2
Zoom avatar
Zoom
07:00:47 PM

Jose Figueredo has joined Public “Office Hours”

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)
Amazon EKS now supports Kubernetes 1.22 | Amazon Web Servicesattachment image

The Amazon Elastic Kubernetes Service (Amazon EKS) team is pleased to announce support for Kubernetes 1.22. Amazon EKS, Amazon EKS Distro, and Amazon EKS Anywhere can now run Kubernetes version 1.22. The upstream project theme for this release is “Reaching New Peaks.” The theme for the release, according to release lead Savitha Raghunathan, is due to what she […]

1
Sherif avatar

Anyone going to KubeCon EU 2022 ? kubernetes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Install and Set Up kubectl on Windows

Before you begin You must use a kubectl version that is within one minor version difference of your cluster. For example, a v1.23 client can communicate with v1.22, v1.23, and v1.24 control planes. Using the latest compatible version of kubectl helps avoid unforeseen issues. Install kubectl on Windows The following methods exist for installing kubectl on Windows: Install kubectl binary with curl on Windows Install on Windows using Chocolatey or Scoop Install kubectl binary with curl on Windows Download the latest release v1.

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

Kubernetes version 1.18 on Amazon EKS will no longer be supported on March 31st, 2022. On this day, you will no longer be able to create new 1.18 clusters. Existing 1.18 clusters will be automatically updated by Amazon EKS to the latest available platform version of Kubernetes version 1.19 through a gradual deployment process after the end of support date.

We recommend you upgrade existing 1.18 clusters and worker nodes to at least 1.19 as soon as practical.

Learn more:

Amazon EKS Kubernetes Versions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)
Using F5's Terraform modules in an air-gapped environment

Introduction IT Industry research, such as Accelerate, shows improving a company’s ability to deliver software is critical to their overall success. The following key practices and design principles are cornerstones to that improvement. Version control of code and configuration Automation of Deploy…

Zoom avatar
Zoom
07:17:16 PM

wasim k has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
kubernetes-sigs/slack-infra

Tooling for kubernetes.slack.com

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Zoom avatar
Zoom
07:33:35 PM

deniz gokcin has joined Public “Office Hours”

sytten avatar

https://github.com/kube-rs/kube-rs https://github.com/linkerd/linkerd2/tree/main/policy-controller Last is linkerd new policy controller using kube-rs, can be a good inspiration

kube-rs/kube-rs

2022-04-08

Matt Gowie avatar
Matt Gowie

Question for next office-hours — How have folks automated AWS IAM Access Key + Secret Key rotation policies (e.g. IAM keys are only active for 1 year)? I’ve always thought about doing it, but never went down the rabbit hole. Huge bonus points if somebody has a simple solution that doesn’t involve lambda

Jeremy (UnderGrid Network Services) avatar
Jeremy (UnderGrid Network Services)

Are you talking for IAM users or for CI/CD automation? If the latter I don’t use them at all

Matt Gowie avatar
Matt Gowie

IAM users is what I’m thinking about.

Matt Gowie avatar
Matt Gowie

Though I would like to it for service accounts as well. I have a client who is using IAM service account users for CircleCI.

Jeremy (UnderGrid Network Services) avatar
Jeremy (UnderGrid Network Services)

I actually use SSO for users and then for my CI/CD as I use GitHub actions I actually use OIDC and grant roles so in both cases I’m using STS credentials that are short lived

1
Jeremy (UnderGrid Network Services) avatar
Jeremy (UnderGrid Network Services)

Alternatively use Hashicorp Vault to hand out dynamic credentials

Jeremy (UnderGrid Network Services) avatar
Jeremy (UnderGrid Network Services)

using an Access Key/Secret Key is my last resort for precisely this reason

Brandon Miller avatar
Brandon Miller

Well, I thought lambda WAS the easy way . That being said, if you want more fine-grained control over the timing you could throw up a cron job on a VM somewhere. Caveat being that you’d end up having to use a service account to auto-rotate the keys, which in turn would need keys, and would quickly turn into key rotation inception. I threw up my own OIDC solution that rotates JWKS at a set interval using cron. It’s a headache for sure… Vault may have this functionality? Not sure, I have only used it a handful of times.

Jeremy (UnderGrid Network Services) avatar
Jeremy (UnderGrid Network Services)

Vault could definitely handle it… just setup the AWS credentials and then request them from Vault and get temporary access just like any other Vault lease.

2022-04-11

2022-04-12

sohaibahmed98 avatar
sohaibahmed98
DataOps platform for Apache Kafka and Kubernetesattachment image

Lenses for your DataOps enterprise platform, to operate with confidence on Apache Kafka with intuitive ui and fine-grained controls

sohaibahmed98 avatar
sohaibahmed98
Aiven - Data infrastructure made simple

Aiven’s fully managed, open source cloud data platform lets you create the data pipelines you always dreamed of – in under 10 minutes. On all major clouds everywhere.

sytten avatar

My life just got better

2022-04-13

Eric Berg avatar
Eric Berg

Question for today: Pros and cons of using the latest EKS AMIs or managing roll-out of new AMI releases manually.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:55 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Brandon Miller avatar
Brandon Miller

hmm I do have a question that maybe is out of scope for office hours, if so feel free to not answer or just point me in the right direction, but are there any examples on the use of helmfile that showcase how one might use it in a “bigger” situation? I.E, multiple releases spanning 1-2 env’s that maybe pulls from a helm repo and overrides some of those values as well. Maybe it’s just a complete misunderstanding on my part but I was finding the examples in the helmfile repo kind-of lackluster, and a lot of the guides I came across on the internet pretty much showcased an entirely different way of doing it.. Not sure if there’s a “standard” or just kind of the wild west lol. Also, I guess a followup would be, if I decide to just roll with Argocd and Gitops is helmfile still relevant in that regard? It seems like a pretty lightweight wrapper overall but due to complete noobness I can’t be sure they’re compatible.

1
Brandon Miller avatar
Brandon Miller

Lol I guess my particular situation is that I’ve learned how to install helm charts from a repo, and can make one from scratch, but somehow am scratching my head on how to create a repeatable CI\CD pipeline in which I can say “I need istio, these are the values I need you to overwrite, please do the thing”

Brandon Miller avatar
Brandon Miller

My first office hours, so apologies if this isn’t the place

deniz gökçin avatar
deniz gökçin

another question for today: what kind of a git repo structure do you recommend if I want to seperate my terraform modules in repository A along with the terragrunt files for some common resources like the vpc, alb some security groups etc. and store my app source code and app specific terragrunt files in repository B(ecr, ecs, app security groups etc.) I also want to note that my ecs hcl file needs dependencies from the common resources(like vpc id)

1
Zoom avatar
Zoom
06:30:34 PM

deniz gokcin has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:36 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:37 PM

Brian Pauley has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:41 PM

Marcos Soutullo has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:42 PM

MALCON MOREIRA has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:44 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:46 PM

dag viggo lokoeen has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:56 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:57 PM

Jim Park has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:00 PM

Arthur Dent has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:02 PM

Andy Roth has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:03 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:05 PM

Jesus Martinez has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:08 PM

Oliver Schoenborn has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:09 PM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:10 PM

Scott Mathson has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:10 PM

Michael Jenkins has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:15 PM

Stevan Arychuk has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:29 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:35 PM

Brandon James Miller has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:26 PM

Jeremy (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:45 PM

Jose Figueredo has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:28 PM

Mazin Ahmed has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:33 PM

Josh B has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:56 PM

dag viggo lokoeen has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:08 PM

Waqar Ahmed has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:18 PM

Marcelo Santoro has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:02 PM

Connor High has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:26 PM

Jeremy Bouse has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:41 PM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:00 PM

Connor High has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:16 PM

Paul Bullock has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:27 PM

Jose Figueredo has joined Public “Office Hours”

Zoom avatar
Zoom
06:41:08 PM

Marin Purgar has joined Public “Office Hours”

Zoom avatar
Zoom
06:41:43 PM

Guilherme Borges has joined Public “Office Hours”

Zoom avatar
Zoom
06:42:46 PM

Shaun Wang has joined Public “Office Hours”

Zoom avatar
Zoom
06:43:22 PM

Allen Lyons has joined Public “Office Hours”

Zoom avatar
Zoom
06:45:15 PM

michael dizon has joined Public “Office Hours”

Zoom avatar
Zoom
06:46:19 PM

Josh B has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Community Note

• Please vote on this issue by adding a :+1: reaction to the original issue to help the community and maintainers prioritize this request • Please do not leave “+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Update from HashiCorp, 2022-02-25

Hi everyone,

In light of how much traction this issue has gained, and the amount of information contained within, @joe-a-t has graciously allowed us to make edits to the description of this GitHub Issue to better reflect its current state, and so that we may pin it to make it more visible. We would like to sincerely thank him for the opportunity to do so. For the sake of transparency, and to not lose any context, the original issue description and title will be retained beneath this update.

Breakdown

On 2022-02-10, the AWS Provider team at HashiCorp released a new major version of the AWS Provider for Terraform (v4.0.0), which included breaking changes to the aws_s3_bucket resource. The update introduced several independent aws_s3_bucket_* resources to manipulate the configuration of S3 buckets, where these configuration options previously were arguments to the aws_s3_bucket resource itself. This issue spawned out of that change, has received quite a bit of valuable feedback, and is being used as a centralized meeting point for ongoing discussions around these changes.

Information around these changes may be found:

Upcoming changes in Version 4.0 of the AWS ProviderTerraform AWS Provider 4.0 Refactors S3 Bucket Resource (blog post)Terraform AWS Provider Version 4 Upgrade Guide

Action Items

Given the amount of feedback that we received after the release, the AWS Provider team has been following this issue (as well as any other avenues of feedback we can find) and taking steps to try to alleviate some of the burden placed on operators with this release. The team continues to investigate additional avenues, and will continue to update this issue with additional information where possible. These efforts include:

Completed

A clarifying comment on what decisions were made, and why they were made • An update to the aforementioned blog post to provide more clarification around how resource configuration has changed (namely with regards to configurable -> read-only/computed arguments • A clarifying comment on when imports are needed and an accompanying update to the migration guide

Currently Investigating

  1. Tooling to help with configuration migration

    We are currently investigating potential tooling options to help operators migrate from legacy aws_s3_bucket resources to the new aws_s3_bucket_* resources. We have identified potential existing tooling and have reached out to the owner of the tool to try to help coordinate efforts to make the tool robust enough to share publicly. This is still in the early phases, so the information we can share on it is relatively limited. Despite this, we feel it’s appropriate to share that we are looking into it, given the large amount of community interest.

  2. Marking deprecated arguments as optional in AWS Provider v4.x

    This change aims at splitting the difference between the behavior in v3.x and v4.x by marking deprecated arguments as optional. The goal here is to allow operators more time to transition before the deprecated arguments are fully removed in v5.x. Optional arguments would display deprecation warnings, but would not prevent Terraform from running, as was the behavior with the initial v4.x releases. Documentation will be written in order to provide as much clarity as possible around how to cope with these changes within configurations.

  3. Backporting aws_s3_bucket_* resources to AWS Provider v3.x

    This change aims to bring the new aws_s3_bucket_* resources into the v3.x series of releases in order to give operators more time to plan and execute migration to the new resources without needing to make the jump to v4.x. This would allow operators to use aws_s3_bucket resources alongside the new aws_s3_bucket_* resources, configuring buckets and updating configurations to the new standards as they see fit, without worry of deprecation notices or failed Terraform operations

    Given that backporting resources in this way falls outside of our normal processes, we would like to be explicit around expectations. Bugs in distinct aws_s3_bucket_* resources will be backported to ensure functionality, but additional features will not be backported. Documentation will be updated as well to ensure that this expectation is clearly communicated.

Original issue information

Title: Change the S3 bucket refactor to be a new resource instead of modifying the existing one

Description

Do not change the existing aws_s3_bucket resource in v4 and instead provide a new aws_s3_minimal_bucket or similar resource that people can use if they want your changes to split out various settings into stand alone resources.

My company has literally thousands of AWS buckets configured through Terraform from hundreds of different root modules. Having to run separate terraform import commands for all of the settings you split out into separate resources per your upgrade instructions in https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-4-upgrade#s3-bucket-refactor is quite frankly an insane demand to make for users.

Right now, our only options appear to be:

  1. Pin to v3 permanently and miss out on all future enhancements/bugfixes/features that the AWS provider publishes.
  2. Spend countless hours running (or making our own tool to run) thousands of terraform import commands in hundreds of Terraform directories.

Please reconsider this change immediately or at least provide tooling to ease adoption.

New or Affected Resource(s)

• aws_s3_bucket

Potential Terraform Configuration References

• #0000

Update 2/22/2022

FYI, I (@joe-a-t) chatted with HashiCorp separately and gave them permission to update the issue (including the title and this comment) if they would like the issue to more accurately reflect the work that they are planning on doing.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@managedkaos

Community Note

• Please vote on this issue by adding a :+1: reaction to the original issue to help the community and maintainers prioritize this request • Please do not leave “+1” or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Update from HashiCorp, 2022-02-25

Hi everyone,

In light of how much traction this issue has gained, and the amount of information contained within, @joe-a-t has graciously allowed us to make edits to the description of this GitHub Issue to better reflect its current state, and so that we may pin it to make it more visible. We would like to sincerely thank him for the opportunity to do so. For the sake of transparency, and to not lose any context, the original issue description and title will be retained beneath this update.

Breakdown

On 2022-02-10, the AWS Provider team at HashiCorp released a new major version of the AWS Provider for Terraform (v4.0.0), which included breaking changes to the aws_s3_bucket resource. The update introduced several independent aws_s3_bucket_* resources to manipulate the configuration of S3 buckets, where these configuration options previously were arguments to the aws_s3_bucket resource itself. This issue spawned out of that change, has received quite a bit of valuable feedback, and is being used as a centralized meeting point for ongoing discussions around these changes.

Information around these changes may be found:

Upcoming changes in Version 4.0 of the AWS ProviderTerraform AWS Provider 4.0 Refactors S3 Bucket Resource (blog post)Terraform AWS Provider Version 4 Upgrade Guide

Action Items

Given the amount of feedback that we received after the release, the AWS Provider team has been following this issue (as well as any other avenues of feedback we can find) and taking steps to try to alleviate some of the burden placed on operators with this release. The team continues to investigate additional avenues, and will continue to update this issue with additional information where possible. These efforts include:

Completed

A clarifying comment on what decisions were made, and why they were made • An update to the aforementioned blog post to provide more clarification around how resource configuration has changed (namely with regards to configurable -> read-only/computed arguments • A clarifying comment on when imports are needed and an accompanying update to the migration guide

Currently Investigating

  1. Tooling to help with configuration migration

    We are currently investigating potential tooling options to help operators migrate from legacy aws_s3_bucket resources to the new aws_s3_bucket_* resources. We have identified potential existing tooling and have reached out to the owner of the tool to try to help coordinate efforts to make the tool robust enough to share publicly. This is still in the early phases, so the information we can share on it is relatively limited. Despite this, we feel it’s appropriate to share that we are looking into it, given the large amount of community interest.

  2. Marking deprecated arguments as optional in AWS Provider v4.x

    This change aims at splitting the difference between the behavior in v3.x and v4.x by marking deprecated arguments as optional. The goal here is to allow operators more time to transition before the deprecated arguments are fully removed in v5.x. Optional arguments would display deprecation warnings, but would not prevent Terraform from running, as was the behavior with the initial v4.x releases. Documentation will be written in order to provide as much clarity as possible around how to cope with these changes within configurations.

  3. Backporting aws_s3_bucket_* resources to AWS Provider v3.x

    This change aims to bring the new aws_s3_bucket_* resources into the v3.x series of releases in order to give operators more time to plan and execute migration to the new resources without needing to make the jump to v4.x. This would allow operators to use aws_s3_bucket resources alongside the new aws_s3_bucket_* resources, configuring buckets and updating configurations to the new standards as they see fit, without worry of deprecation notices or failed Terraform operations

    Given that backporting resources in this way falls outside of our normal processes, we would like to be explicit around expectations. Bugs in distinct aws_s3_bucket_* resources will be backported to ensure functionality, but additional features will not be backported. Documentation will be updated as well to ensure that this expectation is clearly communicated.

Original issue information

Title: Change the S3 bucket refactor to be a new resource instead of modifying the existing one

Description

Do not change the existing aws_s3_bucket resource in v4 and instead provide a new aws_s3_minimal_bucket or similar resource that people can use if they want your changes to split out various settings into stand alone resources.

My company has literally thousands of AWS buckets configured through Terraform from hundreds of different root modules. Having to run separate terraform import commands for all of the settings you split out into separate resources per your upgrade instructions in https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-4-upgrade#s3-bucket-refactor is quite frankly an insane demand to make for users.

Right now, our only options appear to be:

  1. Pin to v3 permanently and miss out on all future enhancements/bugfixes/features that the AWS provider publishes.
  2. Spend countless hours running (or making our own tool to run) thousands of terraform import commands in hundreds of Terraform directories.

Please reconsider this change immediately or at least provide tooling to ease adoption.

New or Affected Resource(s)

• aws_s3_bucket

Potential Terraform Configuration References

• #0000

Update 2/22/2022

FYI, I (@joe-a-t) chatted with HashiCorp separately and gave them permission to update the issue (including the title and this comment) if they would like the issue to more accurately reflect the work that they are planning on doing.

1
Zoom avatar
Zoom
06:48:39 PM

Kris Musard has joined Public “Office Hours”

Zoom avatar
Zoom
06:48:44 PM

shreenu kumar has joined Public “Office Hours”

Zoom avatar
Zoom
06:49:55 PM

Nimesh Amin has joined Public “Office Hours”

Zoom avatar
Zoom
06:52:39 PM

wasim k has joined Public “Office Hours”

Zoom avatar
Zoom
07:15:39 PM

Amer Zec has joined Public “Office Hours”

Zoom avatar
Zoom
07:15:50 PM

Ross Rfd has joined Public “Office Hours”

2022-04-14

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

As requested, here’s the link to the Containers from the Couch livestream I’ll be doing later: https://twitter.com/rothgar/status/1514627176804470787

2022-04-20

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:09 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

deniz gökçin avatar
deniz gökçin

due to many announcements, there was no time for Q&A last week. will you answer questions from last week?

Andy Miguel avatar
Andy Miguel

@deniz gökçin we carry unanswered questions over to the following week, yes

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yep, and we have yours on the list from last week

Zoom avatar
Zoom
06:28:16 PM

Arthur Dent has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:24 PM

Andy Miguel (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:37 PM

venkata mutyala has joined Public “Office Hours”

Zoom avatar
Zoom
06:28:38 PM

Andy Roth has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:13 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:35 PM

Emile Fugulin has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:40 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:02 PM

Igor Miltchman has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:19 PM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:23 PM

Brian Pauley has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:30 PM

Brandon James Miller has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:52 PM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:12 PM

Derek Davis has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:30 PM

deniz gokcin has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:33 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:42 PM

Jim Park has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:36 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:39 PM

Josh B has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:40 PM

Steven Hopkins (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:50 PM

Kris Musard has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:51 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:03 PM

dag viggo lokoeen has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:00 PM

Jose Figueredo has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:05 PM

Oliver Schoenborn has joined Public “Office Hours”

Zoom avatar
Zoom
06:34:57 PM

Yusuf has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:20 PM

Ayobami Bamigboye has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:18 PM

Stevan Arychuk has joined Public “Office Hours”

Zoom avatar
Zoom
06:37:59 PM

Scott Mathson has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:02 PM

Marc Tamsky has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:31 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
06:40:18 PM

Alexandr Vorona has joined Public “Office Hours”

Zoom avatar
Zoom
06:40:54 PM

Michael Jenkins has joined Public “Office Hours”

Zoom avatar
Zoom
06:41:26 PM

Andrew Vitko has joined Public “Office Hours”

Zoom avatar
Zoom
06:41:39 PM

Connor High has joined Public “Office Hours”

roth.andy avatar
roth.andy

Here’s JHipster’s page on the bug bounty stuff they do: https://www.jhipster.tech/bug-bounties/

Zoom avatar
Zoom
06:53:35 PM

Jim C has joined Public “Office Hours”

Zoom avatar
Zoom
06:59:40 PM

Ayobami Bamigboye has joined Public “Office Hours”

Zoom avatar
Zoom
06:59:49 PM

Steven Hopkins (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
07:01:38 PM

Florain Drescher has joined Public “Office Hours”

Zoom avatar
Zoom
07:11:44 PM

Paul Bullock has joined Public “Office Hours”

Zoom avatar
Zoom
07:12:05 PM

Jim C has joined Public “Office Hours”

Zoom avatar
Zoom
07:12:54 PM

Mohammed Almusaddar has joined Public “Office Hours”

Zoom avatar
Zoom
07:26:15 PM

emem emem has joined Public “Office Hours”

sohaibahmed98 avatar
sohaibahmed98
FairwindsOps/reckoner

Declaratively install and manage multiple Helm chart releases

sohaibahmed98 avatar
sohaibahmed98
FairwindsOps/nova

Find outdated or deprecated Helm charts running in your cluster.

sohaibahmed98 avatar
sohaibahmed98
FairwindsOps/goldilocks

Get your resource requests “Just Right”

2022-04-27

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:00:37 PM

@here office hours is starting in 30 minutes! Remember to post your questions here.

Zoom avatar
Zoom
06:29:27 PM

Andy Miguel (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:36 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:36 PM

Christopher Picht has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:41 PM

Andrew Elkins has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:42 PM

Allan Mohr has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:47 PM

Isa Aguilar has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:52 PM

Eric Berg has joined Public “Office Hours”

Zoom avatar
Zoom
06:29:56 PM

David Hawthorne has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:14 PM

Guilherme Borges has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:15 PM

Emile Fugulin has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:19 PM

dag viggo lokoeen has joined Public “Office Hours”

Andrew Elkins avatar
Andrew Elkins

Opinions and thoughts on K8s ingress controllers for high volume deployments.

• traefik (currently using)

• haproxy (currently considering)

• nginx

• ??

1
Andrew Elkins avatar
Andrew Elkins

Running in to some scaling issues when under heavy load. It’s possible that traefik needs further configuration

Andrew Elkins avatar
Andrew Elkins

Curious what the community thinks is the best option or what options should we be investigating.

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

Where is this running? AWS, GCP, Azure, on-prem?

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

High volume deployments = lots of deploys or lots of requests?

1
Andrew Elkins avatar
Andrew Elkins

AWS and Azure. Own k8s cluster on AWS. Main pain point is on deploy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Scaling containers on AWS in 2022attachment image

Comparing how fast containers scale up in 2022 using different orchestrators on AWS

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

nginx-ingress open source version is abysmal for large deployments as adding/removing pods causes a SIGHUP

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

IMO you should go straight to the amazon load balancer controller and work with AWS who can help with any performance issues.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Can you also elaborate on what qualifies as
high volume deployments

Zoom avatar
Zoom
06:30:32 PM

Andy Roth has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:35 PM

Matt Calhoun has joined Public “Office Hours”

Zoom avatar
Zoom
06:30:57 PM

Connor High has joined Public “Office Hours”

Zoom avatar
Zoom
06:31:18 PM

Erik Osterman (Cloud Posse) has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:13 PM

Paul Bullock has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:37 PM

Josh B has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:39 PM

Brandon Miller has joined Public “Office Hours”

Zoom avatar
Zoom
06:32:58 PM

Alex Kiss has joined Public “Office Hours”

Zoom avatar
Zoom
06:33:30 PM

Isaac M has joined Public “Office Hours”

roth.andy avatar
roth.andy

What advice do you have for how to communicate expectations when people decide to use something brand new that is still super beta/rough, are having problems, and are annoyed that things aren’t working?

1
Jonathan Le avatar
Jonathan Le

I’d re-iterate that problems arising from beta products they decide to move forward with that THEY own supporting/troubleshooting them/care and feeding. I wouldn’t roadblock them though.

tim.j.birkett avatar
tim.j.birkett

It depends… who are the people? What is it they’re using?

Zoom avatar
Zoom
06:34:07 PM

Lucas Andrade has joined Public “Office Hours”

Zoom avatar
Zoom
06:35:34 PM

Tim Gourley has joined Public “Office Hours”

Zoom avatar
Zoom
06:36:00 PM

Benjamin Smith has joined Public “Office Hours”

Zoom avatar
Zoom
06:38:06 PM

Ralf Pieper has joined Public “Office Hours”

Zoom avatar
Zoom
06:39:27 PM

Josh B has joined Public “Office Hours”

Zoom avatar
Zoom
06:40:24 PM

Vlad Ionescu has joined Public “Office Hours”

Zoom avatar
Zoom
06:41:36 PM

PePe Amengual has joined Public “Office Hours”

jose.amengual avatar
jose.amengual

looks like is similar to what this does https://github.com/apparentlymart/terraform-aws-tf-registry

apparentlymart/terraform-aws-tf-registry

Terraform module for creating a simple private Terraform registry in AWS with DynamoDB

Zoom avatar
Zoom
06:43:38 PM

Ross Rfd has joined Public “Office Hours”

Zoom avatar
Zoom
06:44:42 PM

Arthur Dent has joined Public “Office Hours”

Zoom avatar
Zoom
06:46:31 PM

Brian Pauley has joined Public “Office Hours”

Zoom avatar
Zoom
06:48:21 PM

Tony Scott has joined Public “Office Hours”

Zoom avatar
Zoom
06:49:21 PM

Matt Gowie has joined Public “Office Hours”

Zoom avatar
Zoom
06:55:41 PM

dag viggo lokoeen has joined Public “Office Hours”

Zoom avatar
Zoom
06:58:46 PM

Justin Davis has joined Public “Office Hours”

Zoom avatar
Zoom
07:00:39 PM

Florain Drescher has joined Public “Office Hours”

Zoom avatar
Zoom
07:01:02 PM

Nimesh Amin has joined Public “Office Hours”

Zoom avatar
Zoom
07:02:39 PM

Michael Jenkins has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
resource "awsutils_expiring_iam_access_key" "default" {
  count   = module.this.enabled && local.create_expiring_access_key ? 1 : 0
  user    = local.username
  max_age = var.iam_access_key_max_age
}
Zoom avatar
Zoom
07:08:45 PM

Andrew Thompson has joined Public “Office Hours”

Zoom avatar
Zoom
07:14:46 PM

Amer Zec has joined Public “Office Hours”

Zoom avatar
Zoom
07:19:01 PM

dag viggo lokoeen has joined Public “Office Hours”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
gofiber/fiber

Express inspired web framework written in Go

3

2022-04-28

Brandon Miller avatar
Brandon Miller

https://css-tricks.com/making-mermaid-diagrams-in-markdown/

https://github.blog/2022-02-14-include-diagrams-markdown-files-mermaid/#<i class="em em-~"</i>text=Mermaid%20has%20been%20getting%20increasingly,feature%20to%20everyone%20on%20GitHub>.

In reference to the question asked yesterday about how to keep expectations realistic during OSS development… I was reading this and it seems like a great solution. Essentially use Mermaid to maintain a roadmap within your readme. First link shows an example roadmap, while the second is the Github-specific implementation details from their announcement post. Haven’t done this yet myself, so I can’t say with 100% certainty that it will be painless but it does seem to be possible :)

Making Mermaid Diagrams in Markdown | CSS-Tricksattachment image

While Mermaid diagrams are not Markdown-exclusive, they is Markdown-inspired. using the same markup abstractions.

Include diagrams in your Markdown files with Mermaid | The GitHub Blogattachment image

Mermaid is a JavaScript based diagramming and charting tool that takes Markdown-inspired text definitions and creates diagrams dynamically in the browser.

    keyboard_arrow_up