best option i can think of would be something like cdktf to generate the .tf configs for you

are you creating the same set of resources in many regions? Or put another way, do you have one set of resources which are created repeatedly with only minor differences?

The best approach would be to split up your resources into many stacks. 3000 resources is too many for a single stack and you will suffer much operational pain doing so.

Apologies, I meant 300. Yes I am creating the same resources in many regions. A typical example of what I am trying to do is shown below. I know that we cannot use count or for_each in providers, is there any way I can do this as a work-around

here is a module that is setup for multi-region. give it a good study to see how to do multi-region in pure terraform… https://github.com/nozaq/terraform-aws-secure-baseline

Thank you. I will check this out

In your example, the only thing that is changing is the region, so you could create multiple executions over the same code and get provider to accept a variable for region which has a different value on each run.

You’ll need to take care with the state file as you’ll end up overwriting it if all you change is the region but if you use terraform workspaces then you can have a state file per region

You also would not need to use a provider alias in this case as each state file would only contain the resources of one account/region all created by one provider

Thank you for this

Hey Ohad

Hey Ohad. Cool, I’ll check it out. Super relevant to what we’re working on (in fact, been looking into env0 as well)

What you can do is, upgrade the s3-logs version in the module main.tf internally for you, from 0.26.0 to 1.1.0 until the PR is merge.

Or the hard way

• terraform init

• sed -i “s/0.26.0/1.1.0/” …

• terraform init

• terraform apply

Thanks @José

@Max Lobur (Cloud Posse) can we prioritize rolling out the release branch manager to this repo

(and any S3 repos)

@Jeremy G (Cloud Posse) let’s discuss on ARB today

Modules terraform-aws-s3-bucket and terraform-aws-s3-log-storage have been updated to work with the new AWS S3 defaults. Other modules dependent on them should be updated soon.

Thanks @Jeremy G (Cloud Posse)

Thanks for the quick response and fix

Running into a weird error when creating a bucket.

module "s3_bucket" {
  source                 = "cloudposse/s3-bucket/aws"
  version                = "3.1.0"
  acl                    = "private"
  context            = module.this.context
  kms_master_key_arn = module.kms_key.alias_arn
  sse_algorithm      = var.sse_algorithm
}

│ Error: error creating S3 bucket (xxxx) accelerate configuration: UnsupportedArgument: The request contained an unsupported argument.
│       status code: 400, request id: XZMWPBJNR1DEBXJQ, host id: CViaAfJ5ZhqVM2t6XViRKzlz+SATKo38dDxSISOQ3nihJM3K6qyWoBVizpP+ywZPrugDBbii/wQ=
│ 
│   with module.s3_bucket.aws_s3_bucket_accelerate_configuration.default[0],
│   on .terraform/modules/s3_bucket/main.tf line 48, in resource "aws_s3_bucket_accelerate_configuration" "default":
│   48: resource "aws_s3_bucket_accelerate_configuration" "default" {

Yes, that is weird, and I cannot reproduce it, so I will need more information if you want me to investigate further.

Reading this https://docs.aws.amazon.com/AmazonS3/latest/userguide/transfer-acceleration.html#transfer-acceleration-requirements

i don’t see govcloud listed

I guess that means it’s not supported?

@Erik Osterman (Cloud Posse) @Andriy Knysh (Cloud Posse) Please review and approve s3-bucket PR 180 to fix @Michael Dizon’s problem above.

@Michael Dizon I hope this is fixed in v3.1.1. Please try it out and report back.

@Jeremy G (Cloud Posse) yeah that worked!! thank you! can terraform-aws-s3-log-storage get that bump also?

The PR is approved, but right now cannot be merged and a new release cut because of another GitHub outage

terraform-aws-s3-log-storage v1.3.1 released

terraform-aws-vpc-flow-logs-s3-bucket v1.0.1 released

terraform-aws-lb-s3-bucket v0.16.4 released

check blocks for validating infrastructure: Module and configuration authors can now write independent check blocks within their configuration to validate assertions about their infrastructure.

i find it funny they are described this way (which is pretty accurate) but the truth is if one of the checks fails, execution continues

Some information if anyone else runs into this issue: https://github.com/confluentinc/terraform-provider-confluent/issues/251#issuecomment-1541025164. I am not sure how to set settings on a cluster that is not of type dedicated. I suppose there might be a way to call the REST API to do so.

I believe that’s it, although there are interesting sub-types, like how dangling resources get deposed

I asked my good friend Chat, Chat GPT, and this is what he came back with. The response sounds reasonable but I have yet to validate…
In Terraform, there are several resource actions that can be reported in a plan. The most common ones are:
• Created: A new resource will be created.
• Updated: An existing resource will be updated.
• Replaced: An existing resource will be replaced with a new one.
• Destroyed: An existing resource will be destroyed.
• No changes: The resource has not changed since the last Terraform run.
Additionally, there are a few less common actions that may appear in a plan:
• Tainted: A resource has been marked as tainted and will be recreated on the next Terraform run.
• Imported: An existing resource has been imported into Terraform state.
• Ignored: A resource has been ignored due to the ignore_changes setting in the configuration.
• Moved: A resource has been moved to a different location within the infrastructure.
It’s worth noting that some of these actions, such as “tainted” and “ignored”, are specific to Terraform and not used in other infrastructure-as-code tools.

@managedkaos iust curious, what are you building?

A plan parser for GitHub actions. I came across one and didn’t like it so I thought I’d put one together. Doesn’t have to be API complete but wanted to cover as much ground as possible.

(That was going to be my guess!)

If you publish it, share it!

I came across one and didn’t like it I have seen quite a few of them, though…

I will admit I didn’t look too far.

But yeah, I’ll definitely share when it comes together.

It’ll be a while before i have a working “action” (i’m just scripting in a workflow at the moment) but here’s a preview:

clearly my summaries are off

Sharing some of my related links:

https://github.com/dflook/terraform-github-actions/

https://blog.jennasrunbooks.com/silence-refreshing-state-highlight-changes-in-github-actions-terraform-plan-output

@Dan Miller (Cloud Posse)

Also, @loren shared this one before: https://suzuki-shunsuke.github.io/tfcmt/

Based on https://github.com/mercari/tfnotify

Supports Slack and GitHub comments out of the box

@managedkaos did you end up building anything? we’re looking into this right now.

I did not build a complete, deployable action…. only some awk code that pares the plan and translates it to Markdown for display in a GitHub Actions job summary. Will see if I can find the code for reference….

https://gist.github.com/managedkaos/77e88902ed35077676b4c91b7e9e1e3b

I’m doing all the plan analysis in plain text but i know it could me better using a JSON plan. Just had to move on to another project.

you’re getting confused between attributes and blocks, one of Terraform’s warts (IMO)

attributes are set in a resource like:

key = val

blocks are set like:

block {
  # attribs...
}

tag is documented as being a block, so remove the =

uuughh.. thanks… I figured it was something I was doing. Just didnt know what.

does

`which terraform` version

Return 1.4.6

?

Maybe something we can bring up on #office-hours today

Generally you want the target bucket to be in a separate region, which means a separate AWS provider, so you have to decide how you want to organize that. You can have a root module with 2 providers, pass the 2nd one into s3-bucket to create the replication destination, then pass the output of that into another s3-bucket instantiation using the default AWS provider to create the primary bucket.

@Erik Osterman (Cloud Posse)

Support for CodeCommit would be nice

1.- anyone that can run terraform can exfiltrate your root credentials, not an atlantis problem per se

i think it’s a CI problem. if i’m running locally, i already have credentials… but if i’m getting the CI credentials, that’s not great

also not sure what might be done to stop that, outside of config parsing and policy enforcement tool

3.- put the UI under OIDC etc and give only access to the people that needs it

4.- it is being develop and there is a github action you could test https://github.com/cresta/atlantis-drift-detection

5.- you can run conftest , what is basic about it?

Ask Cloudposse Again about the status of Atlantis, I’m pretty sure they have changed their minds. Check the releases as well, we have been updating the code pretty often

by the way I’m one of the Atlantis maintainers

When i am reading documentation of digger they have comparison with open source and enterpise tool

That’s the best thing

If they work on solving the issue of both soon they will attract to use it

Does digger has authentication control VCS providers

Like bitbucket user authentication right now not available in atlantis

They only support github

Terragrunt integration

Does digger provide pr request automation same like atlantis

I like my aws SSO setup and permissions, but as a small company we just have permissions like DataTeam/Engineer/Admin for different aws orgs that are our environments, where engineer is read/write to most things we use.

no experience running terraform in a build system as I’m the only one that does it

@Michael Galey thank you for the input, I’m definitely of the mind to keep things simple as possible

as for running/not running in a build system, I get for your use case as the only one doing ops-y things, then it might not be a priority until the team grows

there’s also compliance/security stuff, but that’s another thing that needs to be prioritized

for my own auditing/backup info, i also log all runs and send diffs as events into datadog, the diffs sometimes are beyond the char limit tho

oh, that’s very cool, just a homegrown wrapper around terraform?

yea, i also wrap other terraform stuff to use SSM parameters/aws sso

from my ‘tfa’ (terraform apply)

so it also catches/warns of destructive stuff, and asks for an extra confirm later on

the beauty of bash, very cool stuff, I have a similar alias, but for aws sso+aws-vault+terraform

what do you need aws-vault for? sso means it doesn’t have to vault anything right? or you store even the temp creds in the vault?

I think it’s more habit than anything now to keep aws-vault around. Used it prior to AWS SSO

I have to fix my blog, but I created a post about terraform wrappers https://github.com/jperez3/taccoform-blog/blob/master/hugo/content/posts/TF_WRAPPER_P1.md

I like wrappers, but also want to be aware of how tangled they can become

thanks for sharing! agreed, my setup would be unfortunate to teach a second person, even though it’s technically cool when understood

I think your setup is good and teachable to another engineer. I think the bigger hurdle is organizing the terraform a way where you two can actively work in the same environment without stepping on each others toes

yea for sure

@Andriy Knysh (Cloud Posse) @Dan Miller (Cloud Posse)

@Elad Levi I guess the logging_configuration configuration is part of this variable https://github.com/cloudposse/terraform-aws-firewall-manager/blob/master/variables.tf#L233

which then used here https://github.com/cloudposse/terraform-aws-firewall-manager/blob/master/waf_v2.tf#L51

which, according to https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy#managed_service_data, can be used to set the logging bucket

something like shown here https://docs.aws.amazon.com/fms/2018-01-01/APIReference/API_SecurityServicePolicyData.html (it’s ugly)

search for loggingConfiguration in the wall of JSON there

you will need to provide a similar JSON string to logging_configuration in variable "waf_v2_policies"

having said that, the module was not updated in 1.5 years and prob needs some love

Thanks @Andriy Knysh (Cloud Posse) The problem for me was that I used this block:

        loggingConfiguration = ({
          "logDestinationConfigs" = ["arn:aws:s3:::aws-waf-logs-bucket-name-01"]
          "redactedFields" = []
          "loggingFilterConfigs" = null
        })

And I should have used this:

        logging_configuration = ({
          "logDestinationConfigs" = ["arn:aws:s3:::aws-waf-logs-bucket-name-01"]
          "redactedFields" = []
          "loggingFilterConfigs" = null
        })

As far as I can see there is no need to write it on json string because the loggingConfiguration will be inside jsonencode anyway as you see here:

    managed_service_data = jsonencode({
      type                  = "WAFV2"
      preProcessRuleGroups  = lookup(each.value.policy_data, "pre_process_rule_groups", [])
      postProcessRuleGroups = lookup(each.value.policy_data, "post_process_rule_groups", [])

      defaultAction = {
        type = upper(each.value.policy_data.default_action)
      }

      overrideCustomerWebACLAssociation = lookup(each.value.policy_data, "override_customer_web_acl_association", false)
      loggingConfiguration              = lookup(each.value.policy_data, "logging_configuration", local.logging_configuration)
    })

As the modules usage in the README.md file and the examples suggest, yes. You should have pinned versions and a systematic way to control their upgrades to avoid disruptions due to latest changes.

ok cool, ty @José

there was talk of doing this from terragrunt which we are using to implement said modules and I feel like thats very dangerous

try function?

The issue is you can’t use dynamic in module input params @Hao Wang. Not sure if it’s possible to do differently, but that gives an idea of what I’m trying to accomplish.

Yeah, it is possible, you can pass cluster_addons as a variable with dynamic

https://github.com/terraform-aws-modules/terraform-aws-eks/blob/0f9d9fac93caf239386f47b0f117706ea78c2bec/main.tf#LL412C28-L412C46

may need to use locals variables to compose

So compose the value in a local using dynamic then pass that local var to it

I’ll have to give it a go, new to Terraform/hcl

got it

Awesome, thanks for the suggestion

you are welcome

I am not having any luck with this at all

Looking to dynamically generate cluster_addons, if anyone has any ideas please let me know

I set up EMR before but not used different names

After reviewing the codes, core/master nodes use different labels

Core: https://github.com/cloudposse/terraform-aws-emr-cluster/blob/master/main.tf#L72

Master: https://github.com/cloudposse/terraform-aws-emr-cluster/blob/master/main.tf#L56

and they use different attributes already in the module

May need to pass Name tag

I will try it out. Thank you.

np

Terraform Cloud’s Free tier now offers new features — including SSO, policy as code, and cloud agents

Wow! Includes cloud agents. Amazing. I’ve been lobbying for this as the free tier was unusable as implemented because it required hardcoding AWS admin credentials as environment variables.

Smart move! cc @Jake Lundberg (HashiCorp)

yeah pretty sweet. might make me think about taking another look at TFE. Even with GitHub Actions et. al. you have to hard code creds. Self-hosted runners and dedicated build agents with IAM roles gets you around that though. I guess this is Hashi getting on board.

Even with GitHub Actions et. al. you have to hard code creds. Negative.

https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services

https://github.com/aws-actions/configure-aws-credentials

OK ok i will concede to the openid configuration. I guess i am thinking about arm chair devs/ops

Yes, the dynamic credentials in TFE/C looks quite a bit like Github cloud credentials.

also, when i say “hard code” i mean repo secrets… which leads to the configure-aws-credentials action

i guess my trade offs for the casual dev/ops people are getting them up and running with an AWS key in secrets or going through another hour or so of training to explain and set up the openid configuration.

But overall the pricing model is moving more towards scale of operations which is why we’ll see more traditional paid features be free for smaller teams. And no SSO tax…that’s table stakes these days.

still no list pricing for larger org plans, I guess it’s hard to give up the spice of “how much do we think you can afford”

Would you pay the list price as a larger org?

yes, with more private workers / concurrency

Just my luck, signed up for a paid TFC plan a few months ago because we needed SSO and Teams management.

Not sure, looks a little old, and some providers had decent sized changes in the last 2 years. If you know terraform already, I found it helpful to read some of their modules and compare to my own that did similar things, understanding how they use the ‘this’ object/context object and pass that around etc.

@Michael Galey Thank you so much!

great, will give it a try, can I use gmail to join slack?

Absolutely! This is the link

Cool, joined

What I feel is like your pro feature should be open sources first because there are already tools available which supports these

Like terrakube has already implemented it

Drift detection opa integration

Atlantis with custom workflow can add these features

Scalr is also providing all features unlimited for small business purpose with limited number of runs bit they are not restricting to use there feature and buy it

https://terrakube.org/

@Utpal Nadiger let me know if I miss anything i need something which I can usually pick up that without pro but use your tool as competitor for all open source tool

Which are already fighting get alternative for terraform cloud

Even terraform cloud has enable feature in free tier with unlimited user accounts and rbac agents

And many more

Great move!

Thanks!

Just chiming into the conversation.

i support one customer running a workload with Bitbucket + TF + AWS.

The Bitbucket pipeline tooling is working pretty good for triggers, branch specification (for build targets), and environments for deployment tracking .

i do like the cloudspend feature of Terrateam though!

So our reference architecture is the one thing we hold back. We have some simple examples though

Also, check out our refarch and atmos channels

https://atmos.tools/category/tutorials

https://docs.cloudposse.com/fundamentals/building-blocks/

https://cloudposse.com/services/ ← our paid reference architecture

I’m very familar with ECS, but feel it ie better to migrate ECS to k8s

one step at time

k8s is not the solution for a lot of companies, it all depends on workload/knowledge level etc

Paula what advise are you looking for?

if you go monorepo I will advise to make your rds , ecs alb , etc…modules very flexible so you can Standup any of your services

you could look into using Atmos from cloudposse to make it easier to manage the inputs between all your services/modules

I think the question is “the apply process is taking around 15 minutes”

That’s a very slow apply. Why is it slow? There are generally two reasons:

1. Too many resources in the Terraform configuration
2. Some resources are very slow to change a. And perhaps you have several of these with dependencies causing them to apply linearly The way to improve things depends on the specifics of this answer.

Definitely option 1. There are many resources and I feel like I’m accidentally making a monolith Inside of each service folder i have the creation of the ECR, the task excec role, the container definition, the load balancing ingress, the service, the task, the pipeline, codebuild and the autoscaling (most of it using cloudposse’s modules) with the specific configuration of each service

Answering about k8s, we have not enough knowledge to migrate (startup mode activated). Im gonna check about Atmos

gotcha. One way to think about how to split things up is by lifecycle. You may want to separate the Terraform which sets up the environment from the infrastructure which deploys the application. It sort of sounds like you’ve done that, mostly though. Perhaps you might want to split out the pipeline/codebuild to a second configuration, but that might be pointless complexity.

If these stacks are mostly concerned with things that don’t change often, taking 15mins to run is probably fine. How often do you change ingress rules or autoscaling thresholds? It sounds like a stack you spin up once per service and then change a few times a year

I will say for my experience RDS will take the longest so maybe doing what Alex is suggesting with RDS and have a different deployment lifecycle could be a good option. In my company projects we do not put RDS/Aurora deployments are part of the app because some actions can take a long time

We have encountered the same issue. Our team mitigates it by split monolith into separate independently deployed modules. You can use atmos by cloudposse or terragrunt to manage the dependency graph.

Then you don’t need to apply all resources, just apply the related ones when updating.

You can check out infrastructure reference architecture of cloudposse or gruntworks, which offer you insights to mitigate this issue.

@Paula I’m a bit late to the party but I think the first issue to address is the configuration/folder hierachy

The fundamental problem is you’ve got the relationship inside out. Instead of thinking of a terraform monorepo that contains configuration for many environments > networks > infrastructure > services…

You should think of services containing their own private infrastructure and deployment configuration (whether that’s terraform, kubernetes, ecs service and task definitions, ci/cd config etc) in a service monorepo. Personally I’d put it with the service source code repo, but atleast a repo that is privately owned and maintained by that service.

The problem you’ve got is that you’ve sliced things horizontally by what they are, not the service. This is equivalent to a neighbourhood where each house contains one type of thing for everyone. For example one house contains everybody’s chairs. One house contains everybody’s beds. One house contains everybody’s tables etc.

But of course in reality each house is independent and private to everyone else as dictated by the owners/occupants of that house. And within each house things are they arranged by use case, not furniture type. For example a living room could contain chairs, tables, tvs etc.

The same way a service should contain and control it’s own infrastructure arranged by use case. That’s the whole point of service orientated architecture, in that maintainers of a service can perform the things they need to do end to end without having to involve other people.

I think there are two good options:

1. Just don’t bother. Provider updates almost never fix bugs, and if you write code for a new feature it’s quite painless to update to the latest version on-demand.
2. Renovate with autoMerge

Thanks

wow Renovate is magic

https://www.youtube.com/watch?v=l0YH557eIiE

we’re rolling it out now. But I’d caution that it might just be adding pointless busywork. Does it really matter if you use AWS provider 3.68 instead of 3.72? Is it a waste of time to review PRs upgrading such providers?

And if you enable autoMerge, how do you ensure that problematic upgrades are not merged? With Terraform, your CI probably runs a plan and reports if there are no errors. But “no errors” doesn’t mean “no changes”. A provider upgrade might make significant changes to your resources and it’s very difficult to detect this automatically.

Renovate with Terraform is a bit of a rabbit hole, I’m finding.

Agreed. If you’re concerned about compatibility with shared modules I’d argue to maybe consider the pros and cons of org wide modules.

So since i brought this question up we had built our own little registry thinking we could do a proxy that sat in front of the official terraform registry. And the way it worked is our proxy allowed us to define what versions are actually available from the official terraform proxy. However where it doesn’t work is within modules and anything else that is nested. All the modules we use (ex. CloudPosse modules) would need to also be configured to pull from our custom provider registry. So we came up with another idea today: https://github.com/GlueOps/terraform-module-provider-versions/blob/main/README.md we wrote a module that only defines our provider versions and then reference the module across all our terraform directories.

It’s been 2 hours since we rolled it out and it’s working well so far…

Oh yeah here is the proxy code we came up with: https://github.com/GlueOps/terraform-registry-proxy

versions that it allows right now: https://github.com/GlueOps/terraform-registry-proxy/blob/main/provider-versions.yml

^ we will probably be archiving this repo

Can you talk a bit about what benefit this brings you? What’s the motivation here

Definitely. We calculated we were spending about 3-6 hours a month on just updating all of our terraform directories/repos to use the same/latest terraform providers across all our repos. So speeding up this repetitive process was the primary motivation here.

It’s an interesting pattern i wouldn’t have thought about using a module purely for required versions.

I guess to take a step back, what are you solving by making all your configs using the same provider versions?

Plus I assume you’ll still need to tf init -upgrade and commit all your .terraform.lock.hcl files

That’s a really good point. We will need to revisit if and how we want to handle the .terraform.lock.hcl

RE: standardized/updated configs.

So we actually update all our apps each month. We don’t do every last dependency/package but for all the layers we manage, we try to bring it up to the latest release/patch. Historically, we have worked on teams where we usually ignored updates and have found that falling behind on software updates (not specifically terraform) ends up being a huge nightmare at the most inconvenient times. So for a little over the past year we have been doing updates once a month. It takes about a full day for one of us to get it done.

fyi…
Hey @lorengordon I’m the community manager for the AWS provider. Someone from the team has assigned this to themselves for review, however, they’re currently focused on finishing up the last bit of work needed for the next major AWS provider release. Unfortunately I can’t provide an ETA on when this will be reviewed/merged due to the potential of shifting priorities. We prioritize by count of reactions and a few other things (more information on our prioritization guide if you’re interested).

thanks @loren - it was worth a shot. The problem I have with rankings is it’s not weighted. Some of the issues are level-10 effort, while this is level-0.

cc @Jeremy G (Cloud Posse) @matt

yeah agree

maybe it’ll help to the linked issue also…? https://github.com/hashicorp/terraform/issues/30054

i wonder if “participants” can be tracked as a metric in addition to reactions. a lot of folks will comment, without posting a reaction

could possibly try to frame it as a security enhancement, at least for advanced users, with fine-grained ABAC policies…

I was a lot more excited about this feature before I found out that you cannot use tags to control write-access. You still need a path-based restriction to prevent people who can write to some part of the S3 bucket from writing to (overwriting, deleting) another part of the bucket.

What do you mean “caters for”?

Sorry, I meant builds those 3 resources together.

I feel like I am asking a dumb question though

it is a good question, I came across issues before that combining multiple modules will cause some dependency issue, need to run --target first to create some resources

This Slack is run by the CloudPosse company, who release many open source modules. Searching the repository list at https://github.com/cloudposse/ is a good place to start. There are a couple of hits if I search for sqs and sns. One of them might be suitable.

Otherwise, we’re going to need more info. What are you trying to accomplish with SQS, SNS and CW alarms?

I was looking for a solution to build an SQS queue, with an CW Alert that monitors a DLQ configured to send to an SNS topic. I’ve tried creating my own module as follows:

resource "aws_sqs_queue" "sqs_queue" {
  name = var.sqs_queue_name
}

resource "aws_sqs_queue" "dlq_queue" {
  name = var.dlq_queue_name
  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.dlq_queue.arn
    maxReceiveCount     = 3
  })
}

resource "aws_cloudwatch_metric_alarm" "dlq_alarm" {
  alarm_name          = "dlq-message-received-alarm"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 1
  metric_name         = "ApproximateNumberOfMessagesVisible"
  namespace           = "AWS/SQS"
  period              = 60
  statistic           = "SampleCount"
  threshold           = var.cloudwatch_alarm_threshold

  alarm_description = "This alarm is triggered when the DLQ receives a message."

  alarm_actions = [
    var.sns_topic_arn
  ]
}

Not the cleanest of solutions.

that looks clean to me! What don’t you like about it?

the context must have the following attribute (all or at least one)

namespace = "eg"

stage = "test"

name = "redis-test"

https://github.com/cloudposse/terraform-aws-elasticache-redis/blob/main/examples/complete/fixtures.us-east-2.tfvars

the final ID/name will be calculated as

{namespace}-{environment}-{stage}-{name}-{attributes}

to make all AWS resource names unique and consistent

Ah I see, thank you.

We don’t support it

Also, it’s only for MySQL https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-rds-blue-green-deployments-safer-simpler-faster-updates/

Open to supporting it though

I just did some blue/green RDS upgrades recently, the API/Terraform support was lacking, so had to do it mostly manually, and update our Terraform after the fact to reflect the infra changes. Also, fun fact: you can’t use blue/green with RDS Proxy (yet).

Also be careful with RDS blue green on especially busy databases. It can have a lot of trouble acquiring a lock and eventually just timeout after you initiate the update.

This is my experience on Aurora MySQL at least

@Josh Pollara was this using the AWS managed “Amazon RDS Blue/Green Deployments”? Interesting insight…

Yes it was

Against a very busy database

Additionally, it wasn’t a one time event. We tried multiple times without luck.

Ok, so is a no go for the time being. Yeah it was just a insight, Now I got the facts for not to. Thanks…

What we need is amazon to offer https://neon.tech/ as a managed service

(this is what vercel is now offering)

Also https://planetscale.com/ for MySQL

variable "instance_market_options" is an object, but you use it as a list of objects

same with spot_options

Please ask in refarch

(as this relates to our reference architecture)

It won’t cause problems

Hmm, OK, so ECDSA doesn’t output a public key format? If I change to RSA, it works as expected

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-key-pairs.html#how-to-generate-your-own-key-and-import-it-to-aws

report it as an issue against the github repository. Or if you can’t do that, try #terraform-aws-modules

https://github.com/cloudposse/terraform-aws-vpc/issues/121

seems it is still a limit of terraform since 2017, https://github.com/hashicorp/terraform/issues/13417#issuecomment-297562588

Hm, do you have to run this multiple times??

One approach you can consider is appending a small hash after the name of the keys. ex.: key-name-j3hd1 And create a separate workspace and repo (I’m assuming you are using Terraform cloud) and only run if necessary.

Even if anyone accidentally run it, it will create a new set of keys and won’t change the ones you created before and prbly already using.

I think it solves for now but creates a technical debt with the automation gods.

Also, you could use azurecaf provider for naming conventions and random provider for the small hash.

Could you add the plan where it wants to re-create stuff? As you are using a list there could be an issue with lexical ordering. TF sorts lists alphabetically before looping through them, so you can’t rely on ordinal order. You could work around this by using maps or adding a numeric prefix to each element in the list so you can control the other TF loops through it and how it stores it in state.

Thank you for responses, ill check your suggestions.

More info about the setup. It is a part of a bigger tf script. Im trying to provision a VMs on azure that suppose to host databases. Together with each VM (from the list) there is a keyvault created with secrets for this database. Secrets should be created once.

I need to have the possibility to run it multiple times because there will be need to remove one of the hosts or add another later on. Im using for_each to not relay on count index. While im adding new VM to the list or removing one the secrets related to this machine are created/destroyed but the rest related to other machines that suppose to be unchanged are regenerating once again

Probably using a different state for each key-vault will do the trick but its strictly connected to the VM. While im removing the VMs i would like the key-vault and its secrets to be removed as well.

It seems that this issue is fixed. Thanks. It was some dependency that was created because i was injecting those passwords to “cloud-init. scripts.

anyway im facing another issue now.

variable "vm_names" {
  type    = list(string)
  default = [
   "vm-1-2ds31",
   "vm-2-412ae"
  ]
}

resource "random_password" "password" {
  count           = 2
  length          = 11
  special         = false
  min_upper       = 3
  min_numeric     = 3
  min_lower       = 3
}

resource "azurerm_key_vault_secret" "secret1" {
  depends_on   = [azurerm_key_vault.kv]
  for_each     = toset(var.vm_names)
  name         = "name1"
  value        = random_password.password[0].result
  key_vault_id = azurerm_key_vault.kv[each.value].id
}
resource "azurerm_key_vault_secret" "secret2" {
  depends_on   = [azurerm_key_vault.kv]
  for_each     = toset(var.vm_names)
  name         = "name2"
  value        = random_password.password[1].result
  key_vault_id = azurerm_key_vault.kv[each.value].id
}

I need to create a set of two different random passwords for each VM from the list. Right now with this setup it is creating only 2 passwords and propagating it to all key-vaults that are created

key vaults are created like this

resource "azurerm_key_vault" "kv" {
  for_each                    = toset(var.vm_names)
  name                        = format("%s", each.value)
.
.
.

if i add the for_each to random_password then i cannot specify how many passwords need to be created.

The solution may be another resource for random_password to have it in different object but i cannot do this as it shoudl be created dinamically in relation to the var.vm_names list

@Adrian Rodzik, sorry for the late reply here. Was the latest issue fixed?