#terraform (2023-05)

terraform Discussions related to Terraform or Terraform Modules

Archive: https://archive.sweetops.com/terraform/

2023-05-01

2023-05-02

Afolabi Omotoso avatar
Afolabi Omotoso

Hi I am looking at how to dynamically create resources in multiple regions but as far as I can see, It is not supported yet by terraform. Has anyone tried any work around as I have over 3000 resources to create across multiple regions?

#16967 Dynamically-generated provider configurations based on a collection value

Terraform Version

v0.11.1

Terraform Configuration Files

variable "regions" {
  default = [
    "us-east-1",
    "us-east-2",
    "us-west-1",
    "us-west-2",
    "ca-central-1",
    "eu-central-1",
    "eu-west-1",
    "eu-west-2",
    "eu-west-3",
    "ap-northeast-1",
    "ap-northeast-2",
    "ap-southeast-1",
    "ap-southeast-2",
    "ap-south-1",
    "sa-east-1"
  ]
}

provider "aws" {
  count = "${length(var.regions)}"
  alias = "${element(var.regions, count.index)}"
  region = "${element(var.regions, count.index)}"
  profile = "defualt"
}

resource "aws_security_group" "http-https" {
  count = "${length(var.regions)}"
  provider = "aws.${element(var.regions, count.index)}"
  name = "http-https"
  ingress {
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port = 443
    to_port = 443
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Expected Behavior

Creating a security group on each AWS regions.

Actual Behavior

Planning/Applying fails with

Error: Error asking for user input: 1 error(s) occurred:

* aws_security_group.http-https: configuration for aws.${element(var.regions, count.index)} is not present; a provider configuration block is required for all operations

Steps to Reproduce

  1. terraform init
  2. terraform apply
loren avatar

best option i can think of would be something like cdktf to generate the .tf configs for you

#16967 Dynamically-generated provider configurations based on a collection value

Terraform Version

v0.11.1

Terraform Configuration Files

variable "regions" {
  default = [
    "us-east-1",
    "us-east-2",
    "us-west-1",
    "us-west-2",
    "ca-central-1",
    "eu-central-1",
    "eu-west-1",
    "eu-west-2",
    "eu-west-3",
    "ap-northeast-1",
    "ap-northeast-2",
    "ap-southeast-1",
    "ap-southeast-2",
    "ap-south-1",
    "sa-east-1"
  ]
}

provider "aws" {
  count = "${length(var.regions)}"
  alias = "${element(var.regions, count.index)}"
  region = "${element(var.regions, count.index)}"
  profile = "defualt"
}

resource "aws_security_group" "http-https" {
  count = "${length(var.regions)}"
  provider = "aws.${element(var.regions, count.index)}"
  name = "http-https"
  ingress {
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port = 443
    to_port = 443
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

Expected Behavior

Creating a security group on each AWS regions.

Actual Behavior

Planning/Applying fails with

Error: Error asking for user input: 1 error(s) occurred:

* aws_security_group.http-https: configuration for aws.${element(var.regions, count.index)} is not present; a provider configuration block is required for all operations

Steps to Reproduce

  1. terraform init
  2. terraform apply
Alex Jurkiewicz avatar
Alex Jurkiewicz

are you creating the same set of resources in many regions? Or put another way, do you have one set of resources which are created repeatedly with only minor differences?

The best approach would be to split up your resources into many stacks. 3000 resources is too many for a single stack and you will suffer much operational pain doing so.

Afolabi Omotoso avatar
Afolabi Omotoso

Apologies, I meant 300. Yes I am creating the same resources in many regions. A typical example of what I am trying to do is shown below. I know that we cannot use count or for_each in providers, is there any way I can do this as a work-around

Afolabi Omotoso avatar
Afolabi Omotoso
variable "regions" {
  default = [
    "us-east-1",
    "us-east-2",
    "us-west-1",
    "us-west-2",
    "ca-central-1",
    "eu-central-1",
    "eu-west-1",
    "eu-west-2",
    "eu-west-3",
    "ap-northeast-1",
    "ap-northeast-2",
    "ap-southeast-1",
    "ap-southeast-2",
    "ap-south-1",
    "sa-east-1"
  ]
}

provider "aws" {
  count = "${length(var.regions)}"
  alias = "${element(var.regions, count.index)}"
  region = "${element(var.regions, count.index)}"
  profile = "defualt"
}

resource "aws_security_group" "http-https" {
  count = "${length(var.regions)}"
  provider = "aws.${element(var.regions, count.index)}"
  name = "http-https"
  ingress {
    from_port = 80
    to_port = 80
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port = 443
    to_port = 443
    protocol = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }
}
loren avatar

here is a module that is setup for multi-region. give it a good study to see how to do multi-region in pure terraform… https://github.com/nozaq/terraform-aws-secure-baseline

nozaq/terraform-aws-secure-baseline

Terraform module to set up your AWS account with the secure baseline configuration based on CIS Amazon Web Services Foundations and AWS Foundational Security Best Practices.

Afolabi Omotoso avatar
Afolabi Omotoso

Thank you. I will check this out

Fizz avatar

In your example, the only thing that is changing is the region, so you could create multiple executions over the same code and get provider to accept a variable for region which has a different value on each run.

Fizz avatar

You’ll need to take care with the state file as you’ll end up overwriting it if all you change is the region but if you use terraform workspaces then you can have a state file per region

Fizz avatar

You also would not need to use a provider alias in this case as each state file would only contain the resources of one account/region all created by one provider

Afolabi Omotoso avatar
Afolabi Omotoso

Thank you for this

2023-05-03

ohad avatar

New Podcast - theiacpodcast.com Hi all, my name is Ohad Maislish and I am the CEO and co-founder of www.env0.com We launched yesterday our new podcast about IaC and 3 episodes are already up in the air - with amazing guests such as the CEO of Infracost.io and the CTO of aquasec.com (tfsec+trivy OSS)

5
1
1
1
1
Shahar Glazner avatar
Shahar Glazner

Hey Ohad

wave1
jeffchao avatar
jeffchao

Hey Ohad. Cool, I’ll check it out. Super relevant to what we’re working on (in fact, been looking into env0 as well)

1

2023-05-04

Soren Jensen avatar
Soren Jensen

@Erik Osterman (Cloud Posse) I see there are already 2 open PR’s on the S3 bucket module. The issue is blocking new deployments. It will be much appreciated if one of the 2 solutions are merged into main. https://github.com/cloudposse/terraform-aws-s3-bucket/pulls │ Error: error creating S3 bucket ACL for bucket: AccessControlListNotSupported: The bucket does not allow ACLs

José avatar

What you can do is, upgrade the s3-logs version in the module main.tf internally for you, from 0.26.0 to 1.1.0 until the PR is merge.

Or the hard way

• terraform init

• sed -i “s/0.26.0/1.1.0/” …

• terraform init

• terraform apply

Soren Jensen avatar
Soren Jensen

Thanks @José

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Max Lobur (Cloud Posse) can we prioritize rolling out the release branch manager to this repo

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(and any S3 repos)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy G (Cloud Posse) let’s discuss on ARB today

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Modules terraform-aws-s3-bucket and terraform-aws-s3-log-storage have been updated to work with the new AWS S3 defaults. Other modules dependent on them should be updated soon.

cloudposse/terraform-aws-s3-bucket
cloudposse/terraform-aws-s3-log-storage
2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks @Jeremy G (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
#30380 [New Service]: VPC Lattice

Description

Support for recently announced VPC Lattice

https://aws.amazon.com/blogs/aws/simplify-service-to-service-connectivity-security-and-monitoring-with-amazon-vpc-lattice-now-generally-available/https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonvpclatticeservices.htmlhttps://awscli.amazonaws.com/v2/documentation/api/latest/reference/vpc-lattice/index.html?highlight=lattice

Requested Resource(s) and/or Data Source(s)

☑︎ aws_vpclattice_service ☑︎ aws_vpclattice_service_network ☑︎ aws_vpclattice_service_network_service_association ☑︎ aws_vpclattice_service_network_vpc_association ☑︎ aws_vpclattice_listener ☑︎ aws_vpclattice_listener_rule ☑︎ aws_vpclattice_target_group ☑︎ aws_vpclattice_access_log_subscription ☑︎ aws_vpclattice_auth_policy ☑︎ aws_vpclattice_resource_policy ☑︎ aws_vpclattice_target_group_attachment

Potential Terraform Configuration

TBD

References

https://aws.amazon.com/blogs/aws/simplify-service-to-service-connectivity-security-and-monitoring-with-amazon-vpc-lattice-now-generally-available/https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonvpclatticeservices.htmlhttps://awscli.amazonaws.com/v2/documentation/api/latest/reference/vpc-lattice/index.html?highlight=lattice

Would you like to implement a fix?

None

2
Release notes from terraform avatar
Release notes from terraform
05:33:34 PM

v1.5.0-alpha20230504 1.5.0-alpha20230504 (May 4, 2023) NEW FEATURES:

check blocks for validating infrastructure: Module and configuration authors can now write independent check blocks within their configuration to validate assertions about their infrastructure. The new independent check blocks must specify at least one assert block, but possibly many, each one with a condition expression and an error_message expression matching the existing <a…

Release v1.5.0-alpha20230504 · hashicorp/terraformattachment image

1.5.0-alpha20230504 (May 4, 2023) NEW FEATURES:

check blocks for validating infrastructure: Module and configuration authors can now write independent check blocks within their configuration to v…

2023-05-05

Release notes from terraform avatar
Release notes from terraform
03:33:32 PM

v1.5.0-alpha20230504 1.5.0-alpha20230504 (May 4, 2023) NEW FEATURES:

check blocks for validating infrastructure: Module and configuration authors can now write independent check blocks within their configuration to validate assertions about their infrastructure. The new independent check blocks must specify at least one assert block, but possibly many, each one with a condition expression and an error_message expression matching the existing <a…

Release v1.5.0-alpha20230504 · hashicorp/terraformattachment image

1.5.0-alpha20230504 (May 4, 2023) NEW FEATURES:

check blocks for validating infrastructure: Module and configuration authors can now write independent check blocks within their configuration to v…

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


check blocks for validating infrastructure: Module and configuration authors can now write independent check blocks within their configuration to validate assertions about their infrastructure.

Release v1.5.0-alpha20230504 · hashicorp/terraformattachment image

1.5.0-alpha20230504 (May 4, 2023) NEW FEATURES:

check blocks for validating infrastructure: Module and configuration authors can now write independent check blocks within their configuration to v…

Alex Jurkiewicz avatar
Alex Jurkiewicz

i find it funny they are described this way (which is pretty accurate) but the truth is if one of the checks fails, execution continues

1
Nitin avatar

https://github.com/cloudposse/terraform-aws-elasticache-memcached

curious to know why we can’t modify the security group created by this module. (everything should be known at plan time)

cloudposse/terraform-aws-elasticache-memcached

Terraform Module for ElastiCache Memcached Cluster

2023-05-06

    keyboard_arrow_up