#terraform (2024-02)

please use #pr-reviews

cc @Gabriela Campana (Cloud Posse)

@Boris Dyga the PR is approved, thanks

it is very similar to Atmos, interesting

Ref https://github.com/terraform-docs/terraform-docs/issues/703

Would need to discuss with the team

Thanks Erik. I also opened a thread with opentofu to see if they would be interested.

https://opentofucommunity.slack.com/archives/C05R0FD0VRU/p1706989540737409?thread_ts=1706989540.737409&cid=C05R0FD0VRU

Hey :wave::skin-tone-2: .

Currently, the OpenTofu Core Maintenance team does not intend to fork and maintain 3rd party tooling that is not necessary for the Core OpenTofu project.
In the future, if there’s high demand for such a tool being in high maintenance, we will reconsider this decision. However, right now we do not have the capacity to support a terraform-docs fork alongside the main OpenTofu project so opentofu will not fork

Hi @Erik Osterman (Cloud Posse) , just a friendly ping, did you get a chance to chat with the team?

there is an Ansible provider https://registry.terraform.io/providers/ansible/ansible/latest/docs

It depends on your usecase. This article might also be useful https://spacelift.io/blog/using-terraform-and-ansible-together

Might wanna take a look at this video walkthrough too https://www.env0.com/blog/ansible-vs-terraform-when-to-choose-one-or-use-them-together

the homebrew-core tap only includes open source packages. you can use the hashicorp tap if you want newer versions.

brew tap hashicorp/tap
brew install hashicorp/tap/terraform

That makes sense! Thank you!

I use tfenv but I hear mention that its not regularly maintained anymore or something…havent looked into it but it still works for me.

tfswitch is another option

tfenv is probably dead, here is a discussion thread about it https://github.com/tfutils/tfenv/issues/399

In that same discussion: It’s mentioned that the Tofuutils crew are working on a new tool called tenv, OpenTofu / Terraform / Terragrunt version manager . It might be useful to check.

Cool, thx

Anyone use the asdf Hashicorp plugin for it?

Probably a good extended conversation to have on #office-hours if you can make it tomorrow

Yes, I feel like yor is an anti-pattern. Although I could understand it for legacy code. But everything new, using a null-label pattern (or similar implementation).

Has anyone tried to pre-generate null-label resource names and add them to terraform files as comments? Can you provide a code snippet of the hypothetical example of the comment and code?

also, “in a perfect world”, would you be able do look up a terraform resource name and get it’s AWS resource name? …that name being “unconventional” (non-label generated).

Sure. Here’s an example. Let’s say I have a customer who has problem with eg-prod-bastion-public instance and raised an internal support ticket with instance name. I can simply search for label_id:eg-prod-bastion-public in my codebase. Without it however, it’s a needle in a haystack problem.

module "bastion_label" {
  source   = "cloudposse/label/null"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "x.x.x"

  namespace  = "eg"
  stage      = "prod"
  name       = "bastion"
  attributes = ["public"]
  delimiter  = "-"

  tags = {
    "BusinessUnit" = "XYZ",
    "Snapshot"     = "true"
  }
}

# label_id:eg-prod-bastion-public
resource "aws_instance" "bastion" {
  instance_type = "t1.micro"
  tags          = module.bastion_label.tags
}

also, “in a perfect world”, would you be able do look up a terraform resource name and get it’s AWS resource name? That would be a bonus, but it’s less of a problem for my scenario. The main challenge is finding where a particular resource is defined in the codebase.

Without it however, it’s a needle in a haystack problem. Aha, I see what you mean.

Basically you need annotations that tie deployed infrastructure back to where it is on disk.

If those annotations were in the code, you could search for it.

Since cloud posse deploys the same root modules/modules thousands of times, with strictly configuration via atmos, the code-style comment annotations are not something we’ve considered. E.g. To deploy 20 VPCs, we still have exactly one VPC root module (component). Then 20 configurations on how to deploy it.

here’s a very large terraform codebase (thousands resources) and currently the team relies on cloud resource names (e.g. s3 bucket name) to find those resources in terraform code. Literally, searching bucket name in codebase. So, I think there are 2 challenges.

1. What to do with your existing code base
2. What to aim for in new projects delivered

In new projects delivered, I think https://atmos.tools solves the attribution error very easily because you can enforce tag conventions that pass through to all resources deployed. So everything is tagged with their Stack identifier.

For existing Terraform, would have to take a closer look.

Thanks @Erik Osterman (Cloud Posse), appreciate your feedback. Rolling out something like Atmos would be very hard in a large org.

I’ll update if we find something interesting.

@Jeremy G (Cloud Posse) may know the answer.

I know he’s been working on this and we released a new version today.

Yes, that version is super outdated, and we stopped publishing it under that name. It is still available as cloudposse/platform/datadog but we recommend you upgrade to the dedicated submoudle cloudposse/platform/datadog//modules/monitors

thanks

This is the problem we solve with atmos. What you describing is a configuration management and orchestration problem. You would define your root module one time. Then you define you configuration for that root module as part of a Stack configuration.

See https://atmos.tools/core-concepts/stacks/catalogs#organizations

copy/pasting the resource/module x times with different providers? Sadly, this is what many companies do through the abuse of root modules.

With atmos, there’s no copy/pasting.

Anti-patterns to avoid:

• more than 2 providers in a root module. Only use 2 providers if it’s a hub/spoke relationship.

• Generally use 1 provider.

• Do not use 1 provider per region (e.g. you have poor DR resiliency if terraform root modules expect all regions online)

https://atmos.tools/reference/terraform-limitations

you are on this journey @Gabriel let us know if you need help on this or how to start using Atmos

thanks, I will take a look at it and try to understand how I could solve my problem with atmos.

what I did at the moment is this

module "one" {
  source = "source"
  providers {
    aws = aws.one
  }
  some_var = "one"
  # rest of config is the same
}

module "two" {
  source = "source"
  providers {
    aws = aws.two
  }
  some_var = "two"
  # rest of config is the same
}

...

this is what we call “stage 3” (in that link andriy shared)

see https://registry.terraform.io/providers/hashicorp/aws/5.36.0/docs/data-sources/iam_policy_document#source_policy_documents

here’s an example on how to use it https://github.com/cloudposse/terraform-aws-iam-policy/blob/main/examples/complete/main.tf

it merges policies together using source_policy_documents

Thank you Andriy. I did look at merging documents using source policy documents. But I sum look at the module you shared. Thank you very much.

if you run account-settings that should enabled

so maybe you run it but did no pass true to enable it?

what do you mean “pass true”? how can i do that now?

By the way, i am determining that the account-settings module ran because i see the CloudWatch log role ARN in API Gateway -> APIs -> Settings.

enabled = true

Can i add that as an input? i tried doing that and it didn’t work.

^ when i do this and run the code, custom access logging is still inactive

https://github.com/cloudposse/terraform-aws-api-gateway/blob/main/main.tf#L4

do you have that enabled?

var.logging_level

i have logging_level = INFO

enabled isn’t seeing itself as true

then you should have a log group created for your api

Both execution logs and access logs are disabled in the stage details

you will have to check the plan and see

it looks like i manually created the cloudwatch role instead of using the account-settings module that came with api gw (https://github.com/cloudposse/terraform-aws-api-gateway/tree/main/examples/account-settings)

is that why “enabled” isn’t true?

account-settings is run once per region. i run the api-gw module many times to create new api-gw’s. how can the api-gw enabled variable rely on account-settings module to pass enabled=true if it’s only run once and the other is run many??

no, that is not how it works

the account-setting is run once per region

then you go an deploy your api gateways with loglevel = XXX

that is all you need

and the “enabled=true” follows the future api gw’s that get deployed using the module?

no

the account-setting its a submodule

look at the examples folder

i did. none of the examples show with logging enabled.

i don’t understand how to get “local.enabled” to be true along with logging_level. It looks like both need to be true to create create_log_group as mentioned in your code snipped:

create_log_group       = local.enabled && var.logging_level != "OFF"

I have logging level set to “INFO”.

just pass to your module instantiation enabled = true

the value comes from the context.tf file

I pass these variables to the api gw module and it does not create the cloudwatch logs:

  enabled = true
  xray_tracing_enabled = true #X-Ray tracing
  metrics_enabled = true #Detailed metrics
  logging_level = "INFO"

what about your plan? does it show the cloudwatch group at all?

what version of the module are you using?

No, it creates aws_api_gateway_method_settings and aws_api_gateway_stage

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_api_gateway_method_settings.all[0] will be created
  + resource "aws_api_gateway_method_settings" "all" {
      + id          = (known after apply)
      + method_path = "*/*"
      + rest_api_id = "vu1a6hqjxk"
      + stage_name  = "io"

      + settings {
          + cache_data_encrypted                       = (known after apply)
          + cache_ttl_in_seconds                       = (known after apply)
          + caching_enabled                            = (known after apply)
          + data_trace_enabled                         = (known after apply)
          + logging_level                              = "OFF"
          + metrics_enabled                            = true
          + require_authorization_for_cache_control    = (known after apply)
          + throttling_burst_limit                     = -1
          + throttling_rate_limit                      = -1
          + unauthorized_cache_control_header_strategy = (known after apply)
        }
    }

  # aws_api_gateway_stage.this[0] will be created
  + resource "aws_api_gateway_stage" "this" {
      + arn                  = (known after apply)
      + deployment_id        = "naxrtf"
      + execution_arn        = (known after apply)
      + id                   = (known after apply)
      + invoke_url           = (known after apply)
      + rest_api_id          = "vu1a6hqjxk"
      + stage_name           = "io"
      + tags                 = {
          + "Name"  = "green-parakeet-io"
          + "Stage" = "io"
        }
      + tags_all             = {
          + "Name"      = "green-parakeet-io"
          + "Stage"     = "io"
          + "Terraform" = "true"
        }
      + web_acl_arn          = (known after apply)
      + xray_tracing_enabled = true
    }

Plan: 2 to add, 0 to change, 0 to destroy.

we pinned to this version of the module - https://github.com/cloudposse/terraform-aws-api-gateway.git?ref=0.3.1

that is old….

` + logging_level = “OFF”`

” + logging_level = "OFF"” ??? what do you mean by this?

that is on your plan

so is not reading your value for some reason

you have logging_level = “INFO”

and that should be on the settings

i see that now. ok. thanks for pointing that out. i’ll look into that and then update to newer module.

I’m using terragrunt to pass logging_level as an input

Thank you @jose.amengual it is working now

Do you know if logging_level will be updated to support the selected category in the screenshot?

create a PR , we can review it

ok

It’s a terraform issue. I don’t think your code can add that isn’t there yet in the original resource. I opened a fix with terraform to hopefully get it added. https://github.com/hashicorp/terraform-provider-aws/issues/35863

Terraform responded. I don’t see the variable in your module. I’ll create a PR.

i submitted a pr

link?

https://github.com/cloudposse/terraform-aws-api-gateway/pull/36

please check the comments on the PR

run make precommit/terraform nd commit the changes

ok

For this ChatGPT will be your friend

@Ben Smith (Cloud Posse) @Jeremy G (Cloud Posse) @Jeremy White (Cloud Posse)

@jswc I do not know the direct answer to your question, but I have some tips for you.

• The monitors in our catalog are old and the format outdated. (We are working on updating them.) • If you can update to the latest version of our monitors module, then you can define the monitor in JSON. The advantage of defining the monitor in JSON is that you can use the Datadog website/console to create the monitor, using all the built-in help and shortcuts, and then export the final monitor as JSON and import it into Terraform. See our examples for what that looks like.

@jswc Oh, sorry, I misunderstood your question. No, we do not have a way for you to insert a monitor ID from a monitor created in Terraform into another monitor query. Terraform in general does not let you use outputs as inputs in the same cycle (plan/apply). Probably the best thing to do is create monitors in one component and then use that component’s outputs create either composite monitors or synthetic tests in another component, using the output of the first component as input. We do that via Terraform state right now, and have support for that if you are using atmos and its stacks.

Hi Jeremy, thanks for your replies.

Do I understand correctly that your recommendation around components would be like:

# from <https://registry.terraform.io/providers/DataDog/datadog/latest/docs/guides/monitors>

resource "datadog_monitor" "bar" {
  name    = "Composite Monitor"
  type    = "composite"
  message = "This is a message"
  query   = "${datadog_monitor.metric1.id} || ${datadog_monitor.metric2.id}"
}

# me adding some example monitor whose ID is used above
resource "datadog_monitor" "metric1" {
  name    = "metric 1 monitor"
  type    = "metric alert"
  message = "..."
  query   = "${someMetric} > 10"
}
...

i.e. writing plainer Terraform, not using the CloudPosse terraform-datadog-platform way?

Using the IDs like this reminds me of aws policies/policy_attachments, like

# <https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment#example-usage>

...
resource "aws_iam_policy" "policy" {
  name        = "test-policy"
  description = "A test policy"
  policy      = data.aws_iam_policy_document.policy.json
}

resource "aws_iam_policy_attachment" "test-attach" {
  name       = "test-attachment"
  users      = [aws_iam_user.user.name]
  roles      = [aws_iam_role.role.name]
  groups     = [aws_iam_group.group.name]
  policy_arn = aws_iam_policy.policy.arn
}

I’m pretty confused how doing it this way could be OK, but when we go via yaml the output/IDs are not accessible.

I may be way off the mark, but my instinct is that: • .yaml could have ${paramter1} for use with cloudposse/config/yaml, but can also leave some ${} uninterpolated ◦ so later, the composite query could still see ${datadog_monitor.foo.id} but that doesn’t really consider what’s actually happening so it’s probably nonsense

Looked around more and I think I understand your recommendation better: • keep the setup as is, but ◦ use the output datadog_monitors (https://registry.terraform.io/modules/cloudposse/platform/datadog/latest?tab=outputs) in another module ▪︎ filter irrelevant monitors, pair up the alarms needed for composite • create composite monitor using those pairs I’ll try and read more about what structure datadog_monitors is - that approach may be fair.

Yes, you need one component (root module) to do terraform apply to create the monitors, then another one to take the IDs output from the first one and use them to make a composite monitor.

Nice, thanks Jeremy That could be a nice example in the repo, as it’s quite different from how it’s done with just TF.

this is using the cloudposse aws-sso, account, and account-map component

Yes, it’s using all the components mentioned

if you’re using the root account as the identity administrator with AWS Identity Center, then that’s where the Permission Sets should be as well. Then you can attach a Permission Set to any account under that Organization

he is using identity as the account for role chaining

Alternatively you could also attach the policy doc directly if you merge SqlWorkbench

locals {
  red_shift_access_permission_set = [{
    name             = "RedshiftAccess",
    description      = "Allow access to Redshift",
    relay_state      = "",
    session_duration = "",
    tags             = {},
    inline_policy    = data.aws_iam_policy_document.redshift_access.json,
    policy_attachments = [
      "arn:${local.aws_partition}:iam::aws:policy/AmazonRedshiftDataFullAccess",
      "arn:${local.aws_partition}:iam::aws:policy/AmazonRedshiftQueryEditorV2FullAccess",
      "arn:${local.aws_partition}:iam::aws:policy/AmazonRedshiftFullAccess"
    ]
    customer_managed_policy_attachments = []
  }]
}

data "aws_iam_policy_document" "redshift_access" {
  statement {
    sid       = "SqlWorkbenchAccess"
    effect    = "Allow"
    actions   = ["sqlworkbench:*"]
    resources = ["*"]
  }
  statement {
    sid    = "s3Actions"
    effect = "Allow"
    actions = [
      "s3:PutObject",
      "s3:Get*",
      "s3:List*"
    ]
    resources = [
      "arn:aws:s3:::aaaaaa-redshift",
      "arn:aws:s3:::aaaaaa-redshift/*"
    ]
  }
  statement {
    sid = "RedshiftManagement"
    effect = "Allow",
    actions = [
      "s3:GetObject"
    ]
    resources = ["*"]
  }
}

he is using identity as the account for role chaining then the Permission Sets should be in your identity account as well

and the policies in the same account as the permission set

Thanks, I’ll try with that

@Hans D @Andriy Knysh (Cloud Posse) it’s using Cuelang

Cuelang is cool (as @Hans D knows :slightly_smiling_face: ). In Cue you can describe things, generate things, and validate things all using the same config/model (data is self-validating unlike YAML). But not simple, and not widely used

Yeah, I’ve run into it a few times and Ive heard people sing the praises. Of course it’s one of those tools where I question if the additional complexity it adds only further complicates things, but I tend to question that about a lot. You’re probably not practicing “Infrastructure as Data” anymore if you’re using Cue.

We were looking into https://kcl-lang.io/ internally a tiny bit, which is similar.

Depends on your definition of “Infra as Data”. Personally, I prefer the dynamic parts of Cue over the HCL implementation. And it solves a lot of the repetitive bits that are now all over the place ( I really the attributes/includes also as part of asciidoc for documentation)

Don’t forget Pkl

cue, rego, asciidoc, gomplate and jq/yq … perhaps throw in some go-task (sorry)

i wonder why Google doesn’t promote Cue better. They should have created some frameworks on top of it to increase the adoption (eg a terraform-like framework in Cue). Like Rails for Ruby (before which Ruby was almost unknown)

cue, rego, asciidoc, gomplate and jq/yq

i’d take cue over all of that

lol, Apple are driving their Terraform with pkl, generate JSON, pass it to Terraform. Validation stuff is super nice. env var support, arguably nicer language to write than HCL.

That’s a smart use of pkl+JSON. I’d be curious if they are generating imperative terraform (JSON) code, e.g., without vars, and handling that entirely in PKL.

Please explain a bit more

I think I have down to something simple, I have a variable that might be empty or might be a key value pair, I want to combine it with another key value pair as a map?

I just don’t know if the variable will be a key value pair or an empty string “”