@Boris Dyga thanks, first one approved and merged, the seconds one needs some updates

@Andriy Knysh (Cloud Posse) I’ve updated the module vesrion and the Readme. Please review. Thanks

@Boris Dyga Andriy has comments

Sure, having a look

@Gabriela Campana (Cloud Posse) I’ve run make init make readme but git cannot see any difference. What could be the reason?

@Andriy Knysh (Cloud Posse)

Could it be some issue on the /terratest side? I’ve made a trivial change (empty line), rerun make init make readme and submitted the changes to see if this possibly affects the bot behavior

ping

https://github.com/cloudposse/terraform-aws-budgets/releases/tag/0.4.0

@Boris Dyga thanks for the PRs

You are welcome!

#terraform-aws-modules

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity ? assuming it’s for the account in question.

not really, I have root org account where I’m deploying SSO configuration but it’s propagated to all child accounts so caller_identity data source will provide only ID of root org account as there is only that aws provider if I understand it correctly

we’ve made a simple output-only module, that literally outputs a map of accounts you can for_each over - maybe this works for you as well. otherwise, parse a yaml file, etc, etc.

maybe try to delete these:

• .terraform folder • the lock file • $HOME/.terraform.d folder (where TF caches all the providers)

was t working with TF 1.6 before?

thanks! let me try that now I had it on… 1.5.3 IIRC, and it was happening there too. Then I updated all provider versions to the latest.

if the deletion does not help, try TF 1.5

that tried all 1.5.X versions, no luck. Even tried the last 1.4.X version. Also tried adding “var.dummy_kubeapi_server = null” in the tf manifest as a last chance, per:

provider "kubernetes" {
  # Without a dummy API server configured, the provider will throw an error and prevent a "plan" from succeeding
  # in situations where Terraform does not provide it with the cluster endpoint before triggering an API call.
  # Since those situations are limited to ones where we do not care about the failure, such as fetching the
  # ConfigMap before the cluster has been created or in preparation for deleting it, and the worst that will
  # happen is that the aws-auth ConfigMap will be unnecessarily updated, it is just better to ignore the error
  # so we can proceed with the task of creating or destroying the cluster.
  #
  # If this solution bothers you, you can disable it by setting var.dummy_kubeapi_server = null
  host                   = local.cluster_auth_map_endpoint
  cluster_ca_certificate = local.enabled && !local.kubeconfig_path_enabled ? base64decode(local.certificate_authority_data) : null
  token                  = local.kube_data_auth_enabled ? one(data.aws_eks_cluster_auth.eks[*].token) : null
  # The Kubernetes provider will use information from KUBECONFIG if it exists, but if the default cluster
  # in KUBECONFIG is some other cluster, this will cause problems, so we override it always.
  config_path    = local.kubeconfig_path_enabled ? var.kubeconfig_path : ""
  config_context = var.kubeconfig_context

  dynamic "exec" {
    for_each = local.kube_exec_auth_enabled && length(local.cluster_endpoint_data) > 0 ? ["exec"] : []
    content {

.terraform/modules/eks_cluster/auth.tf. I have nuked the workpaces in jenkins too. I’m going to keep at it today, see what I can come up with

I’ll open an issue with https://github.com/hashicorp/terraform-provider-kubernetes too, just in case they have any feedback.

https://github.com/hashicorp/terraform-provider-kubernetes/issues/2388 lets see if they have anything to say

@AdamP try to pin the kubernetes provider to 2.24.0

here’s what I see as the issue with the latest 2.25.0 released yesterday:

https://github.com/hashicorp/terraform-provider-kubernetes/blob/main/CHANGELOG.md

https://github.com/hashicorp/terraform-provider-kubernetes/pull/2347

looks like they switched to use terraform-plugin-framework , and the new framework has issues with computed lists

see https://github.com/hashicorp/terraform-plugin-framework/issues/713

and we have the computed list here https://github.com/cloudposse/terraform-aws-eks-cluster/blob/main/auth.tf#L118

looks like that new provider framework still does not support computed lists, hence the error

Target Type: []struct { APIVersion basetypes.StringValue
"tfsdk:\"api_version\""; Command basetypes.StringValue "tfsdk:\"command\"";
Env map[string]basetypes.StringValue "tfsdk:\"env\""; Args
[]basetypes.StringValue "tfsdk:\"args\"" }
Suggested Type: basetypes.ListValue

try to pin the kubernetes provider to 2.24.0 and let us know

will do! I have a meeting in a few minutes, I will try right after. Thanks! Will let you know asap

You guys are life savers!!!!!! THANK YOU!! it worked

I swear, whenever I don’t have a provider pinned, weird stuff happens

@AdamP FYI, we’ve just tested a few EKS clusters using the latest version 2.25.1 of the kubernetes provider, and all is OK

prob something else is/was wrong with your cluster (I’m not sure why it started working when you pinned the k8s provider to 2.24.0 but the issue is def something else, or related)

update:

when using the latest kubernetes provider 2.25.1 , the error occurs only when creating a new EKS cluster, it does not occur on updating/modifying existing clusters

so the latest k8s provider broke the eks-cluster module

we’ll update the module to work with the new provider version

fantastic! I just didn’t think to add (and always pin) the kubernetes provider. Thanks again for all the help!

@AdamP You may want to try eks-cluster v4.0.0-rc1 (PR 206), which gets around this whole set of problems by ditching the ConfigMap for the new AWS API. Be sure to read the migration doc!

If you do try it, let us know what you think.

Note: this version is not available via the Terraform registry. Use the GitHub ref instead:

source = "github.com/cloudposse/terraform-aws-eks-cluster?ref=v4.0.0-rc1"

Using this like I typically do is not working for me:

instance_count        = lookup(var.instances_thing, "${terraform.workspace}.thing_count")

should it be

lookup(var.instances_thing, terraform.workspace).thing_count

also, lookup w/o a default value is deprecated, https://developer.hashicorp.com/terraform/language/functions/lookup

Ahh…moving that ) and putting a . after works

so w/o any defaults, it should be

var.instances_thing[terraform.workspace].thing_count


or

var.instances_thing[terraform.workspace][thing_count]

This is still on 1.5.5…not sure if its depd yet in that version

Doing it like that first suggestion you gave works though. Appreciate it.

I knew that dbl lookup was not the way to go but it worked.

It looks like you’re trying to have one configuration for multiple environments and use HCL shenanigans to switch between them

Here’s what I would do:

• Get rid of Terraform Workspaces. They’re not intended to be used in this manner

• Have separate state files or prefixes for each environment

• Have a different directory in your git for each environment

• Call the instance of that module in the root module for each environments directory.

• Write the variables directly into the module block. Don’t use variables for root modules.

Then you can do static analysis, simple and concrete

or just use Atmos (it handles scenarios like that and much more complex)

https://atmos.tools/category/quick-start

atmos

Getting rid of workspaces right now isnt an option for us. But you are right and I hate that we use them. I got it figured out though. Thanks

@Jeremy White (Cloud Posse)

A few things. If zond_id is empty, then the host won’t be populated since DNS won’t be provisioned

As for the ec2-instance for access, I would recommend using the bastion module

in the bastion config, if you set ssm_enabled to true, then you won’t have to set up or connect ssh. Instead, you can just use the aws console to reach redis. Here’s more on using ssm connect with ec2-instances

i’m not sure if i’m following your thoughts. So, I need to define an EC2 to make redis available for AWS Lambdas?

err, it was in response to “not being able to connect to host from my local”. I figured you just wanted a working connection

for lambda, just make sure you give them vpc subnet connections.

then allow their security group to connect to the security group of the elasticache

sorry i already forgot to answer ups; you were right! every reply was so accurate that within an hour or so i could finish my issue, thanks!

Checkit out — terrateam.

https://github.com/GoogleCloudPlatform/terraformer

thanks very much for the quick response @digipandit91 & @RB. I just watched the demo on terraformer, very solid. that’s all I need. thanks again.

Ive use terraformer all the time…solid tool

@Jeremy White (Cloud Posse)

seems like you would need to adjust the module to allow it to configure the AWS_LAMBDA identity type here and then you would want to configure the function attribute like so . As long as you create the lambda with AWS’s recommended policies, it should work smoothly.

Note that the module doesn’t support aws provider v5 yet

Jeremy, sounds good, thanks much!

Create a r53 record and attach it to the sftp. You just need to attach the zone_id which will belong the new record, and the domain_name. Since I do multi-stage I perform a lockup, but those are simple string values. With var values like stage=prd and dns_zone=cloudposse.com the final domain should looks like sftp.prd.cloudposse.com

SWEET THANKS!!

Hey Igor! It really worked with the S3 policy issue (thank you!), but I’m still getting a message error from elastic beanstalk resource.

 Error: waiting for Elastic Beanstalk Environment (e-idnvmxbmxd) create: couldn't find resource (21 retries)
│ 
│   with module.eb_environment_core_api[0].aws_elastic_beanstalk_environment.default[0],
│   on .terraform/modules/eb_environment_core_api/main.tf line 602, in resource "aws_elastic_beanstalk_environment" "default":
│  602: resource "aws_elastic_beanstalk_environment" "default" {

@Igor Rodionov

Include [context.tf](http://context.tf) into your sub module.

Then it will accept the context argument and you can just pass that.

This article may help you: https://masterpoint.io/updates/terraform-null-label/

But you’re getting into the advanced functionality that we allude to at the bottom of that post. And it’s something we haven’t written a post on yet. But the gist is: Include this file into your child module and it should then enable you to do what you want.

When I include this context:

module "this" {
  source  = "cloudposse/label/null"
  version = "0.25.0" # requires Terraform >= 0.13.0

  enabled             = var.enabled
  namespace           = var.namespace
  tenant              = var.tenant
  environment         = var.environment
  stage               = var.stage
  name                = var.name
  delimiter           = var.delimiter
  attributes          = var.attributes
  tags                = var.tags
  additional_tag_map  = var.additional_tag_map
  label_order         = var.label_order
  regex_replace_chars = var.regex_replace_chars
  id_length_limit     = var.id_length_limit
  label_key_case      = var.label_key_case
  label_value_case    = var.label_value_case
  descriptor_formats  = var.descriptor_formats
  labels_as_tags      = var.labels_as_tags

  context = var.context
}

I just end up with a bunch of missing variables that TF is complaining about

do I really have to define them against just for my sub-module?

Yeah. You need to include the whole file. It’s a drop in file to your child modules. It includes all the vars + the this instance so you can drop it in, use it, and accept the vars in the calling root module.

Starting to make sense?

….kinda? What I dont understand is why when we (and this could totally just be our Org doing this) include (in our root project) a

module "this" {
  source  = "cloudposse/label/null"
  version = "0.25.0"

  namespace = var.namespace
  stage     = var.accountname
  delimiter = "-"
}

as our context

without any of the extra variables

Yeah, you’re just creating your own “var.namespace” and “var.accountname” vars in your root module that you’re then using in those arguments.

But then when you want to create that same structure all the way down your hierarchy of child modules that your root module calls… you can either define a bunch of annoying inputs OR you can just drop in the full [context.tf](http://context.tf) file, pass context = module.this.context when you use that child module, and then everything will wire up nicely.

It is complex, so don’t feel discouraged haha.

We have a draft of our blog post on this advanced usage… I’ll share that with you privately

That makes a bit more sense now

Say I had a child module that loops and creates 3 different AWS resources for every object in a map, how can I ensure that I am applying one tag for each resource type as needed?

Right now I’m just passing the tags like

module "tasks" {
  source   = "./datasync_tasks"
  for_each = var.datasync_configuration

  datasync_task_name = each.key
  datasync_src_host  = each.value.src_host
  subdirectory       = each.value.subdirectory

  datasync_agent                     = aws_datasync_agent.default.arn
  datasync_iam_role                  = module.aws_datasync_role.arn
  datasync_destination_s3_bucket     = data.aws_s3_bucket.dest.arn
  datasync_task_cloudwatch_log_group = aws_cloudwatch_log_group.datasync_task_logs.arn

  context = module.this.context

  tags = merge(local.local_tags, var.additional_tags)
}

but say I wanted to tag the datasync locations differently than the datasync tasks, do I have to add that kind of logic into my child module?

I was going to just pass a task tag like tags = merge(local.local_tags, var.additional_tags, module.datasync_task_label) and use it with context = module.datasync_task_label.tags but if I have multiple resource types in the child module, I can’t really do that

oh I guess I could just pass all the tags to the module, and just use what I need for each resource?

do I have to add that kind of logic into my child module? This is a way to do it.

A better way might be to update your datasync_configuration to have an additional attribute called extra_tags and then you can pass that into your tags = merge(local_tags, each.value.extra_tags .

ah, I don’t think that’s what I want to do since I’d just be applying the same tag to each of the created datasync resources, when really what I want is a different tag type for each different resource.

I think defining the tag in the root module context and then just passing it along to the module, and then for each resource type I have use it like tags = merge(module.this.tags, module.specific_resource_tag) with each resource type I’d like to tag separately

*is probably the simplest way

But yeah it’s a lot clearer now

Ah you’re saying tagging certain resources within the child module? Yeah, you would need to create separate labels for each. Example of that here: https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/blob/main/main.tf#L23-L47

it’s not possible to override a provider defined in a child module from a parent module

child modules implicitly inherit the providers from the parent modules (if child modules don’t have their own provider defined)

what do you need to override in https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account/blob/main/requester.tf#L60 ?

what is not working?

I want to override both the requester and the accepter providers mainly to solve the Provider configuration not present as explained here but it’s not supported any work around?

since the providers are defined in the child module, they can’t be overridden.

but… Provider configuration not present error is not because of that. Note that the module works in many deployments. The error is b/c some other issues with Terraform/providers

on the other hand, if you want to test custom changes in the module,:

1. Make a subfolder modules/peering-multi-account in your repo (in the TF component)
2. Clone cloudposse/terraform-aws-vpc-peering-multi-account into the subfolder
3. Make the changes you need to test
4. Reference the module source in your code - instead of source = "cloudposse/vpc-peering-multi-account/aws" use source = ./modules/peering-multi-account

in any case, errors like

The true and false result expressions must have consistent types. The 'false' value includes object attribute

are from expressions like

for_each = local.rule_create_before_destroy ? local.keyed_resource_rules : {}

TF always complains about that. The default {} object does not have all the attributes that the main object has

there is an easy fix (works in all cases):

for_each = { for key, value in local.keyed_resource_rules: key => value if local.rule_create_before_destroy } 

I don’t have any locals declared in my root module, am I supposed to add that to my root module?

I’m kinda confused about that for_each and where it should go.. if it goes somewhere in the .terraform folder, I think that would be a problem when I get this into a pipeline which things will run in a container.. so that directory won’t be there each time the pipeline runs.

Interesting stuff:

• Have the exact same syntax for the security group root module, on my EKS root modules and I don’t get the error.

When I comment out my security_groups.tf file, I can run the root module with create_security_group = true with no errors.

module "elasticache_security_group" {
  source  = "cloudposse/security-group/aws"
  version = "2.2.0"
  //  <https://github.com/cloudposse/terraform-aws-security-group>

  namespace = var.namespace
  stage     = var.stage
  name      = var.name

  allow_all_egress = true

  rules = [
    {
      key         = "SOME KEY"
      type        = "ingress"
      from_port   = 6379
      to_port     = 6379
      protocol    = "tcp"
      cidr_blocks = [var.sg_cidr_blocks]
      self        = null
      description = "SOME DESCRIPTION"
    }
  ]

  vpc_id = module.vpc.vpc_id
}

I don’t think this module works with a security group in the same root directoy of the module, without adding someting not clear from documentation (to me at least)

.
├── CODEOWNERS
├── README.md
├── backends
│   ├── nonprod.tfbackend
│   ├── production.tfbackend
│   └── staging.tfbackend
├── build.groovy
├── data.tf
├── elasticache-redis.tf
├── environments
│   ├── nonprod.tfvar
│   ├── production.tfvar
│   └── staging.tfvar
├── partial-backend.tf
├── provider.tf
├── security_groups.tf
├── subnets.tf
├── variables.tf
└── vpc.tf

maybe I’m just not exactly sure about htat for_each you stated though

additional_security_group_rules works fine though, I’m going that route so I can move forward

that was just an example on how to fix the error

The true and false result expressions must have consistent types. The 'false' value includes object attribute

that for_each is inside the SG terraform module ( I was saying that the module could be improved)

ahh ok cool, thanks!!!

that makes a lot more sense then

should be already in the registry, it usually happens right away since it’s transparent. Not sure why it’s not there, might be some issues with the registry, let’s check a bit later

Thanks

I wonder, are you referencing the same module multiple times in your configuration

I remember someone talking about a bug in Terraform when it came to multiple copies of an external module

many components here https://github.com/cloudposse/terraform-aws-components/tree/main/modules/eks use the https://github.com/cloudposse/terraform-aws-helm-release TF module

the module uses k8s and helm providers

also, this component uses the k8s provider and k8s resources directly https://github.com/cloudposse/terraform-aws-components/blob/main/modules/eks/storage-class/main.tf

impressive docs https://registry.terraform.io/providers/hashicorp/awscc/latest/docs

I’ve seen it before but it looks like it has some resources that may not be available in the aws provider.

ref https://docs.aws.amazon.com/awssupport/latest/user/creating-resources-with-cloudformation.html?icmpid=docs_support_slack#terraform-support-app

@RB you are asking if Cloud Posse will create TF components for that?

(since it’s a Tf provider, anyone can use it with Terraform and Atmos)

not TF components but maybe some modules that may make use of it

i didnt know it was so much more developed and i didnt realize that there are some api calls that only exist in awscc and not in aws provider

i was wondering if there was any rule against using it (such as until awscc is out of beta )

you prob don’t want to use the http provider to call the API directly, should wait for the awscc provider

It would have to be an entirely new module to follow proper naming conventions

E.g. terraform-awscc-foobar

I am not sure when/where the tipping point will be. Most cloud posse development is customer driven. This has not come up.

It will probably help a lot once it’s out of beta.

It’s cool that it creates cloudformation stacks as an output so drift detection is built in

I wonder if they generate the terraform resources too

@Jeremy White (Cloud Posse)

A solution doesn’t come to mind immediately. The most mundane way would be to use lifecycle rules to ignore portions of state. Just understand you would need to fork our module for that to work best.

Thanks.

Try #atlantis instead

When you say VCS backed, does this mean you use terraform cloud or a similar service? And what is driving the build/push process? GitHub actions?

It’s TF Cloud, and the workspace was created as a ‘Version Control Workflow’ as opposed to CLI/API driven

Ahhh ok, and this is just the first push that this is happening?

Really it’s any. I just can’t figure out the proper order of operations here. Since both the TF run and GH Build will trigger at the same time for a commit, I’m just trying to figure out if the VCS Workflow can even do what I want.

I haven’t done it with the VCS workflow yet, but is there a pre-run option to run arbitrary scripts?

My hope is that you can just run a script prior to terraform initializing that looks for the image tag available in ecr in a loop, when it finds the tag, then it proceeds

you can specify a pre-plan run task but that’s shared across the org and not the workflow

Damn

I suppose it could just get changed to API driven, and just power it through GH actions call to TFC

What happens when tf runs? Does it actually attempt the deployment?

It’s just that TF would need pre-knowledge of what the resultant tag would be. Like if GH just re-built the image as latest every time it wouldn’t be a problem, but TF would either to know what the tag was going to be, or the digest and let the ECR deployment flap until the tag existed.

I guess it depends on what your tagging scheme is, but doesn’t terraform cloud expose that info? Eg a short SHA?

It’s just what goes in the TF.

image = "${aws_ecr_repository.this.repository_url}:${some_voodoo_here}"

There may be a way to do this with terraform cloud and vcs, but I’m leaning towards disabling vcs and adding a GitHub job that depends on the completion of the build/push workflow

That’s what I’m leaning towards too, at least for the general case.

Also not sure what you’re using for tagging, but the SHA is handy and unique

Just not very human readable friendly

SemVer

Oh for sure, but what about in lower environments and development?

I’ve used combos of feature brand/SHA as tags for development, then used semver for release after PR(s) merged in

regardless of how it’s tagged, still requires TF to either be psychic, or wait for GH to trigger it

hahaha, it was just a general curiosity question to see how others solve these kinds of problems

In this case, it’s a Flask app so someone can just run it directly from a GH checkout, worrying about an image is just for Fargate’s benefit

adding a GitHub job that depends on the completion of the build/push workflow I’ve got some unfinished work towards this end, just using GH to orchestrate (and bypassing ECR altogether):

1. GH builds and pushes image to its CR
2. GH updates the ECS task definition for the new image URI
3. ECS triggers a deployment upon seeing the new task definition, pulling the image from the GHCR as a private registry

you are correct, this is an omission

here https://github.com/cloudposse/terraform-aws-api-gateway/blob/main/main.tf#L28

the variable principals needs to be added since the https://github.com/cloudposse/terraform-aws-cloudwatch-logs/blob/main/variables.tf module supports it.

then you could override principals in your code when instantiating the https://github.com/cloudposse/terraform-aws-api-gateway module

PRs are welcome

Andriy, thanks for the quick reply! I’ll try to put together a PR!