#terraform (2021-11)

@Grubhold we use components to manage this. please see our docs.cloudposse.com

• iam-primary-roles

• iam-delegated-roles note: our components are often out of date from current usage

i thought thole comments used our upstream role module but it doesn’t appear to do so

basically, primary roles are provisioned in a single account and delegated roles are assumable roles in child accounts. delegated roles assume the assumable roles.

hope that helps!

@RB thanks for your reply. You mean by components the CloudPosse modules? Does this also apply for accounts that are totally seperated (different clients)? I’m not sure if I understood the difference between primary and delegated roles, are those cross referenced from each other?

Check this document out on how stacks and components are used https://docs.cloudposse.com/reference/stacks/

components are root level modules that use cloudposse upstream modules

I think I just found out a hidden gem from me, thanks for referring going through it right now

looks like tg element number is 8 and the tuple only has 2 elements

try changing the tg element number to 0 or 1

Yes, this is not possible at the moment

terraform considers it’s state as the view of the world - if it’s not in state, it doesn’t exist as far as terraform is concerned. this model obviously comes with trade offs and benefits.

you can import an existing resource into the state, but there’s not any automatic import or create as yet

yes, use this var https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/blob/master/variables.tf#L45

https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/blob/master/main.tf#L8

Perfect. Thanks!

Could it be done by using the CP terraform eks stack along with the CP terraform emr cluster stack? Assuming the appropriate perms and policies are set.

I feel like they’re close, but I have no inside info. Before the end of the year would be my guess

And there’s the first beta! Might see the release before end of month

(in Workspace A, i’m doing terraform import blahblah abc123, and on Workspace B, I’m doing terraform state rm blahblah abc123)

not yet, but it has been a discussion topic in the new “config-driven refactoring” feature coming out in tf 1.1… https://discuss.hashicorp.com/t/request-for-feedback-config-driven-refactoring/30730

Yeah no good way. I don’t go the state rm + import route and instead do the state mv -state-out approach, because import is shit and will bite you. I can’t say it makes things easier though.

I was asking for better functionality around this in that config-driven refactoring thread because this is some of the ugliest that terraform gets.

Ugly makefile hack if you’re interested:

## tf/mv-state -- Moves a given resource from one root module to another via `terraform state mv`.
## Example:
## DEST_DIR=monitor \
##		DEST_WORKSPACE=dev \
##		SRC_DIR=k8s \
##		SRC_WORKSPACE=dev \
##		RESOURCE=module.datadog_monitors \
##		make tf/mv-state;
tf/mv-state:
	cd ./components/terraform/$(DEST_DIR); \
		[[ "$(DEST_WORKSPACE)" != "default" ]] && terraform workspace select $(DEST_WORKSPACE); \
		terraform state pull > $(DEST_WORKSPACE)-`date +"%Y-%m-%d"`.tfstate; \
		cd ../$(SRC_DIR); \
		[[ "$(SRC_WORKSPACE)" != "default" ]] && terraform workspace select $(SRC_WORKSPACE); \
		terraform state mv -state-out=../$(DEST_DIR)/$(DEST_WORKSPACE)-`date +"%Y-%m-%d"`.tfstate $(RESOURCE) $(RESOURCE); \
		cd ../$(DEST_DIR); \
		terraform state push $(DEST_WORKSPACE)-`date +"%Y-%m-%d"`.tfstate;

I didn’t know about state pull and state push. thank you matt, this will make this a lot simpler

my guess, having never used this module, is that you need to specify the engine as aurora-mysql to use cluster_family = "aurora-mysql5.7".

Can you please try that and reply with the status?

@managedkaos Worked perfectly! In the github documentation the engine is just like “aurora”.

I will request the correction.

Thanks a lot for the help!

no problem. glad it worked out

Was it 4m net profit for cloud hosted services? That’s much smaller than I would expect… but maybe I’m conflating their licensing with their HCP offering. I know that they charge millions outright for Vault licenses and plenty of big enterprises are paying them big bugs for that.

It’s actually 4m in revenue. Net profit was negative (which means they’re spending more on cloud costs than they’re making).

Yeah, the license business is much better than the cloud business.

Yeah, the license business is much better than the cloud business. I wonder how cloud infrastructure providers can make money if they are also running on top of a IAAS?

Software. People pay for the software a lot more than they’d pay for the infra.

OIC, yes, of course…. and I suppose the branding, etc…

FYI a workaround/hack to prevent this behavior is to provide a random string to private_network_acl_id and public_network_acl_id, which seems to confirm veracity of bug.

I see two options:

1. Fix the current behavior so it’s consistent with the description
2. Modify the private_network_acl_id and public_network_acl_id variables to use a boolean and denote enable (i.e. rather than id) Regardless of solution it might be a good idea to expose the private_network_acl_id and public_network_acl_id as output, which will allow people to utilize ACLs instantiated inside the module in the aws_network_acl_rule resource.

the error says there is another subnet group with the same name my-vpc. changing the name would remove the conflict

if you use the cloudposse vpc module you can take advantage of its null label which allows you to use namespace, environment, stage, name, and attributes to ensure all the resource identifiers in the module will be unique

I’ve changed the name and it worked! the weird thing that there is no resource on/activate with this name - this is an empty new account. Thanks for your help and the tip! I’ll explore it :)

np! happy to help

is this the new hashicorp user group in syd? is the old one gone now? I know old organisers have moved around

hey @Chris Fowles sorry i didnt see this message sorry. yes its a new one, wasnt aware there was an old one…

put something in the data source which must be evaluated at apply-time

like timestamp() or reference to a resource that will always be updated

beautiful, thank you very much! I’ll try this out

please share a working solution in the other places you asked

Will do! I’m initially getting Data resources do not have lifecycle settings, so a lifecycle block is not allowed.

But I think with timestamp() I can get something working here, and will post my solution once I have it finalized

Well, depends_on should not be nested in a lifecycle block, so there’s that….

But also depends_on only orders create/destroy dependencies, iirc. On updates, no more guarantee

oh yeah. You’d want to use an actual attribute from the data source

make the data source echo some static string, use that in the vault data source

This seems to be working: https://github.com/transcend-io/terraform-aws-fargate-container/pull/28/files

It’s pretty ugly though, I’m going to sleep on it so that I don’t have to do the trimprefix + concat to simulate a noop

Fwiw, https://github.com/hashicorp/terraform/issues/8099#issuecomment-252273179

Thanks for that context!

In this particular case though, even though I see in the plan output that the vault value will be read during the apply (see pic), my apply fails when specifying a plan file from more than 15 minutes ago (see second pic)

Is it possible that terraform would be trying to use the provider from the plan during the apply?

Running just a terraform apply without the planfile succeeds, so it’s definitely something to do with the planfile

looks like something to do with the vault provider

These have to be updated to 0.25.0 before tenants can be used

We’ve been upgrading the modules when we need to so we haven’t gotten to all of them yet

https://github.com/cloudposse/terraform-aws-named-subnets/pull/41

New version released that supports tenants

https://github.com/cloudposse/terraform-aws-named-subnets/releases/tag/0.11.3

@RB Thanks for updating that module, the “current” list of the modules that are causing me issues are:

Downloading cloudposse/label/null 0.24.1 for alb.access_logs.s3_bucket.this...
Downloading cloudposse/label/null 0.24.1 for ecs_alb_service_task.exec_label...
Downloading cloudposse/label/null 0.24.1 for ecs_alb_service_task.security_group.this...
Downloading cloudposse/label/null 0.24.1 for ecs_alb_service_task.service_label...
Downloading cloudposse/label/null 0.24.1 for ecs_alb_service_task.task_label...
Downloading cloudposse/label/null 0.24.1 for ecs_alb_service_task.this...
Downloading cloudposse/label/null 0.24.1 for private_subnets.private_label...
Downloading cloudposse/label/null 0.24.1 for private_subnets.public_label...
Downloading cloudposse/label/null 0.24.1 for public_subnets.private_label...
Downloading cloudposse/label/null 0.24.1 for public_subnets.public_label...
Downloading cloudposse/label/null 0.24.1 for route53_url.this...
Downloading cloudposse/label/null 0.24.1 for vpc_endpoints.this...

What’s the best way to get these updated? I could do a PR for each one if that is SOP. Is there a doc on PR requirements I need to follow?

there would need to be a pr for each module to update its context

i know the ecs service one is blocked due to upgrading it’s security group module to 4.0 which will take some time. there is already someone working on unblocking that

OK, sounds like I should holdoff on using tenant for a bit then while that work is going on. If it will help if I generate PRs for the others, let me know. I can copy what you did on the named-subnets one. I’ll use an attribute for now, instead of tenant.

i simply ran make init && make github/init to update the context

if you put in the prs, I’ll review them

most things use the tenant var. just a handful need to be updated

I’ll dig in after lunch.

Ooff – @RB I’m not sure the best way to handle this – looks like a cascading problem. So, for 1 example: • terraform-aws-alb version 0.35.3 with submodule access_logs calls terraform-aws-lb-s3-bucket version 0.14.1. • terraform-aws-lb-s3-bucket version 0.14.1 with submodule s3_bucket calls terraform-aws-s3-log-storage version 0.24.0. • terraform-aws-s3-log-storage version 0.24.0 calls the null-label module version 0.24.1 in [context.tf](http://context.tf) . Someone has already updated terraform-aws-s3-log-storage to use null-label v. 0.25.0 – the version terraform-aws-s3-log-storage with that change is 0.24.1. So, if I update terraform-aws-lb-s3-bucket to use terraform-aws-s3-log-storage v. 0.24.1 then the new version of terraform-aws-lb-s3-bucket will become 0.14.2. Then I’ll have to update terraform-aws-alb to point to that new version. And on-and-on back up the chain.

It’s doable but it seems error prone and tedious. Is there a better way?

I think my enthusiasm for using tenant in our labels has waned .

i think it shouldn’t take more than a couple hours

For EKS you need https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html instead of creating LB by yourself

Ha. Okay. This makes way more sense.

We have a component for this here https://github.com/cloudposse/terraform-aws-components/tree/master/modules/alb-controller

There is also Terrascan and Kics We mainly hear about checkov and we have quite a lot of customers using it together with env0 on the plan.

Why would one use env0 over TFE?

There are a few main differences between TFE and env0 like also supporting Terragrunt, custom flows, drift detection, TTL and scheduling and much more.

You can check out our website for more details - https://www.env0.com/why-env0

• Disclaimer - I am the CTO and co-founder of env0.

I’ve used checkov and tfsec together, there is some overlap, but pretty useful. it gets annoying if you have to put ignores in for both tooling

@Michael Bottoms take a look at https://github.com/iacsecurity/tool-compare

Disclaimer: I’m the CEO of Indeni (company behind Cloudrail)

@Yoni Leitersdorf (Indeni Cloudrail) huge thanks for the link! Found this pretty useful)

Happy to. Let me know if I can help in anything else

Hi Daniel, have you seen this module?

https://github.com/cloudposse/terraform-aws-documentdb-cluster

I have, but I see no mention of global cluster configurations https://aws.amazon.com/es/documentdb/global-clusters/

is it possible to enable a global cluster in the terraform resource itself? i don’t see it here

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/docdb_cluster

There is a PR in the works https://github.com/hashicorp/terraform-provider-aws/pull/20978

ah ok so it doesn’t seem possible yet :(

Too bad. I guess we’ll just have to wait

upvotes help!

Looks like the resource landed in 3.65.0… https://github.com/hashicorp/terraform-provider-aws/pull/20978

we got a ticket to add it to packages https://github.com/cloudposse/packages/issues/1674 for any enterprising contributors

very cool, thanks for sharing, this is a must have for terraform helpers

I’m thinking of adding this to PR checks or cron job

@Mohammed Yahya cool! please do let me know if you find any mismatches with your workflow. my current projects are all not using terraform so it’s not getting any bullet-proofing from my side as a result

(i’m the author of this one)

hey @Sergey! I’ll bring this up on #office-hours today

hey @Erik Osterman (Cloud Posse), awesome, thank you! Sorry I couldn’t make it, I’m pretty swamped rn unfortunately..

https://www.terraform.io/docs/cloud/workspaces/variables.html#scope

similar to spacelift contexts. interesting

It’s most likely because of this alb_ingress_unauthenticated_listener_arns

if you do a targetted apply of -target module.alb first, it should work

that helped a bit since after that I got this error:

│ Error: error creating application Load Balancer: InvalidSubnet: VPC vpc-024b08d14c04aa553 has no internet gateway
│       status code: 400, request id: a5e9dd93-d1a3-49bf-a8d3-0748dcb1afe7
│ 
│   with module.alb.aws_lb.default[0],
│   on .terraform/modules/alb/main.tf line 64, in resource "aws_lb" "default":
│   64: resource "aws_lb" "default" {

with this vpc/subnets code

module "vpc" {
  source  = "cloudposse/vpc/aws"
  version = "0.28.1"

  name = "microservices"

  cidr_block                       = "20.0.0.0/16"
  assign_generated_ipv6_cidr_block = true

  context = module.this.context
}

## Subnets

module "subnets" {
  source  = "cloudposse/dynamic-subnets/aws"
  version = "0.39.7"

  name = "microservices"

  availability_zones = var.availability_zones
  vpc_id             = module.vpc.vpc_id
  igw_id             = module.vpc.igw_id
  cidr_block         = module.vpc.vpc_cidr_block

  nat_gateway_enabled      = true
  nat_instance_enabled     = false
  aws_route_create_timeout = "5m"
  aws_route_delete_timeout = "10m"

  context = module.this.context
}

by my understanding nat_gateway_enabled: true in that case it means internet gateway, right?

try setting enable_internet_gateway = true in the vpc module

same thing

added both internet_gateway_enabled and enable_internet_gateway

but I see that internet gateway was not created

same goes for nat gateway

is the alb module using the same vpc from the vpc module ?

yes

https://github.com/cloudposse/terraform-aws-vpc/blob/a3c4b1598942f3ae7a259d3f2823761a97befbd4/variables.tf#L60-L65

so local.internet_gateway_enabled should resolve to true

and then that true should be passed in here to create the internet gateway

https://github.com/cloudposse/terraform-aws-vpc/blob/a3c4b1598942f3ae7a259d3f2823761a97befbd4/main.tf#L35-L40

can you try a completely fresh terraform module and only provision the vpc and show the plan ?

yeah, gimme few min since i need to manuallyu clear all resources

since tdestroy doesn’t work with -target

it might help to create a new directory

and then create a new [main.tf](http://main.tf) file within that directory

that [main.tf](http://main.tf) should only consume the vpc module

then check the plan and see if it makes sense ( it should show the igw is going to be created)

so the case here, might be that I’m using several files ?

i have no idea what it could be, but if you follow the steps above, you take out a lot of variables

if you can verify the plan above shows the igw then we know the issue is not related to the vpc module.

yea, there’s igw

but with plan for module.alb target there’s no

same goes for subnets

ok so in the new module, now copy over the alb and have it use the vpc module

in the new module, also provision the subnets. do it one at a time.

so one at a time passed

perfect!

then the issue isn’t with the upstream modules, the issue must be with your other root module directory

yeah maybe as well because I’ve got there a 2 modules for different microservices?

and on the other hand the thing I also wanted to ask is how to enable ALB just for a single service rather than for all of them

since communication between them is done through queue

one of the terraform principles is to keep modules (directories) as small as possible

are you putting more than once service in the same terraform directory ?

nope, each service got it own directory

then only two modules where in in gateway service where alb module existed

and on the other hand the thing I also wanted to ask is how to enable ALB just for a single service rather than for all of them it depends on how you have it setup. are you using 1 ALB and multiple services via different listener arns ? or 1 ALB per service ?

1 ALB for gateway, other microservices doesn’t have a listener anr because they use rabbitmq for communication between each other

enabling an ALB for a service i think you want to create alb listeners for only your api services and no listener arns for your non-api services ?

yeah

what are you using to create the listener arns on the alb ?

raw resource or module ? if module, which one ?

hmm i don’t have a listener

the only thing I’ve added for a gateway service was

  # Without authentication, both HTTP and HTTPS endpoints are supported
  alb_ingress_unauthenticated_listener_arns       = module.alb.listener_arns
  alb_ingress_unauthenticated_listener_arns_count = 1

  # All paths are unauthenticated
  alb_ingress_unauthenticated_paths             = ["/*"]
  alb_ingress_listener_unauthenticated_priority = 100

on a module which use cloudposse/ecs-web-app/aws

oh ok so ecs-web-app cp module, uses ecs-service-task cp module

if you want to disable the listener arn, youd have to use the ecs-service-task cp module directly as the ecs-web-app (with web in the name) implies that the service is an http service

@Marcin Mrotek see this one https://github.com/cloudposse/terraform-aws-ecs-alb-service-task

that will make the service task and will not set the ingress on the alb (like you want)

ok so that one should be used on services that I don’t want to use ALB right?

exactly

but auto scalling for taht one would have to be handled on my own, right?

the ecs service task handles auto scaling i believe

oh wait no i dont see autoscaling in the ecs service task

yes i suppose youll have to handle yourself

see https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-autoscaling

you can see how this module and other modules are used directly in the ecs web app module

https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/ce7c53a7eb0c75592b67098a043ebe71eab7ab3c/main.tf#L220-L222

Sure. What are you trying to import ?

If you’re trying to import the ec2 instance and the name of the module is ec2_instance, it should be as easy as this.

terraform import module.ec2_instance.aws_instance.default i-12345678

The full list of resources and their resource names are documented here

thanks

this is due to the null label argument regex_replace_chars

https://github.com/cloudposse/terraform-null-label/blob/488ab91e34a24a86957e397d9f7262ec5925586a/exports/context.tf#L201

try setting that to "/[^a-zA-Z0-9-\/]/" (or similar) to avoid the replacement

Thank you so much, this should work and I should have spotted this earlier, will report back

that’s ok. i think this is a bug. we should create a separate null label reference with this argument just for the log group name.

So i had a look at this, but the above didn’t work, it wasn’t escaping with one slash so I had to add two which gave me a plan

however, it still removes the

/

the one

\ 

gave me tf errors

something really wacky going on with this module

  # module.cloudwatchlogs.aws_cloudwatch_log_group.default[0] will be created
  + resource "aws_cloudwatch_log_group" "default" {
      + arn               = (known after apply)
      + id                = (known after apply)
      + name              = "dev"
      + retention_in_days = 14
      + tags              = {
          + "Name"  = "/aws/kinesisfirehose/aws-waf-logs-dev-app"
          + "Stage" = "dev"
        }
      + tags_all          = {
          + "Name"  = "/aws/kinesisfirehose/aws-waf-logs-dev-app"
          + "Stage" = "dev"
        }
    }

  # module.cloudwatchlogs.aws_cloudwatch_log_stream.default[0] will be created
  + resource "aws_cloudwatch_log_stream" "default" {
      + arn            = (known after apply)
      + id             = (known after apply)
      + log_group_name = "dev"
      + name           = "/aws/kinesisfirehose/aws-waf-logs-dev-app"
    }

I would’ve expected log_group_name to be /aws/kinesisfirehose/aws-waf-logs-dev-app

the Name tag looks correct. the name argument should match that of the Name tag. looks like something isn’t getting cleared. try consuming this cloudwatch logs module in a separate terraform directory and just run a plan.

this looks better, the reason it’s being changed to ‘dev’ is because ‘stage’ is being populated which we use elsewhere, but still can’t get past this issue where the ‘/’ are being stripped by the regex

i could make an MR to the repository so that the log group name/stream name is not validated by the id field?

just let me know thoughts on moving this forward

I’ll have to test this out locally and get back to you. there must be a way to override this. if not, maybe we need a new escape hatch in the module itself

ok let me know if there’s anything i can do to facilitate i’ve tried setting regex_replace_chars = “” regex_replace_chars = null regex_replace_chars = “/[^a-zA-Z0-9-\/]/” regex_replace_chars = “/[^a-zA-Z0-9-\/]/”

so it looks like it needs a custom label

i put in this pr and it shows the correct log group https://github.com/cloudposse/terraform-aws-cloudwatch-logs/pull/26

we’ll have to wait until it’s approved. for now, you can use the source in the pr description until its merged

@David new version released https://github.com/cloudposse/terraform-aws-cloudwatch-logs/releases/tag/0.6.1

Thank you!

usually we allow single feature improvements through a pr. what improvements are you planning?

it sounds like you’re going to make a linode specific module repo?

feel free to pr a branch when it’s ready :)

Yeah the idea is to take this as-is and make it work for Linode, not AWS

we can take all the aws related prs as we are exclusively aws

Good to know. So anything not AWS is basically “feel free to fork but it can’t live with CloudPosse”

Thanks for the clarification!

if we ever branched out to additional clouds, we’d most likely create a separate repo and maintain that like you plan to do

that seems to be the common standard across the terraform ecosystem and that’s why most aws specific repos contain that in the name of the repo

Yep

This’ll be used to spin up first 100 and, when that goes well, 1k instances. AWS pricing for that is a little scary, hence Linode.

@Grubhold check out the docs in https://docs.cloudposse.com/ — That will help you understand your path forward.

Specifically: https://docs.cloudposse.com/tutorials/atmos-getting-started/

what are you using for inputs? are you getting any error messages?

@Matt Gowie Thanks for pointing out, I will go through the documentation to try and understand how it works.

@RB the default.auto.tfvars currently has these inputs I did get a successful plan even though the inputs are just generic currently, but as Matt suggested I’ll go through the documentation to understand. Because I’m not sure how I can use this to bootstrap an account on an existing organisation. What will I need to deploy an account to that organisation etc.

As far as I understand Atmos is working like a wrapper. And the components don’t necessarily need atmos to work unless you want to use the stacks structure. Is that correct?

Because in our organisation’s case its very hard to get such tools accepted

So I’m assuming that if we don’t use Atmos and stacks we need to use the .tfvars file to provide the information needed to create an account under the existing organisation?

Therefore, can I not use the service_control_policies_config_paths to the yaml stacks?

you can use the default auto tfvars instead of atmos but you’re giving up all the yaml.

armos will simply convert the yaml into terraform var inputs for you and then run terraform workspace selection, terraform init/plan/apply

the service control policies config path input for the account component is for yaml that contain service control policies which is very different from the stack yaml configs

@RB Thanks for your reply. It makes sense to me but I’m afraid that we won’t be able to use it on Azure Pipelines that we use for CICD because new tasks pretty much doesn’t get approved at all. I’m afraid Atmos would need it. I might be very mistaken about how Atmos works..

Also, I hope you don’t mind I keep getting this error while trying to provision the account

│ Error: Error creating organization: AlreadyInOrganizationException: The AWS account is already a member of an organization.
│
│   with aws_organizations_organization.this,
│   on main.tf line 91, in resource "aws_organizations_organization" "this":
│   91: resource "aws_organizations_organization" "this" {

I have the root account credentials. What am I missing here? Can’t I just deploy the account on an already existing org with this component?

it looks like you already have an organization created and will have to import the org here

that’s unfortunate that a tool like atmos wouldn’t be approved because that’s the tool that makes our refarch powerful. however, the cool thing about atmos is that it’s just a wrapper for terraform. all it does is deliver terraform var inputs from yaml and so you can substitute the yaml by using default auto tfvars directly

Yes but I won’t give up on it just yet, I’ll see if we can use it somehow in the pipeline. Otherwise indeed we’d just use tfvars then.

Can you please define “import the org here”? I think that’s where I’m stuck

terraform import your organization resource into your current terraform state

!! Ok that is a very good progress. So I imported the organization to my state file, then ran apply and it added outputs as well. But then it says that the real infra matches with the current configuration so nothing changes. Why is it not trying to provision the account I have specified?

@RB Sorry for the ping. I would really appreciate your help with this. I can’t seem to manage to skip organization creation and just create an account under this existing one. I’ve been working on https://github.com/cloudposse/terraform-aws-components/blob/master/modules/account/main.tf to try and skip the organization with no use.

Unfortunately, this module is probably pretty stale

No customers are currently using / paying for maintenance on this module

All our effort is currently on https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation

@Erik Osterman (Cloud Posse) thanks for the response

@Erik Osterman (Cloud Posse) this would be a good one to chat about during #office-hours

wow!

oh, i read “control tower”

but yes, let’s discuss

i dunno how you’d identify outputs to generate them. but here’s a tool that will generate a variables.tf from all var references… https://github.com/YakDriver/scratchrelaxtv

Thanks for that!

Mainly looking for a way to generate all the boilerplate stuff for a module. Say given a main.tf with xyz resource, a tool that will generate a bunch of generic outputs for xyz resource

it’s not for everyone, or every module, but i often just output the entire resource. that might give you a way to generate outputs automatically, since you wouldn’t need to know the attributes… but you’d still have to name the output at least…

resource "aws_iam_role" "this" {}

output "aws_iam_role_this" {
  value = aws_iam_role.this
}

That’s a great idea, thanks for that @loren!

Also consider Yeoman or Cookie Cutter?

I’ve never used either actually

if someone could re-run the terratest, I’m sure it will pass this time

Thanks to @RB ( I think ) for commenting on the PR. I’ve made the changes requested and I think this is ready to merge now

any chance of a review today and re-run of the tests?

I’ve updated the PR again to include the enabled logic!

argh:

Because data.aws_partition.current has "count" set, its attributes must be
46
accessed on specific instances.
47

48
For example, to correlate with indices of a referring resource, use:
49
    data.aws_partition.current[count.index]

not sure exactly what is going on here

@Stephen Tan i pushed an update

thanks

a lot has changed in this module since 0.42.0

have you read over these release notes ? https://github.com/cloudposse/terraform-aws-eks-cluster/releases/tag/0.42.0

one of those options may not be toggled in your arguments

I’m currently running 0.43.2 and its based on the eks example project. I should be using that preferred method. For posterity, I added it and same issue.

I’ll check out kube_exec_auth_enabled…i don’t quite understand though.

try these settings https://github.com/cloudposse/terraform-aws-components/blob/677eb313a7b096aa41025930ba4795849ab677a0/modules/eks/main.tf#L47-L57

checking it out.

kube_exec_auth_enabled = !var.kubeconfig_file_enabled Heh. genius.

That did it. What causes this?

I have another cluster that was deployed the same way but doesn’t have these issues.

@Maya Aravot

As of a few years ago, https://www.oreilly.com/library/view/terraform-up/9781492046899/ was wonderful. I’m not sure how updated it is for newer tf versions, but it explains the details of why things like statefiles, locking, etc. are parts of terraform

A Cloud Guru (soon to be integrated with Pluralsight) has a slew of Terraform classes: https://acloudguru.com/search?s=terraform

And Pluralsight as well: https://www.pluralsight.com/courses/terraform-getting-started

I should also mention, Pluralsight has a free weekend starting today!!!!! https://www.pluralsight.com/offer/2021/q4-free-weekend

Thanks!

this looks as tho it does add subnet table route ids to the transit gateway

https://github.com/cloudposse/terraform-aws-transit-gateway/blob/17e233cce95d5e1830fc85cb0b5d925682170011/main.tf#L89

oh awesome! I wasn’t expecting it to be so well commented. Will hunt a bit deeper in the repo next time

i personally havent tested the beta yet. It might be worth creating an issue with hashicorp/terraform

I see this behaviour with 1.0.10

I wouldn’t expect these logs, I mean

thats interesting. i never noticed that before. do you see this as a security issue ? or just annoying that you have to remove these logs ?

i checked my /tmp directory and i dont see any terraform logs. im using terraform 1.0.9

They appear while running and are deleted afterwards

Security issue, but also it seems to incur performance cost

here seems fine

yay Came across the terraform-aws-transit-gateway module, and looked for inter region peering, but couldn’t find it. Do you support it? )

hmm would that require peering 2 transit gateways ? like this https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_peering_attachment ?

yeap

we don’t support this resource but if you provision 2 transit gateways (with or without our module), you can use the above resource to peer them together

Great thx!!

plz let me know how it goes. I’m curious if this is the only resource needed or if you also have to update route tables or if they propagate automatically

I will.

This is exactly what we’re doing so seems sane to me

Worth bringing up in office hours too

Oh great, awesome to hear. I was spinning my wheels over thinking this

Thanks @RB

It’s just like the backend problem - you need to build your S3 bucket in order to use it for the state: https://github.com/cloudposse/terraform-aws-tfstate-backend

Yep, that’s actually just like it.

sorry, should have threaded

All good. Could hack around that idea, but it would require the first_apply variable. And I’d need to populate that variable with a true / false according to a pre-plan script.

Which would be a bummer.

So not runtime, but yeah possible.

oh you want to do it in a normal terraform workflow…

Yeah — At runtime being the key bit.

i was figuring some kind of wrapper would be in place to handle the logic

still runtime, to me. just runtime of the wrapper

Good point. I wasn’t thinking a wrapper.

Probably doesn’t matter as TF doesn’t have a date comparison function. Which I’m unpleasently surprised about.

invoke a lambda as a data source that manages the date comparison or even just returns the “correct” ami id?

Lambdas make me die inside.

But yes.

That would be a fine way of doing it.

hahaha, they are the glue for everything that doesn’t exist out of the box

an external data source would probably work also

Yeah, was hoping to avoid. I think that’s the way.

But better than the lambda.

write the external data source in python and maybe it can sometimes be a lambda

Hahah yeah, I’ll use the random provider to coin toss determine if it should be invoked using the external data source or via the lambda via the http data source. That’ll be good.

Discussion and likely path forward discussed here: https://sweetops.slack.com/archives/CB6GHNLG0/p1637606453235000

cc @matt

does the firewall manager require the firehose and s3 bucket to be in the same region?

it doesn’t look like you can supply your own s3 bucket if you enable firehose

https://github.com/cloudposse/terraform-aws-firewall-manager/blob/82c2264e4c666ec28467cbca301c587ab4a4731c/firehose.tf#L11

ya if the s3 bucket already exists, then it will error out when creating the same bucket

@Anand Gautam please feel free to open a pr and we’ll take a look!

hey RB, thanks for digging in. I see that bucket name is derived from cloudposse/label/null module -https://github.com/cloudposse/terraform-aws-firewall-manager/blob/82c2264e4c666ec28467cbca301c587ab4a4731c/firehose.tf#L22 Would that not make it unique?

not if the same bucket already exists

are you saying that the bucket it’s trying to create does not exist yet ?

ok I see that this is hardcoded attributes = ["firehose"] I also didn’t pass in these vars in firewall module:

  namespace           = var.namespace
  stage               = var.stage

Once I pass these in, I assume it will make it unique. I will give that a shot

those are passed in via the context

im not sure i fully understand the issue

is the issue

• you have a bucket that you want to reuse that the module is trying to create ?

• you dont have a bucket that you want to reuse and youd like the module to create it and even tho it doesnt exist, it’s failing to create it ?

in either case, it would help to see a plan of the s3 bucket resource when youre using the module

ok it’s working now

I didn’t pass in namespace and stage before so the bucket was not unique

Running into a new error now when applying a wafv2 policy:

│ Error: Creating Policy Failed: InvalidInputException: Error in the SecurityServiceData.ManagedServiceData at [Source: (String)"{"defaultAction":{"type":"ALLOW"},"loggingConfiguration":"{\"logDestinationConfigs\":[\"arn:aws:firehose:us-east-1:xxxxxx:deliverystream/aws-waf-logs-xxxx-xxxx\"],\"redactedFields\":[{\"redactedFieldType\":\"SingleHeader\",\"redactedFieldValue\":\"Cookies\"},{\"redactedFieldType\":\"Method\"}]}","overrideCustomerWebACLAssociation":false,"postProcessRuleGroups":[],"preProcessRuleGroups":[{"excludeRules":[],"managedRuleGroupIdentifier":{"managedRuleGroupName":"AWSManagedRulesLinuxRule"[truncated 145 chars]; line: 1, column: 58]

I added this policy as a default: https://github.com/cloudposse/terraform-aws-firewall-manager/blob/82c2264e4c666ec28467cbca301c587ab4a4731c/examples/complete/main.tf#L70

how my variable looks:

variable "waf_v2_policies" {
  type = list(any)
  default = [{
    name               = "linux-policy"
    resource_type_list = ["AWS::ElasticLoadBalancingV2::LoadBalancer", "AWS::ApiGateway::Stage"]

    policy_data = {
      default_action                        = "allow"
      override_customer_web_acl_association = false
      pre_process_rule_groups = [
        {
          "managedRuleGroupIdentifier" : {
            "vendorName" : "AWS",
            "managedRuleGroupName" : "AWSManagedRulesLinuxRuleSet",
            "version" : null
          },
          "overrideAction" : { "type" : "NONE" },
          "ruleGroupArn" : null,
          "excludeRules" : [],
          "ruleGroupType" : "ManagedRuleGroup"
        }
      ]
    }
    }
  ]
}

Any idea as to the cause?

try outputting the full object and see what it returns

note: use underscores instead of dashes in any resource/data/output names (best practice)

output "github_action_json" {
  value     = google_service_account_key.ci_tools
  sensitive = true
}

yeh i normally do, my bad

thanks for pointing that out

@Steve Wade (swade1987) np.

did outputting the full object allow you to figure out how to output the base64’ed private key ?

i actually just wrote the output to a file for the time being

sure please submit a pr

could you also put the error that you’re seeing in the pr itself? I’m curious because we’ve seen this work in other places

it’s not causing an error - but the entire variable is now missing so the created CodeDeployment Group is missing the tagging

I’m just checking my change now

it’s throwing another issue strangely

you mean the ec2 tag set argument is not provided to the raw resource?

yah

can you share the inputs and plan in the pr description

sure - thank you

np. this may take me until later to get to but let’s try to resolve this today

Hi - I didn’t get around to sending a PR as I ran into a problem when setting up a conditional for the ec2_tag_filter variable. For some reason, setting up

for_each = var.ec2_tag_filter == null ? [] : [var.ec2_tag_filter]

results in the following error when running a plan:

module.notify_slack.aws_sns_topic_subscription.sns_notify_slack[0]: Refreshing state... [id=arn:aws:sns:eu-central-1:615854049996:stargate:0fd866e3-2b52-438e-819b-73b98ee08701]
╷
│ Error: Invalid index
│
│   on .terraform/modules/code-deploy-stargate/main.tf line 184, in resource "aws_codedeploy_deployment_group" "default":
│  184:       key   = ec2_tag_filter.value["key"]
│     ├────────────────
│     │ ec2_tag_filter.value is empty set of object
│
│ Elements of a set are identified only by their value and don't have any separate index or key to select with, so it's only possible to perform operations across all elements of the set.
╵
╷
│ Error: Invalid index
│
│   on .terraform/modules/code-deploy-stargate/main.tf line 185, in resource "aws_codedeploy_deployment_group" "default":
│  185:       type  = ec2_tag_filter.value["type"]
│     ├────────────────
│     │ ec2_tag_filter.value is empty set of object
│
│ Elements of a set are identified only by their value and don't have any separate index or key to select with, so it's only possible to perform operations across all elements of the set.
╵
╷
│ Error: Invalid index
│
│   on .terraform/modules/code-deploy-stargate/main.tf line 186, in resource "aws_codedeploy_deployment_group" "default":
│  186:       value = ec2_tag_filter.value["value"]
│     ├────────────────
│     │ ec2_tag_filter.value is empty set of object
│
│ Elements of a set are identified only by their value and don't have any separate index or key to select with, so it's only possible to perform operations across all elements of the set.
╵
╷
│ Error: Unsupported attribute
│
│   on .terraform/modules/code-deploy-stargate/main.tf line 197, in resource "aws_codedeploy_deployment_group" "default":
│  197:         for_each = ec2_tag_set.value.ec2_tag_filter
│     ├────────────────
│     │ ec2_tag_set.value is set of object with 1 element
│
│ Can't access attributes on a set of objects. Did you mean to access an attribute across all elements of the set?

The tag of the repo is here: https://github.com/StephenTan-TW/terraform-aws-code-deploy/blob/6145b27203c832cd66176369dd3d326a55c14a74/main.tf . When I set up the conditional for ec_tag_filter to not have any conditional:

for_each = var.ec2_tag_filter

then the plan works as “normal”. It’s quite bizarre. Here is the plan output:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.code-deploy-stargate.aws_codedeploy_deployment_group.default[0] is tainted, so must be replaced
-/+ resource "aws_codedeploy_deployment_group" "default" {
      ~ arn                    = "arn:aws:codedeploy:eu-central-1:615854049996:deploymentgroup:stargate/stargate" -> (known after apply)
      - autoscaling_groups     = [] -> null
      ~ compute_platform       = "Server" -> (known after apply)
      ~ deployment_group_id    = "e0942c61-47bd-4098-903f-c5331d181d7c" -> (known after apply)
      ~ id                     = "e0942c61-47bd-4098-903f-c5331d181d7c" -> (known after apply)
        tags                   = {
            "Name" = "stargate"
        }
        # (5 unchanged attributes hidden)


      + blue_green_deployment_config {
          + deployment_ready_option {
              + action_on_timeout    = (known after apply)
              + wait_time_in_minutes = (known after apply)
            }

          + green_fleet_provisioning_option {
              + action = (known after apply)
            }

          + terminate_blue_instances_on_deployment_success {
              + action                           = (known after apply)
              + termination_wait_time_in_minutes = (known after apply)
            }
        }


      + ec2_tag_set {
          + ec2_tag_filter {
              + key   = "Service"
              + type  = "KEY_AND_VALUE"
              + value = "stargate-frontend"
            }
          + ec2_tag_filter {
              + key   = "Service"
              + type  = "KEY_AND_VALUE"
              + value = "stargate-processing"
            }
        }

        # (3 unchanged blocks hidden)
    }

The TLDR is that in order for this to work, we need to remove the conditional from the ec2_tag_filter variable! No idea why this is required

I’ll push a PR based on the WORKING code. I know that this isn’t consistent with CloudePosse conventions, but then it’s something we can work on. I think you are in the US ( I’m in the UK ) so I won’t expect an answer for some time

here is the PR: https://github.com/cloudposse/terraform-aws-code-deploy/pull/13

thanks in advace @RB

thank you for pointing out the flawed logic @RB - I couldn’t see the wood for the trees! It’s so obvious in hindsight. I’ve updated the PR now

this should all just work fine and dandy

in fact - hold off for a while as I test this

@Ben Smith (Cloud Posse)

Hey sorry for the late response, So due to how terraform deals with objects we have to pass all those variables - meaning you have to pass those variables as null or a default value. Also at this time the SLO module is more focused at metric based - this is because its very easy to write a numerator and denominator query. We plan to add monitor based support soon, but it requires integration with our datadog-monitor module and fetching those monitor IDs.

Looks like it’s not able to find the route table in your accepter vpc

https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account/blob/0cad62a594f9a116a3a2a395ea0c8e1a41e541d4/accepter.tf#L66-L71

It uses the found accepter subnets to query the route table

i’d double check the accepter input vars and verify that you have a route table connected to those subnets

do you have any advise on how i can proceed?

did you double check the accepter inputs and verify the subnet route table matches that of the tag you supplied to the module?

my accepter input vars:

accepter_region                          = "eu-west-1"
  accepter_vpc_id                          = "vpc-0a28ca6a26d8df84b"
  accepter_allow_remote_vpc_dns_resolution = "true"
  accepter_subnet_tags                     = { "Name" = "vpc-pcc-dev-private-data-1"}

this mean the route tables query should have result right?

it should as long as vpc-0a28ca6a26d8df84b is in eu-west-1

yes, vpc-0a28ca6a26d8df84b is in eu-west-1

can you try creating a new module and add a tf file trhat contains something like this

provider "aws" {
  alias  = "accepter"
  region = "eu-west-1"
}

data "aws_subnet_ids" "accepter" {
  provider = aws.accepter
  vpc_id   = "vpc-0a28ca6a26d8df84b"
  tags     = { "Name" = "vpc-pcc-dev-private-data-1" }
}

output "subnets" {
  value = data.aws_subnet_ids.accepter
}

what does that return when you do an apply ?

so it looks like it can find it. im at a loss why the module doesn’t find it since its using the same logic

i tried with below:

provider "aws" {
  alias  = "requester"
  region = "eu-west-2"
}

data "aws_route_table" "requester" {
  provider  = aws.requester
  vpc_id   = "vpc-04dcfe9aaafa9a768"
  subnet_id = "subnet-0d74248d90195f61b"
}

output "requester_routes" {
  value = data.aws_route_table.requester
}

The output is

Error: query returned no results. Please change your search criteria and try again
│
│   with data.aws_route_table.requester,
│   on main.tf line 47, in data "aws_route_table" "requester":
│   47: data "aws_route_table" "requester" {

If i comment out the subnet_id attribute in data.aws_route_table.requester , i’ll get some output

provider "aws" {
  alias  = "requester"
  region = "eu-west-2"
}

data "aws_route_table" "requester" {
  provider  = aws.requester
  vpc_id   = "vpc-04dcfe9aaafa9a768"
  #subnet_id = "subnet-0d74248d90195f61b"
}

output "requester_routes" {
  value = data.aws_route_table.requester
}

The output (partially):

Changes to Outputs:
  + requester_routes = {
      + arn            = "arn:aws:ec2:eu-west-2:757726977567:route-table/rtb-0e7c5763cd6513f34"
      + associations   = [
          + {
              + gateway_id                 = ""
              + main                       = true
              + route_table_association_id = "rtbassoc-03d465d3131e67c64"
              + route_table_id             = "rtb-0e7c5763cd6513f34"
              + subnet_id                  = ""
            },
        ]
      + filter         = null
      + gateway_id     = null
      + id             = "rtb-0e7c5763cd6513f34"
      + owner_id       = "757726977567"
      + route_table_id = "rtb-0e7c5763cd6513f34"
      + routes         = [
          + {
              + carrier_gateway_id         = ""
              + cidr_block                 = "194.60.191.6/32"
              + destination_prefix_list_id = ""
              + egress_only_gateway_id     = ""
              + gateway_id                 = "vgw-0c0438f78768596fa"
              + instance_id                = ""
              + ipv6_cidr_block            = ""
              + local_gateway_id           = ""
              + nat_gateway_id             = ""
              + network_interface_id       = ""
              + transit_gateway_id         = ""
              + vpc_endpoint_id            = ""
              + vpc_peering_connection_id  = ""
            },
          + {
              + carrier_gateway_id         = ""
              + cidr_block                 = "194.60.191.20/32"

could it be the way the route tables are setup without explicit subnet association? so it didn’t work with the query?

after adding the explicit subnet association, i managed to create the vpc peering connection

thanks for your help @RB

oh that’s awesome!! it might be good to document this in the readme

glad you got it working! :)

another question, how is it possible to make this module work with for_each?

i tried below

module "vpc_peering_cross_account" {
  for_each = var.remote_peering_data
  source           = "git::<https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account.git?ref=tags/0.17.1>"

but encountered the error :

Error: Module module.vpc_peering_cross_account contains provider configuration
│
│ Providers cannot be configured within modules using count, for_each or depends_on.

ah yes it’s not possible because the providers are defined in the module itself

is there any plan to remove the providers from the module itself so that the for_each/count will work?

not yet, but we would review a pr for this

I use github. What’s wrong with using github as a private source ? You can always create a private module per repo.

I’ve read of citizen but I haven’t personally used it. TBH github has been so easy that when I worked with TFE or Spacelift, i’ve always used github as the point of truth.

I’ve been using Gitlab.

Gitlab is a valid registry so you can hit it as you would terraform cloud and such. https://docs.gitlab.com/ee/user/packages/terraform_module_registry/index.html

What’s wrong with using github as a private source @RB the prob comes with the fact that you can’t have s’thing like

  terraform {    
    required_version = ">= 0.13"    
    required_providers {    
      aws = {    
        source  = "hashicorp/aws"                                           
        version = ">= 3.25.0"                                               
      } 

and so you need to keep have a mechanism to update the version in the module source ... And that is not so much fun when you have a dozen of root modules with various permutations of child modules (scattered across +100 git repos)

@aimbotd that is very neat, wasn’t aware of that, thx a bunch !

You got it. I also wasn’t aware of it until recently. I’ve since moved over a bunch of my projects from GH over to GL. They even have a remote state feature if you’d like GL to manage the remote states. Super easy to get up and running. Honestly, migrating over to GL was one of the easier migrations I’ve had in some time.

i’m starting to consider this too especially when i take into account their CI/CD offer vs GHA which for CD part … is limited (to be polite).

I use dependabot to update the git source ref. Renovate also works

I may or may not have purchased Gitlab Ultimate to exercise all of the features. I’m really enjoying their pipeline and various security related features. I honestly thought GL was subpar because I used it many, many moons ago and just didn’t like it. My most recent role has me pretty embedded into it and I have to say, it’s been lovely.

I use dependabot to update the git source ref. Renovate also works @loren thanks for mentioning , never came across that path but i’ll certainly check it

I use dependabot to update the git source ref. Renovate also works same, it’s super convenient

had a quick look at dependatbot and will need to double check if this works with non public/ private TF rregistry (i.e - spacelift i know has provide one and now GL however not GH). Anyway will crack on with a quick playground to see. thx folks for your help

Be sure to look at renovate also. It’s been moving more quickly with new features than dependabot the last couple years. Dependabot does just enough for me, but sounds like your use case might require more

right, both been marked to look at, much thanks

how do you configure dependabot to do that? Is there specific support for Terraform or can you add your own languages?

fyi @Alex Jurkiewicz - https://sysdogs.com/articles/automating-terraform-updates-with-dependabot although that is using spacelift (and its private registry) which is not what i have/ can use atm . HTH

in .github/dependabot.yml, for each directory with .tf files, add an entry like this (changing the directory as needed):

- package-ecosystem: terraform
  directory: "/"
  schedule:
    interval: daily
  open-pull-requests-limit: 10

so that will make dependabot generate updates for public provider versions in when they are released?

that’s amazing

I do think dependabot added support for managing provider versions recently… Renovate has done that for a while. But personally I only use it for bumping the version in a module’s source url git ref

(fwiw, it also works for the source url in terragrunt.hcl files)

wow, that’s nifty

Have you considered turning your lambda into a Docker container and deploying the lambda as a single container ?

Then you dont have to use a null provider or zip anything up

Have you considered turning your lambda into a Docker container and deploying the lambda as a single container ? negative, not have yet as the cold start is still a prob for us. Maybe in the near future

use this : https://github.com/cloudposse/terraform-aws-ecs-web-app

Thank you very much @jose.amengual

I mentioned it happens intermittently

what version of the aws provider are you using ?

try terraform init -upgrade

the argument override_policy_documents does exist on that data source.

see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document#override_policy_documents

I’m using:

terraform version
Terraform v0.13.5
+ provider registry.terraform.io/hashicorp/archive v2.0.0
+ provider registry.terraform.io/hashicorp/aws v2.70.0
+ provider registry.terraform.io/hashicorp/local v1.4.0
+ provider registry.terraform.io/hashicorp/null v2.1.2
+ provider registry.terraform.io/hashicorp/template v2.2.0

aws v2.70.0 is old, can you upgrade your provider to 3.x ?

not now, I have big infrastructure

have in future plans

the module feels like a bit of a mess. aws trying to do terraform, but pulling from concepts only valid in cdk and cfn

oh lord, and the providers are all hard-coded

the more i poke at it, the more frightened i get

https://github.com/aws-ia/terraform-aws-control_tower_account_factory

Yeah…. no. They can’t even name their module according to the standards. Weak.

Typical aws fauxpen-source… https://github.com/aws-ia/terraform-aws-control_tower_account_factory/blob/main/CONTRIBUTING.md

Thank you for your interest in contributing to the AWS Control Tower Account Factory for Terraform.

At this time, we are not accepting contributions. If contributions are accepted in the future, the AWS Control Tower Account Factory for Terraform is released under the Apache license and any code submitted will be released under that license.

“some big customer asked us enough that we devoted a sprint to this. we don’t plan to touch it again”

My thoughts exactly!

I have migrated org running legacy LZ into a new Org with CT .

We are running without any LZ/CT currently

if ur migrating the accounts of the same org to CT - I think AWS has automation in place from the CT console ?

if u run CT checks it will tell you if it can convert the org or not

Would probably advise against CT. There are many caveats.. for eg. CT logs being expired after a year