#terraform (2021-04)

you will need eks worker to use custom AMI

yea that part is a given. I was talking about if there is support for that in the module, since bottle-rocket is a different in many ways to amazon-eks-node

you may want to give this module a try, https://registry.terraform.io/modules/cloudposse/eks-workers/aws/latest

along with the bottlerocket ami or the ami you build with packer based on bottlerocket ami

we are still waiting for a response from the contributor

if you want this sooner you can create a PR with the same code

@jose.amengual https://github.com/cloudposse/terraform-aws-s3-bucket/pull/82

please look at the tests @Piotr Perzyna

@jose.amengual Could you try now?

merged

Thank you!

np

Bite the bullet… upgrade your provider. I did it last week, it wasn’t too painful. 2.67.0 is way out of date and you’ll miss out on new features and resources.

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-3-upgrade is really helpful.

Check your plan outputs with a fine toothed comb… I successfully deregistered all instances on an ALB because I missed the warning here: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group

Did you figure this out?

I’m not affiliated with CloudPosse, just a member here. Around 1.5y of exp with Terraform and CloudPosse modules, just shy of 10y of experience.

Do you have a code snippet you’re looking to improve @hrishidkakkad

You might want to look at Aurora Serverless for this. It may come out cheaper and less complicated, because you’d still have separation between projects but no dedicated instances.

@Igor Rodionov @joshmyers @Maxim Mironenko (Cloud Posse) maybe you can help to find answer sorry guys for direct mentioning

I will send you few links today

thank you so did like this:

1. created needed IAM permission using kops kops edit cluster
2. created EFS with https://github.com/cloudposse/terraform-aws-efs thank you guy for this module, it is wonderful !
3. deployed efs driver with helm

use modules ?

you mean module per subnet?

when I use TF to do VPC/Subnets I create all that in a separate component/module and then I search/find the subnet to use in each app/service

the search/find = could be remote state share, data lookup or using outputs

just to confirm, so you’d have something like subnets/app_mysql/, subnets/app_cache/, etc. file structure?

not really

I have a vpc/network project

and the app use those resources on their repos

I do not mix app and networking

A vpc and subnet is a lower level than the app so I separate those

it is very strange to “delete” a subnet when you delete an app unless you have a vpc per app

ohh ok, thanks, that clears it up a bit

so you’d use something like

module "vpc" {
  source = "github.com/someuser/vpc"
}

and use it’s outputs within the app project?

so if you look at it from he point of view of a coldstart on a new aws account the first thing you get is a vpc because without it you can’t do anything but usually you create your own because the default is not on your CIDR range and so so set up the fundation for your app to work and after that you do not modify the conectivity for the app to work. Much like on a company there is a network department

I use this https://github.com/cloudposse/terraform-aws-dynamic-subnets

similar to the example

That makes sense to me, so you define all of the networking in a separate project. but then lets say you have a subnet for databases and another for an app, Do you just hardcode the IPs in the app’s project so it knows where the db servers are? There won’t be that many and I can just reference them in the networking project, so maybe it makes sense to just copy paste from one project to another? I’m talking about the subnet “10.8.0.0/24” strings.

Or would your networking project generate an “output file” that your app project could pick up and use variables instead?

Sorry if I’m not being clear, this is all new and a little confusing:) wondering what works better in practice

you can use outputs for sure

if you are deploying app and RDS in a separate subnet then you will need outputs for each

usually you create many private/public subnets per VPC and app and rds use the same subnets but secured by a security group

and sometimes you will have a DMZ were you will allow traffic base on ACLs but all that could be in this networking project

most of the time, you do it once and once connectivity is good you do not touch it again

got it, thank you!

done https://github.com/cloudposse/terraform-aws-ecs-container-definition/pull/93

Yep, pass context or part of context that will name your lambda

Strange, you’d think part of the business model is selling to potential customers

You can DM me

just give me your email

Sending over now. Thanks!

Apologies for possibly not understanding. You’re saying that when you manually fire off an event into SNS, it successfully fires a lambda that sends a message to your slack DM/channel/whatever?

Yes but when I make any changes to RDS nothing happens. The gist above is configured to send events to SNS but it doesn’t seem to be doing it

I’ll try to get around to this sometime tonight if nobody else can pitch in and you’re still having issues. On baby duty right now

You can just use a module block with a source reference pointing to the repo (including the git ref). On init, terraform will pull down the repo. You can reference the files from the .terraform directory

Ah, good call, thanks!

You don’t get a tfvar that way, exactly, but you can use the file and yamldecode functions to pull the value (s) into a local

Hmm, can’t use variables in a source declaration though. Might have to re-think this. I’m calling a module and passing it variables for the repo to check out (have multiple repos depending on the config).

If the git repo or the ref are variables, then no this doesn’t work, but probably nothing else will either. You’re looking at some kind of wrapper, at that that point

Yeah, my initial solution was calling a script with the local-exec provisioner that parsed the vars to checkout the repo.

I think Loren might be saying you can wrap your TF code. maybe have a script that creates the tf file with the module source. Run that script before running terrform plan/apply/etc

Yeah, I understand. Trying to see if I can avoid that.

roger that!

Sidebar question: how many repos are you working with? I know it might not be scalable but what about pulling down all the repos and then using a variable to reference the one for the current configuration?

Right, exactly, cdktf or terragrunt or other external tooling and generating the tf might be preferable to trying to hack the workflow from within a terraform config

Yeah that might work, pulling all the repos. The way I have it set up now is each repo has a google cloud pipeline in it. One module/pipeline/repo that calls another module that handles parsing which repo(s) to check out. I think for now I can do something along the lines of just grabbing all of them every time, its only 6 repos atm.

Or take the opportunity to rethink the larger workflow, really take advantage of the declarative nature of terraform

Otherwise wrapper script or makefile it is

Thanks for the help

Are you sure you even need to clone the repo? can you just use the raw url?

That’s the pattern we use in https://github.com/cloudposse/terraform-yaml-config

ah… never thought of that. No I don’t need the repo, I just need the file.

Thanks @Erik Osterman (Cloud Posse)! Saved me a bunch of time probably dealing with a wrapper or writing a custom module.

1. order of operations - i.e. do you have all the information needed when creating the ASG to also attach the LB at that time?
2. separation of responsibility - i.e. are there different teams/configs responsible for or maintaining the ASG vs the LB?

yes i can

i have this weird issue at present

whereby on first tf execution the attachment works fine

then if i re-execute tf again (with no changes) it wants to remove it

see https://gist.github.com/swade1987/33780145d1052fadc05a0331e4ef5c30#file-ingress-node-group-L7

i can’t work out why :point_up: is happening and i have a suspicion its the aws_autoscaling_attachment

are you passing an empty list here: https://gist.github.com/swade1987/33780145d1052fadc05a0331e4ef5c30#file-asg-calling-L15

if so, change it to null

it’s like security group inline rules vs rule attachments. only one should be used for a given SG

i may be misunderstanding, because i don’t see an aws_autoscaling_group resource in the gist

i see the attachment, not the ASG definition… but going back to the original question, use only one option for attaching the LB to ASG… either the attachment resource OR the ASG resource

if you pass an empty list to an attribute, that typically implies exclusive control of the attribute… so load_balancers = [] means remove all LBs. to get the default behavior of ignoring the attribute, use load_balancers = null

interesting

although the issue seems to be when i am specifying the id

see https://gist.github.com/swade1987/33780145d1052fadc05a0331e4ef5c30#file-ingress-node-group-L7 and then the terraform screenshot

when scaling up the ingress node count in the ASG its removing the load balancer from the ASG configuration which makes no sense to me as nothing else has changed.

Capturing some first impressions as I ramp in hope of pulling together a doc for future rampers.

unfortunately, no such document exists yet

we’re working on some tutorials - i think the next one will be on creating a vpc and EKS cluster

TBH, we probably won’t tackle multi-account tutorial for a while since it’s a master class in terraform and aws

cc: @Matt Gowie

Maybe I should start with something smaller, like a doc on creating a small internal stack rather than a full blown master account.

Is there a baseline or example template I should start from when building my first stack?

I noticed the terraform-yaml-stack-config repo, which looks promising. I’ll start there.

Hey Marc — This is definitely on our roadmap but the full-blown “Here is how you provision many accounts + all the infra for those accounts” with stacks is still a bit of a ways out. That’s what stacks are built for, but it’s a TON of information to convey honestly. We’re trying to do it piece by piece and we’ll eventually work up to that topic, but it’s advanced and similar to what Erik mentioned: It’s a masterclass unto it self.

That said — We are putting together more AWS + Stack example tutorials soon and they should be launching within the coming weeks.

There is no example stack template that is a perfect example, but you can look here for a simpler example: https://github.com/cloudposse/terraform-yaml-stack-config#examples

Thanks Matt! That’s perfect for me. Cheers –

@Marcin Brański might have something to share as well, having just gone through it

I started for my client from ground up with atmos recently (less than 2 weeks ago). I don’t think I’m experienced enough to share tips but if I will have time (which currently I don’t) and client will agree then I can share snippets or whole configuration that we did.

I, personally, used tutorial, example from atmos, read variant and it kinda clicked.

Thanks Marcin – understand about your time. Happy to curate anything you can share. I enjoy that kind of work.

@Marcin Brański it would definitely help, I’m at the same point as @marc but it didn’t click to me yet. Maybe it’s because I’ve been using TF, YAML and some python wrappers in a complete different way, but although I followed the tutorials I can’t figure out how to, for example, create just a pair of accounts (i.e. master - prod) and an VPC/EKS cluster on prod one

Got a VPC running with a master account last night using atmos and geodesic, so I should be good to go. Started a supplemental “tutorial.md” which I’ll submit as a PR to the terraform-yaml-stack-config. Thanks all, for being so welcoming! Cheers –

Yeah! Good for you! Have you already thought of CICD?

I’ve been considering a few options on that side. 1.) Terraform cloud. 2.) JenkinsX.

https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation spacelift

@marc slayton I believe a number of us here (and the wider community is coming to the same conclusion) would suggest against Terraform Cloud. Their pricing model past their “Team” tier is highway robbery and there are better solutions out there. Spacelift as Erik pointed out is once of them.

I’m jumping in to say I’m a little confused with just trying to use the yaml stck config. I don’t need the full atmos stuff, just want to use the yaml configuration for setting backend and components and i’m not having much luck.

Is there anyother example of using terraform-yaml-stack-config repo and how the command with variant or cli is being actually processed to set backend?

I have a go build script doing some of this but I’d love to avoid rework on this if I’m just misunderstanding how to use the yaml stack config option.

Basically what I have right now is each folder loading the stack designated and the resulting output for each “root” module componet looks like this

module "vpc" {
  source                                          = "cloudposse/vpc/aws"
  version                                         = "0.18.1"
  enable_default_security_group_with_custom_rules = module.yaml_config.map_configs.components.terraform.vpc.vars.enable_default_security_group_with_custom_rules
  cidr_block                                      = module.yaml_config.map_configs.components.terraform.vpc.vars.cidr_block
  instance_tenancy                                = module.yaml_config.map_configs.components.terraform.vpc.vars.instance_tenancy

This doesn’t seem to match the more simple output I’ve observed on the other projects.

Can I ask a question here about Sandals?

@marc slayton Nothing stopping you from asking a question! I would suggest starting a new thread if it’s about a new topic.

LOL – thanks for the reply. I was actually just figuring out how to use the Foqal plugin. It looked pretty interesting. Seems like it might be useful in capturing general trends about the types of questions being asked, so you can prioritize certain types of documentation.

Ah I don’t know much about that plugin — I haven’t found the responses useful, but if you’re finding it useful then more power to ya!

@vlad knows more about foqal

that error is from Terraform, not the module

maybe is related to this : https://github.com/hashicorp/terraform-provider-aws/issues/1137

hey @jose.amengual Thanks for responding. I was checking the module at it seems under aws_db_instance default its specifying the identifier as module.this.id .. Sorry, I could heading the wrong path…

module.this.id is the label module we use to name resources so you might want to check that did you pass for name, attributes, stage, environment etc

Lets say I have verified the data mapping between the module and values. Would you be to point me to any other potential mishaps? Or is it that there is missing gap between module expectation and my values?

I will have to look at your plan

but we use this module extensively and we have no issues

Thats my thought also, since its not a major reported issue. Must be something within the configurations.

seems like hyphens may have been the cause. I had removed them from the name, and its throwing another issue. But seems like passing the DBName portion.

This looks like warning related to Atmos

Are you talking about this Atmos? https://github.com/simplygenius/atmos

It doesn’t look like the authors are in this Slack, so you are possibly asking in the wrong place for help?

wow, had never seen that

https://github.com/cloudposse/atmos

this is the one

Yep, I was misinterpreting the warning. Thanks for setting me straight!

For what its worth, Hashicorp said they are removing that warning and they’ll allow you to have whatever you want declared in tfvars now.

Should for sure be in v0.15, I can’t recall if they added it to the recent 0.14 patches

0.14.9 still shows that. I just today upgraded to 0.14.10 so not sure about that but definitely fmt has changed

oh what changed in fmt, I missed that note

In the original config, do an init. then change the config and hit init again

terarform will prompt you that it detected the backend config has changed and ask if it should copy to the new location

it does not remove the old location!

i can then safely remove the old one manaually?

yup

Hm, I dont see why not - the backend is supposed to be separate from the rest of the terraform config.

terraform {
  backend "s3" {
    bucket = "mybucket"
    key    = "path/to/my/key"
    region = "us-east-1"
    ## you'd have to supply your AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY either here or during terraform init
  }
}

then you’d provisiong your azure resources in your standard main.tf.. i’m literally just pulling this from the provider docs.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=2.46.0"
    }
  }
}

# Configure the Microsoft Azure Provider
provider "azurerm" {
  features {}
}

# Create a resource group
resource "azurerm_resource_group" "example" {
  name     = "example-resources"
  location = "West Europe"
}

so your ‘backend’ could be anything.. aws, terraform cloud, terraform enterprise, consul

ok - i was thinking due to using azurerm provider you’d be stuck to using azure blob for backend store and not aws s3 https://www.terraform.io/docs/language/settings/backends/azurerm.html

converting a list to set remove duplicate, for_each takes as input a set or a map . so I don’t see a reason to convert the set back to a list.

Converting back to a list might be necessary for zipmap.

zipmap build a map out of your list, the index is based on a map key and not on a list id ..

Thanks for the explanation. I didn’t understand why it worked.

DevOps Advocate with env0 here. We are currently working on the Azure DevOps integration, including CD and PR Plans. You can use ADO now, but it is just a simple repo hook. We’re just not quite there with the CD / PR plan hooks just yet. I’ve reached out to our CTO to try and get a code-commit date for you.

If you want to DM me your contact info, or just let me know DM’s here are fine, I can keep you in the loop as we make progress.

I built in some PR comment additions to the plan so I’m trying, just figuring better lifecycle in the tool itself would be good. No rush, just exploring options as haven’t caught up on recent updates. thanks for keeping me posted

To clarify, do you mean Azure DevOps as a VCS provider?

I believe they were. We can support it today just as a basic VCS provider. It’s just the extra webhook stuff like PR Plan comments and CD that we don’t have yet. Do y’all have that yet for ADO @marcinw?

Nope.

Sounds like we both have a solid feature request on the board

At env0, it’s not about the deployment job and state to us. It’s about the entire lifecycle of the environment from deploy to destroy. We get compared to a CI/CD tool a lot, but we don’t do CI. We can do CD… But it’s really the day 2+ operational stuff that we do that makes the value come out. Setting TTL’s, environment scheduling, RBAC, Policy enforcement, cost monitoring. All of us TACoS really focus on the whole IaC lifecycle, not just running a pipeline and shoving the state somewhere.

Here are some issues to deal with:

1. planfile is sensitive (secrets likely stored within)
2. order of applies matter, should always apply oldest plan first
3. one-cancels-all: once the planfile for a given root module is applied, any planfiles prepared before that need to be discarded - so if you’re treating planfiles as artifacts, it’s complicated
4. generally want to approve before apply and many systems don’t do this well (e.g. github actions) - and I’m not talking about atlantis style chatops, but a bonafide approval mechanism
5. policy enforcement - where you want a policy that when one project changes, you want another project to trigger

https://docs.spacelift.io/

Hey Marc — does atmos terraform deploy not do what you’re looking for?

Yes, this did the trick! Sorry for the newbie question. Must have missed it in the docs. Cheers –

Well to your credit, the docs on initializing remote state via tfstate-backend isn’t in the docs yet and that will be coming very shortly (PR should be up in the coming couple day).

But glad that did the trick. Let us know if you have any other questions.

Thanks, Matt! – So far, so good. :0)

We’ve been looking at integrating it with Spacelift. It looks pretty decent and the CLI does not seem to leak anything to the API server. The obvious limitation is that usage-based cost estimation is only as good as the input you provide, but flat fees are generally well recognized, at least for AWS, and broken into components.

I’d suggest using it on static artifacts (state and plan) rather than have the wrapper run Terraform for you, because it feels messy and likely duplicates your work.

That said, I’ve mostly looked at it from the integration perspective - inputs, outputs, security and API availability. The CLI feels a bit awkward and inconsistent, especially if you’re outputting machine-readable JSON, but it’s generally not a blocker - you should be able to do what you want after some trial and error.

The team behind it is super nice and very responsive, too.

Thank you Marcin, that is very helpful. Would love to also hear of users using it on a regular basis. We are a vendor, like you, (not competing) and are thinking of integrating Infracost.

Sure thing. Always happy to talk shop and compare notes ;)

Quite possibly

One thing we have encountered is that strict typing can make two otherwise compatible modules totally incompatible due to types. We encountered this with our null label module and have subsequently changed our context object to any.

Still this default function is welcome

I been thinking to reduce number of variables for any module using this via one variable object with optional, gonna test this and see pros cons

I found defaults() hard to use and reason about, when I tried the experiment in 0.14… now, the optional() marker for complex variable objects, that worked perfectly and was very easy

Maybe they’ve fixed defaults though, it was a few months ago and very much an experiment

what’s awkward about the terraform console?

the other thing is you can read your statefile

I did manage to get this going, but not from within atmos. In the end, I rendered the objects I needed using the ‘terraform output’ command. Looks like my multi-account build is working now. I’ll submit a PR with the changes to terraform-aws-components, and also some notes that may help others.

Thanks for the advice, Alex! Cheers –

1. create a sub-module from all the resources

1. Use for_each on the submodule and give it the set of files

right, so you are saying a module is async, interesting.

@Andriy Knysh (Cloud Posse) This looks to have decreased the run time by 30 to 40 percent… Thanks for the pro tip. Still testing but the first result looks good.

ok, its really hard to tell and I think. iam reading it wrong

but maybe 1 minute reduction 10 mins to 9

oh, well. Was worth a try

try just keys(local.myconfig)?

Hello Loren, Thank you, that’s lead me to almost get what I needed. Looks like my initial structure is actually nested. So

}
 myconfig = {
   "/ErrorPages"   = "mys3bucket"
   "/client-assets" = "mys3bucket"
  }
}

Using

flatten([for k,v in local.myconfig :distinct(values(v))])

Gets me to

[
  "mys3bucket",
]

but every time I try and pull this together to something like

{for k,v in local.mys3_configs : flatten(distinct(values(v))) => tolist(keys(v))}

I crash and burn

all i know is that based on what you’ve exposed here and what you said you wanted as a result, keys() is the answer. can’t offer more without seeing the whole data structure

totally understand Loren, sorry, There isn’t really any more to the structure but let me reframe the data above to be less confusing.

okay, I think the confusion is the naming in my example. So let me frame it. Sorry about this

> local.mys3_configs
{
  "api" = {
    "/ErrorPages" = "assets.bucket.dev"
    "/client-assets" = "assets.bucket.dev"
  }
}

keys(local.mys3_configs)
[
  "api",
]

What I need to do is get a distinct list of values aka assets.bucket.dev and use that value to make a new list which will contain all the keys local.mys3_configs.api

This gets me close to what I want.
{for k,v in local.mys3_configs : "randmonword" => { "another_randmon_word" = tolist(keys(v))}}
{
  "randmonword" = {
    "another_randmon_word" = [
      "/ErrorPages",
      "/client-assets",
    ]
  }
}

I’m falling down when I try and make this type of structure flatten(distinct(values(v))) represents the dynamic but distinct list of values covered by values(v) aka assets.bucket.dev tolist(keys(v)) represents the dynamic list of keys I want to add in to a single list. aka [ "/ErrorPages", "/client-assets"]

{for k,v in local.mys3_configs : flatten(distinct(values(v))) => tolist(keys(v))}}
{
  "assets.bucket.dev" = [
      "/ErrorPages",
      "/client-assets",
    ]
  }
}

Still as clear as mud

playing around a little more has got me to:

{for k,v in local.mys3_configs : element(flatten(distinct(values(v))), 0) => tolist(keys(v))}
{
  "assets.bucket.dev" = [
    "/ErrorPages",
    "/client-assets",
  ]
}

But while this gives me what I want, is it correct? or have I just fluked it and a different approach would be safer?

can you just use the data source https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ram_resource_share

ooof, I tried to search for this and wasnt able to find it, that should work

hmm, another AWS account has sent me a RAM invite and I can see it pending, however, the following isn’t working -

data "aws_ram_resource_share" "example" {
  name           = "aws-test-VPN"
  resource_owner = "OTHER-ACCOUNTS"
}

I tried using SELF as well, but I get the following error:

Error: No matching resource found: %!s(<nil>)

  on main.tf line 13, in data "aws_ram_resource_share" "example":
  13: data "aws_ram_resource_share" "example" {

it may only work after you accept the request

but not sure

does the cli work

yeah it seems like its only for after accepting

let me try

nope

{
    "resourceShares": []
}

gotcha

There are a ton of bugs in the terraform ram share accepter, you’re almost better off dealing with it manually

Yes and no. You can deal with it but maybe not the way you would like to. Just dealt with one example today. I used for each in on a module and tried to compute it’s output in locals. So terraform errored out on me. I fixed it with iterating over module.xxx.output in resource instead of iterating over computed local .

Hmm I’m not sure i understand and if that method would apply to my circumstance. My immediate example involves creating a private Route53 zone and sending that zone into a module which will create a DNS entry if the Zone id exits using count. https://github.com/cloudposse/terraform-aws-elasticache-redis/blob/83bd076d932b3bac8203fe9b3a70cac43d8d36db/main.tf#L169

Terraform doesnt know if it will exsist until apply. Usually i deal with this with a feature flag. ie route_53_enabled = true . In this case its not my module its Cloude Posse’s so wondering if there is a better way for conditional resource creation.

ultimately, the for_each key must depend only on user inputs and not on any computed values derived from resources in the same state

@Tom Dugan you can use -target with plan/apply to workaround the problem. looking at that module, the way it is depending on var.zone_id in the enabled variable, and how enabled is used in count, there is no other workaround when you create the zone in the same tfstate

you can certainly move them into different tfstates, and manage them separately, and that will work also

(caveat: i haven’t used this module, so there may be some detail i am unaware of that would support your use case. i’ll defer to any of the cloudposse folks if they chime in)

ah yeah the -target option i was hoping to avoid.

Pretty much this problem is creeping up when we are testing a module which calls that module. To test our module we create the Private zone during the test. In the real call the zone is created in a different state.

I think I will just extract the route53 logic to bypass this issue. Thanks for the insight!

i would suggest, instead of depending on var.zone_id in the enabled variable, the module should accept a var.create_dns_record boolean. i’m not sure how to do that in a backwards-compatible manner though, so not sure it would be accepted

That’s inline with my typical design pattern so I would agree with that approach. The backward compatibility problem is a good note, I’m not sure there is a clean way to solve that,

I’m sure there’s a better way but I do this. for D in */; do cd “${D}” && tflint && cd ..; done

That will only do one level deep though

pre-commit — https://github.com/antonbabenko/pre-commit-terraform

are there any examples of tflint.hcl as i would like to enable a few rules from https://github.com/terraform-linters/tflint/tree/master/docs/rules

Figured it out

Pipe to jq and create a map

sidenote: If there’s a terraform github issue you were able to find about this data-source incompatibility, I’ll happily give it an upvote.

i believe it is more that it’s a whole different api, rather than an incompatibility, exactly… the data source aws_ram_resource_share is based on get-resource-share, but the invitation is returned by get-resource-share-invitations

however, the share accepter resource takes the actual resource_share_arn, which IS returned by aws_ram_resource_share, and then the share accepter resource looks up the invite arn for you using that. the share accepter does not accept the invite arn

yeah that makes sense

i believe you can get the share arn from the invite though, so you’re approach will work, but you’ll need to adjust the query

yeah thats what i’m trying to figure out but jmespath gets ugly, fast

OR you can use a multi-provider approach… use the aws_ram_resource_share data source with a provider that has permissions to read the ram share from the owner account

nice! i like that SCP… now, make sure the trust policy for the Iac role is locked down so only your CI system can assume it… and/or have an explicit deny on all other policies to be able to AssumeRole the Iac role

that’s also the first good use case i’ve seen of paths on iam entities… interesting…

Yeah, Rajeev’s quite creative

We used to use paths for IAM roles, but then some AWS (don’t remember which) service barfed when it wasn’t /

sounds about right @mrwacky

It’s DMS from memory (or at least, it’s also DMS )

To clarify: I can’t find nor imagine an instance where I’d want a different regex than the default.

https://github.com/cloudposse/terraform-null-label/blob/master/main.tf row 48

Ah! That’s a different question. I can’t think of an example right now.

The reason we have this is to support the use-case where a user does not want us to normalize/strip out characters. We (cloudposse) don’t have any such use-case since we’re strict about how name things.

https://github.com/1Password/terraform-provider-onepassword

https://registry.terraform.io/providers/1Password/onepassword/latest

@eric you called it, they released right before office hours

Every week

It doesn’t seem like a significant version. Will test it tomorrow.

it will get auto published today in our packages distribution

Try running terraform state list to get a list of all the modules in your project. If there are a lot of resources you might want to try grepping for module.secondary:

terraform state list | grep module.secondary

That might help you narrow in on the path you need to use to get the value you are looking for.

also note that if you are trying to use a value from a module, the module must publish that value as an output. Check the source for the module to confirm all the things being published. One easy way to do this (especially if you don’t have access to the source) is to print the entire module as output in your project’s outputs.tf:

output "secondary" {
  value = module.secondary
}

Then run terraform refresh to see all the outputs from the module. if the value you are trying to get to is not in there, you can’t get it unless you update the module to publish the value.

thanks, there’s an output, i didn’t know terraform state list (well i did once run it)

i’ll give that a go, cheers

np!

Hey @Matthew Tovbin bring this up in #pr-reviews and somebody will get to it. That module gets left behind a little bit if I remember correctly, so us maintainers just need a bit of a nudge and #pr-reviews is the best place for that.

Thanks!

… moving this into #terragrunt didn’t realize dedicated channel.

This suggests you may need to add more to your data call, excerpt:

jsondecode(data.aws_secretsmanager_secret_version.example.secret_string)["key1"]

hummmmm… interesting… will try that out just now HS ! appreciated.

Error: provider.aws: aws_db_instance: : “password”: required field is not set

it still does not get a value it seems …

but thanks anyway … this is tripping me out !

perhaps the data object isn’t getting right secret? ie

data "aws_secretsmanager_secret_version" "example" {
  secret_id = data.aws_secretsmanager_secret.example.id
}

anyways, good luck!

yeah … I am going on that direction now … take care my man !!!

I was able to sort it out ! the Admin folks are blocking the password field to be set … I went ahead and HARDCODED a string and it is still NOT reading … ALL SORTED !!! MOSTLY APPRECIATED FOR YOUR SUPPORT !!!

That’s a pretty low level question — I’m not sure if anyone will really know off the top of their heads. I’d ask in the HashiCorp Terraform discourse board: https://discuss.hashicorp.com/

Will try to check on the discourse board too

I’m working on a fun open-source tool for Terraform DevOps that should be useful

there are more and more projects operating on the terraform state files. i’d look at how they are doing it

https://github.com/camptocamp/terraboard

no, YAML and JSON have their limitations, but they are supported using the standard library for Terraform. The benefit of TOML is much smaller than the cost of added complexity to your process.

yea, IMO we don’t need another config format.

@Brij S when you say “different for some of the account ids” things get kind of odd. yeah, you could use a for_each on a list of account ids, but if you want to vary the IAM role, you’ll need to change the data to a map. that way each item in the for_each loop will have an id and and IAM role associated with it.

Another option would be creating a module for each IAM role, that way you can associate the IDs with the module that has the IAM role they need.

there are a few ways you can approach this. just need to figure out which one is best for your use.

Maybe ping mitchelh on twitter? Seems like it should be an easy one, and it seems like he’s able to get an eyeball on useability things like that more than comments on stale issues :/

Not a bad idea. I’ll do so.

Always feel like an ass doing that type of thing but went ahead and did it anyway.

2 years seems like a reasonable period to wait before escalating to a more drastic measure

Yeah agreed. It’s also the type of thing that likely burns people constantly, but the vast majority of module consumers aren’t going to actually look up this issue and give it a .

I believe this is a limitation of HCL2 not of Terraform, per se

(you might get a response which asks you to re-raise in a different repo)

Like with any PR It can be few hours up to infinity. Depends on how complex the PR is, if best practices are there, tests if any and if functionality added makes sense.

What do you want to implement? If you need guidance let me know

@Steve Wade (swade1987) we’re pretty good about providing guidance so if you put something up then it’ll likely get eyes on it and you’ll get direction if something needs to change. If nobody responds quickly then feel free to ping in #pr-reviews or ping me directly.

i realised that what i was wanting to do doesn’t make sense for the module

so we just forked what we needed

Doesn’t look like it @Brandon Metcalf — Feel free to submit a PR and post it here or #pr-reviews and I’ll give it a review.

yes but they are separate comparison expressions when you set it up like that:

cookie_behavior = (local.myvalue == "none" || local.myvalue == "whitelist" || local.myvalue == "all") ? local.myvalue : null

or you can use a list with contains():

cookie_behavior = contains(["none", "whitelist", "all"], local.myvalue) ? local.myvalue : null

Thank you Loren,

Does https://github.com/cloudposse/terraform-aws-elasticsearch do what you want?

if anybody has the same issue here. my problem was

resource "aws_cloudfront_origin_request_policy" "example" {
  name    = "example-policy"
  comment = "example comment"
  cookies_config {
    cookie_behavior = "none"
     cookies {
       items = []
     }
  }

leaving the cookies {} section in place when none is set caused the error. Same with either of headers_config & query_strings

Now to find a way to use dynamic to exclude those sections completely if they are set to none.

I did that using v0.30.0 version of that module. Add such options beside other required parameters specified in docs.
advanced_security_options_enabled = true
advanced_security_options_internal_user_database_enabled = true
advanced_security_options_master_user_name = “master”
advanced_security_options_master_user_password = “pass”

thank you!

Look at our strategy for CIS: https://github.com/cloudposse/terraform-aws-config/tree/master/modules/cis-1-2-rules

Here is my similar bash hackery around this — https://github.com/Gowiem/DotFiles/blob/master/terraform/functions.zsh#L70

Though I honestly don’t use that much anymore.

nice.

Probably a good question for #aws or read through AWS docs on the subject.

What help do you need @Kim?

Ask some specific questions and I’m sure folks can help out. Definitely be sure to check the examples/complete directory if need direction on how to use it generally.

In fact, i need to know how to start using the project stp by stp

@Kim I’d check out some tutorials from HashiCorp Learn — The modules track might help https://learn.hashicorp.com/collections/terraform/modules

Going step by step through how to execute Terraform and use a module / talk through the AWS process is a bit much to go into in a forum like this unfortunately, so doing some research on your own and then circling back with specific questions like “Hey why am I getting this error” will allow me or others to help you.

@O K the AWS ElasticSearch service is known to be slow / crappy. I’ve had plenty of issues with it taking hour+ deploy times and terraform timing out. I would guess that your issue is due to AWS ElasticSearch and not due to the module.

I’d suggest trying again tomorrow

Thank you, man

changed instance from t2.small -> t2.medium and it deployed in 15 min! probably more memory is needed or AWS knows the stuff

AWS is weak: They allow you to use t2.small instances, but they basically don’t work. Like I’ve had issues like you just ran into and just general day to day memory failures due to trying to use too small of instances. It’s like AWS wants to show off that you can keep your ES clusters cheap, but in reality…. that shit don’t work.

ES is extremely memory hungry so I’d rather them just come out and say: Hey we don’t allow you to use cheap instances because ES is just way too memory intensive. You have to use these expensive boxes over here.

BTW, what do you think about https://aws.amazon.com/blogs/opensource/introducing-opensearch/

Not sure if I have a fully formed opinion. But I hope that causes them to put more effort into ES as a managed service as I’ve been pretty unhappy with it so far. I’ve lost clusters and data because clusters can just get into a failed state and the AWS documentation just says “Hey contact support to fix”. That’s BS in my mind. So if they start addressing issues like that then I’ll be happier.

Then again, if I never had to use ElasticSearch ever again, I’d be happy.

You can specify a folder path in the key param when specifying the s3 state backend

Thanks!

Yup, this is the best way

send a PR, but there should be similar

Ya, they should be similar.

The NLB module was a contribution from some other organization

Submit the PR and we’ll review in #pr-reviews

Is there a non-saas version?

Not today. The API calls are made directly from our own AWS account. What’s your thinking?

I was going to ask the same thing. Looks pretty cool

Alex - there’s driftctl but need to understand what you determine as drift. In TF, you can use meta argument to intentionally ignore changes . Also, resources are often created out of events (e.g. lambda function creates an s3). What’s your definition of config drift?

in this case I mean that I need to detect any resources in the AWS account which were not created by the terraform code @Charles Kim

driftctl seems good, thanks! @Charles Kim

Try out driftctl. It has some issues but the team is rather responsive. disclosure: i work at Cloudrail and we’re focusing on solving security issues resulting from config drift. Here’s a sample: https://indeni.com/blog/identifying-iam-configuration-drift/

@Charles Kim did you use driftctl with Atlantis

This sounds really cool. Will you be able to query the entire S3 backend, or just one state file at a time?

@Igor You can import sync your S3 backend locally, and then run a query on all Tfstate files at the same time, it’s been really helpful for me

Let me know if I can add you! Hopefully I can hear thoughts from you if you get a chance

I don’t have any immediate use cases for this, but I’ll keep it in mind. Wishing you success with the launch.

if something is missing in the module that this resource supports aws_docdb_cluster, you can open an issue or a PR

ok, thanks so much!

TLS is possible to enable/disable using https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/docdb_cluster_parameter_group

which in the module is this var

<https://github.com/cloudposse/terraform-aws-documentdb-cluster/blob/master/variables.tf#L84>

ok, I try with the variable “cluster_parameters”

Sorry one question, if I use for example :

variable "parameters" {
  type =list(object({
    apply_method = string
    name         = string
    value        = string
  }))
  default     = [{"apply_method"="true","name"="tls","value"="disabled"}]
  description = "List of parameters for documentdb"
} 

How to I could use in my main.tf? I get values with the method for_each ?

or

module "documentdb_cluster" {
  source  = "......"
  version = x.x.x

  cluster_size                    = var.cluster_size
  
  cluster_parameters = var.parameters

the module does for_each on the list of objects

ok,thanks!

PR: https://github.com/tfsec/tfsec/pull/692 Repository: https://github.com/tfsec/tfsec

If you’re into cloud security, would be happy to connect with you on Twitter https://twitter.com/mazen160 :)

Looks amazing!

Hey Amit, check out the variable here: https://github.com/cloudposse/terraform-aws-elasticsearch#input_create_iam_service_linked_role

That enables / disables creation of the role.

The elasticsearch_user resource that is created can be found here: https://github.com/cloudposse/terraform-aws-elasticsearch/blob/master/main.tf#L68

There is logic which you could use to disable that resource if you want.

That value should be bool — Are you trying to pass it as a string?

If it’s still trying to create that role then I’d look at the logic behind that variable and try to trace it back. If there is a bug, either open an issue or put up a PR and then post in #pr-reviews.

thanks, I already did as bool, I will dig deeper

Seems like there’s an issue open for the issue: https://github.com/terraform-providers/terraform-provider-aws/issues/5218

Actually, I misread, but does give some insight into the parameter.

For reference, the code is here: https://github.com/cloudposse/terraform-aws-elasticsearch/blob/0541281379ae1b916fe4e19c884336fd10a328f5/main.tf#L62

Definitely could see how it’d fail as a string even if specified as "false" (non-empty string is truthy, I assume). Looks like using = false should work. Good luck!

Does

resource "google_secret_manager_secret_version" "main" {
  count       = length(var.secret_data)
  secret      = google_secret_manager_secret.main.id
  secret_data = var.secret_data[count.index].secret_data
  enabled     = var.secret_data[count.index].enabled
  depends_on = [google_secret_manager_secret_version.main]
}

Not do what you want?

no you can’t use depends_on on the same resource block to order the creation… you can order them, but only with separate resource blocks. another “kinda” option is to use -parallelism=1

I get a cycle error when I use depends_on on the same resource block. I’ll try to use parallelism=1 to see if that maintains order of the list.

I just tested with parellism=1 . It doesn’t preserve the order of the list. So secret version 7 is created first instead of secret version 1.

i can imagine a module to try to make it easier, but it would still involve multiple resources, each using depends_on for the prior one. say 10 resources, accepting a list of up to 10 secrets. obvs the number is arbitrary. and if you have more than that number of secrets, invoke the module more than once using module-level depends_on for ordering

I think you’re right. I’ll need to have the logic for the depends_on on a module level. Then the user invokes the secret-version module multiple times with a chain of depends_on.

Or maybe I don’t understand how port 0 is supposed to work?

Unlelss I’m mistaken from and to port “0” should mean all ports. the protocol is “tcp” which will allow only TCP traffic https://github.com/cloudposse/terraform-aws-mq-broker/blob/master/sg.tf

Part of the code that I’m seeing

resource "aws_security_group_rule"   description              = "Allow inbound traffic from existing Security Groups"
  from_port                = 0
  to_port                  = 0
  protocol                 = "tcp"
  type                     = "ingress"
}

whelp I guess I’m mistaken…for wide open ports we’re using from 0 to 65535. I have to be going crazy

Doesn’t protocol need to be “all” or “-1” for this to work?

I’m testing a RabbitMQ broker now with nc -z -v <hostname> 5671 and it doesn’t work with “tcp” and port 0 .

but works with port set to 5671 explicitely.

looks like you’re right

Okay, I thought I was loosing it. I’ll submit a PR shortly.

FYI: https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/sg.tf may show what they were attempting

https://github.com/cloudposse/terraform-aws-rds-cluster/blob/a51d466251f9584236606007aefe1578638ce684/main.tf#L25

might be more suited to this particular project (specifying a single port)

Yeah, in my issue I eluded to refining the ports further:
Even this is a bit wide open IMO when it should be restricting traffic to only the ports needed but at least this will fix it.

There’s only 2 supported brokers I think, ActiveMQ and RabbitMQ. Maybe my PR should include the specific ports you think?

Or I could fix the immediate issue and follow up w/a 2nd PR to restrict ‘em.

I personally think it should be restricted to a default port like they do with the RDS cluster.

In a previous PR I was conversing with “Gowiem” and he said he was using this module so now I’m wondering how he’s using it in this state

Are you interested in becoming a maintainer?

Yes. https://github.com/rennokki

Cool There is some info on the docs site here: https://docs.cloudposse.com/community/faq/

I’ll kick off the conversation with the contributor team and we’ll get back to you.

sure

Hey Alex! That’s great. Let’s talk week after next. I will DM you.

Thanks, going through the aws bootstraping tutorial. Makes sense so far. For some reason migrating tfstate to the s3 backend is failing though, not sure why.

TF not finding the state bucket

I can ls, cp, rm from the bucket so it’s definitely there

Huh possibly a region issue? I’d compare the region of the bucket vs the stacks/catalog/globals.yaml backend config + [backend.json.tf](http://backend.json.tf) file.

Also, I believe atmos will pass the -reconfigure flag when doing an tf init to deal with issues of the backend config changing… but I’m not 100% sure on that so maybe try deleting your .terraform directories.

If you can find out what went wrong, please let me know. I’ve gone through that workflow a few times when writing it up.

@Matt Gowie I think the script that invokes yq sets the region to uw2, when it created the state files in ee2. I was going to submit a PR to the tutorial. I hit this too.

Ah damn, maybe I missed that one in a last minute switch to show dealing more than one region. @Joe Hosteny thanks for the info.

No problem. I think workspace_key_prefix sounds like it needs to be set too? I discussed with @jose.amengual in another thread. Haven’t checked into it this weekend any more, but I noticed the literal “env:” in the state file s3 path

@Joe Hosteny — I put up a PR for this. Mind giving it a review since you should be able to approve? https://github.com/cloudposse/tutorials/pull/5

Thanks Approved - that’s what I did as well

Thanks @Joe Hosteny!

Thanks. I’ll test tomorrow.

Thanks Ryan!

if you mean the hostname of the instances, sure the ASG doesn’t care…

if you mean multiple DNS names resolving to one ASG, sure… the certificate needs to be a wildcard or have a SAN for each valid name, and you need to set up the target group rules to match on the host

I would like each instance to have a unique hostname containing the AZ

If all you want is a TAG then you can configure that in the AWS config (see the doc). If you want a fully qualified domain name, then you may have to set something up to read the instance data and create DNS entries. https://docs.aws.amazon.com/autoscaling/ec2/userguide/autoscaling-tagging.html

You can add multiple tags to each Auto Scaling group. Additionally, you can propagate the tags from the Auto Scaling group to the Amazon EC2 instances it launches.

Checking it out @Florian SILVA

We have a potential community member who reached out (@Alex Renoki) who is interested in helping maintain this module so hopefully that module starts getting more attention from folks who are actually using it soon. Problem we have today is that none of the current maintainers are big into beanstalk so it’s hard to review PRs.

Sounds good to me it can be merge I understand the second problem. Since beanstalk is quite complex It’s not easy to review these PR. I’m currently working on it and doing some modifications depending on my needs and seeing if some issues are linked so I’m a bit aware of some things. Just trying to fix minor fix when it annoys ^^

@Florian SILVA I might as well help reviewing some issues if needed. I’m not sure how to help with this maintenance issue

From what I saw, there are interesting and easy PR that has been open. The main issue on this module for me is that it is mainly working for application load balancer. I would not recommend using this module for other cases. I got some issues with the classic load balancer and the network type seems not working good enough but I saw some PR if I remember well so maybe starting by reviewing these could be a good beginning.

@Florian SILVA merged. Thanks for the ping.

Just saw it thank you for the efficiency

Np — I can usually get to things if people complain loudly enough, but overall there is a continuous flood of PRs each week so it’s easy to miss em. Particularly for modules that we / I don’t actively use.

depends a lot on your use-case. Personally, we only alarm on the cluster health metric

if a service wants more specific alarms, it’s done at the application level

makes sense i am trying to work out the healthy alarm

specifically the threshold

is this right it does not feel right

You want > threshold, not >=

unhealthy is zero though

Any value below 1 for HealthStatus is reported as 0 (UNHEALTHY).

oops, I had it backwards. Well 1 is healthy then. So you want Less Than a threshold of 1

or Less Than Or Equal To 0

makes sense

thanks man

hope you’re well too

ty u2

looks like this is expected. Although, I’m unclear if this behaviour is due to a difference in functionality between launch configuration and launch templates. I vagally remember reading that one of them can’t be updated in place. I’ve always used a custom module that changes the ASG when the template changes.

I’ll have to look at the difference between our custom module and the registry based one. would still welcome any comments or points, while I’m doing this comparison.

Looks like my custom module uses the hack mention here: https://github.com/hashicorp/terraform-provider-aws/issues/4100 name = "asg-${aws_launch_configuration.this.name}" but I assume I can’t do that on the module because it will be circular.

For anybody that has the same problem / question. I’ve found a way to do it. Basically call the module twice. Once to create the launch configuration or launch template Set the module not to create the asg

module "asg_config" {
  source  = "terraform-aws-modules/autoscaling/aws"
  version = "~> 4.0"

  create_lc  = true
  create_asg = false

  name = "${var.client}-${var.service}-asg"

then use a second module to create the auto scaling group

module "asg" {
  source  = "terraform-aws-modules/autoscaling/aws"
  version = "~> 4.0"

  launch_configuration = module.asg_config.launch_configuration_name
  use_lc               = true
  create_asg           = true

I still look to have an issue with the ASG creating the asg but doesn’t wait for the node to come up before destroying the old one.

module.asg.aws_autoscaling_group.this[0]: Creation complete after 2s
module.asg.aws_autoscaling_group.this[0]: Destroying...

So not perfect but I’ll keep looking for a solution on that one. Hope this helps somebody else.

Did you ever try configuring the instance_refresh block?

HI Tim, apologies only just seen this. I’ve not configured instance_refresh block on this module but have used it before. What’s your question?

can you try. volume_handle = format("%s::%s", aws_efs_file_system.jenkins-efs.id, aws_efs_access_point.jenkins-efs.id)

perfect cheers!

AFAIU, yes.

The per region part is painful. I know that the Cloud Posse folks have automated some of that via turf: https://github.com/cloudposse/turf

Because doing so via Terraform is very painful supposedly.

Good question — I don’t believe so. But might be a good one to add to the feature request list? cc @Andriy Knysh (Cloud Posse) + @Erik Osterman (Cloud Posse)

We have some scripts/ansible to run multiple components in parallel, but its ugly

Yeah — I can imagine so. I would think that atmos very likely could support that considering it’s a golang binary under the hood (created by Variant) and doing things in parallel like that is one of golang’s biggest selling points, but I’m pretty sure it’s not supported today.

Thats what I thought, but wasn’t sure if that was just due to the documentation being very new and in-progress

it supports running one workflow at a time with sequential steps

I would say our primary focus is using CD platforms to parallelize the runs. Atmos is primarily focused on local execution during development.

Parallel execution is limited by policies. E.g. in spacelift, we use rego policies to determine executions. We don’t want to re-implement that in Atmos - out of scope.