#terraform (2020-10)

Good to see you mentioning Checkov here - but this tool can definitely be used for both build and runtime; especially if you look at BridgeCrew’s SaaS offering which will hook back into your repos and remediate both operational issues, as well as your original code. Relationships between modules deffo still an issue.

I like the bridgecrew offering. Definitely a good option.

For sure it’s not OSS, but it rounds off the LEFT < - > RIGHT

I’m thinking DevSecOps should be followed here. for my TF projects I use unit test ( tf fmt tf validate and tf lint) then Integration testing using Terratest. For security I’m using Checkov and terraform-compliance, all of this should fall into a pipeline

I noticed the same thing actually

Yeah, it’s degraded for sure.

i mentioned it a couple days ago in the hangops terraform channel, there are a number of maintainers there. they’re aware, and trying to figure it out. it was fine when they first switched, but something happened on the google side and now they need to request a re-index or something

One thing I was thinking about this is that it seems they’re trying to reference modules without the prefix — Like aws_instance would now just be instance under the aws provider. But folks are continuing to search aws_instance and it harder to find it seems.

for now, it’s actually faster to search for “terraform aws” and then use the left side scroll bar to find the resource. it’s much better than it used to be, now that they group by service. https://registry.terraform.io/providers/hashicorp/aws/latest/docs

Yup dumpster fire

Yeah, to loren’s point — That does the trick OR “terraform aws_YOUR_RESOURCE” does help narrow it down.

This is getting worse and worse:

someone noted in the hangops channel that their robots.txt is blocking everyone

Usage in your environment or usage globally?

globally would be nice

could also answer this question @Matt Gowie

https://sweetops.slack.com/archives/CBW699XE0/p1601574548034000

So each module has a download count here: https://registry.terraform.io/browse/modules

it’s possible that we could add a new terraform module with a null resource for analytics. maybe default enable_analytics=true and people can turn it off if they like

But we also pulled it into an Excel file recently that’s easy to sift through:

ah but that is only for public terraform modules

i was thinking of something like homebrew analytics

@RB the git clone / download metrics from GH would be the way to do it for CP modules I believe.

ooo cool. how did you build this excel file ?]

The registry has an API

So built a script to scrape it into a CSV

Was very useful for our development efforts to know what modules are most common and we should support

ah very cool

would your team consider building it into a google spreadsheet instead ?

that way it can be updated on a cron

always support all the ClousPosse modules

It’s actually in a google sheet, but we can’t share documents directly from the Drive (org policy). Can potentially find a place to put it and auto-update.

Where’s the source of the data?

That you’re trying to load into those volumes

in the project’s directory

I copy it in the docker file

I fixed it docker configuration had a couple typos. Thanks

Would love to hear some early feedback / comparison if anybody tries it out.

i signed up for an account for the promo, anyway. but similar, would love to hear others’ experiences

@Sebastian Stadil can probably help with that maybe we can do a demo on office hours

Demo would be awesome. This seems to have a struck a nerve — the reception on Reddit looks well received: https://www.reddit.com/r/Terraform/comments/j3c225/scalr_public_beta_is_live/

The reception has been incredibly encouraging. all you guys.

We were thinking about doing an AMA too, if anyone thinks that could be valuable.

https://www.reddit.com/r/devops/comments/j3sdj8/terraform_cloudenterprise_alternative/

check out the comment by danekan

• You e-mailed me that password in plain text.

yes , that is because the token that github needs require admin access for webhooks on repos

hmm i am… lemme double check

and if you remove the access then it will try to create the webhook again even though is still there

mind me asking is this sort of thing documented anywhere…? this is really uh… convoluted

this is no a module problem

this is a githug provider thing

which is incredible annoying

github changes their APIs all the time, version 3.0 made a huge amount of changes

… just as annoying as terraform in general…

and as you know the provider is alway behing the api releases

I would not blame TF in this case

go check cloudformation and you will fell beter about TF

i prefer cdk

AWS apis are crap too

cdk is cloudformation under the covers

iono but my exp with tf has been nothing but headaches… tf syntax comes and goes with each version and theres a delay to support the latest from aws. I havent found a “golden bible” for best practices and all of these open source tf modules are all over the place. thats just my 2c ¯_(ツ)_/¯

and yes, im full admin to the repo, but still it’s trying to target personal namespace and not the org

pretty sure no expressions are allowed in the providers attribute of a module block

I believe that was what I found when I tried to that.

Yeah, was just wondering if there’s some sort of hacky way to accomplish the result

use terragrunt, with generate block(s) to create the provider configs

ah, so that’s where terragrunt comes in.

I would still have n number of module blocks though yeah?

one place, anyway

i could imagine being able to also generate a root config that contains the module blocks. that would be interesting. haven’t tried that

cdktf might be another option

cdktf might be a rabbit hole. The last time I tried working with it, it was pretty frustrating.

well cdktf is really new. there’s going to be some raw edges, and a learning curve. but fundamentally, the issue here is that this part of the tf/hcl config must be static by the time terraform is actually executing. so to make it dynamic, you have to generate/template the tf files. terragrunt can do that, cdktf is built to do that, or you can write your own using any template language you like

That’s useful info. @Sean Turner, I didn’t mean to hijack your thread. Just wanted to give you a head’s up that cdktf might lead to some time-creep.

No worries. I’m going to keep an eye out for a github issue around this and follow it closely as this would be a massive paradigm. Not too keen on trying tfcdk at the moment either, definitely going to give it a little more time to grow :)

@loren, do you have an example that shows how terragrunt can be used for generation? I’ve looked into it twice now, and didn’t get as far as seeing how it could be done. The example I found is too fancy. I’m looking for something simple so I can grok it quickly.

i don’t think you can use terragrunt just for generation… you have to buy into the whole terragrunt approach

That’s what I thought. And I haven’t had time to make that purchase yet .. which is why I’ve skipped over it twice now. But I think I have an idea how it works, I just want to see the end-game for an easy example to justify the jump.

We have at least temporary (feasible) work-arounds for all of the following things. What I really want are more control structures and templating options, that TF doesn’t include, sometimes by design.

in the terragrunt generate block, the contents attribute accepts an expression that must evaluate to a string. you have access to all terraform functions in the terragrunt.hcl file. such as templatefile and the hcl template language. here’s an example where i accepted a complex object as input, and transformed it to another complex object written to a tfvars file (in order to avoid some issue i no longer even remember ) https://github.com/plus3it/wrangler-watchmaker/blob/master/release/copy-bucket/terragrunt.hcl#L13-L25

if you’re not familiar with terraform string templates, https://www.terraform.io/docs/configuration/expressions.html#string-templates

I’m using the string templates. Thanks for the link to your example!

that’s kind of the basis for the solution i’m envisioning here… using the for loop in a string template to dynamically construct provider blocks, referencing values from terragrunt locals… and the locals could be sourced from a yaml file, if that’s your jam, with yamldecode()

The terraform-aws-eks-workers module uses the original self-managed auto-scaling groups with EC2s

the terraform-aws-eks-node-group module implements the fully managed node groups

note, you can mix and match. we’ve deployed clusters that use both.

we used the eks-workers for running jenkins and the eks-node-gruop for everything else (for example)

Please don’t cross post unless your cross-posting the link:

https://sweetops.slack.com/archives/CDYGZCLDQ/p1601895829001800

(or we get multiple answers all over the place)

Thank you for the response!

https://www.terraform.io/docs/backends/types/s3.html#s3-bucket-permissions

https://www.terraform.io/docs/backends/types/s3.html#dynamodb-table-permissions

ah nice, thanks! I’m guessing if you only need to run terraform plan, a least-privelleged read-only policy would grant:

• s3:ListBucket • s3:GetObject • dynamodb:GetItem • dynamodb:PutItem • dynamodb:DeleteItem

assuming even terraform plan tries to grab a lock on the state before running so it needs ddb access too

when it comes time for terraform apply, I’m guessing the policy will pretty much need to be full admin in order to create resources?

the plan probably needs same access as apply but RO

and apply needs access to any resources that yoiu are creating

theres certain aws services i dont use or dont want my pipelines to have access to, so i basically have an admin user with explicit denys

Their CHANGELOG is where I look

@Jaeson what’s your objective?

I often find myself trying to understand the for..each functionalities that were introduced in TF 12. The only real documentation that seems usable to me (from hashicorp) for this is embedded in one of their blog posts. … I’m usually pretty focused (or fighting to be focused) on solving a task, so it didn’t dawn on me until this morning that I’ve been kind of getting the information in pieces – for example, there seems to be no link from one post to the next in the series… you kind of just have to hunt them all down.

TLDR; I guess I was hoping for a better way to understand how to use the changes brought into TF12.

Aha, yea, I follow.

One thing that helps me is often to google for cheatsheets

might help to set TF_DEBUG environment variable so you can get some more details

TF_LOG ? I didn’t find anything when looking for TF_DEBUG . Not being a jerk, just confirming.

Nvm .. it’s in the log:
Use TF_LOG=TRACE to see Terraform’s internal logs

It doesn’t seem to give me much more than I already had.

haha, sorry - maybe I got it wrong.

np, I appreciate the hint. I think it might be something I did to my variables file.

I have a few modules, they are to install kubernetes and some products in the cluster, one module per product. My code is about to install K8s and one or multiple of these products, using the modules. My first approach for this design is to have a directory for the modules and one directory for the code using the modules. The second design idea is to have one directory per product and they can be used as modules or not, this way the only reused code (which may be the real module) is the code to provision the K8s cluster.

The question is: What’s the best practice with Terraform using modules?:

1. Have a directory or repo for the module, then the code using this module .. and maybe other module(s)
2. Have a directory for the module to be use by other codes … and use the same module code to be executed directly, not as a module

Using option (1) my repo would be like this:

terraform
├── all_products
├── p1
├── p2
├── p3
└── modules
    ├── k8s
    ├── p1
    ├── p2
    └── p3

So, the code in all_products will use all the modules, The code in p? uses the module k8s and the module for p?

Using option (2) the repo would be like this:

terraform
├── all_products
├── p1
├── p2
├── p3
└── modules
    └── k8s

The code in all_products uses the code in p* as modules, the code in p? use the module k8s … if other external code want to install p? uses the code in p? as a module

What do you think is better: (1), (2), both are accepted or is there a 3rd option?

you seem to be doing like if it was a monorepo

I personally do not like the aproach

I prefer modules that are instantiated by a project from a main.tf file and then this file pull the modules needed an then run TF somehow

Thanks @jose.amengual , it’s the same or similar answer I’ve received from other people (different Slack)

np

Close — you want to provide an empty array:

for_each = false ? [1] : []

I believe that will do the trick for you.

You might be mixing count / for_each.

ah.

That was it. Thanks so much. I guess I didn’t have the brain for this today – I’ve confused three things as I’ve been working at this: count, array bounds, and even flipped the logic. …

Happens to the best of us!

i’ve done it, makes a lot of sense for cross-account workflows (or cross-region, cross-provider)

usually i have a larger module around it, and this is a nested module in that project, sometimes with a dedicated module for the cross-account workflow

another benefit i’ve been finding recently, is with module-level for_each… putting the resource in a module let’s me document the variables cleanly. then i can manage multiples with module-level for_each and complex objects

here’s an example for a ram-share… the top-level module handles all the “owner” config for a new ram-share, and there is a nested module for the cross-account principal association workflow, https://github.com/plus3it/terraform-aws-tardigrade-ram-share

Totally agreed with module level for each documentation. It took me a second to wrap my head around the new way of doing but now it feels much cleaner as everything is var.foo instead of each.value.foo

Yes. It would be nice if you could break up lines with a backslash like in shell code. Perhaps there is an open ticket with terraform? If not, it would be a good one to write up

I already have enough “will never get worked on” open issues in that repo

it’s all about the number of issues with the most emojis

https://github.com/hashicorp/terraform/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc

and most commented

https://github.com/hashicorp/terraform/issues?q=is%3Aissue+is%3Aopen+sort%3Acomments-desc

don’t be discouraged!

in many places, you can use parens to get multi-line support, works fine in the ternary ? : also… e.g.

foo = (
  local.test) ? true : (
  local.anothertest) ? false : (
  local.end
)

just note the careful placement of opening and closing parens

i don’t think it’s possible to avoid this… the acme provider resources can’t access the aws provider credentials. any such creds need to come from the config of the acme resource

i found a workaround where i use a local-exec provisioner on a null resource to assume the role i want, and with aws cli. then i grab the output and parse it

but it’s a hack

yeah sure, you don’t actually have to use vars

nvm. When I took a closer look at this, I decided to set up the replication from DR to prod as part of the step to switch over to DR.

Won’t cover all AWS services, and I personally have not used this (just found it recently) but you could check into LocalStack https://github.com/localstack/localstack

there’s a way to configure terraform provider for it

thanks Zach had a look at localstack EKS module is not supported in community version

hah. enterprise gonna enterprise

I’d just like to qualify expensive. For most businesses, the most expensive resource is the human resource. E.g. A business is paying $1M/year for AWS and $20M a year for humans/payroll. AWS is letting you pay for fractional usage to run some tests. This is a very scalable way to manage the cost of testing. Instead, reimplementing this with something like localstack trades this predictable cost of using AWS with a highly variable, unpredictable cost of using an AWS emulator. You’re going to need to manage the additional associated techdebt and solve for the inevitable inconsistencies and bugs.

use count or for_each on all the resources in mock_[api.tf](http://api.tf) , and in the expression test a variable to turn the resources on/off

or with tf0.13, drop mock_[api.tf](http://api.tf) into a subdirectory, modules/mock-api/main.tf, and call it with a module reference from ./main.tf, and use count on the module reference with a variable to turn it on/off. this way you don’t mess with every resource in the module, just the reference to the module

This is a pretty simple module with a few canned policies.

If you want to do anything more probably better just to use the raw resiyrces

hey, just using it as guide but my question is more about functionality, how I could use the aws_appautoscaling_policy with custom metrics?

No mention of it in the documentation https://www.terraform.io/docs/cloud/workspaces/agent.html

no subcommand available to test health

For monitoring the health of the agents, right?

I came up with this, which is pretty ugly:

locals {
  combinations = setproduct(["role1", "role2"], ["policy1", "policy2", "policy3"])
  combination_map = {for item in local.combinations : "${item[0]}-${item[1]}" => item}
}
resource aws_iam_role_policy_attachment {
  for_each = local.combination_map
  role = each.key[0]
  policy_arn = each.key[1]
}

You can remove the intermediate variables:

resource aws_iam_role_policy_attachment {
  for_each = {for item in setproduct(["role1", "role2"], ["policy1", "policy2", "policy3"]) : "${item[0]}-${item[1]}" => item}
  role = each.key[0]
  policy_arn = each.key[1]
}

Hi, I cannot come with a better solution than your last code block…

makes sense to me. note that you can multi-line it to make it more readable

welcome!

I know you referred to Erik, but I’ll add what helped me: Terraform course in Udemy (not expensive at all) and a free account with AWS and CircleCI. (or TF Cloud, if you prefer)

That’s a great question! I don’t think I can answer you as comprehensively as I’d like to… I think @antonbabenko would probably able to direct you to the best learning materials.

Here’s my “high level” pitch: • Learn by doing, not just by reading. First identify what you want to achieve (because you need a goal), then read and research enough to get started and go from there. • Study our terraform modules. I’d like to think every single one of our modules is a reference example for how to design and implement composable, re-usable, testable modules. • Get started early writing tests. It’s a habit hard to introduce later. We use terratest and everyone of our modules has a simple example of that. • HashiCorp has invested heavily in their online curriculum and even offers certifications now. Their docs are free, check them out here: https://learn.hashicorp.com/terraform • For Terraform CI (github actions are sufficient to test). For a proper terraform CD workflow, I think your best bet is to start with a SaaS solution and learn from that. Your options are Terraform Cloud, Scalr, Spacelift, and maybe Env0 (haven’t checked these guys out yet). Terraform CD is non-trivial to do well. You can easily stick it in any pipeline, but a well-built terraform CD pipeline will have a terraform plan → planfile → approval → apply workflow. You’ll need to stash the planfile somewhere and the planfile may contain secrets.

• Checkout our weekly #office-hours → cloudposse.com/office-hours (podcast.cloudposse.com and youtube.com/c/cloudposse) they are free and you can ask questions and get answers from our community of experts.

• Hangout in watering holes like this one. You’ll learn a lot in a short amount of time.

That is a great and very detailed answer I usually give to people myself but in shorter way. I personally see a lot of value in reading documentation from A to Z (or to K-Keywords) when I am learning something.

Also, seeing open-source projects and try to contribute there is very good learning point… Though it requires more than one PR to start enjoying this process and see value in it.

You can also do workshop materials yourself for free - https://github.com/antonbabenko/terraform-best-practices-workshop

Yes/no. Strictly speaking, it’s not possible.

You have a few options though. Terraform supports passing backend parameters via environment variables. We used to use this extensively.

The other option is you can easily generate your [backend.tf](http://backend.tf) as a JSON file (E.g. [backend.tf](http://backend.tf).json ) This is the route we’re taking today because it’s easier for developers to understand.

Got any examples of that first solution?

http://github.com/cloudposse/tfenv

I don’t have any issues with doing it as an env var

We used this simple cli to make it easier

Oh, and lastly, #terragrunt can also do this for you. But if you’re not using it, it’s a heavy handed solution for simply managing the backend config.

Yeah, my intention here is to keep things as simple and straightforward as possible as it’s fairly minor in usage

Here was our original post on this pattern: https://www.reddit.com/r/Terraform/comments/afznb2/terraform_without_wrappers_is_awesome/

Here’s the terraform docs https://www.terraform.io/docs/commands/environment-variables.html#tf_cli_args-and-tf_cli_args_name

https://archive.sweetops.com/terraform/2019/01/#f16aa2b1-259f-47d9-ba6c-ed1d21bac05a

Haven’t seen anything for that - not sure if it’s possible natively. Of course, anything can be engineered using APIs, etc. I don’t think that’s your goal though?

Right. I’m trying to ensure it’s as accessible as possible to reduce friction while still promoting a clean git history and workflow. I’ll see what I can swing, just was hoping for something. I might preview the terraform command cli library I used for packer as it calls the api without terraform cli directly. It might be able to help.

You could have Atlantis to just comment the plan on a PR bases and have tf cloud executed it

Sounds like more infra to manage. :-) I found a github action and with terraform api and libraries I bet I could figure out how to extract this from terraform cloud. Wish me luck

Good luck

if it suits your needs, and the names are consistent and unique, should be fine

Thank you for your reply. Yes, it does seem to work fine…so far. I’m trying to see if anyone out there can see a flaw in my plan that I cannot.

we use namespace to identify the company. We add it to all the resources for consistency

We use company-prd(or stg etc)-region(or global)-thing.

this is our standard pattern to provide standard inputs to all modules, and simplify module instantiation

https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf

so now all standard inputs (those that we use in all modules) are in one place (in [contect.tf](http://contect.tf))

you don’t have to provide namespace, environment, stage, name` when calling modules https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/main.tf#L57

you can override any of the standard inputs anytime, e.g. https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/main.tf#L7

Thank you for your answers. I like your label modules and this is a great application.

Yes you can output the entire module as an output

The module that uses the above module would then be able to reference the output using module.whatever.alb.whateveroutputname

We ran into this bug and the conclusion was “we can’t” The WAFv2 resources are a bit hacky, as I’m guessing you found out they can’t be infinitely nested as what the AWS API supports

There is a depressing discussion in one of the tickets about this limitation, where it was suggested to convert the Terraform resource representation from HCL blocks to inline JSON. The hashicorp response was “this would fix the problem but it’s ugly, so we won’t do it”

Thanks, Alex!! That’s good to know.

found out what that error message is.

*InsufficientEncryptionPolicyException* This exception is thrown when the policy on the S3 bucket or KMS key is not sufficient. HTTP Status Code: 400

please look at this module for the required permissions https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket

and this is a working example of using both Cloudtrail and Cloudtrail bucket modules https://github.com/cloudposse/terraform-aws-cloudtrail/blob/master/examples/complete/main.tf

thank you @Andriy Knysh (Cloud Posse)

not quite sure what you’re asking, but does this answer your question?

data pagerduty_user default {
  email    = "[email protected]"
}

resource pagerduty_escalation_policy default {
  name  = "My Escalation Policy"
  teams = [
    data.pagerduty_team.default.id
  ]

  # Notify Alex
  rule {
    escalation_delay_in_minutes = 5
    target {
      id = data.pagerduty_user.default.id
    }
  }
}

you can then create a pagerduty_service resource and specify the escalation policy’s id as the value for escalation_policy

I want to add the subscribers only, not the actual responders. Basically, I see a lot of cc’d folks in chat when activity is on pagerduty already. I’d like to be able to define the subscriber/observer list so they get updated on new incidents but not as a responder.

gotcha. We wanted this too. It’s difficult with how Pagerduty works. What you are after is to set a response play on your service

sadly, this isn’t available via the Terraform provider yet

I saw we found the same GitHub ticket @sheldonh

Great minds think alike

• Atlantis lacks triggers, you cannot have one workspace trigger another. TFC does.

• Atlantis lacks webhooks, so you cannot easily integrate it with other pipelines or continuous delivery frameworks. TFC does.

• Atlantis can only be self-hosted. TFC is also a SaaS.

• TFC cannot comment on PRs (Sometimes this is nice). Atlantis can.

• Moving in/out of TFC is equally difficult.

• TFC does not let you bring your own container without using on-prem runners. Atlantis was a phenomenal first-stab at gitops with terraform. It was pioneering software in a time when most terraform was run locally. However, as release engineering has evolved, there’s clear need for coordinated rollouts spanning tool chains (standard app deployments, db migrations, serverless deployments, infrastructure deployments, etc). Coordinating this via multiple disjoint PRs, is not scalable teams grow. Thus the atlantis-flavor of gitops (operations by github comments), isn’t as optimal in larger in team environments with lots of services.

• Atlantis lacks triggers, you cannot have one workspace trigger another. TFC does. How does this work? workspace 1 triggers a plan/apply of workspace 2 after workspace 1 has been applied?

Correct… https://www.terraform.io/docs/cloud/workspaces/run-triggers.html

won’t auto apply though

Gotcha

Cheers @Erik Osterman (Cloud Posse), hope you’re good!

yes! had many fun projects this year…. the tfc being among them

@Erik Osterman (Cloud Posse) if atlantis supported similar functionality than TF cloud for workspaces and plus commented the pr back, would that be what is needed to make atlantis better?

I wonder if TFC workspaces is lacking features that will be nice to have but since do not use it I’m unaware

Main thing I think Atlantis is missing is triggering projects via API and not just PR

How do TFC workspaces work different to normal workspaces? Guessing this means once your in TFC land, there is no easy getting back out…

I should probably have a poke myself…

Also with Atlantis ALL THE COMMENTS, 0.14 with concise diffs should help this though

we also have 2 forthcoming PRs for TFE/TFC and for the TFC agents.

https://github.com/runatlantis/atlantis/pull/997

there is an intention to add api calls

but the lack of more active maintainers makes it pretty slow

thats cool! hadn’t seen that PR. it’s from back in august though… not optimistic.

maybe is time to fork atlantis into another project with another name for good

heh, call it atlanta

peplantis

lol

That answer should be pinned imho.

Good rundown, @Erik Osterman (Cloud Posse)

Any idea how tfc enterprise pricing works? Experimenting with tfc free plan and secret management across multiple workspaces is toilsome considering we have 50 odd state files (directories -> tfc workspaces). Agent like your cloudposse module is ideal. No access secret keys..

no one knows about prices that is a huge problem with Hashicorp

@kskewes we found out a few things, and trying our best to publish at https://remotebackendpricing.com/ (pending addition of best community understanding of Hashi prices)

First, Terraform Enterprise is priced on number of workspaces, and as a product is being phased out in favor of Terraform Cloud (Business Tier).

Second, Terraform Cloud (Business Tier) is priced per user, plus a fee for a number of applies per month, plus a fee for the number of concurrent runs you want.

Third, they discount / bundle things into an offering based on their guessed ability to pay, i.e. your funding level, or size of your company, which is why prices fluctuate so much from one company to the next.

If you get a quote and want to help others know pricing, PRs are welcome at https://github.com/Scalr/remote-backend-pricing

Cheers, sounds like need to contact sales team. FWIW, I see private scalr pricing is also hidden - Is there a runner we can use with the published pricing?

so basically they have a way to screw up big customers because they have more money instead of make the prices transparent

@kskewes price will be published before the end of next week

When working at EA I remember we asked about Vault enterprise and when we told them we were EA the tone in the conversation changed and they sent us a quote for 2 Million a year

(and yes it pains me to not have it out in the open yet)

Cool cool, no biggie. I understand that at some point customers will engineer ways to reduce cost by circumventing published processes - shared users/etc.

TFC almost looks like it could be a per minute pricing like Gitlab CI minutes

maybe plus per user that need console access/etc and if need be some base. But I have no idea of their costs and constraints, just throwing something out there based on a similar “remote execution as a service”.

“plus a fee for a number of applies per month” any idea what this is?

You mean how much it is?

Yes

Your best spot for these is #pr-reviews. But happy to check it out.

Thanks

@Cody Moore unfortunately, it looks like there are issues with running our tests against this repo due to the module targeting 0.13 but we’re using 0.12 to run the tests. This is the first I’ve seen this, so I’ve brought it up with the rest of the contributor team. We’ll get this merged once I can get that sorted out and those tests pass.

And I spoke too soon. Easier than I thought — Getting it sorted now.

Awesome thanks!

This enables terraform for business users to run their plans using a custom docker image (or the official one), as well as using IRSA

HCP Vault announcement

HCP Consul

Consul 1.9

(this is great! keep ‘em coming)

Zero trust focused products

Watching, but not stoked on anything yet.

What Armon is talking about now is interesting though..

Here we go. https://www.hashicorp.com/blog/hashicorp-boundary

haha #called-it

Now I’m excited.

https://github.com/hashicorp/boundary

Seems like AWS SSM Session Manager, but cloud agnostic.

Though it does more protocols. Looks like it can do anything TCP (redis, postgres, ssh, etc.) https://www.boundaryproject.io/

Yeah, which will be sweet! I can throw away my ugly make target that creates a SSM session to port forward database access.

I love Zero Trust but one of the principles of Zero trust is the identification of the machine/computer itself and without a way to Catalog for example a users laptop there is no way to reconcile the identity of the machine after the registry change because a new software is installed, which is key for this concept to work. The way reconcile instances is by using packer and other tools to build amis or use containers but what about a users phone or laptop?

oh man and we just signed with banyan’s zerotrust solution

oh well. it’s not like boundary is the first open source zt solution

Ask @Erik Osterman (Cloud Posse) about open source IDPs

it is 0.1 you need to wait at least for 0.13.0

lol

I hope Since its HashiCorp, it won’t just die

Curious how it compares to https://gravitational.com/teleport/

Nomad goes 1.0 on 27th and namespaces will be open source in 1.0

really nice keynore for nomad

Got Boundary up and running in AWS and was able to connect from laptop

so basic functionality seems to work fine

though missing 3rd IDP

for now only local user/password

nomad is free? I never used and I thought it was all paid

it is free and there is enterprise version that has some additional feature

Namespaces in OSS is great news

boundary looks great - it covers off (on the label) a bunch of use cases i’m looking for access control atm

New Keynote is going on now and the website for the new product is up!

https://www.waypointproject.io/

HashiCorp Digital Day 2 — Continuing yesterday’s thread

Yet another CI/CD platform??

it is something else

seems to be replacement for a bash scripts we all making to glue things together

but let’s see

Yeah. It’s still push-based which is interesting considering that the bleeding edge now is GitOps where you have something running in the cluster, pulling the latest info about what to deploy

I like that it’s using Buildpacks

remember otto ?

will waypoint go like otto ?

Please keep mentioning otto (in the comments). @RB What is that?

it’s an old very dead hashicorp project

https://www.ottoproject.io/

almost like waypoint is a rebranded otto

I’m late , waypoint is like atlantis?

ohhhh is a ci-cd…..

another ci tool

It seems to be only a CD tool. You do execute builds with it, but it isn’t for CI.

So you might execute a waypoint up as part of your CI tool it seems.

Otto looked cool at first when it appeared, but then people soon realised how utopian it was))

I’m not sold yet. This an area where people do a lot of customizatons to fix their workflows

so let see deep dive section

also, not sure what is the game plan there. As a company you supposed to make money. I can understand tools like packer that do not provide any profit but they are useful for the company itself. This one is something else…

and not really their are

hm yah they’ve already got a github action for it in fact

waypoint entrypoint = ngrok but opensource

One of the Terraform 0.14 major functionality targets is “Concise Diff” — I like the sound of that.

+1

waypoint up, links to test envs, test to terraform - it looks like all those releases are Pulumi inspired

trying to take away reasons to move from HashiCorp ecosystem

“test to terraform” — What’re you referring to there? What’d I miss? I tuned out after 2-3 sessions.

terraform 0.14 will have test provider that you can use to run HCL defined tests for terraform modules

Damn, sounds awesome… how did I miss that.

Did you see any information released about that?

heh, sigh… exciting news about the test provider. but my level of interest in rewriting all of our tests is zero

I couldn’t find any info in writing. They showed it as an experimental feature during presentation. Probably more info will come later

waypoint i think you can think of like a super powered hcl makefile

personally i’m keen to start playing with it because i’ve typically found makefiles to be non-transparent to those that are unfamilar with them

https://github.com/mumoshu/variant2

the other superpowered HCL makefile alternative

nice! Thanks for sharing @Erik Osterman (Cloud Posse)

@antonbabenko talks about the testing provider https://youtu.be/nphb0utdKEY?t=666

IMO looks like it is complimentary to terratest

In its current form - yes, I agree, but later Terraform testing framework (to be developed) should be able to check it everything what is in tfstate, and allow to run “plugin” which does subset of assertion via HCL which currently can be done in terratest.

Let’s come back to this in 2-3 months. ;)

Watched the walkthrough — good stuff! Happy to get some more information on that subject. I could definitely see that pattern being valuable. If that was built first class into Terraform instead of through a provider then that would be sweet. I’ll for 0.15.

userdata is base64 encoded

so whatever value you get you will need to decode it

I tried base64 -d and putting the value but the result is non-sense. I’m thinking the user_data number shown is a hash into some table of text data. Maybe that data is base64 encoded.

maybe is truncating it

ohhhh that is on a plan?

mmm I think the output will be truncated, I guess you will have to compare at git level

This user_data is generated by kops, I just noticed (first time!) a folder gets created in the module, /data, that has the scripts created by kops and those are referenced by user_data in the launch configs. Indeed we do save those in git so I was able to see what changed, would have been nice to get such diff in the terraform plan but git diff is better than just the hash that I was seeing before!

Thanks for you help @jose.amengual

np

oh boy, we’re on to betas already?

Huh they’re releasing beta versions earlier than I would have expected.

well that’s an interesting feature… @Erik Osterman (Cloud Posse) @antonbabenko plays into the versioning discussion from office hours yesterday… https://github.com/hashicorp/terraform/pull/26524

that neat! thanks for point it out.

wow, beta1 already??

https://discuss.hashicorp.com/t/terraform-0-14-the-dependency-lock-file/15696

The first iteration of this for Terraform 0.14 covers only provider dependencies. Tracking version selections for external modules will hopefully follow in a later release, but requires some deeper design due to Terraform’s support for a large number of different installation methods for external modules.

Going to need to git gud at updating… Sounds like a make target.

they are just reminding us that anything that is painful, we should do more of until it’s not painful any more

e.g. it’s painful to update core version pinning on 100s of modules

if anyone has feedback or sees any patterns we can improve, please let me know! (or open a pr )

It would be pretty neat to export the cert ARN as a thing of it’s own, instead of the entire certificate object. My IntelliJ can’t really see that the object will be of the cert type and thus can’t do autocompletion for it

oh that’s interesting. i’ve never relied on autocompletion in modules, so never occurred to me. seems like a pain to keep up with and document all the attributes someone might want though

i wonder if autocompletion of object attributes is something the language server might be able to offer

found an issue pretty close, asked for clarification on this specific use case… https://github.com/hashicorp/terraform-ls/issues/93#issuecomment-710057056

https://github.com/hashicorp/terraform/issues/6632

https://registry.terraform.io/providers/hashicorp/aws/latest/docs#ignore_tags

thanks!. the information will be useful, but the point was to keep it quiet for the change on timestamp tag and not to prevent it.

Hello @Luis Muniz, if you switch from one tfstate to an other you should use terraform init -reconfigure

https://www.terraform.io/docs/commands/init.html

if the point is fetching again the remote state, terraform refresh will get the update. by default terraform plan will do a refresh

Hi, thanks for replying Pierre-Yves. The issue is that the backend is not configured, because the module wants to add the s3 bucket

i’m stuck in this pre-initialization limbo

refresh does not do anything, because the backend has not yet been switched to s3

it’s stuck in the local backend, when I run the plan, it tries to create an s3 bucket and dynamo tables that already exist

refresh does not do anything, because the backend has not yet been switched to s3 that’s something you do by editing the .tf file and updating the backend

exact if I have resources in two terraform state, to switch from one to an other I do: terraform init -reconfigure -backend-config=

Thanks, I was misunderstanding how to use the module, and trying to re-generate the terraform.tf file (it was not under sourtce control)

early adoptions signs , which is good lol

First-class problems definitely a good sign

if you are going to use TF cloud, then use workspaces but if you do not then you could use same repo with multiple backend-configs ot a repo per customer

it depends on how similar are your customers

It also depends on what will happen when you have TF code updates. Will you update all customers, or find yourself with different customers running different versions?

I’d also point out that versioning the parent code, and calling those specific versions for your customer execution plans, is important. Gives you an easy mechanism to perform selective applies.

@Yoni Leitersdorf (Indeni Cloudrail) yes sir , I can think of use cases where customers might have different versions running @Chris Wahl thanks for pointing that out

@jose.amengual thanks for always replying and helping me , so can you provide some more insight how to approach it while rolling out updates for a customer if we only have one repo ? Will the concept of having different back-end configs work ? And can you think of any scalibiliy issues with this ?

this question have many ways to answer it right

I will tell you my preference

for example lets say you have customer with wordpress+mysql

you have a module to build wordpress and another one for mysql

if you want to manage customer you could have one repo for all your customers that have that conbination

and you could have a tree structure like

and then you instantiate the the main.tf per customer using the modules you already have (using versions)

and you ini terraform by doing terraform init --backend-config=clientA/backend.tf

obviously you might have more parameters, like vars file etc

that is one way

you can have a backend.tf per customer or a backend file for all customers that matche the criteria wordpress+mysql

but that will get real messy if you have many customers and with slightly different configs

I do not like that much, I like the CloudPosse way were there are modules and metamodules

so I will prefer to have a mysql+workpress module

and have a meta module terraform-aws-wordpress-mysqlthat uses those other modules and that is flexible enough to accept slight different configs per customers and you can use the same way to init the module for all your customers and use a custom backend.tf file that can be templated by an automated job and the data could come from a key/value store or db or whatever

just like this module https://github.com/cloudposse/terraform-aws-ecs-web-app

and if you need environments per customers you can have another module for the environment itself that will call the meta module etc and so on

is like a chain of module dependecies

but the shallow the dependency chain the better

that is my humble opinion

there must be others that have done this for customers/many teams with different workflows

Thanks again

Hi @nileshsharma.0311 @jose.amengual, I am using one project and one tfstate per tools and one tfstate for each env. this makes smaller tfstate and limit access to project , to only allowed people

yes, that is another thing too, the bigger the state the longer deployments take and the bigger the blast radius

What’s the bucket names you’ve tried, can you use a name_prefix instead

Bucket names are global and must be unique

Using Java style domain names are what I prefer.

com.mydomain.logs

Unless someone is actively trying to sabotage for you they are usually free..

I am using the v0.31.0 module from https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment and the error line is from the main.tf in that module.

I see. I guess your name, namespace etc combination clashes with what someone else is already using then. I think that’s probably something that should be made overrideable in that module.

I’ve avoided using some public open source modules because of that actually

some of them just assume you get a new vpc everytime

That’s the main thing that slowed me down trying to figure out all the right subnets to use. Terraform wasn’t used to manage the VPC deployment so it wasn’t quite so straightforward to quickly deploy

we have specific vpcs and just use data lookups on them by name in all our modules

I tried that and mostly ok but one had duplicate names . Maybe I could get the subnets dynamically by filtering for public/private attribute

yup! we do that too. Add a tag to the subnet ‘tier’ and look it up that way

One of the guide thats good to read https://gruntwork.io/guides/networking/how-to-deploy-production-grade-vpc-aws/ Another good topic is about placement of ec2 ecs k8s lambda etc and interconnecting it. Maybe aws guide for architect pro will have some answers for you?

This is the entire output for the plan command:

I didn’t create an issue in GitHub because I’m not sure if this is because something on my end or specific to the update 0.8.0

Hey @x80486 — I was the one that merge and releases 0.8.0. It passed our tests, but maybe something cropped up that our tests didn’t catch.

Can you check out the below PR / code and see if targeting that will fix this issue that you’ve run into? https://github.com/cloudposse/terraform-aws-acm-request-certificate/pull/27

I’m getting this one now:

Error: Unsupported attribute

  on .terraform/modules/acm_request_certificate/main.tf line 44, in resource "aws_acm_certificate_validation" "default":
  44:   validation_record_fqdns = aws_route53_record.default.*.fqdn

This object does not have an attribute named "fqdn".

What I did was to copy over the files from that branch (that’s in Merge Request now), and changed the source value by pointing to the directory under .terraform/.. (the following file snippet has the correct URL anf reg tag for 0.8.0 which I didn’t use for testing the changes you asked me to)

This is my super-tiny Terraform configuration:

locals {
  tags = {
    Environment = terraform.workspace
    Terraform   = "true"
  }
}

provider "aws" {
  profile = terraform.workspace
  region  = var.aws_region
}

resource aws_route53_zone "default" {
  name = var.domain_name

  tags = local.tags
}

module "acm_request_certificate" {
  source = "git::<https://github.com/cloudposse/terraform-aws-acm-request-certificate.git?ref=tags/0.8.0>"

  depends_on = [aws_route53_zone.default]

  domain_name                       = var.domain_name
  process_domain_validation_options = true
  subject_alternative_names         = ["*.${var.domain_name}"]
  wait_for_certificate_issued       = var.wait_for_certificate_issued
  zone_name                         = var.domain_name

  tags = local.tags
}

module "cloudfront_s3_cdn" {
  source = "git::<https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn.git?ref=tags/0.35.0>"

  depends_on = [module.acm_request_certificate]

  acm_certificate_arn      = module.acm_request_certificate.arn
  aliases                  = [var.domain_name, "www.${var.domain_name}"]
  allowed_methods          = ["GET", "HEAD", "OPTIONS"]
  compress                 = true
  dns_alias_enabled        = true
  error_document           = "not_found.html"
  namespace                = var.company_prefix
  name                     = var.name
  origin_force_destroy     = true
  parent_zone_id           = aws_route53_zone.default.zone_id
  stage                    = var.stage
  use_regional_s3_endpoint = true
  website_enabled          = true

  tags = local.tags
}

The [variables.tf](http://variables.tf) file:

variable "aws_region" {
  description = "The AWS region to deploy to"
  type        = string
  default     = "us-east-1"
}

variable "company_prefix" {
  description = "The company's name prefix"
  type        = string
  default     = "tld-domain"
}

variable "domain_name" {
  description = "The FQDN name of the Website (e.g.: example.com)"
  type        = string
  default     = null
}

variable "force_destroy" {
  description = "Delete all objects from the bucket so that the bucket can be destroyed without error"
  type        = bool
  default     = false
}

variable "name" {
  description = "The identifier/name of the application or solution"
  type        = string
  default     = null
}

variable "stage" {
  description = "Stage, e.g.: prod, staging, test, dev, sandbox, etc."
  type        = string
  default     = null
}

variable "wait_for_certificate_issued" {
  description = "Whether to wait for the certificate to be issued by ACM (status change from PENDING_VALIDATION to ISSUED)"
  type        = bool
  default     = false
}

The .tfvars file:

force_destroy = true

domain_name = "domain.tld"

name = "domain"

stage = "sandbox"

wait_for_certificate_issued = true

My plan target looks like this:

.PHONY: plan
plan: init workspace init ## Create a Terraform execution plan
	@terraform $@ -compact-warnings -input=false -lock=true -out=$(WORKSPACE)-plan.out -refresh=true -var-file=$(WORKSPACE).tfvars

## Private Zone

.PHONY: init
init:
	@terraform $@ -input=false -lock=true -verify-plugins=true

.PHONY: workspace
workspace:
	@terraform $@ select $(WORKSPACE) || terraform $@ new $(WORKSPACE)

@Matt Gowie, did you have a chance to take a look at this one?

Hello, I use the plugin “terraform” from “Anton Kulikov”. I don’t know if it’s the best but my need was the support of 0.12 tf version

Hashicorp has taken over. Update to their latest one as it’s an officially maintained plugin now.

I’m sure someone else has a much better setup than I do but esentially on each PR we run: terraform fmt -resursive -check -diff terraform validate

after merging to master we output the plan and have to approve before applying

manually running tfsec and other tools as needed. trying to figure out how to automate these - daily, weekly, etc..

links: https://github.com/antonbabenko/pre-commit-terraform https://github.com/tfsec/tfsec

We have a Makefile that’s used to call different targets: lint, validate, plan, apply, tfsec, etc…

We only have a handful of parent TF projects so far. Very scalable for all of our app deployments right now. It seems like this is a scalable solution for IaC. No complaints so far.

Do you have Spinnaker running terraform in pipelines MattyB?

Wrt OP.. we use jsonnet for pipeline code etc and spin CLI to upsert Spinnaker. There’s no state so we have to do manual removals. We haven’t wired this to CI yet as haven’t setup x509 or basic auth to Spinnaker. Manually updating with Makefile leverages our gcloud oauth.

there is not a good way of dealing with that. If you have complex types on both sides of the ternary operator, both sides need to be of exactly the same types

you can 1) specify in the variable definition AND provide the exact object types on both sides (and it means not only the types of the objects, but the types of all the fields)

2) use jsonencode and jsondecode to work with strings

this is not a solved case yet https://github.com/hashicorp/terraform/issues/22405

Yes, it was a mismatched sub-type. The error was real, but the error message is worse than old GCC

on the plus side, they’ve been getting better and better with the error messages. this was way worse before!

Hello, if you change a resource name it will be destroyed first and recreated

I don’t want the resource name to depend on these variables

I didn’t experience it myself I would suggest to see if there is a trick with a null_resource to do this https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource

Calculate some kind of hash for thise variables and append it as postfix to field that will recreate the resource you want to depend on it. Im not using much elasticbeanstalk so i dont know which field would be the best but name would be fine imo

the hash is a good idea. i’m also unfamiliar with EB, but i know if you are using ec2 you can force recreation by modifying the userdata, even with just a comment in the script/cloud-init config…

The only attribute that forces recreation is name. And a lot of our infra glue depends on that being predictable I was hoping for some cool Terraform workaround

Interesting. What is the exact resource?

https://www.terraform.io/docs/providers/aws/r/elastic_beanstalk_environment.html

The cloudformation equivalent mentions a few properties that require replacement, so maybe you can work a replacing update implementation around one of those? https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-beanstalk-environment.html

If not, you can always taint the resource, though that’s a bit outside the gitops-style workflow I try to prefer

Yeah, good idea. I will try and use cname prefix – the DNS name can be weird

yep. I can set cname_prefix like:

resource random_id eb_cname_prefix {
  keepers = {
    use_shared_alb  = var.use_shared_alb
  }

  byte_length = 1 # Increase this as we add more keepers
}
resource aws_elastic_beanstalk_environment foo {
  cname_prefix = "myapp-${random_id.eb_cname_prefix.hex}"
  settings = ... settings including LoadBalancerIsShared ...
}

Thanks for the ideas all!

what environment variables have you added? Do they work with the aws CLI?

AWS_ACCESS_KEY_ID AWS_DEFAULT_REGION AWS_SECRET_ACCESS_KEY

i think i needed to also export AWS_REGION to get terratest to work (without setting the values in the provider config). i do not know why. i briefly inspected the source but couldn’t find a reference

i export all these in my local profile:

AWS_SDK_LOAD_CONFIG=1
AWS_DEFAULT_REGION=us-east-1
AWS_REGION=us-east-1

did you check the ecs console ?

you could have a problem with the container not starting and continuously deploying

The deployment is added to the list, but it has a 0 pending and 0 running count. Nothing is logged out in the console and no new task is started

In the events tab there’s any message?

Actually no. All the events seems to be from before the deployment

weird.

Yes indeed. Not really sure how to troubleshoot it

so is the deployment % 0?

what happens if you bump it up to 50%?

I’m not sure which % you are referring to. It looks like this in AWS

Are you talking about the minimum and maximum healthy percentage?

If so: yes, that was it

Seems like it’s spinning up now after i set it to 100% and 200%

yes that is what I was talking about

deployment healthy min/mac percentage

is is 0 it does not do anyting

Running a test deployment now to make sure that it is fixed

That was it! Thanks

2,5 hours of my life gone, but it works after all

you are working on ECS, expect 50% of your life gone troubleshooting this AMAZING service

This was still required in 0.11, 0.12 at a minimum. We have these annoying stubs as well

something like data.azurerm_subnet.module[*].id I guess. See https://www.terraform.io/docs/configuration/expressions.html#splat-expressions

This was the solution in the end: [for subnet in data.azurerm_subnet.module : subnet.id]

Thanks for the suggestion!

This repo is out of date AFAIK. The CP folks are looking to update it with their latest and greatest, but it hasn’t come to fruition yet. Erik has mentioned an EOY target.

Yea, so we’ve gutted the old reference architecture to make sure no one follows that.

We have a new one coming out this year, with bits and pieces trickling out.

Thanks for the reply. I’ll wait for the new architecture. Cheers

Thanks Steve, that is my point. I’ll probably use AKS & Terraform and will use helm at most . ( mainly because there is more example and documentation )

you will need both to create the namespace and secret for flux to use to communicate with your repo

I didn’t know about flux (so I am currently reading the doc), what will be the benefit vs a standard helm pipeline ? my understanding is:

• a classic pipeline will push a release to kubernetes vs

• flux running on kubernetes will watch and fetch the version to deploy it on kubernetes ( should move the thread to #kubernetes but I don’t know how )

I think the fargate module was not updated to TF 0.13 requirements. We’ll get to it ASAP

No rush! Was just curious!

Figured it was updated as the requirements reflected terraform 14

@Aumkar Prajapati this latest release fixes the Fargate module https://github.com/cloudposse/terraform-aws-eks-fargate-profile/releases/tag/0.6.0, should work with TF 0.13

look at the complete example on how to use it https://github.com/cloudposse/terraform-aws-eks-fargate-profile/tree/master/examples/complete

Sweet! Thank you!

From the tools linked I guess you want to output to be JSON? In that case terraform show -json [path-to-file] but I think that is only valid for the most recent plan…

error_message = "Valid values are ultrawarm1.medium.elasticsearch, ultrawarm1.large.elasticsearch and ultrawarm1.xlarge.elasticsearch."

Try ending the error message with a period, if that doesn’t work maybe the other periods in the string are causing a problem

makes sense

This is the code from Terraform’s actual repo about this:

// looksLikeSentence is a simple heuristic that encourages writing error
// messages that will be presentable when included as part of a larger
// Terraform error diagnostic whose other text is written in the Terraform
// UI writing style.
//
// This is intentionally not a very strong validation since we're assuming
// that module authors want to write good messages and might just need a nudge
// about Terraform's specific style, rather than that they are going to try
// to work around these rules to write a lower-quality message.
func looksLikeSentences(s string) bool {
        if len(s) < 1 {
                return false
        }
        runes := []rune(s) // HCL guarantees that all strings are valid UTF-8
        first := runes[0]
        last := runes[len(runes)-1]

        // If the first rune is a letter then it must be an uppercase letter.
        // (This will only see the first rune in a multi-rune combining sequence,
        // but the first rune is generally the letter if any are, and if not then
        // we'll just ignore it because we're primarily expecting English messages
        // right now anyway, for consistency with all of Terraform's other output.)
        if unicode.IsLetter(first) && !unicode.IsUpper(first) {
                return false
        }

        // The string must be at least one full sentence, which implies having
        // sentence-ending punctuation.
        // (This assumes that if a sentence ends with quotes then the period
        // will be outside the quotes, which is consistent with Terraform's UI
        // writing style.)
        return last == '.' || last == '?' || last == '!'
}

thanks guys appreciated

Basically, valid unicode characters ending with period.

also, you can dramatically simplify the validation condition using contains()…

and the can() is totally unnecessary here…

contains() with just a list?

yep… condition = contains([<list of valid values>], var.ultra_warm_instance_type)

https://www.terraform.io/docs/configuration/functions/contains.html

oh that is way better, thanks man appreciated

and you can break lines using parens also, to help readability, if you like

legend thanks man appreciated

You should manage desired size outside of the stack. It is usually a variable with such a high variability that it’s usually not desirable to manage it in the base Terraform stack.

It makes sense, I see the reasoning now. When enable_cluster_autoscaler is enabled the desired count changes, and it could lead to breaking changes on many new apply/deployments.

I have enable_cluster_autoscaler disabled so it didn’t pop in my mind at first.

thanks for the reply.

a followup question - when having the autoscaler disabled, how does one manage the desired count? I am not really keen on manually adjusting variables in aws cli/console

Perhaps by setting the min_size and max_size fields to your desired number? We do this (for one pool that CAS doesn’t handle).