#terraform (2021-02)

https://www.terraform.io/docs/language/dependency-lock.html

Terraform CLI to create a dependency lock file and commit it to version control along with your configuration

i think the lockfile workflows are still a “work in progress”… for example, in reusable modules, where perhaps you run tests, it probably doesn’t make sense to commit the lockfile alongside the tests

and if you are deploying the same config, same version across many many accounts each with their own tfstate, it doesn’t really make sense to commit the lock file for each one of them. a little orchestration is needed to create/update the lock file centrally, and put it in place before init/plan/apply

i posted my thoughts on a similar upgrade a couple weeks back… https://sweetops.slack.com/archives/CB6GHNLG0/p1610817829068500

Yes your thoughts make a lot of sense. Our infrastructure is composed of 100s of small terraform components each with its own state file. Untli now we always used default version of terraform providers and didn’t worry about versioning. The new approach makes sense but I probably want to keep the same version of providers for all my infra.

I’m not sure with this new approach I’m i supposed to read the changelogs of all providers I use and only update manually? This seems like quite a lot of work because I’m a single devops engineer in our team…

if you weren’t bit by provider versions before, i wouldn’t feel bad about just adding the lockfile in .gitignore

another pain point is that init only adds hashes for the platform you are on now. if your platform and your CI/teammate’s platforms are different (osx vs linux vs windows), the hash changes! so you actually need to run providers lock for each platform

fortunately, everyone uses geodesic to run terraform so never happens.

Please see our explanation here:

https://docs.cloudposse.com/community/faq/#what-is-our-versioning-strategy

For 0.x releases, it’s not conventional to bump major.

thanks Erik.

I’m putting my faith here https://www.infracost.io/

Interesting how’d you find it?

still in early development but awesome community support and look promising, using it already but not much resources, but more than the ^^ one

We are all in AWS and they look like they have a fair amount of resources covered

Now to work out how to hook this into Atlantis

https://www.runatlantis.io/docs/custom-workflows.html#running-custom-commands

You beat me too it

Can’t think of a better way off the top of my head. I don’t think terraform supports negative indexes like in some languages (e.g. split(".", each.value)[-1])

That said, I think it would be a lot more readable if you broke it down into locals.

Hrmm… but with the each there, guess that won’t work either

I suppose you could call reverse and then pick index 0

since you aren’t assigning a default with lookup, you can at least just use the index…

var.mime_types[split(".", each.value)[length(split(".", each.value))]]

(not setting a default with lookup is deprecated, anyway)

You can also use `

var.mime_types[replace(each.value, "/^.*\./", "")]

beat me to the replace option… i’m sure there’s some regex that’ll work

relying on the greediness of .* like that ought to work

I think I like the reverse option, it’s readable to me content_type = lookup(var.mime_types, reverse(split(".", each.value))[0])

either way, maybe leave a comment for your future self or teammates explaining what is happening

haha, good point

alternatively… (to beat a dead horse)

locals {
  mime_extension = "/^.*\."
}

...

var.mime_types[replace(each.value, local.mime_extension, "")]

that feels pretty readable

that’s how golang devs think… you get only the most basic tools in stdlib, and from there everyone reimplements common utils functions over and over in slightly different ways

What would endswith do?

endswith("abcdef", "f") == true

this is a bit of a legit question… example from python:

>>> 'abcdef'.endswith('fed')
False
>>> 'abcdef'.rstrip('fde')
'abc'

Well, you can do that easily enough with Terraform. Yes they do not have that exact function, but my feeling is that if you can do it without too much trouble then it is good enough.

> length(regexall("[fed]$", "abcdef")) > 0
true
> substr(trim("Xabcdef","fde"),1,-1)
abc

I will grant that the subst is kind of a hack, but it works and I would rather see Hashicorp add something that cannot be hacked togeter, such as tr

$ echo 0123456789 | tr '[0-9]' '[g-p]'
ghijklmnop

Lol, I repeat, https://sweetops.slack.com/archives/CB6GHNLG0/p1612219807176600?thread_ts=1612219729.176500&cid=CB6GHNLG0

That would be a good feature request. template_string

You could use a tmp file that you use, dump the template string to, use template file, and then delete the file

Very convoluted but I imagine you could create the file using the local_file resource (https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) and then use templatefile() to template it

ideally the solution would not involve a resource as that involves state and the create/destroy lifecycle

The type of value you pass to this argument must be a set of strings. What value do you think the lookup is returning?

i didnt passed any value for this *vpc_zone_identifier*

values i am passing from seperate file called *clusterProperties.env*

TF_VAR_worker_groups_launch_template=$(cat <<EOF
[{
    name                                     = "worker"
    asg_desired_capacity                     = "1"                                           # Desired worker capacity in the autoscaling group.
    asg_max_size                             = "5"                                           # Maximum worker capacity in the autoscaling group.
    asg_min_size                             = "0"                                           # Minimum worker capacity in the autoscaling group.
    on_demand_base_capacity                  = "15"                                           # Absolute minimum amount of desired capacity that must be fulfilled by on-demand instances
    on_demand_percentage_above_base_capacity = "75"                                          # Percentage split between on-demand and Spot instances above the base on-demand capacity
    subnets                                  = "${TF_VAR_subnets}"                           # A comma delimited string of subnets to place the worker nodes in. i.e. subnet-123,subnet-456,subnet-789
    ami_id                                   = "${TF_VAR_ami}"                               # AMI ID for the eks workers. If none is provided, Terraform will search for the latest version of their EKS optimized worker AMI.
    asg_desired_capacity                     = "3"                                           # Desired worker capacity in the autoscaling group.
    asg_force_delete                         = false                                         # Enable forced deletion for the autoscaling group.
    instance_type                            = "${TF_VAR_worker_instance_type}"              # Size of the workers instances.
    override_instance_type                   = "t3.2xlarge"                                   # Need to specify at least one additional instance type for mixed instances policy. The instance_type holds  higher priority for on demand instances.
    on_demand_allocation_strategy            = "prioritized"                                 # Strategy to use when launching on-demand instances. Valid values: prioritized.
    on_demand_base_capacity                  = "7"                                           # Absolute minimum amount of desired capacity that must be fulfilled by on-demand instances
    on_demand_percentage_above_base_capacity = "75"                                          # Percentage split between on-demand and Spot instances above the base on-demand capacity
    spot_allocation_strategy                 = "lowest-price"                                # The only valid value is lowest-price, which is also the default value. The Auto Scaling group selects the cheapest Spot pools and evenly allocates your Spot capacity across the number of Spot pools that you specify.
    spot_instance_pools                      = 10                                            # "Number of Spot pools per availability zone to allocate capacity. EC2 Auto Scaling selects the cheapest Spot pools and evenly allocates Spot capacity across the number of Spot pools that you specify."
    #spot_max_price                           = ""                                            # Maximum price per unit hour that the user is willing to pay for the Spot instances. Default is the on-demand price
    #spot_price                               = ""                                            # Cost of spot instance.
    placement_tenancy                        = "default"                                     # The tenancy of the instance. Valid values are "default" or "dedicated".
    root_volume_size                         = "50"                                         # root volume size of workers instances.
    root_volume_type                         = "gp2"                                         # root volume type of workers instances, can be 'standard', 'gp2', or 'io1'
    root_iops                                = "0"                                           # The amount of provisioned IOPS. This must be set with a volume_type of "io1".
    key_name                                 = "${TF_VAR_ssh_key_name}"                      # The key name that should be used for the instances in the autoscaling group
    pre_userdata                             = "sudo usermod -l peks ec2-user && sudo usermod -d /home/peks -m peks && echo 'peks ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers.d/90-cloud-init-users && sudo groupmod -n peks ec2-user && sudo mkdir -p /goshposh/log && chmod -R 777 /goshposh && chown -R 1000:1000 /goshposh && echo ${TF_VAR_friday_pub_key} >> /home/peks/.ssh/authorized_keys && echo ${TF_VAR_peks_pub_key} >> /home/peks/.ssh/authorized_keys"       # userdata to pre-append to the default userdata.
    additional_userdata                      = "yum install -y <https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm> && systemctl start amazon-ssm-agent"                                            # userdata to append to the default userdata.
    ebs_optimized                            = true                                          # sets whether to use ebs optimization on supported types.
    enable_monitoring                        = true                                          # Enables/disables detailed monitoring.
    public_ip                                = false                                         # Associate a public ip address with a worker
    kubelet_extra_args                       = "--node-labels=kubernetes.io/role=worker --kube-reserved=cpu=200m,memory=256Mi,ephemeral-storage=1Gi --system-reserved=cpu=200m,memory=256Mi,ephemeral-storage=3Gi --eviction-hard=memory.available<500Mi,nodefs.available<10%"
    autoscaling_enabled                      = true                                          # Sets whether policy and matching tags will be added to allow autoscaling.
    additional_security_group_ids            = "${TF_VAR_worker_additional_security_group_ids}"                                            # A comma delimited list of additional security group ids to include in worker launch config
    protect_from_scale_in                    = true                                          # Prevent AWS from scaling in, so that cluster-autoscaler is solely responsible.
    #suspended_processes                      = ""                                            # A comma delimited string of processes to to suspend. i.e. AZRebalance,HealthCheck,ReplaceUnhealthy
    #target_group_arns                        = ""                                            # A comma delimited list of ALB target group ARNs to be associated to the ASG
    #enabled_metrics                          = ""                                            # A comma delimited list of metrics to be collected i.e. GroupMinSize,GroupMaxSize,GroupDesiredCapacity

  }
]
EOF

)

is this from a public terraform module?

Yes

for thsi value where should i pass the variable *vpc_zone_identifier*

can you link to the public terraform module you are using?

i can’t send that is confidential ?

ok, so the module is private then.

In your EKS module, the value being provided to vpc_zone_identifier needs to be a set of strings. You will need to read the module’s code to understand what value is currently being used and how to fix it.

as of now I am not passing this variable anywhere

variable “vpc_zone_identifier”{ type = string }

Two things:

1. The code snippet in your original error message shows the value is being set with a lookup function. That function has a default value which is probably getting used in this case.
2. The variable specification you just posted requires a value to be set. Terraform would error if you didn’t provide a value. So you must be setting something.

Could you be using the wrong username?

(don’t ssh as root user)

which user should I be using?

AWS documentation says hadoop@…

In the applications I have Spark and Zeppelin

I think amazon linux uses *ec2*-*user*

copy your terraform code here

and your input variables

without that is pretty hard to help you

I have this solution working but using the https://github.com/kubernetes-sigs/aws-load-balancer-controller from within the cluster. Works like a charm

Take a look at this https://github.com/sajid2045/eks-alb-nginx-ingress

The aws LB controller is what I use. Do you mind posting what your ingress yaml looks like?

BTW it looks like I’m not the only one to face this issue: https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/1791.

@OliverS I am using a helmfile based on the cloudposse helmfiles. Take a look here: https://gist.github.com/reixd/914a19f2835690cca36db306025dcc85

Thanks but this is not the AWS LB Controller, it is the nginx-ingress controller (which is used by the older AWS ALB Ingress Controller).

the var is defined as type map(map(map(any)))… believe it or not, any does not actually mean absolutely anything… each object in the map needs to be the same type and have the same attributes

though, i think the root_directory key should not be specified like that… only keys should be posix_user and creation_info

this is a strange way to type the object IMO, since each attribute is required. it would be much more clear to use an actual object({...}) type

the indexing into var.access_points while also using for_each is also strange… i feel like it should just be:

gid            = each.value.posix_user.gid
uid            = each.value.posix_user.uid
secondary_gids = each.value.posix_user.secondary_gids

Thanks for the pointers! I’ll look into tomorrow: it is getting late here

Just confirming that variable formatting works perfect. Many thanks

BigSur killed my screen so I’m a bit skeptical that all the tools will work on a M1

I would wait all of the tooling isn’t quite ready yet. Somewhere there is a GitHub ticket tracker with brew

My friend got one and ended up sending it back because it was too much of a pain

However if you do end-up getting one please update us on your hard won experience :)

it is already sounding far from ideal

Agree to what @pjaudiomv said, haven’t seen / heard a success story for M1 mac used for dev / devops teams . I have heard same return stories just like Patrick’s friend did

i’d go with the m1 and use a local VM or a remote dev environment as an interim solution. the speed and power improvements seem worth it IMO

Also there’s Roesetta2 https://support.apple.com/en-us/HT211861

I think apple wants paying customers to do the beta test and rollout lol

https://twitter.com/mitchellh/status/1328877463078801408

parallels appears to work also… https://twitter.com/bradfitz/status/1354874746953814016

yes, I could do that too

are you setting id_length_limit to 28?

(for the null label module)

Nope not setting that i figured out it was the name variable that forms part of the id. I changed that and it works but need to figure out how to construct the id to want I want

If you want a specific thing for the id, maybe just hardcode it

Following this, @Mohammed Yahya please do let me know if you get this from other sources

is there anything better than terraform yet? that is a question I ask

when comparing community sizes there is nothing to compare to TF

there is many other things to include in the comparison table when evaluating

I agree with you, also We need this ADR to show C-Level why they need Terraform, also compare to CDK and Pulumi which start gaining popular

I get the impression a lot of pulumi’s popularity is pulumi hyping itself

true

I can’t prove that of course, but I see a lot of blog spam and ads from them

well they pioneered CDK, that’s count for them

Oh yah, it might be a great product! I’m not even disputing that

That was an easy one for us. As far as open source tools go, it has the highest market share, the highest level of community support, and the least amount of vendor lock

I’ve learned to word this stuff as “This is the tool your technical team is recommending.” rather than “Is it okay if we use this?“.

Sorry for the off-topic, I would like to ask @Mohammed Yahya if you could share some details about your ADR workflow. I’m interested in how it works for you and your team, how often you do this(for which type of decision), where you store records (git/confluence, dedicated repo/with the code), any tooling you are using. Basically, the question is about your feelings if it really helps you and your team/org. If you have any examples of your ADRs that you can share I will appreciate it a lot.

One of our client tried ADR approach, but it didn’t last long. Probably the majority didn’t see any value in following this and ones who did gave up because they were only a few. But it’s a different story. As for me I like the idea.

We use ADR for breaking the ice and destroy the silo of a specific technology, the team can disagree on choosing between two technologies like the famous one, should we use Jenkins and why ( some hate Jenkins, others not so we break the tie using an ADR) it helps a lot knowing what drive the company choices from the start we initially use a repo called technology-adrs inside that a punch of md files describe an ADR for each choice we need to make

tech-adr
|-- cloud.md
|-- iac.md
|--cicd.md
|--frontend.md
|--backend.md
|--sec.md

now we are thinking to move to stackshare.io for our stack listing and choices, here a sample I just create https://stackshare.io/mhmdio/decisions/105668378243793712 this will make us transparent, easy with on-boarding and the team will understand why we choose something over the other.

but I could not find any decent ADR for Terraform, I want to know why Enterprises using it, working with Enterprise clients it is hard to convince them into move and change the way they dealing with Cloud or migration to cloud.

i was, but i updated to the latest module version and it was fixed

Ah! Didn’t notice.

yes, in tf 0.14, the provider has the ability to mark attributes of resources as sensitive. if you output such an attribute, the output must also be marked as sensitive

These outputs are only generated in the ecs-container-definition module and are then fed into an ecs-alb-service-task

We aren’t outputting them ourselves though

same deal the modules need to treat them as sensitive

I see some module do have the sensitive = true for some of the outputs

Seems like I’ll have to make some PR’s then to add this to the ones I mentioned

I would have assumed that the tests would catch this, or my usecase is just completely different from those tests

tests are hard

Its hard to cover all usecases, I know all too well I’m afraid haha

my default version in atlantis is set for 0.12.30 but im trying to run a plan in a module that has the following block and atlantis cannot seem to interpret as using 0.13 or 0.14. module is applied using tf 0.14.5.

terraform {
  required_providers {
    aws = { 
      source = "hashicorp/aws"
    }   
  }
  required_version = ">= 0.13"
}

source: https://www.runatlantis.io/docs/terraform-versions.html#via-terraform-config

error from atlantis.

Warning: Provider source not supported in Terraform v0.12

  on versions.tf line 3, in terraform:
   3:     aws = {
   4:       source = "hashicorp/aws"
   5:     }

A source was declared for provider aws. Terraform v0.12 does not support the
provider source attribute. It will be ignored.


Error: Unsupported Terraform Core version

Maybe just setting it in the repo atlantis.yaml is a workaround for now

@RB Struggled with this today as well - atlantis an only work with = and not with >= right now. https://github.com/runatlantis/atlantis/issues/1217

There’s a secret replace command that needs to be run

terraform state replace-provider "[registry.terraform.io/-/aws](http://registry.terraform.io/-/aws)" "[registry.terraform.io/hashicorp/aws](http://registry.terraform.io/hashicorp/aws)"

Looks like op might need to replace the null provider tho

this ought to show you all providers from the config and the tfstate, so you can map out what to replace:

terraform providers

geez, thanks so much guys, I did not know these commands, they did the job (I did the replace on null, local, template, and aws)

i do this and have had good luck

rm -rf .terraform/

# upgrade from tf12 to tf13
tfenv use latest:^0.13
terraform init -upgrade
terraform state replace-provider "registry.terraform.io/-/aws" "hashicorp/aws" -yes
terraform apply
tf 0.13upgrade -yes
terraform init
terraform apply

# upgrade from tf13 to tf14
tfenv use latest:^0.14
terraform init
terraform apply

some of it might be extra

¯_(ツ)_/¯

what’s tfenv

https://github.com/tfutils/tfenv

there’s also https://tfswitch.warrensbox.com/Install/

more chars in the command line and fewer stars tho

what are some of the bennies with tfswitch over tfenv ?

haven’t seen tfutils before but seems like its very similar

and has you said less chars haha

Just had a look, both have excellent capabilities if you have to switch between several terraform versions regularly

BTW just saw this in the upgrade docs for 0.14:
Terraform v0.14 does not support legacy Terraform state snapshot formats from prior to Terraform v0.13, so before upgrading to Terraform v0.14 you must have successfully run terraform apply at least once with Terraform v0.13 so that it can complete its state format upgrades.

seems to match my commands above ^

if you have a tfstate s3 with versioning on, you can look at previous versions

i havent used it but this might come in handy https://github.com/camptocamp/terraboard

ah nice, thanks! yea I was inspecting the tfstate hoping to see if there was a lastRun type param that might also include any errors that occured but I don’t think that exists

(we do use versioned s3 state backends)

for last run information, you’d need a CICD for terraform like atlantis

@RB have you ran terraboard in aws? whats the recommended way.. fargate? ECS?

https://sweetops.slack.com/archives/CB6GHNLG0/p1612369406254400?thread_ts=1612369362.254000&cid=CB6GHNLG0

ah I went straight to the github page rather reading

it’s on my todo when time permits

ill be free by Q3 maybe haha

+1 for Terraboard, run it locally for a start using docker

We use terraboard in dev environments, very convenient for developers to understand what’s in their TF states or search resources by type/name

Go ahead. Suggest you post the content in replies on a thread and not as direct messages in the channel, to avoid creating noise for people.

Sure, thanks! Here’s the post on SO. I’m posting it just to avoid duplicating text since I tried my best to describe scenario I’m in there, but I’ll gladly discuss this topic here if anyone is for it, and post my resolution as an answer on SO later on. https://stackoverflow.com/questions/66024950/how-to-organize-terraform-modules-for-multiple-environments

Responded: https://stackoverflow.com/a/66033272/3830413

Nikola thanks for posting! I have the same question: what’s the simplest real-world solution to a basic Terraform setup? The SO answers seem to all say “it’s complicated” Yoni thanks for answering.

Yeah I have added my own answer since my avoidance of terraform workspaces led me to this rat race for which i didn’t find an answer. So as one person in the comments replied, the workspaces were created to solve this issue of multi-env project. I successfully did what i wanted to do, but at the moment with the local state keeping. This all is very muh work (more r&d) in progress, but I think i realized the path i want to go with. I’ll try and widen my answer with an example on the SO answer i created a bit later on.

i recall someone using a combination of import -allow-missing-config and something else, maybe state show?, to write out a near-working config…

here you go: https://asciinema.org/a/VVW2jx7jtFahLEyPObB0mABJX

found it in the slack archive… https://archive.sweetops.com/terraform-0_12/2019/08/#68874db3-6e97-4810-a3e5-6797c15eeab4

much rejoicing

Thanks @loren I’ll take a look

yup, simple, and works wonderfully

Do you have var.apply_config_map_aws_auth = false?

nope, We are using the default “true” value. we found a workaround where removing the resource from the state solves the issue, but we would obviously need a better option. If I set that variable to false, I still need to apply the configMap (only this time separately or manually) right ? so what is the difference ?

If you were setting apply… to false then the provider would not get configured, so I thought that might be what was happening. Seems like a bug in the Kubernetes provider. Did you check for open issues?

yes in that case the configMap will not be created and the provider won’t do anything, In the open issues regarding this problem I only found the workaround with terraform state rm . I can try working with this var, but I still need the configmap executed

You might try setting var.apply_config_map_aws_auth = false to create the cluster, then set it to true and update the cluster. It is possible there is a new race condition or something. @Andriy Knysh (Cloud Posse) would you please review the EKS cluster auth in light of the new Kubernetes provider?

@Michael Koroteev You might try updating to the current eks-cluster module, v0.34.1. I just tried it and it worked for me.

this

dial tcp [::1]:80

looks like it tries to connect to a local host cluster (and obviously fails)

Right. Question is: does this seem something that could be new due to the new v2.0 Kubernetes Terraform Provider ?

This looks like the same issue I’m having, in my case isn’t ipv6 but still trying to call localhost:

GET /api/v1/namespaces/kube-system/configmaps/aws-auth HTTP/1.1
Host: localhost
User-Agent: HashiCorp/1.0 Terraform/0.14.6
Accept: application/json, */*
Accept-Encoding: gzip

---: timestamp=2021-02-10T00:25:35.225Z
2021-02-10T00:25:35.226Z [INFO]  plugin.terraform-provider-kubernetes_v2.0.2_x5: 2021/02/10 00:25:35 [DEBUG] Kubernetes API Resp
onse Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 503 Service Unavailable

@Andriy Knysh (Cloud Posse) @Erik Osterman (Cloud Posse) looks like a lot of people are having issues with EKS-cluster since the new v2.0 Kubernetes Terraform Provider. I tried but could not reproduce the problem. Let’s put our thinking caps on.

Behavior suggests

data.aws_eks_cluster.eks.*.endpoint

Is null. Could be a bug where the provider is not waiting for the data.

I wonder, @Michael Koroteev @nnsense Are you using Terraform 0.14? Seems it is more susceptible to https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs#stacking-with-managed-kubernetes-cluster-resources See also: https://github.com/hashicorp/terraform/issues/4149 .

See also https://github.com/hashicorp/terraform-provider-kubernetes/issues/1095

Yes I’m working with terraform 0.14 I will try using the latest version of the module and let you guys know. anyway, I checked in the state itself and the data.aws_eks_cluster.eks.*.endpoint field contains the actual value.

Me too (14.5 or something). It’s strange, yes the ENDPOINT variable if set as output is indeed showing the right value, but if I enable TRACE it clearly shows localhost as endpoint (it even shows my local server answer, the same I get if I run curl post myself locally). Basically I have the same config as your complete examples, with the addition of the iam role thing. If I apply all good. If I then refresh, it shows unauthorised with the 503 to (it seems) localhost. If I destroy, exactly after having destroyed the nodes, it fails again with unauthorised , then I run it again and it destroys the rest of the things until only one module exist, that I usually delete with rm : module.eks_cluster.kubernetes_config_map.aws_auth_ignore_changes[0] (leaving kubernetes_config_map_ignore_role_changes = true) Should I try with v0.13?

@nnsense we did test the new changes with k8s provider on TF 0.13 (did not test it completely on 0.14)

yes, please try 013, we are going to deploy with 0.14 and find the issues if they exist

Thanks! I’m trying as we speak

“As we speak” was incredibly optimistic… 10 minutes to destroy the old one, 10 minutes to create the one with tf0.13,, :D

Still creating... [6m20s elapsed]

Hey, with tf 0.13 refresh worked without throwing that error… mmmmhh… Interesting!

ok thanks @nnsense

we are going to deploy/destroy with TF 0.14

Thanks!!

Re: kubernetes provider issues with terraform 0.14

Error: Get "<http://localhost/api/v1/namespaces/kube-system/configmaps/aws-auth>": dial tcp [::1]:80: i/o timeout

Posted more updates here: https://github.com/cloudposse/terraform-aws-eks-cluster/issues/104#issuecomment-792520725

Seems like no good fix is available yet. Anyone solve this?

Note the fact that Terraform has an example for how to do auth with the v2 of the provider: https://github.com/hashicorp/terraform-provider-kubernetes/tree/master/_examples/eks

Creation, deletion, and upgrades worked without any issues for me, using that code. Buuuuuuut I have no idea how complex the migration path is

@Andriy Knysh (Cloud Posse)

thanks @Vlad Ionescu (he/him) I’ll look into that today

Note the fact that my comment saying that got 3 reactions on GitHub No idea why, but beware!

I still maintain that it worked fine for me on a new cluster scenario. And it worked fine for the students on my “Running containers on AWS” course

no worries, we’ll figure it out. Our current module also works in many cases, but does not work in some for some people

If you have any questions, I’m here!

We have released terraform-aws-eks-cluster v0.37.0 which resumes support for Kubernetes Terraform provider v1. We have rolled back to using the v1 provider in our root modules until the connectivity issue with the v2 provider is resolved. That is the best resolution to this issue we have to offer at this time.

We recommend using terraform-aws-eks-cluster v0.38.0, terraform-aws-eks-node-group v0.19.0, and edit the [versions.tf](http://versions.tf) in your root module to include

terraform{
  ...
  required_providers {
  ...
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 1.13"
    }
  }
}

I’m getting

ERROR: Post “http://localhost/api/v1/namespaces/kube-system/configmaps”: dial tcp 127.0.0.1 connect: connection refused

reliably on initial creation.

I’m using 0.38.0 of the module and the kubernetes provider is 1.13.1. I’m only creating the cluster (no nodegroups) initially as I’d like to keep my workspaces smaller and focused in Terraform Enterprise.

I used an earlier version of the module and never had this issue.

Module is pretty challenging to use in the current state.

Oh, and the error is in the resource “aws_auth_ignore_changes”.

My issue turns out to be wrong versions. Specifically, I switched to the versions in the examples/complete version 0.38.0. I think the change that fixed it for me was kubernetes provider >= 2.0.

any of namespace, environment, stage, name are optional

you can set just one of them, or a few, or all of them

the module will generate IDs based on the pattern namespace-environment-stage-name, but the order of these are configurable as well

So if I exclude any of them they won’t be used?

yes

at least one is required

Sweet, thanks so much for the advice.

note that we use the pattern namespace-environment-stage-name to uniquely identify all the resources for an organization (hence the namespace which is usually an abbreviation for the org)

this is useful for consistency (ALL your resources are names the same way), and also for naming AWS global resources like S3 buckets (which are global and the names could not be reused b/w accounts)

I also assume namespace comes into play for eb url?

for all global resources. if we name all of them using the pattern, there is a very little chance of naming conflicts (if somebody else is using the same pattern and the same values, which is very unlikely)

is there a way to use a different pattern for env name and global resources?

I’m just a little stuck since I’m working in old code, and a whole lot else will need to change if the env name has to change.

all the values are optional

you can just use name which can be anything

you can also change the delimiter from - to whatever you want

You can also use label_order to change the order of the labels.

everything in state file is not production grade ready

I would very secure the s3 bucket that host the state file, and enjoy using the Terraform hype - so the TLS Terraform provider will make it easy to generate Key/certs when needed

How about AWS secrets manager?

Or use vault?

Thank you for your responses. I want to generate the SSH key pair and store in secrets manager. Maybe I’ll use the local-exec

You still need to use aws eks update-kubeconfig to generate your kubeconfig file, and you need to generate it with the --profile you want to use to access the cluster.

You force it to scale down by reducing max_capacity

compact([""]), https://www.terraform.io/docs/language/functions/compact.html

or maybe a for loop if you need more conditions:

[for item in <list> : item if !contains(["", null, false], item)]

Thanks,

But I already have it set here:

target_groups = join(", ", compact([try("${aws_alb_target_group.internal_ecs_tg[0].arn}", null),try("${aws_alb_target_group.external_ecs_tg[0].arn}", null),]))

That was set in locals, but when I’m calling it from aws_cloudformation_stack and there are internal and external target groups, I got [""] .

i can’t really follow that

OK, thanks, indeed it’s complicated, oh love Terraform

Thanks again!

yea, something to help manage it

@loren stumbled across this too https://sweetops.slack.com/archives/CB6GHNLG0/p1610492345159400

Where I think we’d want to see this is in the TACOS

so i didn’t look that closely last time

not working as expected BTW

that’s cool - it will literally consume the statefile and compare that with what’s running.

did you kick the tires?

yes

the issue is it get 97% of the state file as drift, where it should be the opposite

look like it dose not recognize resources in modules. just resources without modules

we’ve been using Fugue for this

Looks like the modules issue is being fixed in the next release.

I would love it if it could just grab all state files from the S3 location, instead of having to specify them one-by-one using –from cli attribute

Hi all, I’m Gerald, part of the driftctl team. Only just joined this slack and thrilled to see your discussions! If you don’t mind, I’ll update some of the posts with more recent informations. Feel free to comment if needed

https://sweetops.slack.com/archives/CB6GHNLG0/p1612544451305500?thread_ts=1612543469.302600&cid=CB6GHNLG0 @Mohammed Yahya the tool now reads resources from modules within the tfstate, (which was indeed not the case before the 0.5.0 release). So you should probably get significantly lower drift % if you retry it.

Thank you, I will give it a shot, and awesome tool BTW

https://sweetops.slack.com/archives/CB6GHNLG0/p1612801916339600?thread_ts=1612543469.302600&cid=CB6GHNLG0 @Igor it’s now possible to read all tfstate files within a S3 or a folder if stored locally. Much more convenient

BTW we had a recurring bug that caused the tool to hang and it seemed to take ages to run while it was basically just stuck. We finally fixed it last week in the last release so I hope you’ll get a better experience now. (we still have issues with SSO causing freezes though as you can see in some of our issues on GH. Working on it )

Thanks @Gerald - welcome and glad you’re keeping us up to date.

Ping me and maybe we can get a demo on #office-hours

Looks like @antonbabenko is doing an episode this week dedicated to this. https://www.youtube.com/watch?v=-BS65owCCmQ

Thanks @Erik Osterman (Cloud Posse)

(we’ll have a terraform module coming out soon - but no specific instructions for azure)

https://registry.terraform.io/modules/cloud-security-labs/sso/aws/latest

Thanks guys

Why is this needed vs tfenv tgenv?

terraenv author here, I created this tool to solve below problems

• single tool to do terraform and terragrunt version management

• available as pip, brew, docker image, osx and linux binaries ( tfenv and tgenv are bash scripts and not binaries )

Why not simply work with them to create a single tool based on their existing code? I don’t have any issue with your tool, just trying to reduce duplicate work.

Sometimes these tools get to their limitations and it is good to create a new tool. Personally speaking I’ve now migrated from tfenv to tfswitch ( https://tfswitch.warrensbox.com/ ) Reason beeing that tfenv did not properly parse *.tf files to detect the terraform version from there for me/us. Additionally tfenv did add considerable overhead in time for executing terraform - tfswitch ( incombination with direnv ) works almost instant to switch

Quite often https://xkcd.com/927/ happens

https://blog.gruntwork.io/how-to-manage-terraform-state-28f5697e68fa

But I could azure blob storage without any db help and locking works

is not enough create a locking file on s3?

I’m afraid I don’t have any knowledge of azure services. I have always used AWS and with TF been using S3 as remote backend for state files and dynamodb for locking.

https://www.terraform.io/docs/language/settings/backends/azurerm.html

This backend also supports state locking and consistency checking via native capabilities of Azure Blob Storage.

thats why

Don’t talk too loud or the next version of the provider will require a lambda too

Azure blob has a native “locking” concept, while S3 does not. That’s why the AWS backed uses a second service to implement homegrown locking

https://docs.microsoft.com/en-us/rest/api/storageservices/lease-blob

Do you have var.apply_config_map_aws_auth = false?

Generally, no, Terraform uses moderately strict typing most of the time. There is very little you can do with a value that might be a string or a list. Usually with these options you use a list of strings and just have one string in the list.

Instead of doing this by terraform-templating the CF, can you write the CF to accept an input parameter and use a CF condition to set AWS::NoValue?

No Because, both values in condition must exist, but in some cases I do not have target group.

the CF parameter list-type accepts a comma-separated string. you can detect an “empty” list in a CF condition with something like this:

Conditions:
  UseTargetGroupArns: !Not
    - !Equals
      - !Join
        - ''
        - !Ref TargetGroupArns
      - ''

and then in the CF ASG resource:

      TargetGroupARNs: !If
        - UseTargetGroupArns
        - !Ref TargetGroupArns
        - !Ref 'AWS::NoValue'

on the TF side, you pass in either an empty string, or a comma-separated string. because it’s a string either way, the TF conditional syntax will work

Thanks, Ioren, but it’s late for that, I’ve managed to set it up via templates.

I find CF very hard and do not understand it at all.

Thanks a lot!

the CF parameter for the target group looks like this:

  TargetGroupArns:
    Default: ''
    Description: >-
      Comma-separated string of Target Group ARNs to associate with the
      Autoscaling Group; conflicts with LoadBalancerNames
    Type: CommaDelimitedList

Empty strings didn’t work for me for example, I’ve tried that

i guarantee it works been doing this for years

i’m literally copying from a template i already have

damn, wish I had this 2 days ago, spent my whole weekend on it. Are you also creating ASG via CF from Terraform?

but i totally agree on CF being difficult and hard to understand, especially as the use case becomes more advanced

yes, the ASG is defined in CFN, for exactly the kind of use case you describe, where we want to use the CFN UpdatePolicy, which is a service-side feature of CFN that terraform on its own cannot implement…

here’s the template. pick out what you need, ignore what you don’t… feel free to ask questions if you need a hand… https://github.com/plus3it/terraform-aws-watchmaker/blob/master/modules/lx-autoscale/watchmaker-lx-autoscale.template.cfn.yaml

This is gold Ioren! Thanks, I’ll save this for future. But it looks like template works also for my use!

I just need to sort out some evaluation, but it works with a little hard coding!

Thanks again

I’ve managed to set it up with template. Here’s part of template file:

%{ for tg in tgs ~}
      TargetGroupARNs: [${tg}]
%{ endfor ~}

And in templatefile function I have:

tgs = try(tolist([internal_tg.*.arn[0]]), [])

So if there is no internal_tg, it will skip TargetGroupArns!

that works also… we made it a point to avoid templating in the CF, so folks could, if they want, just use the CF directly without terraform

Lost 3 days But, as I said I find CF very hard to debug, and to be honest I am little afraid. But I guess some people will use your module. I would if I knew.

@endofcake

I’m glad this old blog is still useful

@endofcake saved my ass a year ago, but I wanted to improve to have ALB healthchecks, and also to set grace period to 300 secs

Thanks!

When I last look at this, I ended up doing something similar to https://terraspace.cloud/ and the roles/profiles pattern from puppet. The general ideas are: • app (your tf code for your application) • stack (your environment) • config (it has the per-environment setup) • modules (shared modules across multiple projects) The difference between app and modules for me is that the app defines the infrastructure specific to your application, whereas a module can be shared across multiple apps (for example a tagging module or a label module).

The stack contains the instantiation code for the app per environment. This gets duplicated across multiple environments, depending on the parameters passed.

I havent had much time since then to review this pattern, but hopefully it can help a bit.

You still need to use modules, and calling the same module multiple times with different values, IMHO that is not RY code. it’s like calling the same function in your app code. I tried various ways to implement my IaC, all of them has pros and cons. see what fit your needs, I would start with question: Mono repo Vs Multiple Repos for architecting my IaC?

then You can choose the tools: like Terraform vs Terragrunt Vs [env0 - scalr - spacelift - TFC] choosing the tool will affect how you layout your repo/repos. Personally I would go with Vanilla Terraform, and stacks approach like: • App stack • Data stack • Network stack I called it micro stacks approach - for different envs for sure I have a DRY code, but that give me more control over my envs and that’s fine with me

@Bob I have two recommendations A terraform book from the author of terragrunt https://www.amazon.com/Terraform-Running-Writing-Infrastructure-Code-ebook-dp-B07XKF[…]/dp/B07XKF258P/ref=mt_other?_encoding=UTF8&me=&qid=1612814961 and an article/video https://www.hashicorp.com/resources/terraform-workflow-best-practices-at-scale

#office-hours

@Mohammed Yahya - the approach you described reminds me of terraservices - https://www.hashicorp.com/resources/evolving-infrastructure-terraform-opencredo

Is it similar or am I misunderstanding your approach ?

@Patrick Jahns the slides looks very helpful explaining how anyone starting with Terraform evolve into his own patterns, also read this book, it’s awesome I strongly recommend it https://infrastructure-as-code.com/book/

Thanks for sharing - will add it into my reading list. Just thought your approach sounded similar to the terraservice approach

Thanks I’m glad you like it, actually my journey was similar to them

I suppose we all go through different (similar) stages of learning - by being part of communities like these here I try to skip some of the learnings - only to find myself tipping over some of the pain points eventually at a different stage

I may be misunderstanding, but isn’t this what aws_iam_policy_attachment does?

Specifically, if you look at the documentation: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment

This means that even any users/roles/groups that have the attached policy via any other mechanism (including other Terraform resources) will have that attached policy revoked by this resource.

aws_iam_policy_attachment manages all attachments of a policy, but does not manage all the policies attached to a role. it’s flipped

i care more about the role than where the policies are attached. and being able to detect drift if someone comes along and attaches a new policy to the role and thereby changes its permission-set

you might need to run with trace logging on so you can see what API request/response data the AWS provider is sending. Perhaps there’s a bug and it’s not looking for the same resource you are seeing in the console

please don’t double post questions in different channels. You can link to a message instead to consolidate responses

If that variable is within a string you’ll need to use ${var.tier}:

terraform_state_file               = "s3state/${var.tier}/terraform.tfstate"

thanks Andy, I will check

wee that works: + “terraform_state_file” = “s3state/test/terraform.tfstate”

thx a lot !

Best bet is to share the output for the error you’re getting

All our modules are continually tested with integration tests to verify they work

I understand that. I’m bumping up against an issue of not knowing exactly what’s required to get a working setup. I set logging_enabled to “false” but it still complains of not being able to create the logging s3 bucket

Thank you for the reply BTW

Please share the literal example/error. E.g. https://sweetops.slack.com/archives/CUJPCP1K6/p1612902288011800

Erorr are here: https://pastebin.com/7YNtjPXb

This is the code; https://github.com/gcw/foosa/tree/main/terraform-aws-cloudfront-s3-cdn-master/examples/complete

So this is in no way a terraform problem. The bucket already exists… often this happens if you provision a root module without a statebackend

the example above doesn’t include a state backend, increasing the odds you’ll accidentally reprovision the same resource

HCP provides free state backends for terraform. That will be the easiest way to get up and running.

Ok. Thank you

Now just need to figure out how to take our non-Terraform-managed-AWS-SSO configuration and migrate it

@Erik Osterman (Cloud Posse) Awesome !! very clean and simple

@Yoni Leitersdorf (Indeni Cloudrail) create terraform templates and import terraform import them, I did it last week, here is a sample script I used:

I took a slightly different approach when creating my module to combine assignments and permission sets all in one. https://github.com/glg-public/terraform-aws-single-sign-on

@matt would it make sense to do this maybe in the root of our terraform-aws-sso module? … combining the submodules?

Thank you for sharing, I have been looking into openVPN replacement, I took a look at Hashicorp Boundary, but I choose AWS client VPN, but this one looks awesome, I will give it a shot

@Mohammed Yahya out of interest what EC2 instance type do you use for OpenVPN? We’re using a t3.small with around 50-80 users connecting throughout the day. We’ve had latency issues reported over lockdown (!) but looking at network monitoring from the server, nothing stands out as an obvious cause. I’m considering just trying to increase the instance type to a t3.medium or t3.large to get better baseline network performance.

@Andy I’m using the managed AWS VPN service. AWS Client VPN. way much better performance than ec2 openvpn and more secure

OK. Looks expensive though Or are there smart ways to manage that?

# 100 users
# $0.05 per user per hour
# 8 working hours in a day
# 253 working days in a year
100 * 0.05 * 8 * 253 = $10,120 per year

yes it’s, but worth it for the simplicity, performance and integration with SSO like Okta or AWS SSO

or check Tailscale I just learn about it today

follow @Tailscale on twitter also, the devs are quite active

@loren this is awesome. Stoked to see this — Thanks for sharing.

Unfortunately for me, Tailscale got declined during a secuirty review by a client’s auditing team so I don’t get to use it with that client. But this will be sweet, because I was hoping to have a better way to manage those ACLs than just copy / pasta a json document around.

One day.

bummer on the security review did they at least say what they would have needed to accept it? i wonder if the tailscale team would be interested in that?

Tailscale failed the security review because one of the founders refused to fill out an 90 question security review questionnaire. I talked with him about it and he just said they didn’t have the time and my client’s team just refused to try after that and considered them too small of a company. It was a sad way for them to get rejected.

i kid of course! put the problem out there, this community is awesome

it’s hard to pick where to start a thread when a lot convo happens in the channel, but you can be explicit about starting a thread, just post start thread here, or something and sorry i can’t help more with the eks problem. there are quite a few eks users here though, so i do think someone will be able to help

there is also #kubernetes, and sometimes cross-posting can help, in moderation

Cool, thanks I’ll keep this thread

Yeah I really hope somebody can help, I really don’t know what else to try, the next step would be to fork the module git and try to fix it but I don;t want to end up using my repo and anyway I have the feeling the fix is simple

Let’s see into kubernetes channel

Why not bombarding everyone with my problms after all

it is very easy to fork and use your own work with git:// sources, so that is very viable

For testing yes, to use it at work not so much

and also sets you up to pr the fix, if you figure out it is a bug upstream!

create a work org, and fork it there, then use that for work

Trust me, If I don’t find an easy fix, I will do exactly that

I need these modules to work, I;’m not going to rewrite the whole thing to change them

modules without automated tests have not been upgraded

@matt

I would suggest using the Terraform module syntax to simplify…

https://registry.terraform.io/modules/cloudposse/config/aws/

Awesome that works, thanks!

use for_each? still technically multiple resources, but you don’t have to spell them out individually

thx @loren i will check this out.

but wouldn’t it fail since target_id only accept a single value (string)?

with for_each, the entire resource is duplicated, and the attribute remains a string:

resource "aws_lb_target_group_attachment" "test" {
  for_each = toset([<your list of instance IDs>])

  target_group_arn = aws_lb_target_group.test.arn
  target_id        = each.key
}

i see

thx for the knowledge

if you are creating the instances in the same state, do not use the instance ID in the for_each expression. instead use an identifier that maps back to each instance:

resource "aws_instance" "test" {
  for_each = toset(["foo", "bar"])

  ...
}

resource "aws_lb_target_group_attachment" "test" {
  for_each = toset(["foo", "bar"])

  target_group_arn = aws_lb_target_group.test.arn
  target_id        = aws_instance[each.key].id
}

note how the for_each expression is using the same keys for both resources, and how in the attachment we index into the instance resource object

I originally attempted to use the data source below and feed that into the target_id. i guess can still do this and fee that into the for_each ?

data "aws_instances" "test" {
  instance_tags = {
    env = "stage"
  }

  filter {
    name   = "tag:name"
    values = "xyz*"
  }

technically data sources are ok, as long as they do not themselves depend on resources created in the same tfstate

got it

very helpful Loren thx you!

run it in CICD

add security checks to the pipeline also

also try #terragrunt

@Erik Osterman (Cloud Posse) thanks

See #airship @maarten

one is considered self-managed and uses ASGs the other is for AWS managed node pools

Thanks got it. Despite my comment, I prefer to user workers, but the nodes aren’t joining the cluster. I’ve tracked it down to this error:

Tag "KubernetesCluster" nor "[kubernetes.io/cluster/](http://kubernetes.io/cluster/)..." not found; Kubernetes may behave unexpectedly

The “kubernetes.io/cluster/${var.cluster_name}” = “owned”

tag is not propagating to my nodes.

Is this a bug or user error?

@Erik Osterman (Cloud Posse) QQ: what module you advise to add if I need autoscaling with node-group like the workers module does? https://github.com/cloudposse/terraform-aws-ec2-autoscale-group ? Apparently node-group module isn’t creating scaling policy, or I’m misunderstanding how that works?

autoscaling in kubernetes requires a controller. Doesn’t matter which node group flavor.

look up the aws kubernetes cluster autoscaler

Thanks for taking the time to answer :slightly_smiling_face: I will check that, anyway my issue was my blame eventually, I’ve deployed the autoscaler using the helm chart and I didn’t set the region

the question is more custom_header.value.value might be using reserved keyword?

I wonder if this have to be something like custom_header.value.custom_header_value or something like that

This is tf 0.12.24

I figure it out

the other items need to have all the values

it’s @marcinw

eek thanks!

As much as I like such idea I think I could never trust SaaS to give RO credentials to whole AWS. This could wuickly go wrong and be exploited. Are there any opensource solutions like this?

I agree, I tested it, it’s early alpha, check driftctl

https://medium.com/faun/how-to-catch-your-infrastructure-drift-in-minutes-using-driftctl-d22802998b06

whats the difference to using datasoucre with i.e. ssm parameter store ?

I use null-label and [context.tf](http://context.tf) also in some non-cloudposse things where it makes sense to be able to pass just that (the full context) between various modules. It’s neat.

Ah ok. But you can also just use it without [context.tf](http://context.tf) , right? Or does that not make any sense? Though in that case one would have to define stuff like the namespace and so on as variables manually

I add [context.tf](http://context.tf) to every module I want to be able to accept incoming context…

It does both the variable + output additions all-in-one, which is the neat thing about it.

Alright, thanks for the help

This probably doesn’t mean much to you but an example:

module db_context {
  source = "../db-context"

  kubernetes_cluster            = var.kubernetes_cluster
  cluster_identifier = var.cluster_identifier
}

module service_context {
  source = "../service-context"

  context   = module.this.context
  delimiter = "_"
  kubernetes_cluster   = var.kubernetes_cluster
}

resource random_password default {
  length  = 32
  special = false
}

resource mysql_database default {
  name = module.service_context.id
}

# ...

all of the involved modules have the [context.tf](http://context.tf) copied in. Makes it really convenient.

I was asking because I’m setting up the infrastructure for a product where every env / stage has it’s own account and I was wondering if I should adapt my TF code to use null-labels. Kind of becomes redundant if everything atm is separated by account already to still have the environment in the id values

more details here: https://archive.sweetops.com/search?query=terraform-terraform-label

the [context.tf](http://context.tf) pattern is a game changer for working with a lot of modules.

https://www.youtube.com/watch?v=V2b5F6jt6tQ

We’re probably at or near the point we can sunset terraform-terraform-label . I believe :100: of our HCL2 modules use terraform-null-label now so we can use the [context.tf](http://context.tf) pattern. @Maxim Mironenko (Cloud Posse)?

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_organization_admin_account

@matt heads up

• does it create multiple resources that all work together to create a final “thing”

• am I going to use this again

• does it need standardization of names, tags, etc

Similar logic to converting code to a library. The first time, write it in-place, hardcoded. The second time, copy and paste. The third time, consider common code.

If we need to change this later, what are the extension points and possible migration options (tfstate?)

We haven’t wrapped any cloud posse modules but considered it… Until for each came along of modules and thus now we definitely won’t.

Takes me 10 mins to write it anyway

if it takes 10 mins, then there is no need for such a module

a good useful module takes a day to write + examples, tests, docs etc.

and since modules are supposed to be reusable in many projects by many people, there is no way around that ^

otherwise it will create more problems than it solves

Modules give you an opportunity to write tests and version smaller components of your infrastructure. I feel that is very valuable. But I would recommend reusing high quality community modules as much as possible. As @Andriy Knysh (Cloud Posse) mentions, maintaining good modules takes a fair amount of work, and specialization/experience in some obscure terraform details that may not add business value for you

yes, it’s similar to any other programming language. In your own code, you use functions to aggregate some common functionality and make the rest of the code DRY(er). But functions in a public library are completely diff things, they are for public consumption and everything else matters even more than the code itself (docs, examples, tests, tutorials, etc.)

If we need to change this later, what are the extension points and possible migration options (tfstate?) Well, there are good manual escape hatches for converting a resource from manually managed to part of a repo, eg terraform state mv. The workflow looks like this:

1. Migrate your TF code from aws_rds_cluster resources to cloudposse-rds-cluster module
2. Run terraform plan and see what your RDS resources get shifted from, eg from aws_rds.cluster.main to module.rds.aws_rds_cluster.primary[0]
3. Run terraform state mv $old $new and then re-run terraform plan to see how many changes the module still wants to make. You’ll often find modules want to change things that require a rebuild, like the name or other important stuff. If you own the module it’s easy to add lifecycle directives to ignore changes in name. If you don’t own the module, this brings me to my more important point:

It’s not always worth migrating existing pre-module uses to use the module, unless the fit is 100%. Nothing is worse than an in-house module with 100 options each used by a single consumer.

#office-hours

https://www.youtube.com/watch?v=nBcum5BIR9g

Use one of the subnet modules

A vpc module will make a new vpc

Oh I never thought of that

Let me try to use the subnet modules

I am looking at 2 cloudposse subnets

1. https://github.com/cloudposse/terraform-aws-dynamic-subnets
2. https://github.com/cloudposse/terraform-aws-named-subnets

as far as I can tell, they are identical to a large degree

which one should I use?

Use whichever fits your use case best. I usually use the dynamic one

ah got it thanks

unrelated: @roth.andy do you think we should create a terraform-aws-static-subnets module for manually defining subnets?

Is that not what the named subnets module does?

Idk I’ve never used it