#terraform (2020-11)

I don’t think a terraform plan will catch those kinds of conflicts.

Exactly right, you would need to configure vault to assume the role, not your terraform provider

Or write your script to assume the role, which we’ve done before. Gimme a sec to find an example….

here’s the setup in python: https://github.com/plus3it/terraform-aws-tardigrade-security-hub/blob/master/modules/accepter/security_hub_accepter.py#L106-L119

Cheers again mate. Was just thinking I’m going to make my error print the output of get caller identity

and here’s how we get that role_arn from the aws provider… https://github.com/plus3it/terraform-aws-tardigrade-security-hub/blob/master/modules/accepter/main.tf#L26-L35

Ah this makes sense. Will try shortly :)

So I got assume role working which is great, I hadn’t done that before. But now I think my problem is that i am returning a map of map (see above), not a map of string values as specified in the docs. Will need to flatten things out which is a bit messy looks like. Thanks for this

Yep, doesn’t error out when I get rid of nesting

> data.external.this.result
{
  "Device" = "tmpfs"
  "FSType" = "tmpfs"
}

You can return a json-encoded object, then use jsondecode() on the result

Ah, even with nesting? I think all of the values need to be strings per the docs. Though this would be great! https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/data_source

Yes, you create a map where all of its keys have json-encoded values, that makes them all strings!

Ooooh damnnnnn. Will give it a crack tomorrow

Or, at that point, a map with a single key and your entire complex structure json-encoded as the value… Maybe a bit easier to refactor that way

Smart :)

Didn’t feel like waiting, got it working, I owe you a beer :)

Glad it turned out

First, I would recommend using terraform-null-label. We started terraform-terraform-label as an alternative for terraform-null-label that would not use the null_resource ; however, since 0.12 shipped with many new featuers, we ended up dropping null_resource from the null label (so the name of the module is a bit of a misnomer now).

In the new null label module, we support a context variable. This makes it very easy to define many label names.

See this example: https://github.com/cloudposse/terraform-aws-dynamic-subnets/blob/master/nat-instance.tf#L1-L7

Hi @Erik Osterman (Cloud Posse) thanks for your reply. I’m looking on to terraform-null-label module but can’t find if that (and how) will work in my scenario. When I want to have one resource:

resource "aws_cloudwatch_metric_alarm" "elb-5xx-anomaly"

with multiple alarm_name taken from label module and multiple LBs defined in variable:

      dimensions = {
        LoadBalancerName = var.monitored-elb-ids[count.index]

How to configure it?

how to make all this work yikes. How kubernetes works? That’s more than a few slack messages lol

or did you have anything a bit more specific

Excuse me, I explained wrong, my English is not native, I take the opportunity to say that if something seems rude, it is not my intention

I am trying to automate the deploy in terraform

the cloudposse modules helped me a lot

But, in my understanding, for an application to work I have to create a listener, give that listener a rule that points to a target group (I always did it this way)

My problem is that I think I’m in a bad way or I can’t make it work well

For example, if I use node groups, I cannot give them the necessary security groups

If you have a question about something specific, like a particular module you are using, that will help get you the right answers. The best way to get the answers you are looking for on most OSS slack workspaces is to ask targeted questions that people are able to answer quickly. Asking “how does all this stuff work” generally just gets ignored since people don’t have time to walk you through everything

I think my problem is this type of networking within amazon or maybe the operation of eks itself, so I asked if there was an example of how to use the modules up to that point

I never asked a specific question, just asked if there was a more detailed guide

Ask and you shall receive

https://aws.amazon.com/blogs/apn/aws-networking-for-developers/ https://www.youtube.com/watch?v=hiKPPy584Mg http://aws-de-media.s3.amazonaws.com/images/AWS_Summit_2018/June7/Coral/AWS%20Networking%20Fundamentals.pdf

As far as the actual CloudPosse modules, the README that is in each repo is the documentation for that module

Most (all?) of them also have a very good example in an examples folder

Let me understand, I ask about specific terraform modules and eks in particular, I explain that I know how to make it work, but I don’t understand how to make it work with the cloudposse modules and do you give me amazon networking tutorials?

I can make it work by hand, that’s not a problem

Yes, the modules have good examples on how to create the cluster, but not how to make any services available

Does what I’m saying make sense? Or am I letting something go

• What kind of services are you running?

• How do you want them made available? Do you want them accessible from the outside internet, or something else?

• What do you want to use to get traffic in? ALB, ELB, something else? None of this stuff has anything to do with the cloudposse modules, they just build the cluster. This is all “how do I do this stuff in Kubernetes”

• Are you using any kind of service mesh? or nginx-ingress controller? Or alb-ingress controller? Gloo? Traefic? Any of the other dozen ways to do ingress and routing in a k8s cluster?

Or just creating a k8s Service of type LoadBalancer? (this is the easiest way to start, but one of the most expensive)

https://kubernetes.io/docs/concepts/services-networking/

@Sebastian Borrajo you question is how to deploy apps and services on the EKS cluster (including apps and system services like nginx-ingress etc.). This is a separate topic, but in short, you can use helm and helmfile to do it

https://helm.sh/

https://github.com/roboll/helmfile

can I use helm to solve these networking problems I’m having? thanks, it’s a very good way to keep going

@roth.andy for the moment I want to use as simple as possible, simple web app, outside, alb

The Ghost helm chart is a simple blogging app. I’ve used it often when doing simple deployments to k8s. It has other optional stuff like databases and Ingress that you can enable if you want to test that stuff too

If you want to use ALB check out https://github.com/kubernetes-sigs/aws-load-balancer-controller

https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html

The one it comes with by default creates ELBs

a.k.a. Classic Load Balancer

I’m going to read about both options, thanks a lot to both! With alb ingress I tried a tutorial from amazon but referring to cf stacks and get stuck, this could work

now that ALB ingress v2 is released, I’d go straight for that one. .

@Sebastian Borrajo Hablas español?

@jose.amengual Si! el ingles me cuesta un poco, pero lo intenteo de llevar con lo poco que se

@Erik Osterman (Cloud Posse) Thank you! I’m going to try version 2

TLDR To those english speakers , this is where the conversation will be encrypted in spanish from now on

Lo que te decian mis compañeros es que los modules de cloudposse setean todo lo que necesitas para un EKS cluster

en la carpeta examples/complete vas a ver un ejemplo que crea todo

incluyendo las redes VPC, etc

despues tienes que hacer un deployment y eso lo puedes hacer con helm que es lo mas facil

pero para cualquier EKS cluster lo minimo que necesitas es tener un VPC, subnets, nat gateways y todo lo que corresponde desde el punto de vista de redes

y eso no es solo cierto para EKS

NO HAY producto en amazon que no necesite un VPC

Cloudposse tambien tiene modules para eso con ejemplos

(i remember we started #terraform-es but it didn’t get much traction)

Si! yo monte todo con el ejemplo puesto en cloudposse, pero mi mayor traba fue como deployar el producto mas tarde, tengo entendido que helm te resuelve estos problemas de ruteo no?

(por estos problemas me refiero a que por ejemplo, para deployar una app yo tengo que hacer un listener, un listener rule y a eso ponerle un target tengo entendidor que ALB ingress resuelve este problema)

si esa parte te la hace el helm chart

pero de una u otra manera te tienes que leer el manual de como deployar apps en K8s por que en algun minuto vas a tener que modificar el chart o el deployment para que funcione de la manera que quieras

Muchisimas gracias pepe, voy a indagar sobre eso

@Erik Osterman (Cloud Posse) yes, there is not much traction in it

una ultima pregunta, por manual te refieres a cual? es un link? un libro?

me refiero a la documentacion general de como hacr deployments en k8s

cualquier tutorial sirve

despues de que uses el helm chart y veas lo que creo

vas a tener que entender lo que hizo para poder modificarlo

a veces el helm chart hace todo lo que necesitas

it should create an A record in the DNS zone and point <local.domain.local.namespace_domain> to [s3-website.ca-central-1.amazonaws.com](http://s3-website.ca-central-1.amazonaws.com)

So this is working as expected then? Hmm, the site doesn’t seem to come up despite everything being there and the direct s3 endpoint working

do you see the A and AAAA records in the DNS zone?

A record

try to use some external tool like https://mxtoolbox.com/DnsLookup.aspx to check the record

Well I destroyed and recreated the whole thing and now it essentially just says Not found

Despite the s3 endpoint having files being hosted

DNS has TTL

Guess I gotta be a little patient haha

if it cached somewhere, you’ll not see the new record for the TTL time

Alright new update, it just times out now

Any ideas @Andriy Knysh (Cloud Posse)?

how do you access the site using the A record?

can you share it?

I’ll dm ya

Something I have used that has been pretty good is to just grep the output looking for hashes.

terraform plan | grep "#"

yea, there was also scenery, but it too was not updated.

with 0.14 they are also improving plan output

i would be weary of any tool right now because the plan output is seemingly changing with every new version

yeah. I hoped some tools would use the JSON representation of plan, which I’m assuming hasn’t changed so much

That’s true…

I just tried 0.14 and the output is great though

Terraform will perform the following actions:

  # module.eb["770"].aws_elastic_beanstalk_environment.main will be updated in-place
  ~ resource "aws_elastic_beanstalk_environment" "main" {
        id                     = "XXX"
        name                   = "XXX"
        # (17 unchanged attributes hidden)

      + setting {
          + name      = "HostHeaders"
          + namespace = "aws:elbv2:listenerrule:SharedAlbRedirect"
          + value     = "XXX"
        }
      - setting {
          - name      = "HostHeaders" -> null
          - namespace = "aws:elbv2:listenerrule:SharedAlbRedirect" -> null
          - value     = "YYY" -> null
        }
        # (86 unchanged blocks hidden)
    }

Those 86 unchanged blocks used to take up sooo much space! Now my diffs fit in a single terminal window.

that’s awesome! I can also be posted as comments now to PRs too since sensitive data will can be filtered out

I found this tool which creates a colored JS Graph which I can study in the browser. Added to a make target

I’m having trouble finding in the state the problematic resource so I can remove it

obviously I do not want to remove my cluster

I can go up to 0.31.0 version of the module so far no issues

so… here are the steps to solve this: (yes, it’s b/c of the removal of the provider from the very old label module)

1. Copy the module locally

1. In the module, add the new version of the label module w/o removing the old one

1. Place the module into modules folder in your solution and reference the module (instead of the remote one)

1. Update all the code to use the new label.id etc.

1. terraform plan should shown no changes w/o destroying anything else

1. terraform apply

1. Delete the old label from the code

1. terraform apply

is there a easy way to find exactly the provider used in the submodule, like in this case the label module?

Yea search for this error in the archives

Someone else has a simpler solution I think

https://www.google.com/search?q=site%3Aarchive.sweetops.com+provider+has+been&rlz=1CDGOYI_enTH613TH613&oq=site%3Aarchive.sweetops.com+provider+has+been+&aqs=chrome..69i57j69i58.17871j0j4&hl=en-US&sourceid=chrome-mobile&ie=UTF-8

I am on my phone so it’s hard to search, but I recall seeing someone run some command to migrate the provider from - to HashiCorp

https://archive.sweetops.com/terraform/2020/09/#00e05a19-07eb-42c0-9a5b-f197ac9c49ed

That’s the command I was thinking of

ahhh cool

I’m having a very weird issue with this project

if I use version 0.31.0 of the module with tf 0.13.5 I can run apply no problem

if I use version 0.32.0 the apply command hangs for ever trying to connect to mysql

stays on a look of waiting to connect

I use a mysql user provider

but if I lower the module version to 0.31 it works

very strange

Sounds like it’s related to this PR: https://github.com/cloudposse/terraform-aws-rds-cluster/pull/80

Using inline rules was “our bad” and shouldn’t have made it through code review back in the day. Switching from inline rules to resources, isn’t supported by terraform.

You might need to delete those inline rules first.

interesting

but the weird thing is that the security group had not been changed yet

https://github.com/cloudposse/terraform-aws-rds-cluster/issues/83

this fails on a plan, which I found it weird

so the steps to reproduce is:

1.- run plan with module version 0.31.0 and confirm plan/apply works
2.- change module version to 0.32.0
3.- run terraform init
4.- run plan to confirm it times out again

I have yet to be able to apply with version > 0.31.0 of the module

Is this with atlantis on fargate?

no, this is from my laptop

I even thought it was my connection, restarted and such and same issue

I was going to run it on atlantis to see if is still a problem there

have you tried setting TF_LOG=debug?

yep, the mysql provider tries to connect to mysql all the time

until it times out after 5 min

and if I lower the module version everything works

it is VERY strange

the module does not have anything to do with the mysql user provider

I figure it out

I tried aws provider 2.7 and 3x and nothing

there seems to be a problem with getting the .endpoint (hostname) of the cluster from the module that sometimes could not resolve

so once I changed

provider "mysql" {
  # endpoint = module.datamart_writer_cluster_us_east_1.endpoint
  endpoint = "xxxxxx.us-east-1.rds.amazonaws.com"
  username = var.datamart_db_user
  password = random_string.db_password.result
}

the plan worked

now since the label module changed the cluster identifier changed so now it wants to destroy everything but that is another issue

So without hard coding the hostname, what hostname was it connecting to in the debug output?

localhost

which makes no sense since the module.endpoint was used

but now that I successfully run plan and since the plan wants to destroy the cluster because of a name change , my guess there is a race condition of the plan trying to calculate the new endpoint name instead of using it from the state

or something along those lines

actually the reason why the cluster is being recreated is because of the inline security group stuff but I will follow Andriy guide to fix it

mmm that was not it

the cluster_identifieer seems to be the problem but even if supplied still wants to delete everything

@jose.amengual I recommend to copy the module locally into modules folder, reference it from your code, and make changes locally to find the issues - this will allow you to iterate fast and find/fix the issues

I found the version that is breaking the changes

is 0.32.0

we added the contex.tf

I think this could be

now I know…

is this thing

module.datamart_writer_cluster_us_east_1.aws_rds_cluster.primary[0

the introduction of primary and secondary cluster

the id in my state looks like module.datamart_writer_cluster_us_east_1.aws_rds_cluster.default[0]

By the way, what was the reason behind adding primary and secondary?

https://github.com/cloudposse/terraform-aws-rds-cluster/pull/79

they are not created at the same time, only one or the other

it should not affect any names or IDs

primary and secondary are not a very descriptive/good names for that

those are just for regular cluster (“primary” by itself) or a cluster that is part of a Global Cluster (hence “secondary”)

(any of those can have their own secondary read replicas)

I use Global RDS too, and I was using this module with global rds but I can see how it can make things easier

the primary and secondary were changed in the instance and cluster ids and that is why my plan wants to destroy everything

Sounds like you don’t have permission to modify that zone. I think there is a setting to disable automatic DNS. If not, we’ll accept any PRs to do that.

Thanks Erik. Yes, in the meanwhile I figured that out.

I was trying to create DNS records within a hosted zone created for service discovery (in the Elastic Container Service). Apparently this is not allowed since, as the error says, can only be managed through Cloud Map.

A very simple workaround is to disable the hostname option when creating the module (it is there) and just use the Elasticsearch endpoint directly. Thanks anyway for the reply!

no, we can go with => 2.0

you are welcome to create a PR

Thank you for the fast answer. I’ll definitely create a PR

should look like this https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/blob/master/versions.tf

just add or delete the not needed providers

and post the link to your pr in the pr-reviews channel afterwards

https://sweetops.slack.com/archives/G014YEKDH4K/p1598576380037300?thread_ts=1598366674.030800&cid=G014YEKDH4K

https://github.com/okgolove/terraform-aws-ssm-parameter-store/pull/1/files

ssm module uses the same providers so just copy-paste

(wrong org okgolove)

miss click lol

now it’s better

https://github.com/cloudposse/terraform-aws-ssm-parameter-store/pull/19

some comments

lol

https://github.com/cloudposse/terraform-aws-ssm-parameter-store/pull/20

I’ve literally just raised this, randomly

Aren’t the required_providers in https://github.com/cloudposse/terraform-aws-ssm-parameter-store/pull/19 going to break TF 12?

Ah, I see the Terratest output, you get a warning

are you guys on a race?

I will close this last Pr then, is that ok?

@Mikhail Naletov

closed

(when we run tests, just need to make sure to add terraform/0.13 label)

ohhh I though we did not have to do that since we were keeping compatibility with 0.12

I don’t feel we have to keep 0.12, it’s more a nice to have

since 0.14 is dropping any day, we need to drop 0.12 soon

Thanks folks!

Any chance of getting a release cut? ;D

I can do that

done

Thanks!

Thank you for merging the fix

Did you try setting use_fullname to false?

will try that now. i also see regex_replace_chars which would explain this but it seems to be defaulted to null

actually. that thing i just said seems to be the problem. the default is null in the table but it says:

If not set, "/[^a-zA-Z0-9-]/" is used to remove all characters other than hyphens, letters and digits.

so to close this out, a working example is

module "xxx_yyy" {
  source = "git::<https://github.com/cloudposse/terraform-aws-ecr.git?ref=tags/0.29.0>"
  name   = "xxx_yyy"
  regex_replace_chars = "/[^a-zA-Z0-9-_]/"
}

i would expect that the default character set should be the amazon-supported characters

What is not working? Can you share an error?

Hi it’s not that it’s not working. Since my user data reference a single script file I wanted a way where I can reference either script a or b depending on the type of instant I spin up.

Hope that clear

And did you try doing it with the method you mentioned?

I did but it’s not able to find the script path which I defined in my variable

So it’s giving you an error message?

Yes I’m not in front of my laptop unfortunately

We’ll need the error message to help you. I suggest you send it when you’re at your laptop.

although the error states invalid function i’m pretty sure i’m not using this correctly.

Error: Invalid function argument

  on ..\..\..\..\..\terraform-aws-ec2\main.tf line 135, in resource "aws_instance" "linux":
 135:   user_data               = filebase64("${path.module}/scripts/${var.linux_user_data}.sh")
    |----------------
    | path.module is "../../../../../terraform-aws-ec2"
    | var.linux_user_data is ""

Invalid value for "path" parameter: no file exists at
..\..\..\..\..\terraform-aws-ec2\scripts\.sh; this function works only with
files that are distributed as part of the configuration source code, so if
this file will be created by a resource in this configuration you must instead
obtain this result from an attribute of that resource.

How did you define the variable?

in my variables.tf i’m using

variable "linux_user_data" {
  description = "User data script"
  default     = ""
}

in the values are defined in my non-prod.tf

linux_user_data = "grafana"

and my module is set to use something like this

module "example_test01" {
  source                          = "../../../../../terraform-aws-ec2"

Let’s do a sanity - remove the default from your variable. This will cause TF to break if it can’t find a value for that variable.

this is what i get

❯ terraform.exe plan -var-file="non-prod.tfvars"

Error: Missing required argument

  on main.tf line 1, in module "example_test01":
   1: module "example_test01" {

The argument "linux_user_data" is required, but no definition was found.

i also tried adding the var to the module and same error

actually this time it sees the file but it’s looking in the wrong path

Error: Invalid function argument

  on ..\..\..\..\..\terraform-aws-ec2\main.tf line 135, in resource "aws_instance" "linux":
 135:   user_data               = filebase64("${path.module}/scripts/${var.linux_user_data}.sh")
    |----------------
    | path.module is "../../../../../terraform-aws-ec2"
    | var.linux_user_data is "granfana"

Invalid value for "path" parameter: no file exists at
..\..\..\..\..\terraform-aws-ec2\scripts\granfana.sh; this function works only
with files that are distributed as part of the configuration source code, so
if this file will be created by a resource in this configuration you must
instead obtain this result from an attribute of that resource.

it should looking in the current dir .\scripts\

so i updated to used cwd

user_data               = filebase64("${path.cwd}/scripts/${var.linux_user_data}.sh")

and now i get

Error: Invalid function argument

  on ..\..\..\..\..\terraform-aws-ec2\main.tf line 135, in resource "aws_instance" "linux":
 135:   user_data               = filebase64("${path.cwd}/scripts/${var.linux_user_data}.sh")
    |----------------
    | path.cwd is "D:/Users/sotath/Desktop/repo/mso/terraform-aws-ec2/environment/non-prod/example_test/test"
    | var.linux_user_data is "granfana"

Invalid value for "path" parameter: no file exists at
D:\Users\sotath\Desktop\repo\mso\terraform-aws-ec2\environment\non-prod\example_test\test\scripts\granfana.sh;
this function works only with files that are distributed as part of the
configuration source code, so if this file will be created by a resource in
this configuration you must instead obtain this result from an attribute of
that resource.

it’s the correct location path now. but errors out

D:\Users\sotath\Desktop\repo\mso\terraform-aws-ec2\environment\non-prod\example_test\test\scripts\granfana.sh

exists?

it does

oh wait

why is it picking up granfana instead of linux_user_data = "grafana"

spelling is off

That’s what you passed to your module

It’s visible in the above screenshot

from a few minutes ago

ah hell how did i miss that

i see it now

sweet! it works

omg!

sorry for the troubles

Happy to help

thx you

@Andriy Knysh (Cloud Posse)

i’ll check it out

@Mikhail Naletov reviewed the PR, LGTM, a few comments

@Andriy Knysh (Cloud Posse) hello! I’ve fixed the PR. Could you review again?

bump

@Mikhail Naletov thanks again. I left a few comments

Agree - releases are nice.

Curious if you’ve tried removing that object from the state file and importing?

Yeah. Going back and forth between import and state rm …

about to try destroying the entire state file. … that’s definitely not my preferred option, but I just tried to recreate it with tf 12, and it just worked …

What version of terraform are you on?

I’ve had problems with terraform 0.13.2 and import. I think they’re known / potentially fixed in later patch versions of 0.13

I feel import is pretty badly broken in 0.13, especially and completely when using count/for_each on modules. they’ve closed issues indicating the bugs are fixed in master and will be part of the 0.14 release, but that’s small solace in the interim

I was able to fix my issue by importing with tf12 and then copying over the contents of the state file generated by tf12 into the working state file I was using with tf13. … this leaves me with working with unreproducible TF, but that seems to be the best they can do right now.

Interested to see where this goes, seems hashicorp is trying to improve visibility of their internal priorities and how they merge community contributions… https://www.github.com/hashicorp/terraform-provider-aws/tree/master/ROADMAP.md

Lulz didn’t realize I was still in thread…

It does make more sense posted to the channel …

I love the permission set support. Was just missing that recently.

The data has been accurate so far (2 quarters)

check the release notes of versions between your previous version and the latest

however, that error looks like it comes direct from the AWS API

“InvalidParameter” is a common AWS API error string

Yes whts weried is my previous provider version is 3.13.0 (October 29, 2020)

I will try running plan with TF_DEBUG=1 to rule out the issue

Is that the same as TF_LOG=trace? I also suggest -parallelism=1 which makes the logs much easier to read

yes I suppose

I use -parallelism=100 I will setting it to 1 to debug

Looks like this was a provider bug https://github.com/hashicorp/terraform-provider-aws/releases/tag/v3.14.1

Are you using TF 0.13? You can do something like this:

terraform {
  required_providers {
    restapi = {
      source  = "fmontezuma/restapi"
      version = "~> 1.14.0"
    }
  }
}

It does require you to register it.

or you can place the binary into terraform.d/plugins/linux_amd64, and TF will find it

this is the name format that TF expects: terraform-provider-shell_v0.1.3

It looks like the AWS provider is installed to

.terraform/plugins/registry.terraform.io/hashicorp/aws/3.7.0/darwin_amd64/terraform-provider-aws_v3.7.0_x5

I wonder if I can simply replace the binary with my own…

More docs on this: https://www.terraform.io/docs/extend/how-terraform-works.html#plugin-locations

they are installed in that location, yes

but terraform.d works as well as a static location

I tried to put my custom compile of aws provider in there as terraform.d/plugins/darwin_amd64/terraform-provider-aws_v3.14.0, but Terraform still installed the “real” version from the internet

you prob need to change the name or the version number so TF would not find it in the registry and look into terraform.d (although we did it some time ago and things could have changed)

dang. Why is this so hard? Does Hashicorp benefit keeping it difficult?

Also keep in mind there’s a hash there, it may be comparing it (I don’t know… didn’t read the code):

% cat selections.json 
{
  "registry.terraform.io/fmontezuma/restapi": {
    "hash": "h1:dvLIvjzP1nGHcimSkM4mSLvuJ7yI+3aV/mZAWHu4EXs=",
    "version": "1.14.1"
  },
  "registry.terraform.io/hashicorp/aws": {
    "hash": "h1:3gkfYjOVSHc3g/eXnk/JnRuoYtoDRu1oV3YPmBnuVtY=",
    "version": "3.12.0"
  }
}

Figured it out. You can force install of packages using terraform init -plugin-dir – the plugin-dir argument is required to install “official namespace” packages from a local cache

example [main.tf](http://main.tf):

# order doesn't matter
provider azure {} # only available online
provider aws {} # available in local cache

Your local cache is a directory with the following file:

registry.terraform.io/hashicorp/aws/3.14.0/darwin_amd64/terraform-provider-aws_v3.14.0

Install aws provider:

$ terraform init -plugin-dir terraform.d/plugins
Initializing the backend...

Initializing provider plugins...
- Using previously-installed hashicorp/aws v3.14.0
- Finding latest version of hashicorp/azure...

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider
hashicorp/azure: provider registry.terraform.io/hashicorp/azure was not found
in any of the search locations

- terraform.d/plugins

Then you can install all other providers:

$ terraform init

(well, azure is a bad provider name, if you use a real provider things work )

One trick I sometimes leverage is configuring the resource in the console (by hand) and then importing the resource to see how it was configured. I was not able to find another timeout value in the provider documentation.

The internet

But more specifically you can try the CloudPosse GitHub org

I always expect a bit of cynicism, and arrogance , from all the narcistic, genius people out there who have it all figured out. You mate, are the product of the world we live in today..

thanks for the answer

are you asking if it’s possible to write terraform code that works with 0.12 and 0.13? It is

when you say a handfull is like 10-20, or 100s?

if is 10 20, I will make a list and put them all there and if there is a module that create the group of record per each domain I will for loop over the list and use the module to make it really small and simple

but the same apply if you have 1000s in a json file, then you need an additional step to decode it and iterate

If you manage the data with another tool, json is much easier to deal with than HCL. For humans, you might as well keep it HCL so the system has fewer moving parts

Thanks for the advice!

tag your ami as part of your packer build then in your terraform use a datasource for the ami to look up the ami id

i think you need to use name actually for the terraform datasource (off the top of my head)

but the principle remains

try asking in #packer channel

Hey. Do we know each other?

Thanks for this. i did try a similar approach. but didnt work, maybe what i did is just wrong. so what i did is at the packer json file, i have "ami_name":"ebs_backed_ami_by_packer", and in variables.tf i have

variable "ami_name_filter" {
    description = "Filter to use to find the AMI by name"
    default = "EBS-*"
}

and created a separate ami.tf that have this

data "aws_ami" "ami" {
  most_recent      = true
  owners           = ["var.ami_owner"]

  filter {
    name   = "name"
    values = ["${var.ami_name_filter}*"]
  }

so eventually was able to use it at terraform

resource "aws_launch_configuration" "ASGlaunchconfig" {
    image_id = data.aws_ami.ami.id

it looks to work .

now my issue when ran in gitlab ci, i get this message data.aws_ami.ami: Refreshing state... Error: UnauthorizedOperation: You are not authorized to perform this operation. status code: 403, request id: ebbbd4fe-097f-4f78-b3bd-9ab1abac9f49 Error: UnauthorizedOperation: You are not authorized to perform this operation. status code: 403, request id: cd24c1fd-47b4-47c2-a4b6-8da6371efa9f

Cleaning up file based variables 00:01 ERROR: Job failed: exit status 1

i read somewhere that somehow i have not given packer enough permissions to let terraform use probably the ami i created. i added this on the iam user i used to exec packer and terraform { “Version”: “2012-10-17”, “Statement”: [ { “Sid”: “VisualEditor0”, “Effect”: “Allow”, “Action”: “iam:”, “Resource”: “” } ] }

but not sure / and fully understand what it means. not working too

Unfortunately not….

(we use kinsta.com!)

Thanks for letting me know! Based on the pricing I’ve seen from hosted solutions, it seems like a great business to be in. xD

It’s quite a tough business to be in. I used to run the infrastructure and hosting platform for ghost.org - managing 10’000s of application instances, databases and the customers that come along with them is interesting.

We’d built a scheduler, billing, routing and caching layer out of Ruby, NodeJS, LXC, MySQL and Redis that was, at times, quite funky. We usually got about 200 - 250 sites on a single instance. Databases were “sharded” ( blogId % 2 == 1 ? "db01" : "db02") over 2 sets of primary/secondary MySQL replicas…

About 1200 of the DBs were double encoded UTF8 in Latin1 which was fun to fix…

Business wise, there are many competitors including self-hosting and “1-click apps” like those on DigitalOcean. That’s not necessarily bad though as DigitalOcean love OSS and donate back to many of the apps that they provide.

Then backups… don’t get me started on backups…

@t_humphrey Thanks for that insight. That’s actually the site that led me to the question. Currently I run no where near that size of infrastructure, but on AWS am getting just under 100 sites on their own t2.micro instances for < $1 / instance, without reserve instances. The margin between that and Ghost’s basic package seems quite large..

Self hosting Wordpress is easy. Keeping it hardened and patched is what we pay for. If there’s ever any issue, they fix it and my weekend isn’t ruined cleaning up some hack :-)

What difference is that exactly?

second the request to clarify. done a lot of upgrades from 0.12 to 0.13 and haven’t (yet) seen a big problem with for expressions…

Hmm.. I seem to be mistaken. The problem I was having was within the vpc module, the terraform-aws-alb example had pinned the reference to tags/0.8.1, changing that master seems to have cleared the error I was having.

setup chamber in which way?

you mean setting a user to use chamber or something else?

Yes. When using the reference architecture and creating the ‘child’ accounts it looks like the bucket and kms was setup for chamber but not the user. I am trying to add the user but not on the 0.11 versions that were used during the initial provisioning

Here is what I am having some issues with… I used the reference-architecture repo to get started so everything was based on terraform 0.11 But since then I have setup EKS and ECR which those repos are on TF 0.12+ so I modified my Dockerfile for the child stage to use the newer TF… so I am kind of version split and it is just awkward. Maybe I am just not doing it correct either…

I don’t want to have some of the directories in my conf/ to be TF 0.11 and some to be TF 0.12… that just seems wrong, but maybe that is expected at this point in time??

I use chamber in project with tf 0.13 and 0.12

I do not use the chamber user module

I give access to the kms key to the ecs task/instance profile to pe able to read the secrets

that is how I do it

Yeah makes sense… I am trying to use with codefresh so I don’t think that will work… I think i will have to also have a user as well

that module is the one that is still in 0.11?

It looks like it… I forked it and started trying to upgrade it but bumped into some issues quickly so figured I would ask what others were doing before going to far down the rabbit hole. Terraform is not my strong suite yet, done the basics for a few years but nothing like how cloudposse is doing things so I am still climbing the steep learning curve

sorry I mean to say which module?

Well I started at https://github.com/cloudposse/terraform-aws-components/tree/master/deprecated/aws/chamber (which used to be terraform-aws-root-modules)

did you notice the /deprecated/ ?

it has not been updated in a while

you are basically looking for an upgraded version of this module https://github.com/cloudposse/terraform-aws-iam-chamber-user

you could open a PR and we can review it

I am working on the PR, but having some issues with modules that are used inside this repo. I am going to have to back burner this for a little but will make note to come back to it.

Hi Rel, I have a bit of experience with this. For now we are using a monorepo containing Terraform, Helm charts, Helmfile config and CI config.

If you make sure the configuration is separated well, for example:

repo/
  terraform/
  helm/
    charts/
    helmfile.yaml
  ci/

then it’s not too difficult to filter git history and have your CI watch for changes based on path filters.

and how to do you tag the repo?

Do you also combine infra helm(files) with application helm(files)?

We consider the state of the entire repo to represent the state of all the infra, so each commit can update TF, cluster config and Helm

Because of this I would keep application Helm charts in the application repos

But if the helm chart are in the application repos, the devs may checkout a helm chart in a different version, than the current tagged version available on master.

Yeah so you have to decide who has responsibility over the charts and release cycle - application developers or ops

try posting this code as a code snippet, it will be a little easier to read

Done. Thanks.

I suspect it’s because your top-level context module is being initialised without any useful values

eg, none of the variables that it loads are being set

From what I know of the null label module, you need to set at least one of namespace/environment/stage/name

Hi Alex. Thank you. I set all values for namespace/environment/stage/name. That error is gone. But, I got another error, Error: Error creating Elasticache Replication Group: InvalidParameterValue: Number of node groups cannot be less than 1. Do you have any idea?

I think this question can be answered by reading the README for the module: https://github.com/cloudposse/terraform-aws-elasticache-redis

Hi Alex, It works. Thank you very much!

if is not in the module variables then most probably no but is easy to do with a Dynamic

PRs are welcome

mmm I have never done this but I guess you add the redirect rules in cloudfront instead of the bucket

Actually no, I’ve done it by hand, and you have to do it in the bucket for the redirect to work… In the terraform module I thought when you add the redirect_all variable the bucket would be created with the redirect, but it seems it doesn’t… So I don’t seed what that option actually does…

what is it you’re reading?

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster

https://www.reddit.com/r/Terraform/comments/dvibmk/aurora_database_clone/

https://github.com/hashicorp/terraform-provider-aws/issues/2321

oh it’s an aurora feature

interesting. i’ve used the snapshot and create workflow. clone is new to me

clone is hours faster

makes sense

I clone a 600GB cluster in 3 min

there is a linked pr implementing an underpinning piece… looks active as of a few days ago, so might get some traction on the clone issue… https://github.com/hashicorp/terraform-provider-aws/pull/7031

ohhh I did not see that one

if that gets merged that will solve my problem

oh it wasn’t linked after all. two prs implementing the same feature. i found this one just by searching pulls for “clone”

@jose.amengual fyi aws provider v3.15.0 just dropped with support for the aurora rds point in time restore feature

resource/aws_db_instance: Add restore_to_point_in_time argument and latest_restorable_time attribute (#15969)

https://github.com/hashicorp/terraform-provider-aws/releases/tag/v3.15.0

NO FING WAY…..

I literally hacked my way to do this with local exec

I have been testing it…….

hate it when that happens lol

I was about to merge the PR

I guess I have more work to do

thanks for letting me no @loren

no prob!

this is a half done implementation

they do not allow you to specify the security group

it defaults to the default vpc security group

Dang. So close! Open a new issue I guess

I’m testing it, the docs are not so clear bu I was able to pass the SGs

but I need to see if my data is there now

it works

Nice! Fwiw, instead of depends_on in the instance resource, can you pass through the cluster_identifier from the cluster instance?

Yes, I think that is possible

Did you try invoking the example from the examples/complete?

Yes

@Erik Osterman (Cloud Posse) That’s exactly what i did but i removed the VPC & Subnets from it and then ran it.

@Jeremy G (Cloud Posse) error i am getting is this

module.autoscale_group.aws_autoscaling_group.default[0]: Creating... Error: One of `id` or `name` must be set for `launch_template

`

@aravind.kandalam498 Where did you start? I mean, what is the root module, what module is it calling.

https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/tree/0.7.3

calling the above module.

You are using 0.7.3 with Terraform 0.13?

Yes i tried that and then i went back to 0.5.0 with TF .12

Are you setting mixed_instances_policy ? If so, to what?

well i was not planning to use mixed_instances_policy but this is what i had in the example

mixed_instances_policy = {
  instances_distribution = null
  override = [{
    instance_type     = "t3.2xlarge"
    weighted_capacity = null
  }]
}

I could be wrong, but this seems to me to be an issue with your environment. In particular, cached modules, providers, and or state. Clear out any .terraform or .module directories. Also, try module version 0.7.0 with the latest Terraform 0.13.x

sure will that try that again.

is there a version you recommend with TF .12?

Error: Unsupported Terraform Core version

  on .terraform/modules/autoscale_group.label/versions.tf line 2, in terraform:
   2:   required_version = "~> 0.12.0"

Module module.autoscale_group.module.label (from
git::<https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.16.0>)
does not support Terraform version 0.13.0.

Looks like 0.7.0 is using null-label that needs 0.12.0 and cant run with 0.13.0

Try 0.7.1 with TF 0.13. or 0.6.0 with TF 0.12

will do

Roll back all the way to 0.3.0 with TF 0.12 if you are still stuck.

k

We will fix the module if you find and isolate the bug, but so far the module looks good to me.

Thanks again for your help. i will try it out

@Jeremy G (Cloud Posse) you were correct.

Would you please be more specific? What fixed the issue for you?

Sure! I went ahead and deleted .terraform folder which had the modules as well. I then downloaded tf 0.12.29 using tfswitch and used 0.6.0 root module to build without the mixed_instance_policy.

I also verified modules were working with 0.12.29

Hooray!
Terraform will now support reading and writing all compatible state files, even from future versions of Terraform. This means that users of Terraform 0.14.0 will be able to share state files with future Terraform versions until a new state file format version is needed. We have no plans to change the state file format at this time.

finally

Yeah, this will be interesting. I wonder if the recommended approach of pinning an explicit terraform version in root modules will go away with v14

i believe they are deprecating that terraform hcl block

The version argument inside provider configuration blocks has been documented as deprecated since Terraform 0.12. As of 0.14 it will now also generate an explicit deprecation warning. To avoid the warning, use provider requirements declarations instead. (#26135) hmmm or perhaps just the version arg for providers. i can’t find the terraform hcl block referenced anywhere for 0.14 or 0.15 changelog

You can call out to a script using remote exec (assuming you need to do this on a server)

Thank you. suggested. using “Azure VM Extension resource with a custom PowerShell script

Haha - mostly since when we started it wasn’t available

but the reason I laugh, is this literally came up at standup today

We’re probably going to start switching things over.

What’s the benefit?

Hurrah

IMHO it’s a bit more comfort to read in code

I think our module stats will go through the roof once we do that everywhere

Yeah, quick Google search https://registry.terraform.io/providers/hashicorp/aws/latest/docs#shared-credentials-file

@rei to clarify this is to run remotely on terraform cloud (previously named terraform enterprise)

What about cloud agents?

https://github.com/cloudposse/terraform-kubernetes-tfc-cloud-agent

Using this, you can deploy it kubernetes with IRSA

This way it also runs in your VPC and can manage things like database users with the postgres/mysql providers

(this is an alternative to “using aws creds environment variables yet”, not necessarily a solution to using “profiles”, however, I think that could also work if you mount a configmap with the profile settings, since terraform uses the AWS SDK, it should “just work” )

@btai ok, got it

If somebody has this working… I would love to hear about it.

I’ve had to disable the Terraform extension’s LS. The error throwing was getting annoying.

They’ve mostly fixed the error spam in the past couple of weeks. But yeah, it’s still borderline useless. I don’t get completions beyond the built in vscode “other tokens from this file”

It seems to assume your terraform code is in the root directory, and follows modern standards exactly (eg using the 0.13 provider version constraints, etc)

My colleague was informing me today how well his terraform things in intellij was working.. gee thanks

For me this is the main (if not only reason) to use intellij. There are some plugins for VSCode for Terraform and Terragrunt ( https://github.com/4ops/vscode-language-terraform ), but they are rather limited and provide limited static completion.

I’ve also given IntelliJ a stab for these reasons. It beats having TF docs open in a browser or parsing the provider schema.

I just tried vscode again this morning and it managed to autocomplete (in order) lifecycle, ignore_changes = [ name ] give it another try everyone!

In a single repo workspace?

Most of my auto completion is via dictionary history and not due to functioning language support. Omg look at this though!!? https://github.com/hashicorp/terraform-ls/blob/master/CHANGELOG.md

0.10.0 (Unreleased)

FEATURES:

Support module wide diagnostics (#288) Provide documentation on hover (#294) ENHANCEMENTS

I was testing an internal module repo. So a repo with terraform files in the root directory, and basic terraform init having been run

I just tested a monorepo which has some terraform configurations in sub-directories, and no dice there

man it just keeps getting worse here … falling like a deck of cards hehe

the vscode terraform-ls just added support for docs on hover… https://github.com/hashicorp/terraform-ls/releases/tag/v0.10.0

It’s not great… Like cmon gimme a link please.

Haha I shouldn’t rag on them. They’re trying. I’m sure that’ll get better with time.

Haha yeah that’s not so helpful

still doesn’t work for monorepos. In fact it’s even worse – crashes repeatedly if you make the panels appear. It’s been months now so I guess they have no interest in supporting this

I completely gave up for now … here’s my new combination …

At least it works. And no longer randomly I get “You have no formatter for terraform installed” and other depressing things.

Hahaha not disappointed in you @mfridh — Disappointed in terraform and you having to go to such lengths to have a useable environment in the biggest editor of today. It’s weak on their part. They need to step it up.

It’s slowly getting better. For instance, it doesn’t crash constantly in monorepos now!!

I know what you meant @Matt Gowie .

The completion itself is pretty bad to nonexistent in my current setup but it’s ok. I take stable and predictable over the old situation.

Maybe that warning about using master branch was a real one? I’m shocked .

are you reffering to this var https://github.com/cloudposse/terraform-aws-rds-cluster/blob/master/variables.tf#L185 ?

yup . Am just adding a serverless ? conditional on it …

I think I found another bug in the example too …

family = "mysql5.7 change to: family = "aurora-mysql5.7"

we build serverless using this module without issues

version 0.35.0

Oh, I can too if I explicitly set those variables. So the bugs are very minor, no problemo.

howdy PePe btw

hello

we automatically test on AWS only examples/complete

the rest of the examples are for reference only

thanks for finding the bug with the encryption for serverless

we need to update that example (set the var to true)

like I said, no biggie, but can’t hurt to fix those!

come on @mfridh we want a PR

yeah I’ll do that later.

technically the correct thing could be to have that value not set… would that mean passing an explicit null in the case where this is a conditional?

if serverless set to null if not set to current default

merged and released @mfridh

I’ve got this:

resource "kubernetes_persistent_volume" "sona_efs" {
  metadata {
    name = local.efs_name
  }
  spec {
    access_modes = ["ReadWriteMany"]
    capacity = {
      storage = "50Gi"
    }
    volume_mode = "Filesystem"
    storage_class_name = "efs-sc"
    persistent_volume_reclaim_policy = "Retain"
    persistent_volume_source {

    }
    csi = {
      driver = "efs.csi.aws.com"
      volume_handle = aws_efs_file_system.efs_file_system.id
    }
  }
}

but it doesn’t like the csi block (I also tried putting it inside of persistent_volume_source

found a similar question here https://stackoverflow.com/questions/64630508/persistent-volume-source-configuration-for-terraform-kubernetes-persistent-volum#

I see there is a csi type: https://github.com/hashicorp/terraform-provider-kubernetes/blob/7cc826c9e55af33aee11165a425155e4f8cc2ae1/kubernetes/structure_persistent_volume_spec.go#L972 but for some reason terraform doesn’t like it

I solved it - answered in the stackoverflow post

Good evening, Please can somebody help confirm if this is the best approach to what I’m trying to achieve? It is working but I’ve seen similar things in past version of TF with count that appear to work in the similar way and cause really issues when the source data is changed or reordered. I’m trying to use the below collection of objects to be a data structure I use throughout my code, this is just a stripped back version of a much bigger structure. The aim is to say for each one of the collect_lists entries create a log_group based on the keys and values, dynamically decided if it needs a custom KMS key. If true, create one and then like the key ID to the log_group.

logs_map = {
  files = {
    collect_list = [
      {
        "file_path"          = "C:\cms\logs\iis\W3SVC1\*"
        "log_group_name"     = "cms/{environment}/iis"
        "log_stream_name"    = "{instance_id}_{local_hostname}"
        "retention_in_days"  = 7
        "use_custom_kms_key" = true
        "auto_removal"       = true
      },
      {
        "file_path"          = "C:\maint\logs\iis\W3SVC2\*"
        "log_group_name"     = "maint/{environment}/iis"
        "log_stream_name"    = "{instance_id}_{local_hostname}"
        "retention_in_days"  = 7
        "use_custom_kms_key" = true
        "auto_removal"       = true
      },
      {
        "file_path"          = "C:\cms\website\App_Data\Logs\*"
        "log_group_name"     = "cms2/{environment}/umbraco"
        "log_stream_name"    = "{instance_id}_{local_hostname}"
        "retention_in_days"  = 7
        "use_custom_kms_key" = true
        "auto_removal"       = true
      },
    ]
  }
}

resource "aws_cloudwatch_log_group" "loggroups" {
 for_each = { for key, value in var.logs_map.files.collect_list: key => value }
  name = replace(each.value.log_group_name, "{environment}", var.environment)
  retention_in_days =  each.value.retention_in_days
  kms_key_id = each.value.use_custom_kms_key == true ? aws_kms_key.log_group[each.key].arn : ""
 # tags = var.tags
}

resource "aws_kms_key" "log_group" {
  for_each = { for key, value in var.logs_map.files.collect_list: key => value if value["use_custom_kms_key"] == true }

  description             = "KMS key used to encrypt log files. One per log group. ${element(split("/", each.value.log_group_name), 0)}"
  deletion_window_in_days = 30
  enable_key_rotation     = true
  #tags = merge({ Name = "${element(split("/", each.value.log_group_name), 0)}-default-kms" }, var.tags)
}

While this works, it feels far too risky and susceptible to error if the items in the list change order. Am I mistaken when I thought terraform 0.13+ changed the way it stored items with the state? Rather than being array based index: aws_cloudwatch_log_group.loggroups[“0”] aws_kms_key.log_group[“0”] It used a name as a unique identify instead? Assumedly something like: aws_cloudwatch_log_group.loggroups[“mylog”] aws_kms_key.log_group[“myotherlog”] So when used within a for_each loop the resources created would be easier to refer to later on and it didn’t matter if the order changed. Am I completely way off with my resources configuration or even my data layout? Should I be doing the above in a different way. Thank you for taking the time to read this. Sorry, its was so long.

if you use count on a list, and the items in the list change, then TF will try to recreate everything (this could be ok in some cases, but not acceptable in other cases)

for_each does not have that problem

use for_each whenever possible (especially when you create many instances of the same resource)

also, resources with count are lists, so you have to use a list index to get one item

resource with for_each are maps, so you have to use a map key to get one item

https://www.terraform.io/docs/configuration/resources.html#referring-to-instances-1

Hi Andriy thank you for taking the time to reply and for the information.

I get count and for_each are different and I’ve used them for a while but I’ve never need to link between the resources like I’ve done in my example.

Given what you’ve said, is it fair to assume that the each.key of the newly created KMS will always be in sync with the each value of the log group?

Sorry if it’s obvious, I’m a little paranoid that I set this up and somebody comes along and adds a new value to my data structure e.g log group number 4 and all of a sudden TF updates the existing resources to use the wrong keys.

To put you in the spot. Would you be happy doing it as I have or would you restructure in some way.

Again thanks in advance.

i think your concern is coming from an ill-advised for_each expression…

for key, value in var.logs_map.files.collect_list: key => value

key is not an actual key in a map here. it is the index of the list. using the index of the list in this way makes the for_each subject to all the same problems as count

you could instead do something like:

for item in var.logs_map.files.collect_list: item.file_path => item

note the use of item.file_path to create the key of the map. this becomes the resource label/identifier in the tfstate. you don’t have to use file_path, just any attribute that is unique over all the items. in all lists of objects, i often require a name attribute that must be unique, and use that as the for_each key

Okay, I’m going to need more coffee. So, are you suggesting that a better data structure would be to do away with the list which I used to symbolise each log_group. Eg one item for each log I want. And change it to map/object so I have a key rather than the numerical index. Which would help do the matching later to ensure “key A”match “log group A”.

Sorry Loren, if I missed your point.

no change to the data structure is necessary, just the for_each expression

Okay, I’ll go have a go. Would that be in both log group and the kms key or just one or the other ?

both, they should use the same key so you can index into the map the way you have it: kms_key_id = each.value.use_custom_kms_key == true ? aws_kms_key.log_group[each.key].arn : ""

thank you, I’ll go do some more testing.

using file_path as the key, this makes your resource address look like:

aws_cloudwatch_log_group.loggroups["C:\\cms\\logs\\iis\\W3SVC1\\*"]

if you do not like that resource address, then you might want to update the data structure with a new attribute that you use as the key… e.g.

logs_map = {
  files = {
    collect_list = [
      {
        "name"               = "cms_w3svc1"
        "file_path"          = "C:\\cms\\logs\\iis\\W3SVC1\\*"
        "log_group_name"     = "cms/{environment}/iis"
        "log_stream_name"    = "{instance_id}_{local_hostname}"
        "retention_in_days"  = 7
        "use_custom_kms_key" = true
        "auto_removal"       = true
      },
      ...
    ]
  }
}

which adds a unique name attribute to each item. and then your for_each like:

 for_each = { for item in var.logs_map.files.collect_list: item.name => item }

and your resource address would look like:

aws_cloudwatch_log_group.loggroups["cms_w3svc1"]

hopefully that better demonstrates the relationship between the for_each expression and your resource address…

@loren you have firmly hammered the nail in to my thick skull, its finally clicked. Thank you for talking the time to answer the question and my thanks also to @Andriy Knysh (Cloud Posse)

I can clearly see now the naming is working and that it is indeed no longer based on an index count and It was my own doing that was name it in that way.

i’ve converted a lot of code to for_each, it is totally worth it. good luck!

Hello, I have read all the thread, and wants to thanks for taking time to address the question and explain every thing. I have runned in the same need previously and made it by changing datastructure to a map of map

managing the map of objects directly certainly works also (instead of converting the list to a map with the for expression). i haven’t quite figured out why i prefer a list of objects, but something about it just works better for me

that’s right. If you check the github issues this is a known bug. Specifically the bug is that all the values have a default, so it seems to work but doesn’t really

We wanted to create a MySQL server, then create users/databases within it, all in one Terraform configuration. This is not possible, and you have to split your configuration in two

Actually I have to do the opposite. I have to combine it… So that I create the initial RDS cluster and all services that need to use it from the same configuration

I wanted to fetch the endpoint, master username,password from parameter store to be able to split the creations …

i suggest you provide these values as variable inputs. The system you use that runs Terraform can load the parameter and pass in the data

Yeah that’s the method I guess. These things always make me want to try pulimi…

Ok, I did some more digging.

You can use dynamic data in the provider. But it only works at initial creation. In the Refresh pass it does NOT populate the values at all.

2017 bug … https://github.com/hashicorp/terraform/issues/11264

actually… seems like it’s maybe possible if I don’t use a module for the data reading… (I was using the cloudposse ssm parameter module)

I had issues loading the data from terraform remote state data source.

I want to advise you not to go down this path. It’s a rabbit hole and is not a workflow well supported by Terraform (configuring providers with dynamic data). If you read the Terraform docs, it’s specifically called out as something to avoid:
You can use expressions in the values of these configuration arguments, but can only reference values that are known before the configuration is applied. This means you can safely reference input variables, but not attributes exported by resources (with an exception for resource arguments that are specified directly in the configuration).

Hey :wave: @Alex Jurkiewicz I just needed a clarification on this thread. Do you mean we should avoid using terraform_remote_state as it is not well supported by Terraform yet and it can cause problems? I am using something like below.

data "terraform_remote_state" "rds" {
  backend = "s3"
  config = {
    bucket  = "somebucket"
    key     = "terraform.tfstate"
    region  = "eu-west-1"
    profile = "default"
  }
}
locals {
  dbInstanceEndPoint = data.terraform_remote_state.rds.outputs.endpoint
}
module "container" {
  source                       = "[email protected]:cloudposse/terraform-aws-ecs-container-definition.git?ref=0.44.0"
  container_name               = var.container_name
  ...
  ...
  ...
  environment = [
    {
      name  = "DATABASE_HOST"
      value = local.dbInstanceEndPoint
    }
  ]
  privileged = false
}

I am planning to use terraform_remote_state extensively in my use case. Like creating each service Like RDS, ASG, ALB separately and use the outputs via terraform_remote_state into other services like ECS.

Thanks

Terraform remote state is great, use it lots. My advice is “only configure providers with static data”. That is, values derived only from input variables or hardcoded values. Don’t attempt to configure a provider with data source attributes.

Your example seems fine

It works, probably, if done in the root module. Dynamically in sub module is explicitly advised against in terraform and results in horrible experiences… (unfortunately)

@mfridh I am not sure if I understand your point correctly. Are you talking about making provider configuration dynamic? Can you share a code example please?

Your example is fine and this thread was more about the case I mention with provider instantiation in sub modules. A practice I certainly in some cases would like if it worked, for flexibility reasons, but it’s just not possible in a sane way today.

Take a look at dynamic blocks: https://www.terraform.io/docs/configuration/expressions.html#dynamic-blocks you’d be able to do it with that.

yeh

here’s an example from the config aggregator i was just working on:

hmm that’s very creative

complex object types can be null‘d! very convenient for this kind of thing

here are the variable defs that work with that config…

variable account_aggregation_source {
  description = "Object of account sources to aggregate"
  type = object({
    account_ids = list(string)
    all_regions = bool
    regions     = list(string)
  })
  default = null
}

variable organization_aggregation_source {
  description = "Object with the AWS Organization configuration for the Config Aggregator"
  type = object({
    all_regions = bool
    regions     = list(string)
    role_arn    = string
  })
  default = null
}

I’m pretty sure I added that in long time ago

This sure looks like it’s mandatory for expiration to always be on… https://github.com/cloudposse/terraform-aws-s3-bucket/blob/master/main.tf#L51-L53

yes, but no reason to be on a dynamic you could send a PR

or set expiration to 0 that i think is unlimited

Yeah, a dynamic seems more idiomatic with the rest of the module. Glad to know I wasn’t misreading things. Surprised me that others haven’t hit this before. Must not be a lot of users of lifecycle rules. :)

we use it

but we always set expiration

do you have some code you can share? it’s easier to update example code to point you in the right direction than to come up with something totally from scratch

though, if you look at this thread, we’re basically talking about exactly this use case… https://sweetops.slack.com/archives/CB6GHNLG0/p1605395904242600

That’s fair. This is how I’m doing it currently with count:

resource "aws_instance" "advprx_instance" {
  count = var.proxy_counts[var.environment]
  ami                         = var.override_ami != "" ? var.override_ami : var.AMI_map[var.environment]
  instance_type               = var.instance_type
  vpc_security_group_ids      = var.security_group_map[var.environment]
  subnet_id                   = var.subnet_map[var.environment]
  associate_public_ip_address = "false"
  key_name                    = var.keypair_name_map[var.environment]
  iam_instance_profile        = aws_iam_instance_profile.win-ec2-profile.name
  user_data                   = base64encode(data.template_file.advprx_user_data.rendered)

  credit_specification {
    cpu_credits = "standard"
  }

  # Give instance unique name
  tags = merge(local.proxy_tags, { "Name" = "${var.host_name}-${count.index + var.override_start_index + 1}" } )
}

# Attach the generated instances to the appropriate target group
resource "aws_lb_target_group_attachment" "advprx_instance_alb_attachment" {
  count            = var.proxy_counts[var.environment]
  # how do I use for_each or for for this?
  target_group_arn = local.target_group_arn
  # target_id        = aws_instance.advprx_instance[each.key].id 
  target_id        = aws_instance.advprx_instance[count.index].id
  port             = 80
}

and what is the object structure in var.proxy_counts?

literally just a number?

if you are creating a literal multiple of identical things, then count is a fine approach. for_each makes more sense when each item has some unique identifier as one of its properties. in your code, we could construct that identifier for the “Name” tag, but it would still be based on the index and so may not avoid classic limitations of count

Yes, it’s literally a number.

Oh. It wouldn’t avoid the limitations of count? That’s what I was after. Specifically this limitation: if I decrease the count, or increase the count, I didn’t want resources recreated.

increasing the count by one will add one instance, decreasing the count by one will destroy one instance (the last one, you can’t pick)

Ok. Fair enough. Thanks!