#terraform (2019-08)

terraform Discussions related to Terraform or Terraform Modules

Archive: https://archive.sweetops.com/terraform/

2019-08-30

2019-08-29

Phuc avatar

hi Guys, I have some issue when running test on module creating s3

Phuc avatar

Here is the code in my main file

Phuc avatar
module "s3_bucket" {
  source                              = "git:xxxxx/terraform-modules/aws-s3.git?ref=terraform_0.11"
  enabled                             = "true"              
  user_enabled                        = "false"         
  allowed_bucket_actions              = []                                                             
  policy                              = ""              
  force_destroy                       = "false"        
  versioning_enabled                  = "true"          
  allow_encrypted_uploads_only        = "false"         
  sse_algorithm                       = "AES256"        
  kms_master_key_arn                  = ""             
  namespace                           = "test"
  name                                = "frontend"
  stage                               = ""
  attributes                          = []
  delimiter                           = "-"              
  tags  = {
    "BusinessUnit" = "XYZ",
    "Snapshot"     = "true"
  }
}
Phuc avatar

and here is in variables.tf

variable "namespace" {
  type        = "string"
  #default     = ""
  description = "Namespace (e.g. `eg` or `cp`)"
}

variable "stage" {
  type        = "string"
  #default     = ""
  description = "Stage (e.g. `prod`, `dev`, `staging`)"
}

variable "name" {
  type        = "string"
  #default     = ""
  description = "Name  (e.g. `app` or `db`)"
}

variable "delimiter" {
  type        = "string"
  default     = "-"
  description = "Delimiter to be used between `namespace`, `stage`, `name` and `attributes`"
}

variable "attributes" {
  type        = "list"
  default     = []
  description = "Additional attributes (e.g. `1`)"
}

variable "tags" {
  type        = "map"
  default     = {}
  description = "Additional tags (e.g. `{ BusinessUnit = \"XYZ\" }`"
}

variable "acl" {
  type        = "string"
  default     = "private"
  description = "The canned ACL to apply. We recommend `private` to avoid exposing sensitive information"
}

variable "policy" {
  type        = "string"
  default     = ""
  description = "A valid bucket policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy."
}

variable "region" {
  type        = "string"
  default     = ""
  description = "If specified, the AWS region this bucket should reside in. Otherwise, the region used by the callee."
}

variable "force_destroy" {
  type        = "string"
  default     = "false"
  description = "A boolean string that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable."
}

variable "versioning_enabled" {
  type        = "string"
  default     = "false"
  description = "A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket."
}

variable "sse_algorithm" {
  type        = "string"
  default     = "AES256"
  description = "The server-side encryption algorithm to use. Valid values are `AES256` and `aws:kms`"
}

variable "kms_master_key_arn" {
  type        = "string"
  default     = ""
  description = "The AWS KMS master key ARN used for the `SSE-KMS` encryption. This can only be used when you set the value of `sse_algorithm` as `aws:kms`. The default aws/s3 AWS KMS master key is used if this element is absent while the `sse_algorithm` is `aws:kms`"
}

variable "enabled" {
  type        = "string"
  description = "Set to `false` to prevent the module from creating any resources"
  default     = "true"
}

variable "user_enabled" {
  type        = "string"
  default     = "false"
  description = "Set to `true` to create an S3 user with permission to access the bucket"
}

variable "allowed_bucket_actions" {
  type        = "list"
  default     = ["s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload"]
  description = "List of actions the user is permitted to perform on the S3 bucket"
}

variable "allow_encrypted_uploads_only" {
  type        = "string"
  default     = "false"
  description = "Set to `true` to prevent uploads of unencrypted objects to S3 bucket"
}

`

Phuc avatar

the issue is, if I run terraform validate to see the resource which will be created. I will come up with this error :

  Test_s3_module terraform validate       

Error: Required variable not set: namespace



Error: Required variable not set: stage



Error: Required variable not set: name

but I already declared the value for each of that value in main.tf this error wont show up if I put default value under the variables file like this:

variable "namespace" {
  type        = "string"
  default     = ""     <------ this line
  description = "Namespace (e.g. `eg` or `cp`)"
}

variable "stage" {
  type        = "string"
  default     = ""    <------ this line
  description = "Stage (e.g. `prod`, `dev`, `staging`)"
}

variable "name" {
  type        = "string"
  default     = ""    <------ this line
  description = "Name  (e.g. `app` or `db`)"
}
Phuc avatar

I’m testing on TF v0.11

Phuc avatar

Updated, I figure out the meaning of variables, The issue solved

:--1:1
Brij S avatar
Brij S

has anyone tried github actions for terraform? https://www.terraform.io/docs/github-actions/getting-started/index.html

johncblandii avatar
johncblandii

I’m actually looking to do this soon. I get tired of having people re-commit/push due to TF fmt failing on CI. Looking to automate an fmt commit.

Brij S avatar
Brij S

yeah, I followed their getting started……but it doesnt work

johncblandii avatar
johncblandii

(unsure if possible, but i’m going to try it out soon)

Brij S avatar
Brij S

it is possible according to their docs

johncblandii avatar
johncblandii

welp…then there is that

Brij S avatar
Brij S

but I set it up to be invoked on PR’s

Brij S avatar
Brij S

I tried it last night and it didnt even work

Brij S avatar
Brij S

their documentation is a bit confusing

Todd Lyons avatar
Todd Lyons

I’m a bit flummoxed. I have a [00_remote_state.tf](http://00_remote_state.tf) file that I’ve used all over the place that configures s3 for remote state for various vanilla terraform projects. I’m trying to use it now in a new project. terraform init downloads the latest aws plugin and says it succeeds, but it’s not creating the key (dns-nonprod/terraform.tfstate) that I’ve told it to up in the S3 bucket. I swear this used to work. Shouldn’t terraform init create the key up in s3? Running any terraform plan or terraform apply errors with state not found:

data.terraform_remote_state.ops_s3: Refreshing state...
data.aws_route53_zone.qa_example_net: Refreshing state...
data.aws_route53_zone.qa2_example_net: Refreshing state...
data.aws_route53_zone.dev_example_net: Refreshing state...

Error: Unable to find remote state

  on [00_remote_state.tf](http://00_remote_state.tf) line 1, in data "terraform_remote_state" "ops_s3":
   1: data "terraform_remote_state" "ops_s3" {
Todd Lyons avatar
Todd Lyons

Does this behavior sound familiar to anybody?

Todd Lyons avatar
Todd Lyons

I have tried forcing older aws plugin version that worked fine previously. 2.19 and 2.14 (2.25 is latest). No change in behavior. I can paste the file here if requested. It’s just not making any sense to me.

aknysh avatar
aknysh
  1. terraform versions are the same or different?
Todd Lyons avatar
Todd Lyons

Running terraform 0.12.4, same as I have been for weeks.

Todd Lyons avatar
Todd Lyons

Thanks for the feedback, BTW.

aknysh avatar
aknysh
  1. Does the user you are using to provision have the permissions to access the remote state?
Todd Lyons avatar
Todd Lyons

I just created a file in that S3 bucket, and then deleted it. I do have create access.

Todd Lyons avatar
Todd Lyons

I ran the init with TF_LOG=debug. I see it checks if the file exists up in S3. But then it never tries to create it.

johncblandii avatar
johncblandii

question: I am trying to build a module wrapping a resource and don’t want to provide every single argument for the resources blocks.

resource "type" "name" {
  some_block {
    blah = true
  }
}

I want to instead say:

resource "type" "name" {
  some_block = var.some_block
}

I did see possibly using a loop, but I’m not seeing a definitive answer or direction.

aknysh avatar
aknysh

dynamic blocks (TF 0.12 only)

aknysh avatar
aknysh
cloudposse/terraform-aws-emr-cluster

Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS - cloudposse/terraform-aws-emr-cluster

johncblandii avatar
johncblandii

perfect

johncblandii avatar
johncblandii
05:52:59 PM
aknysh avatar
aknysh

if the var is not a list of objects (and you don’t want i to be a list), then you could do something like this:

aknysh avatar
aknysh
dynamic "bootstrap_action" {
    for_each = [var.bootstrap_action]
    content {
      path = bootstrap_action.value.path
      name = bootstrap_action.value.name
      args = bootstrap_action.value.args
    }
  }
aknysh avatar
aknysh

if you want conditionally add the block depending on some bool expression:

aknysh avatar
aknysh
dynamic "bootstrap_action" {
    for_each = var.add_block ? [var.bootstrap_action] : []
    content {
      path = bootstrap_action.value.path
      name = bootstrap_action.value.name
      args = bootstrap_action.value.args
    }
  }
johncblandii avatar
johncblandii

nice

johncblandii avatar
johncblandii

so we still need to define the content of each individual one, but we do not need to worry w/ defining all vars

johncblandii avatar
johncblandii

that’s lovely

johncblandii avatar
johncblandii

and this works with blocks of blocks?

johncblandii avatar
johncblandii

can you for_each in a content block?

aknysh avatar
aknysh

yes

johncblandii avatar
johncblandii

good deal

johncblandii avatar
johncblandii

spot check (if you don’t mind):

resource "aws_msk_cluster" "this" {
  cluster_name           = var.cluster_name
  kafka_version          = var.kafka_version
  number_of_broker_nodes = var.number_of_broker_nodes
  tags                   = var.tags

  dynamic "client_authentication" {
    for_each = var.client_authentication
    content {
      dynamic "tls" {
        for_each client_authentication.tls

        content {
          certificate_authority_arns = client_authentication.tls.value.certificate_authority_arns
        }
      }
    }
  }
}

https://www.terraform.io/docs/providers/aws/r/msk_cluster.html

aknysh avatar
aknysh

var.client_authentication should be list(object)

aknysh avatar
aknysh

tls inside it should be list(string)

aknysh avatar
aknysh

and add this:

aknysh avatar
aknysh
dynamic "tls" {
        for_each = toset(client_authentication.value.tls)
        iterator = item

        content {
          certificate_authority_arns = item.value.certificate_authority_arns
        }
aknysh avatar
aknysh

TF 0.12 has difficulties with list(string), needs it to be set(string)

johncblandii avatar
johncblandii

so i take it we need to flesh out the object in the variable declaration as well?

Error: Invalid type specification

  on ../variables.tf line 18, in variable "client_authentication":
  18:   type        = list(object)

The object type constructor requires one argument specifying the attribute
types and values as a map.
aknysh avatar
aknysh

yea, need to specify the var correctly:

:--1:1
aknysh avatar
aknysh
cloudposse/terraform-aws-emr-cluster

Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS - cloudposse/terraform-aws-emr-cluster

Todd Lyons avatar
Todd Lyons

@aknysh I thought I had found it. In my s3.conf used for terraform init -backend-config=s3.conf, I had put dns-nonprod/terraform.tfstate. And in my [00_remote_state.tf](http://00_remote_state.tf), I had put dns_nonprod/terraform.tfstate.

Todd Lyons avatar
Todd Lyons

I fixed the second one, and now it’s giving me the same error. This is perplexing, still working through it. Thanks for the sanity questions previously.

Todd Lyons avatar
Todd Lyons

In my s3.conf, I have:

bucket = "foo"
key = "dns-nonprod/terraform.tfstate"
region = "us-east-1"
Todd Lyons avatar
Todd Lyons

Is it possible to make the terraform_remote_state configuration read that in for the config = { ... } section? I hate having two different sources of truth (one for terraform init - the s3.conf, and one for everything else)

Robert avatar
Robert
07:19:21 PM

@Robert has joined the channel

Robert avatar
Robert

@Todd Lyons with the consul backend you can do something like this:

terraform {
    backend "consul" {
        address = "consul.vault:8500"
        scheme  = "http"
        path    = ""
    }
}
export BACKEND_KEY=project/environment/name
terraform init -backend-config="path=$BACKEND_KEY" "$TERRAFORM_DIR"
Robert avatar
Robert

Maybe you could do the same with the key.

Todd Lyons avatar
Todd Lyons

I apologize, but I don’t quite understand the “do the same with the key” comment. I’m still experimenting though, I may get better results now that I’m no longer dealing with the chicken/egg issue.

:--1:1
Robert avatar
Robert

No worries

Robert avatar
Robert

Robert avatar
Robert

Todd Lyons avatar
Todd Lyons

It sounds like I can only do it with something remote, not a local file. I tried reading it in using the local_file resource and then all manner of tomap() and split() and such things. Some things flat out errored, some things acted like they were going to work but then complained about the attempt I was making.

Todd Lyons avatar
Todd Lyons

I also found the original issue I was having. It turns out that the key in S3 isn’t actually created until the first apply is run. So when I was doing a terraform plan, the data lookup wasn’t finding it. I wonder how the heck it ever worked.

1
Todd Lyons avatar
Todd Lyons

Hmmm, maybe now that I got past that original issue, I can retry some things.

oscar avatar
oscar

What is your goal again? And the issue

Todd Lyons avatar
Todd Lyons
I want to configure my s3 bucket name, key, and region in one place, that can be used both by terraform init and by terraform plan apply refresh output, etc. I think I just figured it out (now that the previous issue was working on is resolved).
Todd Lyons avatar
Todd Lyons

I have a file, s3.conf, that has:

$ cat s3.conf
bucket = "foo"
key    = "dns-nonprod/terraform.tfstate"
region = "us-east-1"
Todd Lyons avatar
Todd Lyons

I initialize like this: terraform init -backend-config=s3.conf

Todd Lyons avatar
Todd Lyons

And my remote state is configured like this:

data "local_file" "s3" {
  filename = "${path.module}/s3.conf"
}

terraform {
  backend "s3" {
    config = tomap(data.local_file.s3.content)
  }
}


\# Default provider works for the various pieces of the terraform initialization
provider "aws" {
  region  = "us-east-1"
}
Erik Osterman avatar
Erik Osterman

this is really cool.

Erik Osterman avatar
Erik Osterman

for the longest time, interpolation wasn’t supported in this context.

Erik Osterman avatar
Erik Osterman

I thought it still wasn’t

Erik Osterman avatar
Erik Osterman

@aknysh I think we could benefit from this too.

aknysh avatar
aknysh

could be useful, agree

Todd Lyons avatar
Todd Lyons

So far, it seems to be working.

Todd Lyons avatar
Todd Lyons

The simple solution above was just muddied by my initial error: attempting to set a data “terraform_remote_state” for an S3 key dns-nonprod/terraform.tfstate that didn’t exist yet, because I hadn’t yet run an apply (couldn’t run an apply because init failed, because that file didn’t exist yet because I hadn’t yet run an apply, circular dependencies FTW).

Todd Lyons avatar
Todd Lyons

I don’t know why, but I could swear that terraform init used to create that S3 key with a minimal tfstate file. I must have been wrong. This has all been with the latest aws module and terraform 0.12.4.

Todd Lyons avatar
Todd Lyons

Sorry for spamming the channel. I’ll exercise restraint from now on.

aknysh avatar
aknysh

i would put the backend files into a separate folder, provision it first (w/o specifying remote backend obviously b/c it does not exist yet), then add the remote backend config to the code, then terraform init will ask you to import the old (local) backend config into the remote one

aknysh avatar
aknysh

after that, don’t touch the tf-backend folder

aknysh avatar
aknysh

for all other modules, use diff folders

aknysh avatar
aknysh

although outdated, the doc above will give you an idea on what needs to be done to provision the remote backend without having a remote backend to provision it in the first place

aknysh avatar
aknysh

here is the project structure that we usually use https://github.com/cloudposse/testing.cloudposse.co/tree/master/conf

cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

aknysh avatar
aknysh

tfstate-backend is in separate folder and gets provisioned separately and only once

aknysh avatar
aknysh
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

aknysh avatar
aknysh

the script to provision the backend locally and then enable remote backend https://github.com/cloudposse/terraform-root-modules/blob/master/aws/tfstate-backend/scripts/init.sh

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Todd Lyons avatar
Todd Lyons

I’ll surely study them. If I’m having trouble, I’ll bug my co-worker Jon and see if he can fill in the gaps.

Todd Lyons avatar
Todd Lyons

Thank you for all you’ve shown me.

johncblandii avatar
johncblandii

I’ve seen some chatter on the pre-commit-terraform working with tf 0.12, but it seems to have a problem with a simple something = var.something declaration

johncblandii avatar
johncblandii

I’m on 1.19 of pre-commit-terraform so that should support it just fine

aknysh avatar
aknysh

never used that version. why not switch to the latest 0.12?

johncblandii avatar
johncblandii

pre-commit-terraform version

johncblandii avatar
johncblandii

TF 0.12 p-c-t 1.19

johncblandii avatar
johncblandii
hookid: terraform_docs

2019/08/30 10:51:44 At 41:5: error while trying to parse object within list: At 42:40: Unknown token: 42:40 IDENT null

^ it seems terraform-docs is not able to properly parse the code

johncblandii avatar
johncblandii

Running outside of pre-commit shows:

~/Work/terraform-aws-kafka (git: feature/DEVOPS-557-kafka-module) (tf: default): terraform-docs md document .
2019/08/30 10:53:08 At 2:28: Unknown token: 2:28 IDENT var.cluster_name
johncblandii avatar
johncblandii

line 2:

resource "aws_msk_cluster" "this" {
  cluster_name           = var.cluster_name
  ...
}
aknysh avatar
aknysh

that error is when you try to parse 0.12 code with TF 0.11

johncblandii avatar
johncblandii

but i’m on tf 0.12

johncblandii avatar
johncblandii
: tf --version
Terraform v0.12.7
+ provider.aws v2.25.0
aknysh avatar
aknysh

maybe you have two of them installed

johncblandii avatar
johncblandii

I am using tfenv

aknysh avatar
aknysh

make sure in the Dockerfile :

aknysh avatar
aknysh

\# Install terraform 0.11 for backwards compatibility
RUN apk add [email protected]


\# Install terraform 0.12
RUN apk add [email protected] [email protected]==0.12.3-r0
aknysh avatar
aknysh

if you are using geodesic for that

johncblandii avatar
johncblandii

nopers. all local

2019-08-28

Kiran avatar
Kiran

Can I know estimation time for this PR to be merged - https://github.com/cloudposse/terraform-aws-multi-az-subnets/pull/16 ? Also TravisCI is failing because it checks this TF12 upgrade PR in TF11 binary.

aknysh avatar
aknysh

we will review it ASAP. We started converting the module to 0.12, but did not have time to finish it. We also adding tests for the module and for the example (bats and terratest) and Codefresh Ci/CD pipelines to deploy the example on AWS account

Kiran avatar
Kiran

Sounds good Thanks buddy.

Brij S avatar
Brij S

Hello all, I’m trying to do acm cert validation(multiple at a time) and I’m running into some issues.

resource "aws_route53_record" "cert_validation" {
  count   = length(var.certs)
  name    = aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_name
  type    = aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_type
  zone_id = aws_route53_zone.zones[count.index].id
  records = ["${aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_value}"]
  ttl     = 60
}

the above code..kinda works? when I apply this, I get the following error:

Error: Invalid index

  on ../modules/bootstrap/acm_validation.tf line 4, in resource "aws_route53_record" "cert_validation":
   4:   type    = aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_type
    |----------------
    | aws_acm_certificate.cert is tuple with 2 elements
    | count.index is 0

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/bootstrap/acm_validation.tf line 6, in resource "aws_route53_record" "cert_validation":
   6:   records = ["${aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_value}"]
    |----------------
    | aws_acm_certificate.cert is tuple with 2 elements
    | count.index is 0

The given key does not identify an element in this collection value.


Error: Invalid index

  on ../modules/bootstrap/acm_validation.tf line 6, in resource "aws_route53_record" "cert_validation":
   6:   records = ["${aws_acm_certificate.cert[count.index].domain_validation_options.0.resource_record_value}"]
    |----------------
    | aws_acm_certificate.cert is tuple with 2 elements
    | count.index is 0

The given key does not identify an element in this collection value.

But when I do destory, it seems its creating at least 1..

module.nonprod.aws_route53_record.cert_validation[1]: Still destroying... [id=Z3FZBH8XNPPPYT__3aedbf37656ebde46d6db19a4f38212c.test-api.dev._CNAME, 30s elapsed]
module.prod.aws_route53_record.cert_validation[1]: Still destroying... [id=Z2IAS5UODUXPHA__73016316fbab15ae3d5db4d2b9b240c8.test-api.com._CNAME, 30s elapsed]
Brij S avatar
Brij S

any ideas?

aknysh avatar
aknysh

what version of TF? what’s type of var.certs? If it’s a list of string, how many items are in it?

Brij S avatar
Brij S

Terraform v0.12.5

Brij S avatar
Brij S
variable "certs" {
  default = ["apps", "api"]
  type    = "list"
}
aknysh avatar
aknysh

you are mixing TF 0.11 code with TF 0.12

aknysh avatar
aknysh

that’s why it’s not working

aknysh avatar
aknysh

(yes, TF is not so smart)

Brij S avatar
Brij S

oh, what do you mean

Brij S avatar
Brij S

like my var is setup wrong?

aknysh avatar
aknysh

var types and interpolations in TF code

Brij S avatar
Brij S

im not sure I follow, what would I need to change?

aknysh avatar
aknysh

not sure if that’s the reason of the errors, but first convert everything to TF 0.12 syntax

aknysh avatar
aknysh

type = list(string)

aknysh avatar
aknysh

0.12 ^

Brij S avatar
Brij S

it worked!!

party_parrot1
cool-doge1
sarkis avatar
sarkis

@aknysh is a wizard

1
aknysh avatar
aknysh

hi @sarkis

1
1
Brij S avatar
Brij S

thank you both! indeed wizards. Wouldve never noticed the 11 vs 12 syntax. D’oh

sarkis avatar
sarkis

just went to the top of my TF debug playbook as well - thanks to @aknysh

:--1:1
Erik Osterman avatar
Erik Osterman

#office-hours starting now! join us here https://zoom.us/s/508587304

Brij S avatar
Brij S

how would this line be turned into ‘tf 12 syntax “*.${var.domain}-${element(var.certs, count.index)}.${var.aws_env == “prod” ? “com.” : “dev.”}”`

aknysh avatar
aknysh

it’s already TF 0.12 syntax since you are using string concatenation with interpolation

Brij S avatar
Brij S

yes, I ran the terraform12 upgrade command and it didnt change

aknysh avatar
aknysh

also you could use this: format("*.%s-%s.%s", var.domain, var.certs[count.index], var.aws_env == "prod" ? "com." : "dev.")

aknysh avatar
aknysh

whatever looks better for you

Brij S avatar
Brij S

oh cool, didnt know I could do that

Brij S avatar
Brij S

thank you again !!

Chris Montes avatar
Chris Montes
aknysh avatar
aknysh

did not see it before

aknysh avatar
aknysh

you can try to run terraform taint

Joshua Snider avatar
Joshua Snider

Hi everyone! We are currently using the terraform-null-label module for labels in Terraform but we are running into an issue when updating our code to v0.12 We pass “context” between modules so we have a variable defined in the module called “tags_context” and type: map This was upgraded to type: map(string) by Terraform but then the plan doesn’t work What type should the context variable be in v0.12?

aknysh avatar
aknysh

@Joshua Snider see #announcements

Joshua Snider avatar
Joshua Snider

nvm, it was answered in #announcements

Sharanya avatar
Sharanya

Did anyone Come across NPM memory Issues ?

Daniel avatar
Daniel

Hey folks! I see that there’s a Cloud Posse container definition module, but I’m wondering if there’s an easy way to make a container_definition a reusable variable (or similar) that still supports interpolation. Use case: I have an ECS service that I run as both Fargate and EC2 using two different modules and don’t want to duplicate the container definition to keep it dry.

PePe avatar

the container definition for Fargate and ECS are slightly different

PePe avatar

in Fargate the Task have to set memory and CPU and the container definition too

PePe avatar

but in ECS EC2 that is not required

PePe avatar

and there is some other differences on the network setup etc

PePe avatar

I think is sane to have them separated

Daniel avatar
Daniel

Hmmm, with 15+ envvars/secrets and 4 environments it feels very anti-dry to repeat it so many times.

PePe avatar

you have them in variables ?

PePe avatar

what is so not DRY about that ?

PePe avatar

if you had them hardcoded I will agree

PePe avatar

and you have 4 environments ?

PePe avatar

so if you separate those for ENVs in it’s own TF

PePe avatar

would you call it DRY ?

Daniel avatar
Daniel

In my structure I have a directory per env for an application

PePe avatar

we have some cluster with like 20 different task defs

Daniel avatar
Daniel

Perhaps I should switch to using tfvars?

PePe avatar

but in reality they should be it’s own thing

PePe avatar

I use tfvars

Daniel avatar
Daniel

Hmmm

Daniel avatar
Daniel

I think if I reassessed how I structured this it would help my DRY concerns

Daniel avatar
Daniel

Thanks for the feedback

PePe avatar

we populate the tfvar from SSM parameter store

PePe avatar

when necesary

PePe avatar

or jenkins does it

PePe avatar

from other secret/parameter store

Daniel avatar
Daniel

Does it automatically create the tfvars based on what is in parameter store? I have my secrets name spaced with app/env/secret_name and was thinking I could write something to automatically grab all the params set for an app and put together the secrets block

PePe avatar

you can use chamber for that

PePe avatar
segmentio/chamber

CLI for managing secrets. Contribute to segmentio/chamber development by creating an account on GitHub.

PePe avatar

it can export to tfvar

:--1:1
PePe avatar

so you can do something like

chamber write test-ec2-helloworld ecs_parameter_secret password333
PePe avatar

where

test-ec2-helloworld

is your app/service

Daniel avatar
Daniel

Very interesting

PePe avatar
chamber export test-ec2-hello -f tfvars                                                               1350ms  Wed 28 Aug 15:42:11 2019
ecs_parameter_secret = "password1"
ecs_parameter_string = "NOTSECRET"
PePe avatar

or you can use chamber as ENTRYPOINT in your containers

PePe avatar

but I don’t know how usefull is that since now you can use SM or SSM parameter store directly in the task def

Brij S avatar
Brij S

I’m trying to use a local variable as the bucket name:

resource "aws_s3_bucket" "remote_state" {
    bucket = local.bucket_name
    force_destroy = var.force_destroy
    acl = "private"
    
    versioning {
      enabled = var.versioning_enabled
    }

    tags = local.required_tags
}

local var is :

locals {
  bucket_name = "account-${var.aws_env}-${project_domain}-${var.aws_region}"
}

but when I try to run terraform plan I get the following error:

50:   bucket_name = "account-${var.aws_env}-${project_domain}-${var.aws_region}"

A reference to a resource type must be followed by at least one attribute
access, specifying the resource name.
Brij S avatar
Brij S

what does that mean?

aknysh avatar
aknysh

Missing var in project_domain

Brij S avatar
Brij S

missing var?

Brij S avatar
Brij S

I have project_domain declared as a var

aknysh avatar
aknysh

var.project_domain

Brij S avatar
Brij S

omg!

Brij S avatar
Brij S

@aknysh cant believe I didnt see that! Thanks!

1
aknysh avatar
aknysh

Use a good editor with TF error detection, like JetBrains IDEA with TF plugin, or VS Code

Brij S avatar
Brij S

I use vscode with the terraform plugin

Brij S avatar
Brij S

ive been told its ‘alright’ though

PePe avatar
cloudposse/terraform-aws-rds-cluster

Terraform module to provision an RDS Aurora cluster for MySQL or Postgres - cloudposse/terraform-aws-rds-cluster

PePe avatar

does not work on TF 0.12

aknysh avatar
aknysh

Maybe, only examples/complete tested in 0.12 and have automatic tests for them

PePe avatar

I had to do

assume_role_policy = "${data.aws_iam_policy_document.enhanced_monitoring.json}"

to make it work

PePe avatar

which I found it weird

PePe avatar

I’m new to 0.12 and I posted this in the 0.11 channel

aknysh avatar
aknysh

Weird indeed

Brij S avatar
Brij S

@aknysh one more question if you dont mind! I have a module declaration as follows:

module "s3-remote-state-bucket" {
  source = "../../modules/remote_state"

  versioning_enabled = var.versioning_enabled
  force_destroy = var.force_destroy

  aws_env = "nonprod"
  aws_region = var.aws_region
  aws_account_id_nonprod = var.aws_account_id_nonprod
  aws_account_id_prod = var.aws_account_id_prod

  providers = {
    aws = "aws.nonprod"
  }                                                         
}

in my variables.tf file in the same folder I have:

variable "aws_env" {
  description = "aws account environment"
  type = string
}

Note: no default value. but when I run apply I get asked for the aws_env:

 terraform apply
var.aws_env
  aws account environment

  Enter a value:

I dont want to use a default variable for this, this worked as intended with tf11 - anything new with tf12 maybe?

aknysh avatar
aknysh

Don’t believe 0.11 didn’t ask you for a missing value :)

aknysh avatar
aknysh

Nothing changed in 0.12

Brij S avatar
Brij S

im providing it in the module declaration though, strange

aknysh avatar
aknysh

But maybe the var value was provided in tfvar file, or on command line, or in ENV var

Brij S avatar
Brij S

huh, so if I remove it from the vars file and provide the value it works as intended

Brij S avatar
Brij S

wild..

2019-08-27

Brij S avatar
Brij S

Hey all! New to terraform, I’m trying to do some acm cert validation using : https://www.terraform.io/docs/providers/aws/r/acm_certificate_validation.html

In the link, they show the following r53 record for cert validation being created:

resource "aws_route53_record" "cert_validation" {
  name    = "${aws_acm_certificate.cert.domain_validation_options.0.resource_record_name}"
  type    = "${aws_acm_certificate.cert.domain_validation_options.0.resource_record_type}"
  zone_id = "${data.aws_route53_zone.zone.id}"
  records = ["${aws_acm_certificate.cert.domain_validation_options.0.resource_record_value}"]
  ttl     = 60
}

Specifically, aws_acm_certificate.cert.domain_validation_options.0.resource_record_name , I create my acm certs(Specifically 2) like this:

resource "aws_acm_certificate" "cert" {
  provider          = "aws.acm"
  domain_name       = "*.${var.domain}.${element(var.certs, count.index)}.${var.aws_env == "prod" ? "com." : "test."}"
  validation_method = "DNS"

  tags = "${local.required_tags}"

  lifecycle {
    create_before_destroy = true
  }
  count = "${length(var.certs)}"
}

How would I be able to reference each cert for validation?

AWS: aws_acm_certificate_validation - Terraform by HashiCorp

Waits for and checks successful validation of an ACM certificate.

Erik Osterman avatar
Erik Osterman
cloudposse/terraform-aws-acm-request-certificate

Terraform module to request an ACM certificate for a domain name and create a CNAME record in the DNS zone to complete certificate validation - cloudposse/terraform-aws-acm-request-certificate

Erik Osterman avatar
Erik Osterman

This might serve as a good reference point.

Brij S avatar
Brij S

oh this looks like a good idea, thank you

mrwacky avatar
mrwacky

This looks correct

Brij S avatar
Brij S

what looks correct?

Brij S avatar
Brij S

I dont know how to reference each cert for validation

sarkis avatar
sarkis

@Brij S you are missing a resource - let me find it

sarkis avatar
sarkis

ah, I see what you mean … you will need to somehow loop over all of your certs and specify a aws_acm_certificate_validation resource for each one…

Brij S avatar
Brij S

yeah.. thats the tricky part

Brij S avatar
Brij S

not sure how to facilitate that

sarkis avatar
sarkis

@Brij S are you using terraform 0.12+?

Brij S avatar
Brij S

no, TF 0.11

sarkis avatar
sarkis

it might be impossible to do this, since the count for resource "aws_route53_record" "cert_validation" will not be able to be <computed>, i.e. can’t dynamically set to the cert count iirc

PePe avatar

What are you guys toughs on Terraform enterprise ?

Erik Osterman avatar
Erik Osterman

@johncblandii

johncblandii avatar
johncblandii

I like it. I think there are some corners for sure where they could improve, specifically around integrating with other systems without creating custom CLI solutions

johncblandii avatar
johncblandii

The private module registry is legit.

1
johncblandii avatar
johncblandii

PR integration is legit

sarkis avatar
sarkis

private module registry is almost a requirement if you plan on having modules live in their own repos, the dependency management between all of them using just git tagging (only option non-enterprise) is such a pita

johncblandii avatar
johncblandii

yeah, it definitely can be

johncblandii avatar
johncblandii

i can demo any parts of it to anyone who wants to check it out

johncblandii avatar
johncblandii

we terraformed all of our workspaces so all projects are basically just reusable TF modules

PePe avatar

I will interested on that demo for sure

sarkis avatar
sarkis

@johncblandii so you aren’t using workspaces for separation between environments? or did i read that wrong

johncblandii avatar
johncblandii

we are

johncblandii avatar
johncblandii

dev, uat, support, prod, training, etc

johncblandii avatar
johncblandii

all different workspaces in TFE

:--1:1
johncblandii avatar
johncblandii

@PePe email me: [email protected]

sarkis avatar
sarkis

i read somewhere (i think official docs) that workspaces shouldn’t be used for this but they just work so perfectly for DRY

johncblandii avatar
johncblandii

TFE is workspaces

johncblandii avatar
johncblandii

literally

sarkis avatar
sarkis

yea my guess is outdated docs or it wasn’t official and im mistaken

johncblandii avatar
johncblandii

oh and the remote execution is legit

johncblandii avatar
johncblandii

such a timesaver for devs who don’t have write access to prod to test fixes with prod secrets

johncblandii avatar
johncblandii

write locally, use vars configured on Cloud

PePe avatar

sounds awesome

:--1:1
PePe avatar

is it worth it ?

Brij S avatar
Brij S

@sarkis right! that makes sense, any thoughts on how to go forward with this then

sarkis avatar
sarkis

@Brij S i’d say if you want to continue to go with this dynamically, TF 0.12 upgrade may be the only option… otherwise only way I can think of solving this in 0.11 is to have static cert_validations - hopefully someone can prove me wrong here

Brij S avatar
Brij S

ok, if I was on TF 0.12, how would you go about it

loren avatar
loren

you just count over the variable input again, length(var.certs)

Brij S avatar
Brij S

but how would i reference it?

Brij S avatar
Brij S

aws_acm_certificate.cert.domain_validation_options.0.resource_record_name refers to..one cert?

loren avatar
loren

you want to create a route53 record for each certificate?

Brij S avatar
Brij S

yep

Brij S avatar
Brij S

because for me aws_acm_certificate.cert contains two certs (corresponding to two zones)

loren avatar
loren
resource "aws_route53_record" "cert_validation" {
  count = "${length(var.certs)}"

  name    = "${aws_acm_certificate.cert.domain_validation_options.*.resource_record_name[count.index]}"
  type    = "${aws_acm_certificate.cert.domain_validation_options.*.resource_record_type[count.index]}"
  zone_id = "${data.aws_route53_zone.zone.id}"
  records = ["${aws_acm_certificate.cert.domain_validation_options.*.resource_record_value[count.index]}"]
  ttl     = 60
}
ciastek avatar
ciastek

I had a problem with domain_validation_options when using multiple names from aws_acm_certificate resource - the order of domain_validation_options were undefined. Not sure if it’s still a problem.

Brij S avatar
Brij S

@loren this is awesome, would the following be the same too then?

resource "aws_acm_certificate_validation" "cert" {
  certificate_arn         = "${aws_acm_certificate.cert.arn}"
  validation_record_fqdns = ["${aws_route53_record.cert_validation.fqdn}"]
}
loren avatar
loren

something like that, can also use element(...) interpolation

loren avatar
loren

need the wildcard to reference all the resources…

resource "aws_acm_certificate_validation" "cert" {
  certificate_arn         = "${aws_acm_certificate.cert.arn}"
  validation_record_fqdns = ["${aws_route53_record.cert_validation.*.fqdn}"]
}
loren avatar
loren

there are also more syntax options in terraform 0.12

Brij S avatar
Brij S

"${aws_acm_certificate.cert.arn}" would be "${aws_acm_certificate.cert.*.arn[count.index]}" ?

loren avatar
loren

validation_record_fqdns is a list, so i think you want a single resource there, not multiple with count. just pass the list of all fqdns to the parameter, rather than a single one (using count.index)

:--1:1
loren avatar
loren

if you do want a aws_acm_certificate_validation resource per cert though, then yes, same setup

Brij S avatar
Brij S

@loren mind if I DM you for more questions? Just want to confirm some things to make sure I understand

loren avatar
loren

of course, i may not be online to respond quickly though. kinda doing this in between work tasks

Phuc avatar

Hi Guys

Phuc avatar

I have some issue when running terraform validdate on this modules terraform-null-label

aknysh avatar
aknysh

hi @Phuc

Phuc avatar

Does anyone expenrienced this before ?

  terraform-null-label git:(0.11) ✗ terraform validate

Error: local.generated_tags: local.generated_tags: zipmap: count of keys (1) does not match count of values (0) in:

${zipmap(
    compact(list("Name", local.namespace != "" ? "Namespace" : "", local.environment != "" ? "Environment" : "", local.stage != "" ? "Stage" : "")),
    compact(list(local.id, local.namespace, local.environment, local.stage))
    )}
Phuc avatar

I didn’t adjust anything

Phuc avatar

just running simple command to validate at first

aknysh avatar
aknysh

what TF version?

Phuc avatar

for 0.11 and below

Phuc avatar

I clone the repo on branch 0.11/master

aknysh avatar
aknysh

how do you use it? show the example

Phuc avatar

actually I didn’t used it yet

Phuc avatar

I just try to validate the code first to see if there is error

Phuc avatar

just simple terraform init then terraform validate

Phuc avatar

and that error coming up

aknysh avatar
aknysh

you need to validate module invocation with all vars provided, similar to https://github.com/cloudposse/terraform-null-label/blob/0.11/master/examples/complete/main.tf

cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Phuc avatar

thank @aknysh, I will try to follow that

Phuc avatar

Hi aknysh

Phuc avatar

Just a smal question need you to answer

Phuc avatar

I see there is 2 modules: terraform-label and terraform-null-label on github of cloudposse

Phuc avatar

It seems they are all for naming conversion.

Phuc avatar

so what is the difference between those modules ?

aknysh avatar
aknysh

null-label and terraform-label are mostly the same in terms of naming convention and generating globally unique IDs for AWS resources

aknysh avatar
aknysh

null-label has much more features, e.g. context, additional tags as list of maps outputs, etc.

aknysh avatar
aknysh

but with TF 0.11, all that complex logic in null-label was sometimes throwing the count can't be computed errors in top-level modules

aknysh avatar
aknysh

that’s why we created a simplified version of it and named it terraform-label

aknysh avatar
aknysh

so if you just need a naming convention and globally unique IDs, both could be used

aknysh avatar
aknysh

but both were converted to TF 0.12 now, so null-label should be OK to use (much less count can't be computed errors`

aknysh avatar
aknysh

so try null-label with TF 0.12, it has more features

aknysh avatar
aknysh

(but both modules are supported)

Phuc avatar

thank Akysh, due to current TF 0.11, I think I will test with null-label

Kiran avatar
Kiran

@aknysh Is there any possibility to support TF12 for https://github.com/cloudposse/terraform-aws-ec2-bastion-server module ? I can’t find any issue regarding this?

cloudposse/terraform-aws-ec2-bastion-server

Terraform Module to define a generic Bastion host with parameterized user_data - cloudposse/terraform-aws-ec2-bastion-server

aknysh avatar
aknysh

yes, we’ll convert it. Did not have time yet, we have more than 100 modules, converted 40+ so far

cloudposse/terraform-aws-ec2-bastion-server

Terraform Module to define a generic Bastion host with parameterized user_data - cloudposse/terraform-aws-ec2-bastion-server

Kiran avatar
Kiran

Thanks for the prompt response

2019-08-26

Cloud Posse avatar
Cloud Posse
04:02:31 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Sep 04, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

party_parrot1

2019-08-23

Chad Hartman avatar
Chad Hartman

Do you need to use version Terraform 0.11 when bootstrapping with the reference-architectures?

gk avatar

Hey folks. Is there a way to use a merge or use a splat-type operator in a terraform child block (not identifier values) ? i.e.

data "aws_ami" "potato" {
  filter {
    ...local.default_filters
  }
}
gk avatar

I’m effectively trying to filter merge(local.thing, {})

cabrinha avatar
cabrinha

So, when you’re creating an ECS service, you either have the choice between using an ALB or not.

cabrinha avatar
cabrinha

If you’re using an ALB, you need to pass in additional objects for the load_balancer parameter, example here: https://www.terraform.io/docs/providers/aws/r/ecs_service.html#example-usage

cabrinha avatar
cabrinha

How would you structure a module to make this object parameterized and optional?

cabrinha avatar
cabrinha
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

cabrinha avatar
cabrinha

just create a ton of these different ecs_service resources based on how they’re configured.

Sharanya avatar
Sharanya

The variable “lambda_settings” is required, so Terraform cannot proceed without a defined value for it. “ - any idea about this

2019-08-22

Sharanya avatar
Sharanya

Error: The role “arnawsiam:role/gc-invoicedataimport-function-role” cannot be assumed. There are a number of possible causes of this - the most common are:

* The credentials used in order to assume the role are invalid

* The credentials do not have appropriate permission to assume the role

* The role ARN is not valid
Sam avatar

IMPORTANT: Upcoming change to AWS Cost and Usage Report Access Control Policies on August 19th

2019-08-21

pericdaniel avatar
pericdaniel

Can’t recall but when you create an AWS SG rule with terraform and you do something like count index vs creating separate rules, which one doesn’t delete the whole sg and create a new one? Instead of creating a new sg every time I’d like to just keep adding/removing ports if needed.

from_port = "80" vs from_port = var.allowed_ports[count.index]

I know in your tf you do the second one. let me know! Thank you!

aknysh avatar
aknysh

@pericdaniel when you create a separate aws_security_group w/o rules, you can add as many aws_security_group_rule as you need w/o recreating the whole SG

aknysh avatar
aknysh
cloudposse/terraform-aws-emr-cluster

Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS - cloudposse/terraform-aws-emr-cluster

sarkis avatar
sarkis

Curious, are you all using or looked into dependabot for terraform module dependencies - https://dependabot.com/terraform/

Dependabot

Automated dependency updates for your Ruby, Python, JavaScript, PHP, .NET, Go, Elixir, Rust, Java and Elm.

Addison Higham avatar
Addison Higham

for the cloudposse modules, I got all these working with 0.12: https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/pull/14 https://github.com/cloudposse/terraform-aws-eks-workers/pull/21 https://github.com/cloudposse/terraform-aws-eks-cluster/pull/20

I forgot to update the version for CI to 0.12, will try and push that out

Upgrade to terraform 0.12 by addisonj · Pull Request #14 · cloudposse/terraform-aws-ec2-autoscale-group

This moves us to terraform 0.12, it is working with our usages of this module, but it hasn&#39;t been tested completely with all options, but does appear valid. note that the examples aren&#39;t po…

Terraform 0.12 upgrade by addisonj · Pull Request #21 · cloudposse/terraform-aws-eks-workers

Note, this depends on cloudposse/terraform-aws-ec2-autoscale-group#14 getting merged and then making a change here to reference that new tag. This does the upgrade and also copies the new arguments…

Upgrade to terraform 0.12 by addisonj · Pull Request #20 · cloudposse/terraform-aws-eks-cluster

This moves this module to terraform 0.12, the example isn&#39;t ported, as some of those modules aren&#39;t 0.12 compliant yet, but this is working with our EKS clusters. I notice there are also te…

Addison Higham avatar
Addison Higham

I am using them and they are working

Addison Higham avatar
Addison Higham

but could use some help to get the rest of the work done (porting examples and adding the new CI stuff)

Addison Higham avatar
Addison Higham

except… looking around, I have no idea how to get it 0.12 to run as part of the things

2019-08-20

Callum Robertson avatar
Callum Robertson

Hi All, I’m creating a bucket policy data resource (https://www.terraform.io/docs/providers/aws/r/s3_bucket_policy.html)

I’m running into an issue where I’m trying to reference the ‘json’ attribute of the data source in a policy account resource, can anyone help me with the below?

AWS: aws_s3_bucket_policy - Terraform by HashiCorp

Attaches a policy to an S3 bucket resource.

Callum Robertson avatar
Callum Robertson
Callum Robertson avatar
Callum Robertson

I’m getting the error that it’s an empty tuple, not sure what I’m doing wrong here..

maarten avatar
maarten

@Callum Robertson what about you do at line16 the same as line 2, so the count’s are in sync

maarten avatar
maarten

or what is the idea there, that you only want to apply the policy when var.upload_bucket_objects is set to true, correct ? What happens now is that you refer to a policy you are not creating hence it fails.

Sharanya avatar
Sharanya

encountered the following error when running terraform apply?

[3:57 PM] “policy” contains an invalid JSON: invalid character ‘a’ looking for beginning of value

mpmsimo avatar
mpmsimo

You’d have to paste your policy here for further debugging. Seems like a syntax or formatting issue.

mpmsimo avatar
mpmsimo

Also not sure what provider this is for?

mpmsimo avatar
mpmsimo

AWS, Google Cloud, Sentinel?

mpmsimo avatar
mpmsimo

If you are using AWS, this tool has helped me in the past for creating sane policy document templates:

Sharanya avatar
Sharanya

Thanks That helped

Callum Robertson avatar
Callum Robertson

Thanks @maarten, I think it’s the case of starting at a problem for to long

Callum Robertson avatar
Callum Robertson

I fixed it with the following if anyone is interested

:--1:2
Callum Robertson avatar
Callum Robertson

2019-08-19

Cloud Posse avatar
Cloud Posse
04:05:24 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Aug 28, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

Sharanya avatar
Sharanya

anybody working with terraform integration with Jenkins cicd pipeline

aknysh avatar
aknysh

we use atlantis and Codefresh to deploy terraform, but here are a few articles on how to do it from Jenkins

:--1:1
aknysh avatar
aknysh
Immutable Infrastructure CI/CD Using Hashicorp Terraform and Jenkins - DZone DevOps attachment image

This extensive article shows you how to create an immutable CI/CD infrastructure with Terraform and Jenkins that will make managing your infrastructure easier.

aknysh avatar
aknysh
CI/CD with Jenkins pipelines, part 1: .NET Core application deployments on AWS ECS attachment image

In theory, deploying a dockerised .NET Core app is easy (because Docker simplifies everything, right?). Just trigger your CI/CD pipeline on…

aknysh avatar
aknysh
Terraform Pipelines in Jenkins attachment image

If you’ve followed my last few posts you have now used Terraform to create a Google Kubernetes Engine cluster, and you’ve deployed Jenkins…

Brij S avatar
Brij S

Hello, I was curious to see if anyone has ever tried to create a cross account aws codepipeline with terraform. If not, It would be great to get some feedback on my current approach

mpmsimo avatar
mpmsimo

Does anyone use software to detect state drift on a recurring basis?

I’d like to start performing infrastructure wide Terraform state drift identification, and work towards making sure I see that excellent “No changes.” messaging after issuing a terraform plan across the board more frequently.

loren avatar
loren

We just have our CI run terraform plan -detailed-exitcode, and alert on job failures…

:--1:1
mpmsimo avatar
mpmsimo

I was looking at the detailed-exitcode option, seems good as I can get an explicit list of terraform projects that are having issues. I’d like to go a step further so I can reduce operator work around identification and fixing of the state drift manually.

mpmsimo avatar
mpmsimo

Only saw a few project on GitHub, none of them maintained.

mpmsimo avatar
mpmsimo
digirati-labs/drifter

Check for drift between Terraform definitions and deployed state. - digirati-labs/drifter

mpmsimo avatar
mpmsimo

This one seemed to be the most relevant for my use case.

loren avatar
loren

Cheap and easy and works was my thinking, can always optimize later

loren avatar
loren

Could save off the plan, and analyze it separately. Maybe whitelist some resources/diffs to run apply automatically

mpmsimo avatar
mpmsimo

The Foqual bot was able to find some information regading GitHub Actions to perform plans, could be a good starting point for me.

mpmsimo avatar
mpmsimo

Yeah, very much MVP for this drift detector - iterate later

mpmsimo avatar
mpmsimo

Thanks for the ideas, I’ll think about this some more and see how to approach this

PePe avatar

Hi, should this module : https://github.com/cloudposse/terraform-aws-cloudwatch-logs when a CMK key gets passed to create a policy to be able to use that key ?

2019-08-17

davidvasandani avatar
davidvasandani

Why did the release cadence change for this module? https://github.com/cloudposse/terraform-aws-vpc/releases

cloudposse/terraform-aws-vpc

Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc

Erik Osterman avatar
Erik Osterman

0.4.2 is a patch release against the last version of the module for terraform 0.11

cloudposse/terraform-aws-vpc

Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc

Erik Osterman avatar
Erik Osterman

0.7.0 is the latest release for 0.12

davidvasandani avatar
davidvasandani

Thanks @Erik Osterman

Norbert avatar
Norbert
Hi, I do a bit of TF (0.12) + AWS. I have created a few modules: VPC, Subnets, IGW, etc. Each module [outputs.tf> some variables, I use S3 backend and I can access exported variables from different modules. All smooth and easy but when I create application load balancer (aws_lb) I can NOT make terraform to output any variable. My <http://output.tf output.tf](http://outputs.tf) (root module dir) looks like this:

output “alb_id” { value = aws_lb.alb.id } output “alb-security-group_id” { value = aws_security_group.alb-security-group.id } output “alb-target-group_arn” { value = aws_lb_target_group.alb-target-group.arn }

The ALB, security group and target group gets created, I can see it in the console but the output is empty, no errors during terraform apply. Why?

Barani avatar
Barani

Hi, Anybody using local persistent volume for any kubernetes_stateful_set. I am trying use affinity with ndoe_selector_term but failing. Kindly guide me if anyone using in this way.

2019-08-16

praveen avatar
praveen

Hi, I am using terraform workspace to create azure windows server’s in more than one environment using one terraform code. For passing hostname and IP address am using the following lookup’s in local

praveen avatar
praveen

locals { location = “${terraform.workspace}” image_id = “${module.image.image_id}” environment = “${module.locals.environment}”

subnets = { eastus2_prod = “${data.terraform_remote_state.shared_networking.eastus2_api_tier_subnet.id}” centralus_prod = “${data.terraform_remote_state.shared_networking.centralus_api_tier_subnet.id}” }

lb_ips = { eastus2_prod = “10.244.160.164” centralus_prod = “10.245.160.164” }

 system = {
eastus2_prod   = [   {
hostname = "wqilpeap101"
ip       = "10.244.160.165"   },   {
hostname = "wqilpeap102"
ip       = "10.244.160.166"   }, ]

centralus_prod = [   {
hostname = "wqilpcap101"
ip       = "10.245.160.165"   },   {
hostname = "wqilpcap102"
ip       = "10.245.160.166"   }, ]

}

subnet_id = “${lookup(local.subnets, format(“%s_%s”,local.location, var.environment))}” lb_ip = “${lookup(local.lb_ips, format(“%s_%s”,local.location, var.environment))}” systems = “${lookup(local.system, format(“%s_%s”,local.location, var.environment))}” }

praveen avatar
praveen

it fails with following error message

praveen avatar
praveen

Error: Error asking for user input: 1 error occurred: * local.systems: local.systems: lookup: lookup() may only be used with flat maps, this map contains elements of type list in:

${lookup(local.system, format(“%s_%s”,local.location, var.environment))}

praveen avatar
praveen

can you help me fix this

maarten avatar
maarten

@praveen you can try: subnet_id = “${local.subnets[ format(“%s_%s”,local.location, var.environment)]}”

praveen avatar
praveen

subnet is working fine

praveen avatar
praveen

issue is with systems

praveen avatar
praveen

should I try it for systems?

maarten avatar
maarten

yes

praveen avatar
praveen

Hey, its working. Thanks a lot for the help

praveen avatar
praveen

trying now

Alex Siegman avatar
Alex Siegman

So, I’m trying to be a bit clever. I have a need to conditionally add statements to an IAM policy document

I did try to do a join on the data.aws_iam_policy_document.<stuff>.json property of multiple data sources, and mixed data sources and already rendered json documents coming in as variables in strings.

I think the root problem with this approach is that both things will render a FULL json document, so it will confuse stuff, such as:

{
  my policy doc
}
{
  my next policy doc
}

Is there a way to conditionally add statement blocks to a data.aws_iam_policy_document ?

The root cause here is that the cloudposse/terraform-aws-s3-module has some built in document handling to set an “encrypted-only” policy, so if I need to do something like add a separate cross-account access principal policy, I can’t, because an s3 bucket can only have one bucket policy attached.

I’ve forked it and attempted my above described fix here: https://github.com/asiegman/terraform-aws-s3-bucket/blob/moar-bucket-policy-0.11/main.tf#L94

But alas, that didn’t work due to the multiple json documents being joined to form invalid json.

I can always not use cloudposse’s module and just build my own resources, but if I could add a clever bit to add arbitrary statements in for stuff like this, I was going to deliver it back to the community

maarten avatar
maarten

@Alex Siegman in 0.12 you can do this with “dynamic” loops. What you could do with 0.11 is using source_json with another aws_iam_policy_document. I don’t really like it so much but it’s a funny hack: https://github.com/doingcloudright/terraform-aws-ecr-cross-account/blob/ab55861e4de158d3bf490976c16a2bebb6661c28/main.tf#L43

doingcloudright/terraform-aws-ecr-cross-account

Terraform module to create an ECR repo with cross-account-access - doingcloudright/terraform-aws-ecr-cross-account

Alex Siegman avatar
Alex Siegman

Oh interesting. What happens if source json is just a blank string then~? I’ll play with it, great lead. Thanks!

maarten avatar
maarten

That won’t work, but you can have one policy statement which would be valid for all your policies, and use that one to start ‘sourcing’ from.

:--1:1
maarten avatar
maarten
cloudposse/terraform-aws-iam-policy-document-aggregator

Terraform module to aggregate multiple IAM policy documents into single policy document. - cloudposse/terraform-aws-iam-policy-document-aggregator

Alex Siegman avatar
Alex Siegman

heck, i can probably just use that, i already have multiple valid documents, i just need to aggregate all their statements in to one

:--1:2

2019-08-15

PePe avatar

Anyone with experience in

resource "aws_ssm_document

and after destroy the document does not seems to be deleted

Sharanya avatar
Sharanya

Having this error on Tf Plan - Error: Missing resource instance key

on .terraform\modules\vpc_peering_cross_account[accepter.tf](http://accepter.tf) line 96, in locals: 96: accepter_aws_route_table_ids = “${distinct(sort(data.aws_route_tables.accepter.ids))}”

Alejandro Rivera avatar
Alejandro Rivera

@Sharanya I just shared a solution I’m using for vpc peering connection x accounts and x regions that might help a couple of posts up, I’ll tag you

SweetOps avatar
SweetOps
06:02:25 PM
Are you using some of our [terraform-modules> in your projects? Maybe you could <https://cpco.io/leave-testimonial leave us a testimonial](https://cpco.io/terraform-modules)! It means a lot to us to hear from people like you.
Bharat avatar
Bharat

How do i set order precedence in terraform? I have main.tf where it destroys a IAM role and deploy’s it again on terraform-apply. I am hitting a error ` aws_iam_role.service_role: Error creating IAM Role DEFAULT-TestingService-ecs-service-role: EntityAlreadyExists: Role with name DEFAULT-TestingService-ecs-service-role already exists`

This is after destroying the role, pretty sure that AWS needs some time to update the cache.

I want to first destroy the services and then create it. Is this possible?

aknysh avatar
aknysh

hmmm…. terraform is used to create resource definitions - how to create them

aknysh avatar
aknysh

terraform knows from its state file what was created

aknysh avatar
aknysh

how are you doing destroy from TF files?

PePe avatar

Hi, do you guys have a preference on using the aws KMS managed key or creating a CMK ?

PePe avatar

the reason I ask that is that after removing Kms_key_id from :

module "ssm_tls_ssh_key_pair" {
  source               = "git::<https://github.com/cloudposse/terraform-aws-ssm-tls-ssh-key-pair.git?ref=0.2.0>"
  namespace            = "${var.namespace}"
  stage                = "${var.stage}"
  name                 = "${var.name}"
  attributes           = "${var.attributes}"
  
  ssm_path_prefix      = "${var.stage}/${var.name}/infrastructure/ssh_keys"
  ssh_key_algorithm    = "RSA"
  ssh_private_key_name = "${module.default_label.id}-private"
  ssh_public_key_name  = "${module.default_label.id}-public"
  #kms_key_id           = "${module.kms_key.key_id}"
}
PePe avatar

I got

Error refreshing state: 1 error occurred:
	* module.ssm_tls_ssh_key_pair.data.aws_kms_key.kms_key: 1 error occurred:
	* module.ssm_tls_ssh_key_pair.data.aws_kms_key.kms_key: data.aws_kms_key.kms_key: error while describing key [alias/test-demo-chamber]: NotFoundException: Alias arn:aws:kms:us-east-1:046894643055:alias/test-demo-chamber is not found.
	status code: 400, request id: 7e387954-2256-4ef2-a40e-b48269259e9c
Maeghan Porter avatar
Maeghan Porter

Hey all, I updated the release of terraform-aws-dynamic-subnets that I’m pulling down and now I’m getting this error:

Error downloading modules: Error loading modules: module dynamic_subnets: Error parsing .terraform/modules/e972fa1c1c4c2e3a44d52f7491016697/label.tf: At 3:25: Unknown token: 3:25 IDENT var.attributes
Maeghan Porter avatar
Maeghan Porter

Any idea what’s going on?

aknysh avatar
aknysh
cloudposse/terraform-aws-dynamic-subnets

Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets

aknysh avatar
aknysh
cloudposse/terraform-aws-dynamic-subnets

Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets

Maeghan Porter avatar
Maeghan Porter

oh ok, I’m on TF 0.11.13 and trying to use release 0.16.0 of the module. That must be why then.

Maeghan Porter avatar
Maeghan Porter

I’ll try version 0.12.0

Maeghan Porter avatar
Maeghan Porter

thanks!

Joe Presley avatar
Joe Presley

I’m looking to move terraform code for a customer from regular terraform to terraform enterprise. I googled but couldn’t find if there are any quirks or things I should know about TFE. My goal is to create a terraform module that can be used for both the open source version as well as TFE.

igeorgopoulos avatar
igeorgopoulos

I have an existing aws ecs cluster (made by hand) and would like to update it by using terraform. Is there any documentation for updating existing aws services (alb, ecr, ecs)?

2019-08-14

Szymon avatar
Szymon

Hi, is there any possibility to setup azurerm app service deployment from bitbucket automatically? scm_type block doesn’t work, as described in this issue: https://github.com/terraform-providers/terraform-provider-azurerm/issues/3696

Do you see any alternatives?

azurerm_app_service unable to configure source control. · Issue #3696 · terraform-providers/terraform-provider-azurerm

Community Note Please vote on this issue by adding a :–1: reaction to the original issue to help the community and maintainers prioritize this request Please do not leave &quot;+1&quot; or &quot;me to…

Chris Montes avatar
Chris Montes

Hey guys, anyone know how to get terraform-aws-elastic-beanstalk-environment to attach security groups directly to the launch config this module creates? Currently, whatever SGs you list under var.security_groups, they all get added as security group rules of a new SG that this module creates, instead of actually associating the SGs themselves directly to the Launch Config… https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/blob/master/main.tf#L318

Chris Montes avatar
Chris Montes

https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/blob/master/main.tf#L489 I guess the best way is to fork this module and customize it here?

cloudposse/terraform-aws-elastic-beanstalk-environment

Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment

aknysh avatar
aknysh

in all Cloud Posse modules we usually create a new SG and then add existing SGs and CIDRs blocks to it to allow ingress

Chris Montes avatar
Chris Montes

But how would you allow ingress from external IPs, etc? as nested security groups DO NOT work like that….

aknysh avatar
aknysh

we use external SGs and external CIDRs

aknysh avatar
aknysh

not sure if https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment uses both (we used the module more than a year ago)

cloudposse/terraform-aws-elastic-beanstalk-environment

Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment

Chris Montes avatar
Chris Montes

https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/blob/master/main.tf#L486 This setting should be refactored to support a list of paramterized strings

cloudposse/terraform-aws-elastic-beanstalk-environment

Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment

github140 avatar
github140

Hi, I have a general Terraform question. Is anybody using it for deployment of and to HyperV? I found a community provider however I’m a little bit hesitant in using it. Which other gitops based tool would be available for HyperV deployments?

Erik Osterman avatar
Erik Osterman

@github140 saw @chrism talking about hyperv in #geodesic last month

chrism avatar
chrism

Nah it was just that Docker for Windows creates its VM in hyper-v on windows. We use vsphere/aws/azure

Erik Osterman avatar
Erik Osterman

Erik Osterman avatar
Erik Osterman

Our public weekly #office-hours is starting now! Join https://zoom.us/j/508587304

Alejandro Rivera avatar
Alejandro Rivera

Hi, I’m trying to create a aws_vpc_peering_connection between 2 accounts, cross region using assume-role and specifying a aws provider for the aws_vpc_peering_connection_accepter with corresponding region. This fails because the requester tries to look for the accepter vpc in the same region and fails to find the vpc. I’ve used this code to successfully deploy cross account, but on the same region. Using TF version 0.11.13

Alejandro Rivera avatar
Alejandro Rivera

This is what the module I’m using looks like:

data "aws_vpc" "accepter" {
  provider = "aws.accepter"
  id = "${var.accepter_vpc_id}"
}

locals {
  accepter_account_id = "${element(split(":", data.aws_vpc.accepter.arn), 4)}"
}

resource "aws_vpc_peering_connection" "requester" {
  vpc_id = "${var.requester_vpc_id}"
  peer_vpc_id = "${data.aws_vpc.accepter.id}"
  peer_owner_id = "${local.accepter_account_id}"

  tags {
    Name = "peer_to_${var.accepter_tag}"
  }
}

resource "aws_vpc_peering_connection_accepter" "accepter" {
  provider = "aws.accepter"
  vpc_peering_connection_id = "${aws_vpc_peering_connection.requester.id}"
  auto_accept = true

  tags {
    Name = "peer_to_${var.requester_tag}"
  }
}


\#######################

\# ROUTE TABLE UPDATES #

\#######################

data "aws_vpc" "requester" {
  id = "${var.requester_vpc_id}"
}

data "aws_route_tables" "requester" {
  vpc_id = "${var.requester_vpc_id}"
}

data "aws_route_tables" "accepter" {
  provider = "aws.accepter"
  vpc_id = "${data.aws_vpc.accepter.id}"
}

resource "aws_route" "requester" {
  count = "${length(data.aws_route_tables.requester.ids)}"
  route_table_id = "${data.aws_route_tables.requester.ids[count.index]}"
  destination_cidr_block = "${data.aws_vpc.accepter.cidr_block}"
  vpc_peering_connection_id = "${aws_vpc_peering_connection.requester.id}"
}

resource "aws_route" "accepter" {
  provider = "aws.accepter"
  count = "${length(data.aws_route_tables.accepter.ids)}"
  route_table_id = "${data.aws_route_tables.accepter.ids[count.index]}"
  destination_cidr_block = "${data.aws_vpc.requester.cidr_block}"
  vpc_peering_connection_id = "${aws_vpc_peering_connection.requester.id}"
}
Alejandro Rivera avatar
Alejandro Rivera

and this is how I defined the provider:

provider "aws" {
  max_retries            = "5"
  profile                = "${var.aws_profile_name}"
  region                 = "${var.accepter_region}"
  skip_get_ec2_platforms = true
  skip_region_validation = true

  alias = "accepter"
  assume_role {
    role_arn = "${var.accepter_role_arn}"
  }
}
Alejandro Rivera avatar
Alejandro Rivera

So this works perfectly if both VPC’s are on the same region, but when one of the VPC’s is on another region the requester peering connection regions (both accepter and requeter) shows up in AWS console as the same as the requester, thus failing to find the VPC.

Alejandro Rivera avatar
Alejandro Rivera

Is there a way to specify the accepter’s region?

Callum Robertson avatar
Callum Robertson

Hi @Alejandro Rivera I haven’t done this specifically with peering connections, but I have done this with a TGW

Callum Robertson avatar
Callum Robertson

I had to set up a Resource Share that contained my VPC ID and shared it with the requester VPC

Callum Robertson avatar
Callum Robertson

alternatively, you can share the resource within an OU in your AWS organisation

loren avatar
loren

On the peering connection resource, set the argument peer_region?

https://www.terraform.io/docs/providers/aws/r/vpc_peering.html#peer_region

AWS: aws_vpc_peering_connection - Terraform by HashiCorp

Provides a resource to manage a VPC peering connection.

Alejandro Rivera avatar
Alejandro Rivera

@loren Since the peering connection is the one from the account I’m creating this, this one does get the region correctly set, the problem comes with the accepting peering connection which doesn’t take in peer_region but takes in a provider which has the correct region set, but won’t recognize it.

Alejandro Rivera avatar
Alejandro Rivera

@Callum Robertson Thanks!, will try that out and let you know if that helps in this case also.

loren avatar
loren

What you describe is exactly what peer_region exists for

loren avatar
loren

In the requesting account, it creates the peer request, the request must set the region in which the vpc peering connection will be accepted. you then accept it exactly as you are

1
Alejandro Rivera avatar
Alejandro Rivera

@loren omg, trying that out right now and I’ll come back with results

Alejandro Rivera avatar
Alejandro Rivera

@Callum Robertson Didn’t get to try your approach since @loren’s solution worked out, I had misunderstood that value and thought it referred to the requester vpc, thank you both again for the help!

Alejandro Rivera avatar
Alejandro Rivera

@Sharanya You can see the code I’m using at the top, hope it helps

Patrick Beam avatar
Patrick Beam

I’ve been tinkering with this module today. https://github.com/cloudposse/terraform-aws-ec2-instance-group Is there a way to get instances spread across mutiple AZ’s?

cloudposse/terraform-aws-ec2-instance-group

Terraform Module for provisioning multiple general purpose EC2 hosts for stateful applications. - cloudposse/terraform-aws-ec2-instance-group

Erik Osterman avatar
Erik Osterman

One strategy is to provision the module once per AZ

cloudposse/terraform-aws-ec2-instance-group

Terraform Module for provisioning multiple general purpose EC2 hosts for stateful applications. - cloudposse/terraform-aws-ec2-instance-group

Erik Osterman avatar
Erik Osterman

that gives you the most guarantee of even distribution

Erik Osterman avatar
Erik Osterman

incidentally, this is the strategy that kops takes when provisioning ASGs for master nodes.

Patrick Beam avatar
Patrick Beam

Thanks for the response. I like the kops approach as you do get guaranteed provisioning across AZ’s. It just generates a lot more code but it’s a fair tradeoff.

Erik Osterman avatar
Erik Osterman

yea, it’s a trade off but as you say, probably fair and easy to understand what’s going on.

Patrick Beam avatar
Patrick Beam

I think it would be nice to have both options. I’ve forked the repo and time permitting i’ll try and add that feature. I do agree though its easier to read. I find this module far less magical than some of the others I have looked at. Thanks so much for open sourcing and sharing all of these modules.

Callum Robertson avatar
Callum Robertson

@Patrick Beam you could provide a list of availability zones for that region e.g. [“a”, “b”, “c”] and then in your availability_zone input use this:

Patrick Beam avatar
Patrick Beam

I think that will work. The problem i’m seeing is with subnet which is required. I pass the list of subnets created into this module like this.

variable "public_subnet_ids" {}

subnet =var.public_subnet_ids[0] 

without that index position terraform throws an error. When I try and create a new variable subnets with the following.

variable "subnets" {
  description = "A list of VPC Subnet IDs to launch in"
  type        = list(string)
  default     = []
}

\#In the instance resource I changed this.
subnet_id                   = element(distinct(compact(concat([var.subnet], var.subnets))),count.index,)

\#inside the module I set subnet to subnets
  subnets                     = "${var.public_subnet_ids}"

When I plan I get the following error.

Error: Missing required argument

  on instances/instance.tf line 18, in module "versio":
  18: module "versio" {

The argument "subnet" is required, but no definition was found.

I’m curious how that argument subnet is required. I don’t understand how that is defined in the module and can’t seem to track that down in the repo.

Callum Robertson avatar
Callum Robertson

availability_zone = “${element(var.availability_zones, count.index)}”

Callum Robertson avatar
Callum Robertson

You would just have to change that variable to a type = list(string)

Callum Robertson avatar
Callum Robertson

hope that helps

2019-08-13

maarten avatar
maarten

Hi Everyone, I’m having the issue that when I attach a private EIP to an instance in a private subnet,, that associate_public_ip_address gets set to true. This with a subnet with public ip mapping to false. Maybe someone else stumpled upon the same issue ?

joshmyers avatar
joshmyers

@maarten what module is this?

maarten avatar
maarten

“terraform-aws-modules/ec2-instance/aws” but it’s irrelevant, it’s not a module problem

joshmyers avatar
joshmyers

No but I just want to have a look

joshmyers avatar
joshmyers

and what is the actual problem? You don’t want associate_public_ip_address: true ?

maarten avatar
maarten

\# Grafana
module "grafana" {
  source  = "terraform-aws-modules/ec2-instance/aws"
  version = "~> 2.0"

  name           = "grafana-xlt"
  instance_count = 1

  ami                    = var.grafana_ami
  instance_type          = "m4.xlarge"
  key_name               = ""
  monitoring             = true
  vpc_security_group_ids = [module.ec2_sg.this_security_group_id]
  subnet_id              = module.vpc.private_subnets[0]
  #private_ip = "10.0.1.200"
   associate_public_ip_address = false

  user_data = "{\"auth\": [ {\"name\": \"admin\", \"pass\": \"${var.password}\"}]}"

  tags = {
    Name   = "grafana"
    Terraform   = "true"
    Environment = "dev"
  }
}


\#resource "aws_eip" "grafana" {

\#  vpc = true

\#  associate_with_private_ip = "10.0.1.200"

\#}

\#

\#resource "aws_eip_association" "grafana" {

\#  instance_id                  = module.grafana.id[0]

\#  allocation_id = aws_eip.grafana.id

\#}
joshmyers avatar
joshmyers

OK, makes more sense seeing that. saying that, I haven’t come across this

joshmyers avatar
joshmyers

Does the instance then get a routable public IP that you don’t want it to have?

maarten avatar
maarten

yep

joshmyers avatar
joshmyers

Can you deny outside world access via SG rule?

maarten avatar
maarten

It’s a human error I think. I thought the EIP would be private, but it’s actually a public EIP .. associated with a private address

sarkis avatar
sarkis

ah i don’t think you can have a private EIP - you’d need ENI for that … assuming I understood correctly what you want (a private static ip?)

maarten avatar
maarten

Yes, but you can just assign that directly, I wasn’t thinking clearly .

:--1:1
joshmyers avatar
joshmyers

Don’t think that actually works does it?

aknysh avatar
aknysh

EIPs are always public

Hugo Lesta avatar
Hugo Lesta

Hello there, do you have any fargate terraform module to analyze and implement in a production environment?

aknysh avatar
aknysh
cloudposse/terraform-aws-ecs-web-app

Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more. - cloudposse/terraform-aws-ecs-web-app

aknysh avatar
aknysh

it uses Fargate

aknysh avatar
aknysh

used by https://github.com/cloudposse/terraform-aws-ecs-atlantis (which deploys atlantis on ECS Fargate)

cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

aknysh avatar
aknysh

the complete atlantis solution is here https://github.com/cloudposse/terraform-root-modules/tree/master/aws/ecs (uses the two modules above and more)

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Hugo Lesta avatar
Hugo Lesta

@aknysh thankss

2019-08-12

Cloud Posse avatar
Cloud Posse
04:01:16 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Aug 21, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

Vitaliy Lobachev avatar
Vitaliy Lobachev

Hello! I want to share with you resource:for_each and dynamicfor_each construction for terraform 12.6 . Hope this helps you work with large arrays of resources. https://github.com/devops-best-practices/terraform-best-practice/blob/master/s3.tf

devops-best-practices/terraform-best-practice

Contribute to devops-best-practices/terraform-best-practice development by creating an account on GitHub.

:--1:1
Sharanya avatar
Sharanya

Hey people, looking for terraform template on vpc peering ( syntax 0.12) any help plz

foqal avatar
foqal
05:39:38 PM

@Sharanya’s question was answered by <@Foqal>

Vikram Yerneni avatar
Vikram Yerneni

Hola fellas….

Vikram Yerneni avatar
Vikram Yerneni

Quick question here… We have a Terraform RDS module (typical base build format) for the build deployments to use to setup RDS instances in our AWS Setups. Now I am tying to enable alerting (SNS topics with Cloudwatch) within the existing RDS module but not sure how to enable the alerting within an existing RDS module. I found out eh cloud posse GitHub repo (https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms) will give the ability to create the sns topics (please do correct me if I am wrong here), but what I need is to enable the alerting within the RDS module so that the users will create the required sns topics based on their needs. Anyone worked on this kind of typical setups before? Any input helps me for my cause here

cloudposse/terraform-aws-rds-cloudwatch-sns-alarms

Terraform module that configures important RDS alerts using CloudWatch and sends them to an SNS topic - cloudposse/terraform-aws-rds-cloudwatch-sns-alarms

Vikram Yerneni avatar
Vikram Yerneni

or even anyone been through this kind of requirement before?

aknysh avatar
aknysh

you can use the alarms from https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms/blob/master/alarms.tf (update them and add new ones if needed)

cloudposse/terraform-aws-rds-cloudwatch-sns-alarms

Terraform module that configures important RDS alerts using CloudWatch and sends them to an SNS topic - cloudposse/terraform-aws-rds-cloudwatch-sns-alarms

aknysh avatar
aknysh

then you can create an SNS topic in diff module (or manually, or however you need it), and subscribe the RDS instance to the topic https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms/blob/master/main.tf#L20

cloudposse/terraform-aws-rds-cloudwatch-sns-alarms

Terraform module that configures important RDS alerts using CloudWatch and sends them to an SNS topic - cloudposse/terraform-aws-rds-cloudwatch-sns-alarms

aknysh avatar
aknysh

(the module currently always creates an SNS topic)

PePe avatar

and since we are talking about SNS alarms :

PePe avatar

I’m using : “git://github.com/cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms.git?ref=tags/0.6.1>” and I notice when using the newer alb module that target_group_name = “${module.alb.target_group_name}” and target_group_arn_suffix = “${module.alb.target_group_arn_suffix}” are not valid outputs anymore so it can’t be use with this cloudwatch-sns module, are you guys deprecating the use of the cloudwatch-sns-alarms or recommend something else ?

PePe avatar

@aknysh I think that maybe this one it has not be updated to reflect changes on the alb module

aknysh avatar
aknysh
cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms

Terraform module to create CloudWatch Alarms on ALB Target level metrics. - cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms

aknysh avatar
aknysh

what alb module are you using that uses terraform-aws-alb-target-group-cloudwatch-sns-alarms?

PePe avatar

I took that from ECS web app example

aknysh avatar
aknysh

it uses alb_ingress, not terraform-aws-alb-target-group-cloudwatch-sns-alarms at that line

aknysh avatar
aknysh

this uses alarms, but does not use any outputs from it https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/master/main.tf#L170

cloudposse/terraform-aws-ecs-web-app

Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more. - cloudposse/terraform-aws-ecs-web-app

PePe avatar

HAHAHAHAH well so you know I’m dyslexic

PePe avatar

for example

PePe avatar

Thanks again

PePe avatar

maybe I’m too old for this

aknysh avatar
aknysh

haha, it happens feel free to ask any questions

Vikram Yerneni avatar
Vikram Yerneni

Sure @aknysh… Let me give it a try and will update here in the group with the progress..

sarkis avatar
sarkis
:--1:3
Erik Osterman avatar
Erik Osterman

Interesting!

sarkis avatar
sarkis

@Erik Osterman just implemented this.. works really well

Erik Osterman avatar
Erik Osterman

can you zoom?

Erik Osterman avatar
Erik Osterman

I want to see what you did

Erik Osterman avatar
Erik Osterman

The part I’m still miffed about is that I can’t use it to terraform init -from-module

Erik Osterman avatar
Erik Osterman

Basically, I want to be dry across repositories

Erik Osterman avatar
Erik Osterman

I don’t want to be dry just in a single repository

Erik Osterman avatar
Erik Osterman

I want to define my root modules once

Erik Osterman avatar
Erik Osterman

I want to use them all over the place.

Erik Osterman avatar
Erik Osterman

ya, so ugh, i see now that I look closer.

Erik Osterman avatar
Erik Osterman

tfworkspacesettings = yamldecode(local.tfsettingsfilecontent) is the operative line. this is nice. i get what they are doing. we’ll probably use some thing like this.

Erik Osterman avatar
Erik Osterman

but still this really assumes a monorepo infrastructure strategy and depends on workspaces.

Erik Osterman avatar
Erik Osterman

it’s nice though. pretty elegant. very easy to understand.

sarkis avatar
sarkis

sorry missed these messages… looks like you got it though… yamldecode from 0.12.x made this a possibility i can zoom tomorrow if you still need it

Vikram Yerneni avatar
Vikram Yerneni

Also @aknysh, just a quick fyi.. we dont use the IAM policy in the main.tf for the SNS topics creation as we use the user aces at higher levels in our build deployments. Is there other way to comment that part out of the module?

aknysh avatar
aknysh

if you don’t use aws_sns_topic, then you don’t need the policy as well

aknysh avatar
aknysh

comment out of the module: you can fork it and comment out aws_sns_topic and the policy, or you can open a PR and add new var sns_topic_enabled (set to true by default for backwards compatibility). Then use count = "${var.sns_topic_enabled == "true" ? 1 : 0}" for aws_sns_topic and aws_db_event_subscription and aws_sns_topic_policy`

Vikram Yerneni avatar
Vikram Yerneni

Without using the aws_sns_topic, how can I subscribe the RDS instance to the topic https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms/blob/master/main.tf#L20 (from your above statement dude)?

cloudposse/terraform-aws-rds-cloudwatch-sns-alarms

Terraform module that configures important RDS alerts using CloudWatch and sends them to an SNS topic - cloudposse/terraform-aws-rds-cloudwatch-sns-alarms

aknysh avatar
aknysh

you need an SNS topic to subscribe the instance to a topic

Vikram Yerneni avatar
Vikram Yerneni

ok ok

Vikram Yerneni avatar
Vikram Yerneni

let me comment out the section from the module and will try it out

2019-08-11

Leonard Wood avatar
Leonard Wood

Are there any known bugs with the ec2 module, release 0.11?

Leonard Wood avatar
Leonard Wood

deployed an ec2 instance using the ec2 module, and had associate_public_ip_address set to true. Changed it to false from true, which then prompted a redeploy, now it fails with the error: value = coalesce( aws_eip.default is empty tuple aws_instance.default is tuple with 1 element Call to function “coalesce” failed: no non-null, non-empty-string arguments.

Leonard Wood avatar
Leonard Wood

I cant destroy or apply any updates to my entire deployment

Leonard Wood avatar
Leonard Wood

looks like there is a pull request for this specific issue: https://github.com/cloudposse/terraform-aws-ec2-instance/pull/45

Fix `public_ip` output variable error when instance is not enabled. by pkonieczny · Pull Request #45 · cloudposse/terraform-aws-ec2-instance

Terraform changed handling of coalesce function to error out when there are no non-null non-empty elements on the list. This results in an error while configuring an instance with no EIP assigned t…

Leonard Wood avatar
Leonard Wood

does anyone know if there is a temp workaround for this?

maarten avatar
maarten

@LeoGmad you can fork the branch of the PR and use that one ?

Leonard Wood avatar
Leonard Wood

I will try, thank you @maarten

Leonard Wood avatar
Leonard Wood

I was able to successful fork the PR but issue still persists. I would be interested if anyone has been successful with this PR or finding a workaround

Leonard Wood avatar
Leonard Wood

the only way to resolve is to set all instances “associate_public_ip_address = true”

Leonard Wood avatar
Leonard Wood

which I guess isnt a big deal as long as they’re deployed behind a NAT or limited ACL

maarten avatar
maarten

@Leonard Wood ok let me run the example with associate_public_ip_address set to false, and see what can be done.

maarten avatar
maarten

@Leonard Wood see if it works with the new pr

Leonard Wood avatar
Leonard Wood

I tried with the new PR but no luck. I am deploying 2 ec2 instances, one set to true and one to set false - and thats when the issue occurs. All ec2 instance deployments have to be set to ‘true’ for it to deploy.

maarten avatar
maarten

@Leonard Wood make sure to clean your cache. I’m running ‘examples/complete’ with instance_enabled = falseand that works

Leonard Wood avatar
Leonard Wood

Interesting - why the instance_enabled = false flag though?

Leonard Wood avatar
Leonard Wood

I did delete the .terraform directory and re init so that should have cleared the cache

Leonard Wood avatar
Leonard Wood

thanks again @maarten for looking into this too

maarten avatar
maarten

sure, ah I’ve tried differnent options, including setting instance_enabled to false, which was the problem with the original PR.

Leonard Wood avatar
Leonard Wood

but if that flag is still to false, will the instance deploy?

Leonard Wood avatar
Leonard Wood

“Flag to control the instance creation. Set to false if it is necessary to skip instance creation”

Leonard Wood avatar
Leonard Wood

set* to false

2019-08-10

Milos Backonja avatar
Milos Backonja

Do you have module which covers ACM Certificate for CloudFront if Route 53 HostedZone is not in us-east-1?

aknysh avatar
aknysh
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Milos Backonja avatar
Milos Backonja

Thanks @aknysh Of course, its provider thing, I may miss something, but I don’t see how this reference address my use case

Milos Backonja avatar
Milos Backonja

If I see well it presume that Hosted Zone is in same Region

Milos Backonja avatar
Milos Backonja

or i take that back

Milos Backonja avatar
Milos Backonja

sorry

Milos Backonja avatar
Milos Backonja

Hosted Zones are global

aknysh avatar
aknysh

Yes hosted zones are global

aknysh avatar
aknysh

You create a certificate in us-east-1 using different provider

aknysh avatar
aknysh

Then reference the cert ARN when you create a distribution

Milos Backonja avatar
Milos Backonja

@aknysh Thank, I did that, but i messed up something else which made me problem

Milos Backonja avatar
Milos Backonja

provisioning atm, everything looks good so far

Julio Tain Sueiras avatar
Julio Tain Sueiras

regarding the HCL deprecation on github action

Julio Tain Sueiras avatar
Julio Tain Sueiras

is very likely due to the recent news that github action will have built-in CI/CD

Julio Tain Sueiras avatar
Julio Tain Sueiras

and given github is owned by microsoft

Julio Tain Sueiras avatar
Julio Tain Sueiras

is very likely the backend is azuredevops

Julio Tain Sueiras avatar
Julio Tain Sueiras

and until next tuesday, automation for azuredevops pipeline is in yaml

Julio Tain Sueiras avatar
Julio Tain Sueiras

@aknysh after much time, upcoming tuesday is the decided to release the azuredevops terraform provider

Julio Tain Sueiras avatar
Julio Tain Sueiras

decided day*

2019-08-09

Leonard Wood avatar
Leonard Wood

Hi everyone - new to the channel but was hoping to find a solution to a problem I am running into dealing with output variables from modules

Leonard Wood avatar
Leonard Wood

I use the aws vpc terraform module, and there is a specific output that that gets created as a list, specifically the database subnet output. I am trying to reference this output as an input variable for an rds module

Leonard Wood avatar
Leonard Wood

does anyone know how to properly reference an output list generated by a module, as an input variable for another module?

Leonard Wood avatar
Leonard Wood

any guidance or direction would be sincerely appreciated

aknysh avatar
aknysh

outputs types are the same regardless whether it’s a list or a string

aknysh avatar
aknysh
cloudposse/terraform-aws-emr-cluster

Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS - cloudposse/terraform-aws-emr-cluster

aknysh avatar
aknysh

and another example where the list output from the subnet module goes into the elasticsearch module https://github.com/cloudposse/terraform-aws-elasticsearch/blob/master/examples/complete/main.tf#L33

cloudposse/terraform-aws-elasticsearch

Terraform module to provision an Elasticsearch cluster with built-in integrations with Kibana and Logstash. - cloudposse/terraform-aws-elasticsearch

aknysh avatar
aknysh
cloudposse/terraform-aws-rds

Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.

aknysh avatar
aknysh
cloudposse/terraform-aws-rds-cluster

Terraform module to provision an RDS Aurora cluster for MySQL or Postgres - cloudposse/terraform-aws-rds-cluster

Leonard Wood avatar
Leonard Wood

bingo!

Leonard Wood avatar
Leonard Wood

thank you @aknysh

PePe avatar
PePe
04:53:55 PM

any ideas on this ?

and now I think I found a bug , I’m getting this :

Error: Error running plan: 1 error occurred:
	* module.alb_ingress.local.target_group_arn: local.target_group_arn: Resource 'aws_lb_target_group.default' not found for variable 'aws_lb_target_group.default.arn'
PePe avatar

I know the posts from yesterday usually get buried….

aknysh avatar
aknysh

what module are you using? can you share the code

PePe avatar

and now I think I found a bug , I’m getting this :

Error: Error running plan: 1 error occurred:
	* module.alb_ingress.local.target_group_arn: local.target_group_arn: Resource 'aws_lb_target_group.default' not found for variable 'aws_lb_target_group.default.arn'
PePe avatar

mmm that did not work, my complete post is just a higher

aknysh avatar
aknysh
cloudposse/terraform-aws-ecs-web-app

Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more. - cloudposse/terraform-aws-ecs-web-app

aknysh avatar
aknysh
cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

aknysh avatar
aknysh
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

PePe avatar

that is exactly where I got the example :

module "alb_ingress" {
  #source            = "git::<https://github.com/cloudposse/terraform-aws-alb-ingress.git?ref=tags/0.7.0>"
  source            = "../terraform-aws-alb-ingress"
  name              = "${var.name}"
  namespace         = "${var.namespace}"
  stage             = "${var.stage}"
  attributes        = "${var.attributes}"
  vpc_id            = "${var.vpc_id}"
  port              = "${var.container_port}"
  health_check_path = "${var.health_check_path}"
  target_group_arn  = "${module.alb.default_target_group_arn}"

  # Without authentication, both HTTP and HTTPS endpoints are supported
  unauthenticated_listener_arns       = ["${module.alb.listener_arns}"]
  unauthenticated_listener_arns_count = 1

  # All paths are unauthenticated
  unauthenticated_paths    = ["/*"]
  unauthenticated_priority = "100"
}
PePe avatar
locals {
  target_group_enabled = "${var.target_group_arn == "" ? "true" : "false"}"
  target_group_arn     = "${local.target_group_enabled == "true" ? aws_lb_target_group.default.arn : var.target_group_arn}"
}
PePe avatar

when this evaluation happens , for some reason this :

data "aws_lb_target_group" "default" {
  arn = "${local.target_group_arn}"
}
PePe avatar

can’t find the ALB

PePe avatar

and I’m 100% sure the value is correct

PePe avatar

I mean the arn

PePe avatar

but that data resource is the one that fails but is not anywhere else int he code

aknysh avatar
aknysh

Did you provision the ALB?

PePe avatar
module "alb" {
  source                    = "git::<https://github.com/cloudposse/terraform-aws-alb.git?ref=tags/0.5.0>"
  name                      = "${var.name}"
  namespace                 = "${var.namespace}"
  stage                     = "${var.stage}"
  attributes                = ["${compact(concat(var.attributes, list("alb")))}"]
  vpc_id                    = "${var.vpc_id}"
  ip_address_type           = "ipv4"
  subnet_ids                = "${var.subnet_ids}"
  security_group_ids        = [""]
  access_logs_region        = "${var.region}"
  http_enabled              = "true"
  https_enabled             = "false"
  http_ingress_cidr_blocks  = ["0.0.0.0/0"]
  https_ingress_cidr_blocks = ["0.0.0.0/0"]
  certificate_arn           = "${var.certificate_arn}"
  health_check_interval     = "60"
  health_check_path         = "${var.health_check_path}"
}
PePe avatar

that is right before

PePe avatar

yes first

aknysh avatar
aknysh

Look at the root modules ECS folder

aknysh avatar
aknysh

Did you try to run terraform apply second time?

PePe avatar

yes

aknysh avatar
aknysh

Sometimes there race conditions

PePe avatar

I used target module.alb

PePe avatar

then I run the rest

aknysh avatar
aknysh

Where some resources are not created yet

PePe avatar

I understand

PePe avatar

I run target module.alb twice

PePe avatar

then run the rest

aknysh avatar
aknysh

Try running the rest twice

PePe avatar

I did too

PePe avatar

I’m destroying everything again right now

PePe avatar

the thing is : even if I set

target_group_arn  = "${module.alb.default_target_group_arn}"
PePe avatar

to the target group arn itself

PePe avatar

it does not work

aknysh avatar
aknysh

I ran this project https://github.com/cloudposse/terraform-root-modules/tree/master/aws/ecs about 35 times, but never saw it could not find the target group

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

aknysh avatar
aknysh

I saw in many cases that the ALB was not ready yet (it’s slow), so it could not attach the target group

PePe avatar

ALB created :

PePe avatar
module.alb.aws_lb.default: Creation complete after 2m16s (ID: arn:aws:elasticloadbalancing:us-east-1:...ging-demo-droneio-alb/0ff8d366761cb319)
module.alb.aws_lb_listener.http: Creating...
  arn:                               "" => "<computed>"
  default_action.#:                  "" => "1"
  default_action.0.order:            "" => "<computed>"
  default_action.0.target_group_arn: "" => "arn:aws:elasticloadbalancing:us-east-1:234234234234:targetgroup/staging-demo-droneio-alb-default/a11e7a7298308db8"
  default_action.0.type:             "" => "forward"
  load_balancer_arn:                 "" => "arn:aws:elasticloadbalancing:us-east-1:234234234234:loadbalancer/app/staging-demo-droneio-alb/0ff8d366761cb319"
  port:                              "" => "80"
  protocol:                          "" => "HTTP"
  ssl_policy:                        "" => "<computed>"
module.alb.aws_lb_listener.http: Creation complete after 1s (ID: arn:aws:elasticloadbalancing:us-east-1:...-alb/0ff8d366761cb319/fbc88868db406827)
PePe avatar

target group is there

PePe avatar

and same error :

PePe avatar
aws-vault exec hds-admin -- terraform apply -target module.alb_ingress                                                                                                                                               34s  Fri  9 Aug 11:20:21 2019
null_resource.default: Refreshing state... (ID: 938128705396491941)
aws_lb_target_group.default: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...o-droneio-alb-default/c5d32c308b13dee7)
aws_security_group.default: Refreshing state... (ID: sg-0dca458073974ca29)
data.aws_elb_service_account.default: Refreshing state...
data.aws_iam_policy_document.default: Refreshing state...
aws_s3_bucket.default: Refreshing state... (ID: staging-demo-droneio-alb-alb-access-logs)
aws_lb.default: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...ging-demo-droneio-alb/61f67535cafe2201)
aws_lb_listener.http: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...-alb/61f67535cafe2201/8e7ac22fe675d697)

Error: Error running plan: 1 error occurred:
	* module.alb_ingress.local.target_group_arn: local.target_group_arn: Resource 'aws_lb_target_group.default' not found for variable 'aws_lb_target_group.default.arn'

`

PePe avatar

that is after running 3 times the alb creation

aknysh avatar
aknysh

after running the whole thing second time, the ALB is already ready, and it finishes provisioning the rest

PePe avatar

yes, ALBs are really slow

PePe avatar

I’m running this thing again just to make sure I did’t do anything stupid

PePe avatar

just tried from zero and run the alb target 3 times, waited few minutes and tun module.alb_ingress and I get the same error

PePe avatar
null_resource.default: Refreshing state... (ID: 938128705396491941)
aws_security_group.default: Refreshing state... (ID: sg-0dca458073974ca29)
aws_lb_target_group.default: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...o-droneio-alb-default/c5d32c308b13dee7)
data.aws_elb_service_account.default: Refreshing state...
data.aws_iam_policy_document.default: Refreshing state...
aws_s3_bucket.default: Refreshing state... (ID: staging-demo-droneio-alb-alb-access-logs)
aws_lb.default: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...ging-demo-droneio-alb/61f67535cafe2201)
aws_lb_listener.http: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...-alb/61f67535cafe2201/8e7ac22fe675d697)

Error: Error running plan: 1 error occurred:
	* module.alb_ingress.local.target_group_arn: local.target_group_arn: Resource 'aws_lb_target_group.default' not found for variable 'aws_lb_target_group.default.arn'

`

PePe avatar

so I can consistently reproduce the issue

PePe avatar

I can run this :

terraform apply -target data.aws_lb_target_group.default                                                                                                                              
aws_lb_target_group.default: Refreshing state... (ID: arn:aws:elasticloadbalancing:us-east-1:...o-droneio-alb-default/c5d32c308b13dee7)
PePe avatar

with same code is failing in the module and it does not fail outside of it

PePe avatar

so, something is happening here :

locals {
  target_group_enabled = "${var.target_group_arn == "" ? "true" : "false"}"
  target_group_arn     = "${local.target_group_enabled == "true" ? aws_lb_target_group.default.arn : var.target_group_arn}"
}
aknysh avatar
aknysh

what TF version are you using?

PePe avatar
Terraform v0.11.14
+ provider.aws v2.23.0
+ provider.local v1.3.0
+ provider.null v2.1.2
+ provider.random v2.2.0
+ provider.template v2.1.2
+ provider.tls v2.0.1

aknysh avatar
aknysh

i think you are running into some race conditions

aknysh avatar
aknysh
cloudposse/terraform-aws-alb-ingress

Terraform module to provision an HTTP style ingress rule based on hostname and path for an ALB using target groups - cloudposse/terraform-aws-alb-ingress

aknysh avatar
aknysh
cloudposse/terraform-aws-alb-ingress

Terraform module to provision an HTTP style ingress rule based on hostname and path for an ALB using target groups - cloudposse/terraform-aws-alb-ingress

PePe avatar

but

resource "aws_lb_target_group" "default" {
  count       = "${local.target_group_enabled == "true" ? 1 : 0}"
PePe avatar

will only be created if target_group_arn is = “”

PePe avatar

base on the locals evaluation

PePe avatar

so it should not even be trying to create this resource ?

aknysh avatar
aknysh

so yes, you either provide one, or the module will create it

PePe avatar

in my case I’m providing one

PePe avatar

so you think somehow the evaluation is failing ?

PePe avatar

even if I pass the raw arn value still fails

aknysh avatar
aknysh

ok yes, there is a bug in that flow. we always created target group in the module (did not test when you provide one)

aknysh avatar
aknysh

the bug is…

aknysh avatar
aknysh
cloudposse/terraform-aws-alb-ingress

Terraform module to provision an HTTP style ingress rule based on hostname and path for an ALB using target groups - cloudposse/terraform-aws-alb-ingress

aknysh avatar
aknysh

target_group_arn = "${local.target_group_enabled == "true" ? join("", aws_lb_target_group.default.*.arn) : var.target_group_arn}"

PePe avatar

you need to exampling me this a bit more

PePe avatar

are you saying that the data resource : aws_lb_target_group.default output is a list ?

aknysh avatar
aknysh

any output from it is a list

aknysh avatar
aknysh

since it has count

aknysh avatar
aknysh

even with count=1, it’s a list with one item

PePe avatar

ahhhh yes….exactly

PePe avatar

but I thought L3 was referencing :

PePe avatar
data "aws_lb_target_group" "default" {
PePe avatar

and not

resource "aws_lb_target_group" "default" {
PePe avatar

this is where I’m confused

PePe avatar

is there a order of preference ?

aknysh avatar
aknysh

your error was referencing resource “aws_lb_target_group” “default”

aknysh avatar
aknysh

which itself is used in the locals

aknysh avatar
aknysh

which itself is used in data “aws_lb_target_group” “default”

PePe avatar

but local.target_group_enabled is false since I’m passing the target group arn

PePe avatar

so it should have taken the value of var.target_group_arn

aknysh avatar
aknysh

TF always evaluates both sides of ternary operator

PePe avatar

maybe this is some basic terraform I’m missing

aknysh avatar
aknysh
cloudposse/terraform-aws-alb-ingress

Terraform module to provision an HTTP style ingress rule based on hostname and path for an ALB using target groups - cloudposse/terraform-aws-alb-ingress

aknysh avatar
aknysh

thanks a lot @PePe

PePe avatar

this is my first PR where I do not understand the fix lol

aknysh avatar
aknysh

haha

PePe avatar

I’m reading the docs again that evaluation thing is screwing with my head

aknysh avatar
aknysh

TF parses both sides of ? operator in case of true or false, does not matter

PePe avatar

I don’t get the reasoning behind evaluating when the if-statement is already false

aknysh avatar
aknysh

since the target group resource is disabled by providing your own target group, it does not have any outputs

aknysh avatar
aknysh

when TF tries to get an output on non existing resource, it fails

aknysh avatar
aknysh

join(“”, xxx.*.yyy) works because even with non existing resource, it returns an empty string

aknysh avatar
aknysh


I don’t get the reasoning behind evaluating when the if-statement is already false

aknysh avatar
aknysh

blame TF

aknysh avatar
aknysh

that’s how they did it 20 years ago in the mainstream languages

PePe avatar

ok so since the resource has count the output instead of being a string is a list and since we are evaluating strings it fails

PePe avatar

something like that ?

aknysh avatar
aknysh

when you start a new parser/compiler, you have to go through all of that again, and it’s not easy

PePe avatar

I always blame TF lol

PePe avatar

ohhh wait the join is basically acting like a try catch so even is the output is empty it return a sane value

aknysh avatar
aknysh

it returns an empty string even if the list is empty or NULL

PePe avatar

please tell me this is better in 0.12

PePe avatar

please…..

aknysh avatar
aknysh

you are lucky

aknysh avatar
aknysh
HashiCorp Terraform 0.12 Preview: Conditional Operator Improvements and Conditionally Omitted Arguments

As part of the lead up to the release of Terraform 0.12, we are publishing a series of feature preview blog posts. The post this week is on the improvements to conditional operator…

PePe avatar

by the way, everything is working perfectly now

PePe avatar

thanks so much

1
aknysh avatar
aknysh

want to open a PR?

PePe avatar

for sure

aknysh avatar
aknysh

thanks for finding it

PePe avatar

this thing has been killing me, I though I was doing something wrong

PePe avatar

I know I have to run few commands before I create the PR

PePe avatar

are those documented somewhere ?

aknysh avatar
aknysh

you mean to rebuild README?

aknysh avatar
aknysh
make init
make readme/deps
make readme
aknysh avatar
aknysh

you need that if you change any variable or outputs or README.yaml (not in this case where you just fix the code)

aknysh avatar
aknysh

but run it anyway

PePe avatar

I see ok

Mike Nock avatar
Mike Nock

Has anyone here used terraform as the CD portion of the CI/CD pipeline? Currently, I’m deploying docker images to ECR with Gitlab, but I’m running into an issue where I need to somehow tell terraform to update it’s workspaces once a new image comes out. Was curious if anyone else had run into this / figured this out

aknysh avatar
aknysh

@Mike Nock you mean that terraform is somehow constantly running and waiting for commands? Or it’s a manual process?

aknysh avatar
aknysh

if manual, you can use https://www.terraform.io/docs/providers/aws/d/ecr_image.html to retrieve information about images

AWS: aws_ecr_image - Terraform by HashiCorp

Provides details about an ECR Image

Mike Nock avatar
Mike Nock

Yea, currently the pipeline is Gitlab > Docker > ECR for CI, and then manually deploying the images by going into TFE and updating the workspace so teraform sees the new image (the CI pipeline retags the image with :production and removes that tag from the old image when deploying to ECR, and terraform is set to only use the image with that tag). I’d like to automate that. So, when the developers push a new feature to master, it build, deposits the image, and then either sends an API call to terraform to update (prefer not doing it this way), or preferably someway of having terraform monitor the tags, and deploy once the tag is removed?

Mike Nock avatar
Mike Nock

Also, for backstory, the reason I prefer not doing the api calls to update, is we are doing self-service terraform where client environments are being created regularly and dynamically, so I wouldn’t have a list of all the workspaces to send individual API calls to, and you can’t send 1 api call to update all workspaces, same as you can’t send 1 api call with a map of variables for the workspace but have to list out each one by one in separate calls.

aknysh avatar
aknysh

that’s interesting (but we at CloudPosse did not use TFE)

Mike Nock avatar
Mike Nock

Understandable, just figured it was worth asking if anyone else had run into it. Thanks!

Hetal S avatar
Hetal S
1 error occurred:
        * module.s3cdn-dev.aws_route53_record.cert_validation: At column 19, line 1: list "local.dvo" does not have any elements so cannot determine type. in:

${lookup(local.dvo[count.index], "resource_record_value")}
PePe avatar

I’m guessing that is only available in providers and should not be used with other terraform code except rds ?

aknysh avatar
aknysh

those are attributes on the provider which you can use to disable some checks if you want to run it faster

aknysh avatar
aknysh

has nothing to do with RDS

aknysh avatar
aknysh

used in some tests

aknysh avatar
aknysh

but not necessary at all

PePe avatar

RDS takes so long I was thinking on adding them to my big TF but I was not sure if they could break something

2019-08-08

Erik Osterman avatar
Erik Osterman
Erik Osterman avatar
Erik Osterman

I wonder what the backstory is here…

Erik Osterman avatar
Erik Osterman

a) why did they launch with HCL support

Blaise Pabon avatar
Blaise Pabon

They had not yet been purchased by MSFT

Erik Osterman avatar
Erik Osterman

b) why did they drop HCL support

Blaise Pabon avatar
Blaise Pabon

MSFT didn’t want to dilute the brand with a bush league API…

Erik Osterman avatar
Erik Osterman

haha, possibly!

aknysh avatar
aknysh

strange that they even started with HCL support

loren avatar
loren

maybe the devs working on that feature were also supporting the terraform github provider, and thought, wouldn’t it be neat if…?

Rich Allen avatar
Rich Allen

Hi all, this is probably a very dumb and novice questions, but I’m having a hard time understanding what I’m doing wrong here. As far as I can tell, when I try to attach a policy document using a role from the CP role repository, the base module is expecting a string?

data "aws_iam_policy_document" "s3_full_access" {
    statement {
      sid       = "FullAccess"
      effect    = "Allow"
      resources = [
        "arn:aws:s3:::${module.static-app.s3_bucket}",
        "arn:aws:s3:::${module.static-app.s3_bucket}/*"
      ]

      actions = [
        "s3:PutObject",
        "s3:PutObjectAcl",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:ListBucket",
        "s3:ListBucketVersions",
        "s3:ListBucketMultipartUploads",
        "s3:GetBucketLocation",
        "s3:AbortMultipartUpload",
      ]
    }
  }

  module "s3-write-role" {
    source     = "git::<https://github.com/cloudposse/terraform-aws-iam-role.git?ref=0.4.0>"

    enabled    = "true"
    namespace = "${var.namespace}"
    stage     = "${var.stage}"
    name      = "${var.name}"

    policy_description = "Allow S3 FullAccess"
    role_description   = "IAM role with permissions to perform actions on S3 resources"

    policy_documents = ["${data.aws_iam_policy_document.s3_full_access.json}"]
  }

I’m wondering if I’m missing something ovbious here, or not understanding how to use this module?

Rich Allen avatar
Rich Allen

Follow up error

Error: Incorrect attribute value type

  on .terraform/modules/s3-write-role.aggregated_assume_policy/main.tf line 23, in data "aws_iam_policy_document" "zero":
  23:   override_json = "${element(local.policies, 0)}"

Inappropriate value for attribute "override_json": string required.
Joan Hermida avatar
Joan Hermida

You are missing something there

Joan Hermida avatar
Joan Hermida

Why error says aggregated_assume_policy ?

Joan Hermida avatar
Joan Hermida

Oh, your source is a git tag

Rich Allen avatar
Rich Allen

I’m sorry, I’m not understanding what the source of confusion is? Could you possibly rephrase?

aknysh avatar
aknysh
cloudposse/terraform-aws-iam-role

A Terraform module that creates IAM role with provided JSON IAM polices documents. - cloudposse/terraform-aws-iam-role

aknysh avatar
aknysh

(I personally did not test the latest changes to the module, so can’t just say what’s the exact issue is)

Rich Allen avatar
Rich Allen

That is the example I’m working from. I’ve looked through it several times. And can’t see a difference. From what I can tell, my only real difference is I don’t have an outputs.tf

Rich Allen avatar
Rich Allen

is is required I have that file, so that the module exposes the outputs to the consumer?

aknysh avatar
aknysh
cloudposse/terraform-aws-iam-policy-document-aggregator

Terraform module to aggregate multiple IAM policy documents into single policy document. - cloudposse/terraform-aws-iam-policy-document-aggregator

aknysh avatar
aknysh

it does not work with one item in the list

aknysh avatar
aknysh

since it checks only for >1

Rich Allen avatar
Rich Allen

well so, I can provide and update, and I’m using 2

Rich Allen avatar
Rich Allen

so the base + full admin, as the example suggests

aknysh avatar
aknysh

policy_documents = [“${data.aws_iam_policy_document.s3_full_access.json}“]

Rich Allen avatar
Rich Allen

the previous was to simplify, I will post an updated stanza

Rich Allen avatar
Rich Allen

I’m now using 2 as the example suggests

provider "aws" {
  region = "${var.region}"
}

module "static-app" {
  source                   = "git::<https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn.git?ref=0.10.0>"
  namespace                = "${var.namespace}"
  stage                    = "${var.stage}"
  name                     = "${var.name}"
  aliases                  = "${var.aliases}"
  parent_zone_name         = "${var.parent_zone_name}"
  default_root_object      = "${var.default_root_object}"
  acm_certificate_arn      = "${var.acm_certificate_arn}"
  cors_allowed_headers     = ["GET", "HEAD"]
  cors_allowed_methods     = ["GET", "HEAD"]
  cors_allowed_origins     = ["*"]
}

data "aws_iam_policy_document" "s3_full_access" {
  statement {
    sid       = "FullAccess"
    effect    = "Allow"
    resources = ["arn:aws:s3:::${module.static-app.s3_bucket}/*"]

    actions = [
      "s3:PutObject",
      "s3:PutObjectAcl",
      "s3:GetObject",
      "s3:DeleteObject",
      "s3:ListBucket",
      "s3:ListBucketMultipartUploads",
      "s3:GetBucketLocation",
      "s3:AbortMultipartUpload",
    ]
  }
}

data "aws_iam_policy_document" "base" {
  statement {
    sid = "BaseS3Access"

    actions = [
      "s3:ListBucket",
      "s3:ListBucketVersions",
    ]

    resources = ["*"]
    effect    = "Allow"
  }
}

  module "s3-write-role" {
    source     = "git::<https://github.com/cloudposse/terraform-aws-iam-role.git?ref=0.4.0>"

    enabled    = "true"
    namespace = "${var.namespace}"
    stage     = "${var.stage}"
    name      = "${var.name}"

    policy_description = "Allow S3 FullAccess"
    role_description   = "IAM role with permissions to perform actions on S3 resources"

    policy_documents = [
      "${data.aws_iam_policy_document.base.json}",
      "${data.aws_iam_policy_document.s3_full_access.json}"
      ]
  }
aknysh avatar
aknysh

working for you?

Rich Allen avatar
Rich Allen

no this is the result of that update

Rich Allen avatar
Rich Allen
17:04 $ terraform plan

Error: Incorrect attribute value type

  on .terraform/modules/s3-write-role.aggregated_assume_policy/main.tf line 23, in data "aws_iam_policy_document" "zero":
  23:   override_json = "${element(local.policies, 0)}"

Inappropriate value for attribute "override_json": string required.


Error: Incorrect attribute value type

  on .terraform/modules/s3-write-role.aggregated_policy/main.tf line 23, in data "aws_iam_policy_document" "zero":
  23:   override_json = "${element(local.policies, 0)}"

Inappropriate value for attribute "override_json": string required.
aknysh avatar
aknysh

what TF version are you using?

Rich Allen avatar
Rich Allen

17:04 $ terraform -v Terraform v0.12.2

  • provider.aws v2.22.0
  • provider.local v1.3.0
  • provider.null v2.1.2
  • provider.template v2.1.2
Joan Hermida avatar
Joan Hermida

Ok, I’m seeing something nasty

Rich Allen avatar
Rich Allen

I hope it’s not something that dumb like I forgot brew update

Joan Hermida avatar
Joan Hermida
cloudposse/terraform-aws-iam-policy-document-aggregator

Terraform module to aggregate multiple IAM policy documents into single policy document. - cloudposse/terraform-aws-iam-policy-document-aggregator

aknysh avatar
aknysh

this module has not been updated to TF 0.12 yet https://github.com/cloudposse/terraform-aws-iam-role

cloudposse/terraform-aws-iam-role

A Terraform module that creates IAM role with provided JSON IAM polices documents. - cloudposse/terraform-aws-iam-role

1
aknysh avatar
aknysh

that’s why the errors

Joan Hermida avatar
Joan Hermida
cloudposse/terraform-aws-iam-policy-document-aggregator

Terraform module to aggregate multiple IAM policy documents into single policy document. - cloudposse/terraform-aws-iam-policy-document-aggregator

Joan Hermida avatar
Joan Hermida
cloudposse/terraform-aws-iam-role

A Terraform module that creates IAM role with provided JSON IAM polices documents. - cloudposse/terraform-aws-iam-role

Joan Hermida avatar
Joan Hermida

Main file

Rich Allen avatar
Rich Allen

so I’m a noobie here, @aknysh where is that documented? I’m not seeing it but that could be true.

Joan Hermida avatar
Joan Hermida

At line 23, you will see the error

aknysh avatar
aknysh

both modules were not converted to TF 0.12 yet

aknysh avatar
aknysh

that’s why TF 0.12 throws the errors

aknysh avatar
aknysh

we’ll get to it soon

Rich Allen avatar
Rich Allen

@Joan Hermida I’m not sure I understand the context there

Joan Hermida avatar
Joan Hermida

Downgrade to the latest 0.11 version

Rich Allen avatar
Rich Allen

@aknysh could you just point out where I can see that? To avoid version mismatches in the future?

Joan Hermida avatar
Joan Hermida
09:11:43 PM
Joan Hermida avatar
Joan Hermida
09:12:34 PM
aknysh avatar
aknysh

so for CloudPosse modules, the ones that were converted to TF 0.12 have hcl2 tag in the repo, e.g. https://github.com/cloudposse/terraform-aws-rds

cloudposse/terraform-aws-rds

Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.

aknysh avatar
aknysh

and you can tell TF 0.11 from TF 0.12 by the syntax

aknysh avatar
aknysh

TF 0.12 does not use any string interpolations https://github.com/cloudposse/terraform-aws-rds/blob/master/outputs.tf

cloudposse/terraform-aws-rds

Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.

aknysh avatar
aknysh

and uses real first-class types like bool, number, list(string), map(string) instead if strings like “string” and “list” https://github.com/cloudposse/terraform-aws-rds/blob/master/variables.tf#L19

cloudposse/terraform-aws-rds

Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.

Rich Allen avatar
Rich Allen

ty, that’s very helpful, I’m getting a few new errors related to the downgrade but I’ll fix these up. Much appreciate the help.

Joan Hermida avatar
Joan Hermida

:–1:

PePe avatar

Hello, I’m using : “git://github.com/cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms.git?ref=tags/0.6.1>” and I notice when using the newer alb module that target_group_name = “${module.alb.target_group_name}” and target_group_arn_suffix = “${module.alb.target_group_arn_suffix}” are not valid outputs anymore so it can’t be use with this cloudwatch-sns module, are you guys deprecating the use of the cloudwatch-sns-alarms or recommend something else ?

PePe avatar

and now I think I found a bug , I’m getting this :

Error: Error running plan: 1 error occurred:
	* module.alb_ingress.local.target_group_arn: local.target_group_arn: Resource 'aws_lb_target_group.default' not found for variable 'aws_lb_target_group.default.arn'
PePe avatar
locals {
  target_group_enabled = "${var.target_group_arn == "" ? "true" : "false"}"
  target_group_arn     = "${local.target_group_enabled == "true" ? aws_lb_target_group.default.arn : var.target_group_arn}"
}
PePe avatar

If I pass the ARN or the module output still fails

PePe avatar

unless I’m doing something really wrong

PePe avatar

I made a copy of the module , removed the data resource and it does work, so I’m guessing this lies : https://github.com/cloudposse/terraform-aws-alb-ingress/blob/0.7.0/main.tf#L6

PePe avatar

don’t really need to be there ? why making a lookup of something I’m already passing and creating that data resource that is not used in that tf ?

PePe avatar

I guess is to check that the target group exists before continuing but in my case I used -target module.alb to make sure I had everything before continuing but somehow still fails

2019-08-07

Milos Backonja avatar
Milos Backonja

Guys, I am looking how can I tag resources dynamically, and without repeating same block of code with one change.


\# Resource 1
tags = merge(
    var.tags,
    map(
      "Name", format("dev-bastion-0%s.${var.domain}",count.index+1),
      "type", "bastion"
    )
  )


\# Resource 2

tags = merge(
    var.tags,
    map(
      "Name", format("dev-app-0%s.${var.domain}",count.index+1),
      "type", "app"
    )
  )

any suggestions are welcome, thanks. Okay I see now that I could just abstract that with module

ruan.arcega avatar
ruan.arcega

i am not sure, but, i guess you could do follow this way: example:

data "null_data_source" "tags" {
  count = "${length(keys(var.tags))}"

  inputs = {
    key                 = "${select(keys(var.tags), count.index)}"
    value               = "${select(values(var.tags), count.index)}"
    propagate_at_launch = true
  }
}

resource "aws_autoscaling_group" "asg" {
  ...
  tags = "${data.null_data_source.tags.*.outputs}"
}
ruan.arcega avatar
ruan.arcega

terraform 0.12 you can do it with foreach operator

  dynamic "tag" {
    foreach = "${local.common_tags}"
    content {
      key   = "${dynamic.foreach.name}"
      value = "${dynamic.foreach.value}"
    }
  }
ruan.arcega avatar
ruan.arcega

i hope it’s helpfull for you

2019-08-06

Hetal S avatar
Hetal S

Need help in accessing values as a list element inside map values in terraform

Hetal S avatar
Hetal S
variable "controller_name" {
        type    = "list"
        default = [{
        z1      = ["EKS-controller1"]
        z2      = []
        z3      = ["EKS-controller1","EKS-controller2"]
        z4      = []
        }]
}
aknysh avatar
aknysh

you can use element() or […] to access the list items

aknysh avatar
aknysh

for maps, The syntax is var.MAP[“KEY”]. For example, ${var.amis[“us-east-1”]} would get the value of the us-east-1 key within the amis map variable.

aknysh avatar
aknysh

for lists the syntax is “${var.LIST}“. For example, “${var.subnets}” would get the value of the subnets list, as a list. You can also return list elements by index: ${var.subnets[idx]}

aknysh avatar
aknysh
Interpolation Syntax - 0.11 Configuration Language - Terraform by HashiCorp

Embedded within strings in Terraform, whether you’re using the Terraform syntax or JSON syntax, you can interpolate other values into strings. These interpolations are wrapped in ${}, such as ${var.foo}.

Hetal S avatar
Hetal S

this is how the variable looks like

aknysh avatar
aknysh

@mmarseglia we converted the module to TF 0.12 https://github.com/cloudposse/terraform-aws-ecr/releases/tag/0.7.0

cloudposse/terraform-aws-ecr

Terraform Module to manage Docker Container Registries on AWS ECR - cloudposse/terraform-aws-ecr

aknysh avatar
aknysh

and this example are automatically tested on CI/CD https://github.com/cloudposse/terraform-aws-ecr/tree/master/examples/complete

cloudposse/terraform-aws-ecr

Terraform Module to manage Docker Container Registries on AWS ECR - cloudposse/terraform-aws-ecr

aknysh avatar
aknysh

can you try with TF 0.12?

mmarseglia avatar
mmarseglia

i’m not sure all the modules I’m using have been converted. they weren’t, last I checked.

aknysh avatar
aknysh
cloudposse/terraform-aws-ecr

Terraform Module to manage Docker Container Registries on AWS ECR - cloudposse/terraform-aws-ecr

mmarseglia avatar
mmarseglia

i built a manifest using elasticbeanstalk app w/ a multidocker container.

mmarseglia avatar
mmarseglia

i am using 7 different modules. i think the elasticbeakstalk ones haven’t been converted yet?

aknysh avatar
aknysh

no, benstalk has not been converted yet

aknysh avatar
aknysh

i mean that we converted https://github.com/cloudposse/terraform-aws-ecr/releases/tag/0.7.0 to TF 0.12 a few weeks ago and it did not throw any policy errors

mmarseglia avatar
mmarseglia

yes, i would like to use the new one. you have done great work converting them in a short time.

mmarseglia avatar
mmarseglia

i’ll figure out a way around this in the short term and look to upgrade that module to 0.7.0

aknysh avatar
aknysh

@mmarseglia try to delete the statement = [] from https://github.com/cloudposse/terraform-aws-ecr/blob/0.6.1/main.tf#L124

cloudposse/terraform-aws-ecr

Terraform Module to manage Docker Container Registries on AWS ECR - cloudposse/terraform-aws-ecr

aknysh avatar
aknysh
cloudposse/terraform-aws-ecr

Terraform Module to manage Docker Container Registries on AWS ECR - cloudposse/terraform-aws-ecr

aknysh avatar
aknysh

(and we tested only 0.12 version recently)

sweetops avatar
sweetops

If someone from Cloudposse gets a chance to review a PR: https://github.com/cloudposse/terraform-aws-datadog-integration/pull/11

2019-08-05

sirhopcount avatar
sirhopcount

Does anybody know how to reference an instance created via a google_compute_instance_group_manager? I’m creating a route via google_compute_route and I need to have the name of the instance created via the group so I can set next_hop_instance.

Cloud Posse avatar
Cloud Posse
04:03:34 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Aug 14, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

Andy avatar

Will it be recorded and shared post the event, as well?

Erik Osterman avatar
Erik Osterman

we’ve just published our first EMR module (by @aknysh) https://registry.terraform.io/modules/cloudposse/emr-cluster/aws/0.1.0

:--1:1
aknysh avatar
aknysh
cloudposse/terraform-aws-emr-cluster

Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS - cloudposse/terraform-aws-emr-cluster

:--1:3
Erik Osterman avatar
Erik Osterman

We’ve been inconsistent about recording them

Erik Osterman avatar
Erik Osterman

if we do, they’ll be posted to #office-hours

Andy avatar

NP. Thanks!

Andy avatar

I am facing an issue with provisioning multiple Windows EC2 instances using Terraform. This is an overview of my TF template (Oversimplified with syntax ignored): Template Overview

Resource EC2 {
count = variable

connection 
type     = "winrm"
host     = "${self.private_ip}"
user     = "${var.username}"
password = "${var.admin_password}"
timeout  = "${var.timeout_tf}"

provisioner remote-exec inline
    1. powershell.exe rename-computer -Machine is rebooted once this is run.
provisioner remote-exec inline
    2. powershell copy platform code from s3 bucket
    3. powershell.exe run DomainAdd.ps1 - Machine is rebooted once this is run.
provisioner remote-exec inline
    4. powershell.exe run PreDeploy.ps1 (DSC script)
}
  • If I set the count of the instance to 1. All the above #1, #2, #3 and #4 provisioning steps work fine. Issue: If I set the count of the instance to anything other than 1 (e.g. 2), Terraform successfully runs #1, #2 and #3 on both the instances and runs #4 on ONLY ONE of the instances.

Observations:

  • After running #3 on both the instances, the remote-exec is able to eastablish the connection with both the instances successfully however runs #4 on only one of the instances.
  • Even after running #4 on one instance, it keeps on showing the following output unless I force TF to stop.
    aws_instance.ec2instance[1]: Still creating... [7m0s elapsed]
    aws_instance.ec2instance[0]: Still creating... [7m0s elapsed]
    aws_instance.ec2instance[1]: Still creating... [7m10s elapsed]
    aws_instance.ec2instance[0]: Still creating... [7m10s elapsed]
    aws_instance.ec2instance[0]: Still creating... [7m20s elapsed]
    aws_instance.ec2instance[1]: Still creating... [7m20s elapsed]
    aws_instance.ec2instance[1]: Still creating... [7m30s elapsed]
    aws_instance.ec2instance[0]: Still creating... [7m30s elapsed]
    aws_instance.ec2instance[0]: Still creating... [7m40s elapsed]
    aws_instance.ec2instance[1]: Still creating... [7m40s elapsed]
    aws_instance.ec2instance[1]: Still creating... [7m50s elapsed]
    

Why is Terraform behaving inconsistently when the instance count is set to anything other than 1? Is there something I might be missing? Any suggestions/pointers will be greatly appreciated! TF_LOG are not showing anything useful.

2019-08-04

2019-08-03

ioyu avatar

hello @here anyone knows and can recommend good libvirt provider other then this one here > https://github.com/dmacvicar/terraform-provider-libvirt

dmacvicar/terraform-provider-libvirt

Terraform provider to provision infrastructure with Linux’s KVM using libvirt - dmacvicar/terraform-provider-libvirt

kskewes avatar
kskewes

I’m using this with good results. What are you wanting to do?

dmacvicar/terraform-provider-libvirt

Terraform provider to provision infrastructure with Linux’s KVM using libvirt - dmacvicar/terraform-provider-libvirt

ioyu avatar

hey @kskewes i have an issue with referencing local image on the server where libvirtd is running

ioyu avatar

\# We fetch the latest ubuntu release image from their mirrors
resource "libvirt_volume" "ubuntu-qcow2" {
  name   = "ubuntu-qcow2"
  pool   = libvirt_pool.default.name

\#  path   = "/home/ivano/ubuntu-qcow2"
  source = "<http://cloud-images.ubuntu.com/releases/xenial/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img>"
  format = "qcow2"
}
ioyu avatar

this is what i mean, if you look at the example from terraform-provider-libvirt/examples/v0.12/ubuntu

ioyu avatar

if i try ‘path’ its always local to the server where terraform is started and spinning up webserver on the remote end so i can use source rather then path is also not ideal, any other ideas?

kskewes avatar
kskewes
terraform/env-dev/libvirt-k8s · master · Karl Skewes / k8s-with-gitlab

Create and maintain a multi-arch Kubernetes cluster utilizing Gitlab CI/CD tools where possible.

ioyu avatar

sweet , thats elegant, thx

kskewes avatar
kskewes

Sorry your other messages didn’t show before I replied but looks like mine will work for you huh :) I haven’t looked at repo on a few months and should look at what can do better with 0.12. any suggestions appreciated, otherwise enjoy!

ioyu avatar

thanks @kskewes will check it out. think i found already few issues e.g ‚count‘ as var name cant be used as it conflicts with the modul and there is also few glitches as to how it works in 0.12 , lastly you still pull the source via http…

kskewes avatar
kskewes

Thanks! Will change count. Plan was to turn it into a module anyway. Re source, I use a local file on the kvm server, per example tfvars.

2019-08-02

2019-08-01

Adrian avatar
Adrian

any workaround for

The "count" value depends on resource attributes that cannot be determined
Adrian avatar
Adrian

I want to use output of MSK bootstrap servers to create R53 CNAME entries

loren avatar
loren

don’t reference an output/attribute in count? can only reference vars and locals (and the locals must be fully deterministic in advance, i.e. cannot themselves rely on outputs/attributes)

Adrian avatar
Adrian

@loren so is it possible to get output from e.g. resource “aws_msk_cluster” and use as count in another resource e.g. “aws_route53_record”

loren avatar
loren

not as the count, no, not as far as i know, you’ll always get count cannot be determined errors

loren avatar
loren

you can use the output in an attribute on another resource, but not in count

loren avatar
loren

and you can set the count some other way where the length is fully deterministic from a var or local without relying on an attr of a resource/data source

loren avatar
loren

say you pass in a var that determines the number of nodes in your cluster… you can use that var to count aws_route53_record, and then reference the attrs of aws_msk_cluster in aws_route53_record attrs

Adrian avatar
Adrian

ye I saw workaround with bash echo

Adrian avatar
Adrian

\# Verify that the count matches the list
resource "null_resource" "verify_list_count" {
  provisioner "local-exec" {
    command = <<SH
if [ ${var.topic_arns_count} -ne ${length(var.topic_arns)} ]; then
  echo "var.topic_arns_count must match the actual length of var.topic_arns";
  exit 1;
fi
SH
  }
Adrian avatar
Adrian

smth like this

loren avatar
loren

here’s the issue with all the fun details, https://github.com/hashicorp/terraform/issues/12570

terraform modules value of count cannot be computed · Issue #12570 · hashicorp/terraform

I was using terraform modules for IAM user creation, add multiple inline policies and multiple policy_arn&#39;s to the user after creation. But now I got an issue where I create an IAM_POLICY and g…

loren avatar
loren

situation is somewhat improved in tf 0.12, so may run into the error less frequently, but it’s still a problem

Adrian avatar
Adrian

thnx for answer

Adrian avatar
Adrian

I’m stupid, already there number_of_broker_nodes

:--1:1
    keyboard_arrow_up