#terraform (2020-01)

terraform Discussions related to Terraform or Terraform Modules

Archive: https://archive.sweetops.com/terraform/

2020-01-31

Igor Bronovskyi avatar
Igor Bronovskyi

tflint error

Igor Bronovskyi avatar
Igor Bronovskyi

what am I doing wrong?

Chris OBrien avatar
Chris OBrien

Can you show your code for setting up mysql_replica_instance_type?

Igor Bronovskyi avatar
Igor Bronovskyi

I set var.hardware default value and everything is ok

Chris OBrien avatar
Chris OBrien

Right. But is there a corresponding entry for var.hardware in mysql_replica_instance_type?

Igor Bronovskyi avatar
Igor Bronovskyi

yes

Igor Bronovskyi avatar
Igor Bronovskyi
variable "mysql_master_instance_type" {
  description = "DB Instance Type"
  type        = map
  default = {
    small  = "db.t2.small"
    medium = "db.t2.medium"
    large  = "db.t2.large"
    xlarge = "db.t2.xlarge"
  }
}
Chris OBrien avatar
Chris OBrien

Well, you gave me ‘mysql_master_instance_type’ instead of ‘mysql_replica_instance_type’. But since they’re likely the same structure, I’ll go with it.

Chris OBrien avatar
Chris OBrien

The error: ‘The given key does not identify an element in this collection value.’ occurs when the key does not have an associated value.

e.g. If ‘var.hardware = “small”’ then everything should work. If ‘var.hardware = “smallish”’ then you will get that error.

Igor Bronovskyi avatar
Igor Bronovskyi

Exactly

Igor Bronovskyi avatar
Igor Bronovskyi

terraform validate say Ok

wannafly37 avatar
wannafly37

Anyone have recommendations/reading suggestions on how you test infra built with terraform?

creature avatar
creature

I watched a Hashicorp video the other day that talked about terratest

creature avatar
creature

curious what other folks are doing also though

wannafly37 avatar
wannafly37

I’m even just curious on HOW testing for infra is done - beyond the tools - the concept of testing infra is new to me

creature avatar
creature
Open sourcing Terratest: a swiss army knife for testing infrastructure code attachment image

Tools to test Terraform, Packer, Docker, AWS, and much more

creature avatar
creature

Just joined a greenfield project where we will be doing this. Just haven’t gotten that far yet. Would love to hear how this goes for you

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we use bats and terratest for all our modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

each module has a complete example

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-fargate-profile

Terraform module to provision an EKS Fargate Profile - cloudposse/terraform-aws-eks-fargate-profile

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-node-group

Terraform module to provision an EKS Node Group. Contribute to cloudposse/terraform-aws-eks-node-group development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and terratest to deploy the example on real AWS account

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-node-group

Terraform module to provision an EKS Node Group. Contribute to cloudposse/terraform-aws-eks-node-group development by creating an account on GitHub.

creature avatar
creature

@Andriy Knysh (Cloud Posse) - are there any books, required reading, Udemy courses or something on proper Terraform testing?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-fargate-profile

Terraform module to provision an EKS Fargate Profile - cloudposse/terraform-aws-eks-fargate-profile

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

don’t know about books/reading, we just implemented out solution (it’s open-sourced), and it works well for us

:--1:1
wannafly37 avatar
wannafly37

interesting stuff - thanks guys, infrastructure testing seems more like end-to-end on your real provisioned infra.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Since terratest is Golang, you can create tests of any complexity, including end-to-end. For example, on EKS you could not only deploy the infra, but deploy Kubernetes apps and test them

loren avatar
loren

Presentation by the gruntwork folks behind terratest, I found it really helpful in understanding the concepts and approaches for infra testing, https://www.infoq.com/presentations/automated-testing-terraform-docker-packer

Automated Testing for Terraform, Docker, Packer, Kubernetes, and More

Yevgeniy Brikman talks about how to write automated tests for infrastructure code, including the code written for use with tools such as Terraform, Docker, Packer, and Kubernetes. Topics covered include: unit tests, integration tests, end-to-end tests, dependency injection, test parallelism, retries and error handling, static analysis, property testing and CI / CD for infrastructure code.

Igor Bronovskyi avatar
Igor Bronovskyi
tflint .
terraform validate
imiltchman avatar
imiltchman

Does ECS provide an SNS topic to subscribe to events like updating service, tasks starting/stopping, autoscaling events?

imiltchman avatar
imiltchman

And if so, is there a terraform example of this that someone can share?

Joe Hosteny avatar
Joe Hosteny

Using the CloudPosse tooling, and root modules using tf 0.12, is there any way at all to run output currently, when using remote state?

Joe Hosteny avatar
Joe Hosteny
Command `terraform output` always fails when .tf files are located in a child directory and remote backend is used · Issue #17300 · hashicorp/terraform

Terraform Version Terraform v0.11.3 + provider.aws v1.8.0 Terraform Configuration Files # aws-stack/backend.tf terraform { backend "s3" { bucket = "my-project" key = "state…

Joe Hosteny avatar
Joe Hosteny

I got around this temporarily by removing the -from-module from the TF_CLI_INIT envvar, after doing the terraform init in the /conf/<module> directory. Then I cd’ed into .module and ran the terraform output and some various state management commands

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Joe Hosteny this is an unfortunate downside as a result of terraform 0.12 not allowing init -from-module=.. in the local directory even the files are just dot files (the way it worked in 0.11)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Unfortunately, I don’t see a clean way around this without a bunch of extra scripting, make targets, or using terragrunt.

Joe Hosteny avatar
Joe Hosteny

Thanks @Erik Osterman (Cloud Posse), I read the thread and it is unfortunate.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is now the officially recommended layout

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

using this pattern, the need for invoking root modules multiple times nearly entirely goes away

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it takes a bit of mind warping to think this way, but in the end, i think it’s going to lead to easier projects to maintain with fewer inconsistencies

Joe Hosteny avatar
Joe Hosteny

I haven’t read that yet, but it seems like it would have significant impact on the tooling? Or is that assumption wrong?

Joe Hosteny avatar
Joe Hosteny

I’ll look that over though

Joe Hosteny avatar
Joe Hosteny

It also seems root module outputs should be fewer in number and in SSM

:--1:1
Joe Hosteny avatar
Joe Hosteny

So that may mitigate for now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, I thinking using SSM is the way to go

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

outputs is really just for human validation and convenience

pianoriko2 avatar
pianoriko2

@here can anyone help with terraform connection to a private cloud. Do I have to write a custom provider?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@pianoriko2, do you mean a way to manage a private cloud platform/api with Terraform?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

…if so, first check the wealth of providers for private clouds

:100:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

alternatively, if the scope of what you want to manage is small/simple and the private cloud provides a standard REST API, you can use this “escape hatch”: https://github.com/Mastercard/terraform-provider-restapi

Mastercard/terraform-provider-restapi

A terraform provider to manage objects in a RESTful API - Mastercard/terraform-provider-restapi

1
Chris Fowles avatar
Chris Fowles

i’m so unsure about how i feel about this provider - i can totally see the need, but i’ve had to fix up so many terraform environments that were full of null resource local_execs that i’m a little terrified of what’s going to come out of it

Mastercard/terraform-provider-restapi

A terraform provider to manage objects in a RESTful API - Mastercard/terraform-provider-restapi

2020-01-30

Laurynas avatar
Laurynas

Hi, has anyone tried Pulumi? I heard a lot of good things about it but I’m not sure if it’s good idea to migrate

loren avatar
loren

there have been a few discussions about it, https://archive.sweetops.com/search?query=pulumi

SweetOps Slack Archive

SweetOps is a collaborative DevOps community. We welcome engineers from around the world of all skill levels, backgrounds, and experience to join us! This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build sweet infrastructure.

Zachary Loeber avatar
Zachary Loeber

I know python enough where I really should dig into Pulumi more. Dut did you know that there is a pulumi terraform ‘bridge’? https://github.com/pulumi/pulumi-terraform

pulumi/pulumi-terraform

A bridge between Pulumi and Terraform. Contribute to pulumi/pulumi-terraform development by creating an account on GitHub.

Zachary Loeber avatar
Zachary Loeber

I have a hard time mentally reconciling shifting to an imperative model for what I believe should be declarative based work.

Zachary Loeber avatar
Zachary Loeber

Plus, I’ve worked with some brilliant infrastructure people who were just awful at coding (even in python)….

Zachary Loeber avatar
Zachary Loeber

And Pulumi’s first class citizen is Typescript. Not my bag of tea….

IckesJ avatar
IckesJ

In general terms how are any of you guys protecting secrets inside tfstates?

• We are currently using the S3 backend with it encrypted so the general tf recommendation referenced here https://www.terraform.io/docs/state/sensitive-data.html is only part of the solution. The solution for pgp is great but only available inside iam_user, iam access key & iam login profile & light sail.

So what about when an RDS instance for example, the admin password I want to be a secret in the tfstate. Other examples are DS Directory Services domain admin password, SSM values, etc.

State: Sensitive Data - Terraform by HashiCorp

Sensitive data in Terraform state.

kj22594 avatar
kj22594

I haven’t used it but I read about a tool called terrahelp that might be useful for this situation. If you end up looking into it I’d be curious to what you think https://github.com/opencredo/terrahelp

opencredo/terrahelp

Terraform helper. Terrahelp is as a command line utility written in Go and is aimed at providing supplementary functionality which can sometimes prove useful when working with Terraform. - opencred…

IckesJ avatar
IckesJ

Thanks - I haven’t seen this but will look at it.

:--1:1
Andrew Jeffree avatar
Andrew Jeffree
segmentio/chamber

CLI for managing secrets. Contribute to segmentio/chamber development by creating an account on GitHub.

Brij S avatar
Brij S

when you want to reference a release(Within github) for a tf module. Do you reference within the link, for example:

[email protected]/example.git?ref=v.1.0

or can you do the following:

  source  = "[email protected]/example.git"
  version = "1.0"
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
example.git?ref=tags/0.1.0
:--1:1
sumit parmar avatar
sumit parmar

How to import Azure Function App in Azure API Management using Terraform?

Pierre-Yves avatar
Pierre-Yves

Hello Sumit, I don’t have experience with it but as I see azurerm_api_management_api has an import block: https://www.terraform.io/docs/providers/azurerm/r/api_management_api.html

Azure Resource Manager: azurerm_api_management_api - Terraform by HashiCorp

Manages an API within an API Management Service.

2020-01-29

Pierre-Yves avatar
Pierre-Yves

Hello, I am using remote tfstate and change are directly made to it with no backup. Do you do tfstate backup and store it remotelly ? i see that terraform refresh has a backup option, but as I am using it in CI local storage is not an option

Adrian avatar
Adrian

S3 bucket versioning isn’t enough?

Pierre-Yves avatar
Pierre-Yves

Hi Adrian, i am using Azure where there is no versioning on Blobstorage, also Azure Snapshot is only for file but not for Blob

Pierre-Yves avatar
Pierre-Yves

I just found that the snapshot is available by file not by storage

mmarseglia avatar
mmarseglia

When using elasticbeanstalk why does every apply result in setting changes on the elasticbeanstalk app even though it looks like nothing changed?

      - setting {
          - name      = "MinSize" -> null
          - namespace = "aws:autoscaling:asg" -> null
          - value     = "2" -> null
        }
      + setting {
          + name      = "MinSize"
          + namespace = "aws:autoscaling:asg"
          + value     = "2"
        }
mmarseglia avatar
mmarseglia

i don’t understand what’s going on there

mmarseglia avatar
mmarseglia

and what that -> null is all about

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

those are nown bugs/issues in the provider

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Module recreates all `settings` on each `terraform plan/apply` · Issue #43 · cloudposse/terraform-aws-elastic-beanstalk-environment

terraform-aws-elastic-beanstalk-environment recreates all settings on each terraform plan/apply setting.1039973377.name: &quot;InstancePort&quot; => &quot;InstancePort&quot; setting.1039973377.n…

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we were not able to solve it at that time

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

did not look into it for the last 3-4 months though, so maybe things could be better now

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

the issue I suppose is that we provide a set of settings which terraform sends to the AWS API to apply. But the API does not apply all of them since some are not relevant to the particular environment you are building

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then terraform reads the settings back and compares with what it has, and always see differences

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

another one, even if the settings are just for the environment and nothing more, they are returned in diff order and terraform still sees differences

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so in short, it could be either one of 1) settings not specific to the environment which AWS just drops; 2) incorrect order. Or a combination of the above

:--1:1
mmarseglia avatar
mmarseglia

thank you for the explanation

creature avatar
creature

hey there sweetops ninjas. I’m working with the terraform-aws-cloudtrail-s3-bucket module and wondering if there’s a trick to adding a custom policy attribute. Docs simply say “string”, but I can’t get anything to stick. It always overwrites the policy with the default.

module "cloudtrail_s3_bucket" {
  source    = "git::<https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket.git?ref=master>"
  name      = "cloudtrail-sandbox-boo"
  policy    = file("policies/cloudtrail-bucket.json.tpl")
}
creature avatar
creature

any examples would be greatly appreciated. I’ve tried the <<EOF pattern also, both of which work with the aws_s3_bucket_policy resource. But these two battle it out, so no idempotency which makes me a sad panda.

creature avatar
creature

trying to force policy to a null value also doesn’t work. I’m going to try rolling my own implementation of cloudtrail_s3_bucket using aws_s3_bucket_policy instead since it works.

2020-01-28

getSurreal avatar
getSurreal

Is the terraform-aws-dynamic-subnets module preferred over terraform-aws-multi-az-subnets or is there a reason to use one over the other?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@getSurreal there’s no one best way to do it because it depends on the customer requirements on what you want to achieve

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s why we decoupled subnets from VPCs

getSurreal avatar
getSurreal

ok. thanks. I guess I need to study them better. On the surface it appears you can get the same results.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

because subnetting is a very opinionated topic, especially in established organizations

Daren avatar
Daren

Do you have plans to update https://github.com/cloudposse/terraform-aws-cloudfront-cdn/releases with support for TF 0.12?

cloudposse/terraform-aws-cloudfront-cdn

Terraform Module that implements a CloudFront Distribution (CDN) for a custom origin. - cloudposse/terraform-aws-cloudfront-cdn

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes. we have a few modules not converted to 0.12 yet, this is one of them

cloudposse/terraform-aws-cloudfront-cdn

Terraform Module that implements a CloudFront Distribution (CDN) for a custom origin. - cloudposse/terraform-aws-cloudfront-cdn

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

will do it as soon as we have time

kskewes avatar
kskewes
cloudposse/terraform-aws-cloudfront-s3-cdn

Terraform module to easily provision CloudFront CDN backed by an S3 origin - cloudposse/terraform-aws-cloudfront-s3-cdn

2020-01-27

sumit parmar avatar
sumit parmar

do we have terraform provider for Ingress controller on kubernetes

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
Kubernetes: kubernetes_ingress - Terraform by HashiCorp

Ingress is a collection of rules that allow inbound connections to reach the endpoints defined by a backend. An Ingress can be configured to give services externally-reachable urls, load balance traffic, terminate SSL, offer name based virtual hosting etc.

sumit parmar avatar
sumit parmar

?

Cloud Posse avatar
Cloud Posse
05:00:12 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Feb 05, 2020 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2020-01-25

2020-01-24

Bradford Toney avatar
Bradford Toney

Would it be unwise to use terraform to install fluxcd at the end of a EKS provision?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

https://github.com/cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms/blob/0.7.0/main.tf#L51

module "alb_target_group_alarms" {
 source = "git::<https://github.com/cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms.git?ref=tags/0.7.0>"
 
 ...
 insufficient_data_actions = []
 ...
}

what solution would fix this when it is set to [] or null?

Error: Error in function call
  on .terraform/modules/core.alb_target_group_alarms/main.tf line 51, in locals:
  51:   insufficient_data_actions = coalescelist(var.insufficient_data_actions, var.notify_arns)
    |----------------
    | var.insufficient_data_actions is null
    | var.notify_arns is list of string with 1 element
Call to function "coalescelist" failed: panic in function implementation:
value is null
goroutine 3185 [running]:
runtime/debug.Stack(0xc000cb2230, 0x25dc320, 0x2d91510)
        /opt/goenv/versions/1.12.4/src/runtime/debug/stack.go:24 +0x9d
[github.com/zclconf/go-cty/cty/function.errorForPanic(...)](http://github\.com/zclconf/go\-cty/cty/function\.errorForPanic\(\.\.\.\))
        /opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/function/error.go:44
[github.com/zclconf/go-cty/cty/function.Function.Call.func1(0xc000cb2568](http://github\.com/zclconf/go\-cty/cty/function\.Function\.Call\.func1\(0xc000cb2568),
0xc000cb2588)
        /opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/function/function.go:239
+0x8f
panic(0x25dc320, 0x2d91510)
        /opt/goenv/versions/1.12.4/src/runtime/panic.go:522 +0x1b5
[github.com/zclconf/go-cty/cty.Value.Lengt](http://github\.com/zclconf/go\-cty/cty\.Value\.Lengt)
cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms

Terraform module to create CloudWatch Alarms on ALB Target level metrics. - cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

It seems compact(coalescelist([], [""])) is needed on that line so [] is the result

cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms

Terraform module to create CloudWatch Alarms on ALB Target level metrics. - cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms

Bruce avatar
Bruce

This may be a dumb question. But how do I get a single Nacl ID from a the data "aws_network_acls" resource to add a route in aws_route for the route_table_id attribute? I’ve tried using element function but that doesn’t work.

marcinw avatar
marcinw

Can you please share the actual code snippet?

Bruce avatar
Bruce

It’s something like this:

data "aws_network_acls" "example" {
  vpc_id = var.vpc_id

  filter {
    name   = "tag:Name"
    values = ["ACL-Name"]
  }
} 

resource "aws_route" "route" {
  route_table_id              = data.
aws_network_acls.ids
  ........
}
marcinw avatar
marcinw
resource "aws_route" "route" {
  route_table_id = element(data.aws_network_acls.ids, 0)
}

What happens when you do that? ^

marcinw avatar
marcinw

Also, are you sure th route_table_id expects a network ACL ID? It seems to expect the output of this resource

AWS: aws_route_table - Terraform by HashiCorp

Provides details about a specific Route Table

Bruce avatar
Bruce

I think your right it’s the case of staring at the problem for too long. :–1:

2020-01-23

Brij S avatar
Brij S

i have a route53 module which creates a route53 zone among some other operations and an acm module, when running terraform apply, I get the following output

module.route53_zone.aws_route53_record.digital_ns: Creating...
module.acm.aws_acm_certificate_validation.this[0]: Creating...
module.acm.aws_acm_certificate_validation.this[0]: Still creating... [10s elapsed]

The acm validation wont pass until that route53 record is created, can I force the order here, or place a dependency on modules?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

terraform does not support dependencies on modules yet as far as I know

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Command: apply - Terraform by HashiCorp

The terraform apply command is used to apply the changes required to reach the desired state of the configuration, or the pre-determined set of actions generated by a terraform plan execution plan.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and do two apply, first with --target for the zone, second just apply for everything else

Rob Rose avatar
Rob Rose

Hey how do you guys work around https://github.com/hashicorp/terraform/issues/4775 when using MySQL instances in a private subnet?

Support SSH bastion host for MySQL and PostgreSQL providers · Issue #4775 · hashicorp/terraform

I&#39;d like to use Terraform&#39;s PostgreSQL provider to provision some databases on an AWS RDS instance in a private subnet (with Terraform running on a host outside of my VPC). It doesn&#39;t s…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  1. practice gitops
Support SSH bastion host for MySQL and PostgreSQL providers · Issue #4775 · hashicorp/terraform

I&#39;d like to use Terraform&#39;s PostgreSQL provider to provision some databases on an AWS RDS instance in a private subnet (with Terraform running on a host outside of my VPC). It doesn&#39;t s…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
  1. run something like atlantis on ECS Fargate
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(@marcinw might have some other ideas )

marcinw avatar
marcinw

Re: 2 -> I’d personally recommend against running atlantis on Fargate for anything non-trivial because with Fargate you have no guarantee that your task will stay up, and if they reap your task while running terraform apply , then best of luck cleaning up the mess. Just get a single EC2 machine for your ECS task and put it there.

marcinw avatar
marcinw

The thing @Erik Osterman (Cloud Posse) probably meant when mentioning myself was that I guess you could put a little Lambda in your VPC and bounce your VPC-internal requests off of it - I’m currently investigating this approach for the Terraform SaaS I’m working on - https://spacelift.io

marcinw avatar
marcinw

Also re: running Terraform in your VPC, it’s a bit of a and problem because something has to set up the VPC itself

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

Hey folks, I have the eks_node_group working, but am hitting a problem with EFS allowing connections. I need to update the EFS security group to allow the node groups sg, but it gets a random sg from the template.

Anyone address this yet?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hmm interesting. What about allowing CIDR blocks in SG?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

or writing the SG from the template into SSM and then reading it from there and adding to EFS SG

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

I think I am going to try the CIDR approach. I left off last night looking at that and looking up the template’s group

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

looks like that might’ve worked just fine

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

thx

:--1:1

2020-01-22

marcinw avatar
marcinw
instrumenta/conftest

Write tests against structured configuration data using the Open Policy Agent Rego query language - instrumenta/conftest

:--1:1
marcinw avatar
marcinw

This is one is interesting because it can operate on HCL, too.

instrumenta/conftest

Write tests against structured configuration data using the Open Policy Agent Rego query language - instrumenta/conftest

Brij S avatar
Brij S

in a brand new aws account with nothing in it initially, how do you all handle creating some sort of iam role/user which carries out tf applies

marcinw avatar
marcinw

Manually

Brij S avatar
Brij S

kind of a chicken n egg situation

1
marcinw avatar
marcinw

Yup. Can’t think of a clever solution to that. One of the reasons I find Google’s IAM more elegant because you can both create a project and add a service account to it in Terraform.

loren avatar
loren

Create the account with aws organizations, assume the role it creates in the account

:--1:3
Chris Fowles avatar
Chris Fowles

we have an org management project that is run against the org root - it handles creating sub accounts and then assuming roles into those to create the baseline IAM setup

Chris Fowles avatar
Chris Fowles

it’s a chicken omelette

1

2020-01-21

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Terraform

Policy-based control for cloud native environments

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

There’s an example of using OPA with terraform. Pretty neat.

Chris Fowles avatar
Chris Fowles

https://github.com/fugue/regula <- similar thing from fugue

fugue/regula

Regula checks Terraform for AWS security and compliance using Open Policy Agent/Rego - fugue/regula

:--1:2
marcinw avatar
marcinw

In GitHub actions, are secrets shared between actions? So if I put my AWS credentials there, does it mean that pretty much anyone with push access to the repo can use it for pretty much any purpose? Or is there a way to enforce some sort of policy there, too?

fugue/regula

Regula checks Terraform for AWS security and compliance using Open Policy Agent/Rego - fugue/regula

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in GitHub actions, secrets are not shared between actions.

nyan_parrot1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If one really wanted to, it’s possible to ensure a user can steal those, but by default github actions tries to make this difficult.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This also makes it a real pain for testing PRs from forks on open source repos.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ah cool, so OPA under the hood

2020-01-20

Pierre-Yves avatar
Pierre-Yves

Hi, do you recommend using multiple tfstate file ? per environnment and per tool as explained here the post below is from 2016 and I wonder if its still the best way to go . ( I am currently struggling by having a single tfstate file ). or should I go with workspace ? which path did you choose ?

https://blog.gruntwork.io/how-to-manage-terraform-state-28f5697e68fa

How to manage Terraform state attachment image

A guide to file layout, isolation, and locking for Terraform projects

Martin Tooming avatar
Martin Tooming

Hey, this article is from 2016, but also updated in 2019 I would also recommend moving to smaller .tfstate files

How to manage Terraform state attachment image

A guide to file layout, isolation, and locking for Terraform projects

Martin Tooming avatar
Martin Tooming

We are not using workspaces atm. To my understanding, workspaces are working with different directories in the backend, so I don’t see any benefit there compared to just use different directories

Pierre-Yves avatar
Pierre-Yves

ok thanks, I will move to multiple terraform state.

vFondevilla avatar
vFondevilla

Yep, multiple tfstate ftw.

vFondevilla avatar
vFondevilla

We’re doing it without workspaces

Pierre-Yves avatar
Pierre-Yves

in parallel of using multiple remote statefile, did you setup a deployment pipeline for each env / component ? or did you stick to a single one ?

vFondevilla avatar
vFondevilla

right now we’re not using a ci/cd pipeline for the infrastructure stuff as it is relatively static for now

Pierre-Yves avatar
Pierre-Yves

hopefully my code was organized by sub module, ( kubernetes, vm_linux, vm_windows, sql server, network, mgmt ). Each will have its own independant tfstate.

how should I migrate from one tfstate to several one ?

with a new terraform init in a sub folder, terraform init; terraform plan list resource already deployed . should I manually amend the new tfstate files?

Cloud Posse avatar
Cloud Posse
05:00:21 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Jan 29, 2020 11:30AM.
Register for Webinar
slack #office-hours (our channel)

imiltchman avatar
imiltchman

Has anybody ever run into an error that just says Invalid Parameter with no other info. The DEBUG output shows a 400 Bad Request from AWS

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think it would help if you share a little bit more context like:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

• what is the terraform provider

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

• how are you authenticating with AWS (e.g. through SSO or access keys)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

• if this was working and recently stopped

imiltchman avatar
imiltchman

I was able to find the issue. It had to do with target group not being connected to the ALB, due to neither unauthenticated_hosts/_paths parameter being passed in to terraform-aws-alb-ingress.

imiltchman avatar
imiltchman

Not sure why TF was swallowing the error message

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Saw a neat demo today by @marcinw of his new SaaS (spacelift.io). They’ve built something similar to Terraform Cloud, but some nice differentiators:

• Integration with Open Policy Agent so you can set policies that operate on the output of the terraform plan, but also other things like time-of-day.

• Bring-your-own-docker-container model so it’s easier to run custom providers and depend on other tools

• No hardcoded AWS credentials. Just grant access to their principal, the way datadog works.

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If it sounds interesting, you can ping him for a demo.

marcinw avatar
marcinw

Thanks for the shout-out @Erik Osterman (Cloud Posse) . If anyone wants a demo or just wants to try it out (it’s in private beta) please give me a shout, either here or through the contact form on https://spacelift.io

Szymon avatar
Szymon

Hi Marcin! I’d like to see the demo.

marcinw avatar
marcinw

Yeah, I got your email alright. I’ll whitelist you and let you play around. If you want a live demo afterwards, give me a shout.

2020-01-18

loren avatar
loren
depends_on cannot be used in a module · Issue #10462 · hashicorp/terraform

Hi there, Terraform Version 0.8.0 rc1+ Affected Resource(s) module Terraform Configuration Files module &quot;legacy_site&quot; { source = &quot;../../../../../modules/site&quot; name = &quot;foo-s…

:--1:3
cytopia avatar
cytopia

CC @maarten

depends_on cannot be used in a module · Issue #10462 · hashicorp/terraform

Hi there, Terraform Version 0.8.0 rc1+ Affected Resource(s) module Terraform Configuration Files module &quot;legacy_site&quot; { source = &quot;../../../../../modules/site&quot; name = &quot;foo-s…

2020-01-17

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
5 Common Terraform Patterns—Evolving Your Infrastructure with Terraform attachment image

Nicki Watt, OpenCredo’s CTO, explains how her company uses HashiCorp’s stack—and particularly Terraform—to support its customers in moving to the world of CI/CD and DevOps.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

“The Terralith” very apropos

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

nice explanation for newcomers to terraform (And why to avoid them)

David avatar
David

What happens when you have an explicit dependency on a resource that has a count of 0?

So something like:

resource some_resource thing {
  count = 0
}

resource other_resource thang {
  ...
  depends_on = [some_resource.thing]
}
loren avatar
loren

i’d say your config is broken

2020-01-16

Naseem avatar
Naseem

Niche question but any GCP users have tried declaring their build steps of Cloud Build triggers in Terraform?

Chris Fowles avatar
Chris Fowles

yeh - it was a bit awkward but ok for simple builds

Naseem avatar
Naseem

interesting, I was looking into making a module to standardize build steps across the org. cloudbuild has no way to sort of share steps across multiple cloudbuild files (e.g. if all builds have a common download encrypted key step and the location of that key happens to change)

Naseem avatar
Naseem

but good to know in advance that it’s not as great as it seems

Chris Fowles avatar
Chris Fowles

It was more that it got difficult for different teams to own their own builds rather than functionally broken things

Chris Fowles avatar
Chris Fowles

probably wouldn’t try doing it again

Milos Backonja avatar
Milos Backonja

Guys, I am using waf regional web acls with fortinet managed rules from marketplace. That fortinet rule set id changes from region to region, maybe even from account to account so option to hardcode id cant work. I can’t find way to dynamically find rule id (over data source). I was only able to find rule id, if i create web acl by hand through console, attach fortinet rule to webacl, and to describe web acl through aws cli which contains rule id inside. Does anyone have similar issue? Any ideas are welcomed. Thanks

2020-01-15

IvanM avatar
IvanM

guys anyone could help pls with small thing? It’s about https://github.com/cloudposse/terraform-aws-codebuild

Everytime I execute terraform plan I can see this diff (without any changes)

      - source {
          - buildspec           = "cicd/swaggerspec.yml" -> null
          - git_clone_depth     = 0 -> null
          - insecure_ssl        = false -> null
          - report_build_status = false -> null
          - type                = "CODEPIPELINE" -> null
        }
      + source {
          + buildspec           = "cicd/swaggerspec.yml"
          + report_build_status = true
          + type                = "CODEPIPELINE"
        }

Any idea how to get rid of it?

cloudposse/terraform-aws-codebuild

Terraform Module to easily leverage AWS CodeBuild for Continuous Integration - cloudposse/terraform-aws-codebuild

Mikael Fridh avatar
Mikael Fridh

I dunno about the module specifically… But seems there is a diff between the state and desired state at least …

report_build_status = false

report_build_status = true

Did you apply the current diff above at least once?

IvanM avatar
IvanM

IvanM avatar
IvanM

it was report_build_status thanks a lot

cabrinha avatar
cabrinha

Seems like using a aws_launch_config and setting spot_price = "" no longer launches spot instances?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Announcing Providers in the New Terraform Registry

Today, we’re excited to announce the beginnings of a new direction for the Registry. We’re renaming it as the Terraform Registry and expanding it to include Terraform providers as …

rbadillo avatar
rbadillo

Hi team, I have a question about launch templates and AWS ASG:

resource "aws_launch_template" "example" {
  name_prefix   = "example"
  image_id      = "${data.aws_ami.example.id}"
  instance_type = "c5.large"
}

resource "aws_autoscaling_group" "example" {
  availability_zones = ["us-east-1a"]
  desired_capacity   = 1
  max_size           = 1
  min_size           = 1

  mixed_instances_policy {
    launch_template {
      launch_template_specification {
        launch_template_id = "${aws_launch_template.example.id}"
      }

      override {
        instance_type = "c4.large"
        weighted_capacity = "3"
      }

      override {
        instance_type = "c3.large"
        weighted_capacity = "2"
      }
    }
  }
}

Can I have an ASG with 2 overrides and not weighted_capacity ?

rbadillo avatar
rbadillo

I think I found my answer by reading this:

override - (Optional) List of nested arguments provides the ability to specify multiple instance types. This will override the same parameter in the launch template. For on-demand instances, Auto Scaling considers the order of preference of instance types to launch based on the order specified in the overrides list. Defined below.

2020-01-14

Vlad Ionescu avatar
Vlad Ionescu

If I have an object of the following type:

type = list(object({
  sqs_arn     = string
  bucket_name = string
}))

is there any way for me to get a list of all the sqs_arns?

I want to say there is but for the life of me I can’t figure it out

Vlad Ionescu avatar
Vlad Ionescu

Got it:

[for i in var.additional_forwarding_configs : i.sqs_arn]
maarten avatar
maarten

works, also:

variable "additional_forwarding_configs" {
  default = [
    {
      sqs_arn     = "1"
      bucket_name = "b1"
    },
    {
      sqs_arn     = "2"
      bucket_name = "b2"
    }
  ]
  type = list(object({
    sqs_arn     = string
    bucket_name = string
  }))
}

output "test" {
  value = var.additional_forwarding_configs.*.sqs_arn
}

2020-01-13

IvanM avatar
IvanM

guys could this https://github.com/cloudposse/terraform-aws-codebuild/pull/50 be merged? Seems that it will work and it will help big time!

Add option to enable LOCAL_CACHE by drselump14 · Pull Request #50 · cloudposse/terraform-aws-codebuild

In order to speed up docker build process in aws codebuild, we can enable local cache for caching docker layer. This PR add option to enable LOCAL_CACHE in aws codebuild

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we’ll review, thanks for the PR

Add option to enable LOCAL_CACHE by drselump14 · Pull Request #50 · cloudposse/terraform-aws-codebuild

In order to speed up docker build process in aws codebuild, we can enable local cache for caching docker layer. This PR add option to enable LOCAL_CACHE in aws codebuild

IvanM avatar
IvanM

well, it’s not mine but thx

Cloud Posse avatar
Cloud Posse
05:01:23 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Jan 22, 2020 11:30AM.
Register for Webinar
slack #office-hours (our channel)

PePe avatar

Hi, I’m using https://github.com/cloudposse/terraform-aws-alb-ingress.git but I need to specify 15 ingress rules some path some host-header for what I understand with this module I can’t define more than

count        = length(var.unauthenticated_paths) > 0 && length(var.unauthenticated_hosts) == 0 ? var.unauthenticated_listener_arns_count : 0
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@PePe yea, I think it could use some refactoring for that use-case. We were a bit constrained with HCLv1 syntax, but I think with HCLv2, it can be improved. When we upgraded it to HCL2, didn’t change the interface or leverage the new features of HCL2.

PePe avatar

ok, yes we could use some dynamics for that

PePe avatar

ok for now I will just do it in plain tf without the module

PePe avatar

I will see if I have some time and send a PR over

2020-01-12

2020-01-11

tamsky avatar
tamsky

Do any folks here have advice to share on how they manage environments that require multi-step terraform apply ? For instance, in our environment, we have dependencies between two different terraform config dirs ( config dir A references resource ARNs that are created and exist in the output of config dir B ). If config dir A executes apply before resources exist in config dir B, we rely on terraform_remote_state with lookup(), and an empty ("") default value.

I’m interested to hear about methods folks have created that track and/or automate the cases where A is waiting for resources in B, and helps the system determine and/or trigger a subsequent apply in dir A.

:--1:2
kskewes avatar
kskewes

We’ve ended up with multiple state files, one per directory and a bring up dependency order. For example VPC early then things like RDS later. So we have B relying on A but not vice versa. We use remote state for RDS to get VPC id and subnets etc. Have built some mechanisms for changing subnets etc but major changes would require new vpc or whatever the lower level is and then a migration. Manageable and typical for infra primitives. Haven’t done CI job for running terraform yet. Current thinking is we’re mostly going to run isolated terraform in changed dir on merge to master.

Could you have CI run A on changes to A or B directories? Might be a noop and slight job duration increase most of the time… Catch if any real changes in the plan output which you’d be checking anyway for B?

maarten avatar
maarten

@tamsky use Terragrunt.. https://terragrunt.gruntwork.io/docs/getting-started/configuration/

dependencies {
  paths = ["../vpc"]
}
Configuration

Learn how to configure Terragrunt.

2020-01-10

Milos Backonja avatar
Milos Backonja
Milos Backonja avatar
Milos Backonja

so this seams like global issue?

Milos Backonja avatar
Milos Backonja

strange, yesterday was working fine with 0.12.18

Milos Backonja avatar
Milos Backonja

but upgrade to 0.12.19 fixed issue

loren avatar
loren

i bet it was related to the checkpoint api being broke yesterday, https://github.com/hashicorp/terraform/issues/23816

Checkpoint provides wrong download URL for Terraform v0.11.19 · Issue #23816 · hashicorp/terraform

https://checkpoint-api.hashicorp.com/v1/check/terraform currently returns the following: { &quot;product&quot;: &quot;terraform&quot;, &quot;current_version&quot;: &quot;0.11.19&quot;, &quot;curren…

Rob Rose avatar
Rob Rose

Hey I was hoping someone could let me know if I’m on the right track. I’m currently setting up a Jenkins pipeline to provision resources for a startup I’m freelancing for. I was planning on using a multibranch repository with each branch for one environment. Is this an alright way to do it? Or should I do something else?

Joe Niland avatar
Joe Niland

Hi @Rob Rose this would fit into the #release-engineering channel.

Branch/env strategy depends on the team’s exact workflow but normally develop to an integration environment and master to Production is the minimum.

Rob Rose avatar
Rob Rose

Thanks @Joe Niland currently the client has a couple customers and they want one production environment per customer plus one develop environment per developer as well as staging. Trying to figure out the best way to orchestrate that all using Jenkins. Do you know of any examples?

Joe Niland avatar
Joe Niland

It sounds pretty standard. Create a pipeline and define variables that will change per environment. Define build and deploy stages.

Joe Niland avatar
Joe Niland
End-to-End Multibranch Pipeline Project Creation attachment image

Jenkins – an open source automation server which enables developers around the world to reliably build, test, and deploy their software

Joe Niland avatar
Joe Niland

Do they definitely need separate branches per customer? That seems like a potentially more difficult way to manage changes across multiple customers.

Rob Rose avatar
Rob Rose

@Joe Niland Don’t definitely need separate branches per customer but I’m not sure how else to persist variables. Not too familiar with Jenkins so I’ll have to keep digging.

Joe Niland avatar
Joe Niland

@Rob Rose in my experience, it’s normally easier to use Env vars for system-wide variables, and then use a database or a config file (if secure) for customer-specific variables

Philip L Bankier avatar
Philip L Bankier

Hey can someone help me terraform an EKS cluster. I’m trying to use the aws_eks_node_group resource but i can’t figure out how to pass it the worker’s security group so when i deploy i get workers that cant connect to the cluster because they dont have the right security group. Is that resource supposed to generate the correct security group or something? Also what security group do i use as the source_security_group_id in the cluster security group rules?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Philip L Bankier have you seen our working example here? https://github.com/cloudposse/terraform-aws-eks-node-group

cloudposse/terraform-aws-eks-node-group

Terraform module to provision an EKS Node Group. Contribute to cloudposse/terraform-aws-eks-node-group development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

see examples/complete of using our module that implements the EKS managed node groups

Joe Hosteny avatar
Joe Hosteny

Hi @Andriy Knysh (Cloud Posse). The previously mentioned NLB module is available at https://github.com/jhosteny/terraform-aws-nlb/. I didn’t realize NLBs could not have security groups assigned when I started, so it has a smaller surface area now. Also, I could not figure out how to get access logs to work with NLBs due to encryption issues (not sure it is possible on NLBs yet), so I left that commented out. Also, tests have not been run, though I modified your ALB tests and expect it should work, or be close to working.

jhosteny/terraform-aws-nlb

Terraform module to provision a standard NLB for TCP/UDP/TLS traffic https://cloudposse.com/accelerate - jhosteny/terraform-aws-nlb

Joe Hosteny avatar
Joe Hosteny

I am using this for a Concourse installation in ECS, and it seems to be working so far (concourse web launches in Fargate and passes ALB and NLB health checks). I haven’t actually run traffic through the NLB yet, so it may need another tweak or two.

jhosteny/terraform-aws-nlb

Terraform module to provision a standard NLB for TCP/UDP/TLS traffic https://cloudposse.com/accelerate - jhosteny/terraform-aws-nlb

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

nice work @Joe Hosteny thanks

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

PRs for Cloud Posse modules are welcome

Joe Hosteny avatar
Joe Hosteny

Hmm, is there a way to issue a PR to transfer a repo? I was not aware of that. I will check it out.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ah no

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you want us to use your repo and put it in cloudposse/terraform-aws-nlb? that would be nice, we’ll look into that

Joe Hosteny avatar
Joe Hosteny

Also, I have some changes that propagate this into terraform-aws-ecs-web-app, as well as some changes allow that module to run init containers (also with volumes available to the main container after they are done). I’ll issue those as several PRs so you can decide if they are worthwhile.

Joe Hosteny avatar
Joe Hosteny

Yes, feel free to just copy it. I ran the tooling to build a proper README for cloudposse, so you should be able to just clone it and upload to GH

:--1:1
loren avatar
loren

very cool, variable validation coming to tf 0.12.20… https://github.com/hashicorp/terraform/issues/2847#issuecomment-573252616

Value assertions / validations in configuration files · Issue #2847 · hashicorp/terraform

It would be nice to assert conditions on values, extending the schema validation idea to the actual config language. This could probably be limited to variables, but even standalone assertion state…

:--1:2

2020-01-09

kj22594 avatar
kj22594

Hi all, someone yesterday asked a question here: https://sweetops.slack.com/archives/CB6GHNLG0/p1578496997120600 and I have a very similar question. My team is currently trying to figure out the best way to deploy resources different AWS account in multiple regions using modules in terraform. For example we’d want to deploy an EC2 instance into account A in us-east-1 and us-west-1 and deploy that same instance into account B in the same regions. Is anyone doing anything like this, and if so how are you structuring your terraform to do so?

Greetings all, We have a number of existing terraform modules that we are looking to expand into a multi-region/multi-env deployment process. Just curious if anyone has any recommendations or instructions on repo configuration/setup for modules that would be deployed concurrently? Thanks in Advance.

slaughtr avatar
slaughtr

Hello! Looking for some pointers here. This deploys the project to codebuild as expected, but doesn’t want to properly link to the private repo. I have to go into the UI and change it from Public repository to Repository in my GitHub account and find it in the dropdown every time. As you can see at the bottom, I’m mirroring what TF is reporting the config as when it’s setup properly in AWS, but that doesn’t seem to fix it. Any help sincerely appreciated, this is a manageable nuisance, but a nuisance nonetheless.

slaughtr avatar
slaughtr

This is copied/modified from cloudposse/terraform-aws-codebuild, if that helps in any way

Tom Taubkin avatar
Tom Taubkin

Hi the example of https://github.com/cloudposse/terraform-aws-ecs-web-app/tree/master/examples/without_authentication doesn’t run.

I keep getting this response:

Error: Error in function call

  on .terraform/modules/web_app.alb_target_group_cloudwatch_sns_alarms/main.tf line 49, in locals:
  49:   alarm_actions             = coalescelist(var.alarm_actions, var.notify_arns)
    |----------------
    | var.alarm_actions is empty list of string
    | var.notify_arns is empty list of string

Call to function "coalescelist" failed: no non-null arguments.


Error: Error in function call

  on .terraform/modules/web_app.alb_target_group_cloudwatch_sns_alarms/main.tf line 50, in locals:
  50:   ok_actions                = coalescelist(var.ok_actions, var.notify_arns)
    |----------------
    | var.notify_arns is empty list of string
    | var.ok_actions is empty list of string

Call to function "coalescelist" failed: no non-null arguments.


Error: Error in function call

  on .terraform/modules/web_app.alb_target_group_cloudwatch_sns_alarms/main.tf line 51, in locals:
  51:   insufficient_data_actions = coalescelist(var.insufficient_data_actions, var.notify_arns)
    |----------------
    | var.insufficient_data_actions is empty list of string
    | var.notify_arns is empty list of string
Tom Taubkin avatar
Tom Taubkin

When trying to replicate locally without using the example I get the same errors

Brij S avatar
Brij S

When using terragrunt’s plan-all command, Ive got the following directory structure

.
├── global
│   ├── [main.tf](http://main\.tf)
│   └── terragrunt.hcl
├── terragrunt.hcl
└── us-east-1
    ├── [main.tf](http://main\.tf)
    └── terragrunt.hcl

[main.tf> inside the us-east-1 folder has a variable which refers to module.route53_zone.zone_id which is an output of the module referred to in main.tf in the global <http://main.tf|main.tf](http://main.tf). However I get the following error:

Error: Reference to undeclared module

  on [main.tf](http://main\.tf) line 15, in module "acm":
  15:   zone_id      = module.route53_zone.zone_id

No module call named "route53_zone" is declared in the root module.

[terragrunt] 2020/01/09 10:50:41 Encountered the following errors:
Hit multiple errors:
exit status 1

Is this even possible with terragrunt? Or am I doing something wrong?

slaughtr avatar
slaughtr

You need to add a dependency block in the .hcl file that’s referencing the other. TG syntax is a bit different.

inputs = {
  vpc_id = dependency.vpc.outputs.vpc_id
}

dependency "vpc" {
    config_path = "../../network/vpc"
}
Brij S avatar
Brij S

now it says

/global/terragrunt.hcl is a dependency of /us-east-1/terragrunt.hcl but detected no outputs. Either the target module has not been applied yet, or the module has no outputs. If this is expected, set the skip_outputs flag to true on the dependency block. 
Brij S avatar
Brij S

based on your example, I have

inputs = {
  zone_id = dependency.route53_zone.outputs.zone_id
}

dependency "route53_zone" {
    config_path = "../global"
}
Brij S avatar
Brij S

do I need mock outputs?

Brij S avatar
Brij S

oh I think do

slaughtr avatar
slaughtr

So the actual terraform (not terragrunt.hcl) for whatever you’re using for your global there should have an output called zone_id - outputs here refers literally to what’s in the outputs for that module.

slaughtr avatar
slaughtr

You can’t (I don’t think) access resources directly in TG. Add an output "route_53_zone_id" with the proper value to your [outputs.tf](http://outputs\.tf) in your global module and then it’s dependency.route53_zone.outputs.route_53_zone_id

slaughtr avatar
slaughtr

You will also need to re-apply your global module for those outputs to be detected

Brij S avatar
Brij S

@slaughtr do you know how I can pass the input to my module? the .tf file in my us-east-1 folder has the following

provider "aws" {
  region  = "us-east-1"
}

terraform {
  backend "s3" {}
}

module "acm" {
  source  = "terraform-aws-modules/acm/aws"
  version = "2.3.0"

  domain_name  = "[tools.domain.com](http://tools\.domain\.com)"
  zone_id      = ??
}

and the .hcl file has

include {
  path = find_in_parent_folders()
}

dependency "global" {
    config_path = "../global"

    mock_outputs = {
    zone_id = "Z3P5QSUBK4POTI"
  }
}

inputs = {
  zone_id = dependency.global.outputs.zone_id
}

how do i pass zone_id input as a variable to the module?

slaughtr avatar
slaughtr

To the acm module? You’d have a variable "zone_id" and then zone_id = var.zone_id

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’m getting below error. Is this the right forum to ask for help?

terraform init
Initializing modules...
Downloading cloudposse/ecs-container-definition/aws 0.21.0 for ecs-container-definition...

Error: Failed to download module

Could not download module "ecs-container-definition" ([ecs.tf:106](http://ecs\.tf:106)) source code
from
"<https://api.github.com/repos/cloudposse/terraform-aws-ecs-container-definition/tarball/0.21.0//*?archive=tar.gz>":
Error opening a gzip reader for
Bernhard Lenz avatar
Bernhard Lenz

Bernhard Lenz [3:15 PM] I’m getting below error.

terraform init
Initializing modules...
Downloading cloudposse/ecs-container-definition/aws 0.21.0 for ecs-container-definition...

Error: Failed to download module

Could not download module "ecs-container-definition" ([ecs.tf:106](http://ecs\.tf:106)) source code
from
"<https://api.github.com/repos/cloudposse/terraform-aws-ecs-container-definition/tarball/0.21.0//*?archive=tar.gz>":
Error opening a gzip reader for

My terraform file contains

module "ecs-container-definition" {
  source  = "cloudposse/ecs-container-definition/aws"
  version = "0.21.0"

The URL does not seem to resolve correctly. Does anybody know here how to get this to work? I believe this worked for me 2 days ago

slaughtr avatar
slaughtr

What do you have for your source value?

Bernhard Lenz avatar
Bernhard Lenz

module “ecs-container-definition” { source = “cloudposse/ecs-container-definition/aws” version = “0.21.0”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhhhhhh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think this was a bug in terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are you running the latest terraform?

Bernhard Lenz avatar
Bernhard Lenz

yeah latest version 0.12.18

Bernhard Lenz avatar
Bernhard Lenz

on windows

slaughtr avatar
slaughtr

Maybe try using a git:: source? IE source = "git::<https://github.com/cloudposse/terraform-aws-dynamodb.git?ref=tags/0.11.0>"

Bernhard Lenz avatar
Bernhard Lenz

wait, the now have .19. Let me try that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hello all, In the last 24 hours, all of our terraform-null-label modules started failing, with the following error, anyone have any ideas?

Error: Failed to download module

Could not download module "s3_bizrewards_dev_label" ([s3_bizrewards.tf:31](http://s3_bizrewards\.tf:31))
source code from
"<https://api.github.com/repos/cloudposse/terraform-null-label/tarball/0.16.0//*?archive=tar.gz>":
Error opening a gzip reader for
/var/folders/1d/gpvdrwrd0y1_d0jv64w76j645nyvdq/T/getter001152442/archive: EOF.
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is what I was thinking about

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

see the thread/discussion below that

Bernhard Lenz avatar
Bernhard Lenz

Thanks for the quick help

Bernhard Lenz avatar
Bernhard Lenz

0.12.18 -> 0.12.19 fixed it

slaughtr avatar
slaughtr
08:51:48 PM

Annoyingly re-sharing this since it got buried. I’m fixing a lot of stuff in the coming days that touches codebuild so it would be great to figure this out before I do that. Seriously, thanks for any help!

Hello! Looking for some pointers here. This deploys the project to codebuild as expected, but doesn’t want to properly link to the private repo. I have to go into the UI and change it from Public repository to Repository in my GitHub account and find it in the dropdown every time. As you can see at the bottom, I’m mirroring what TF is reporting the config as when it’s setup properly in AWS, but that doesn’t seem to fix it. Any help sincerely appreciated, this is a manageable nuisance, but a nuisance nonetheless.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i don’t know specifically what’s causing this and haven’t/can’t look right now, but sounds like it could be related to something like ignore_changes somewhere.

Hello! Looking for some pointers here. This deploys the project to codebuild as expected, but doesn’t want to properly link to the private repo. I have to go into the UI and change it from Public repository to Repository in my GitHub account and find it in the dropdown every time. As you can see at the bottom, I’m mirroring what TF is reporting the config as when it’s setup properly in AWS, but that doesn’t seem to fix it. Any help sincerely appreciated, this is a manageable nuisance, but a nuisance nonetheless.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

maybe grep through .terraform/modules and see if you see something related to that

slaughtr avatar
slaughtr

Hmm didn’t even consider that. I’ll look around and see what I can find. And maybe - even if it isn’t really a “fix” - I can use ignore_changes or something to prevent it from reverting what I do in the console and save myself some headache

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ya, or something like that…

slaughtr avatar
slaughtr

It’s all a hack until it isn’t

1
Brij S avatar
Brij S

Ive got a terraform module that creates resources in two different aws accounts. I handle this by doing the following:

provider "aws" {
  region  = "us-west-2"
  profile = "profile1"
}

provider "aws" {
  region              = "us-west-2"
  profile             = "profile2"
  alias               = "digi"
}

I’m trying to utilize terragrunt to deploy many modules. This becomes difficult since the above method no longer works, has anyone encountered this? If so, how have you got around this. I dont think Terragrunt supports multiple providers like this

David avatar
David

Hey Brij, I’ve used multiple providers before in terragrunt modules without issue.

What error are you seeing?

Brij S avatar
Brij S

no errors yet, Im trying to understand how terragrunt will manage to use different aws profiles

Brij S avatar
Brij S

how have you used multiple providers ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(also we have #terragrunt - might get more feedback there)

Brij S avatar
Brij S

oh woops - thanks

2020-01-08

Phuc avatar

HI Guys

1
Phuc avatar

I’m using terraform-rds at the moment to create postgres RDS. using engine version 9.6.15 Thing work ok with DB_paramter_group, but when it running to create DB_Option_group, it return error as below:

InvalidParameterCombination: Cannot find major version 9 for postgres

I look at AWS document, seem it didn’t have any DB option group yet for Postgres, So how can I by pass this resource in the module?

Bruce avatar
Bruce

Hi team, I am using the terraform-aws-rds-cluster to create an Aurora MySQL read replica of an RDS MySQL instance to transition over to Aurora by using the replication_source_identifier set as the RDS MySQL instance. However the creation hangs on Terraform but is successful in the console. It’s most likely an issue with AWS provider, but I am curious to see if anyone else has come across this issue or have a work around that was successful?

Callum Robertson avatar
Callum Robertson

Hey @Erik Osterman (Cloud Posse) or @Andriy Knysh (Cloud Posse), can one of you tell me how you’re doing terraform md automation? (If you are)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Hi Callum. What’s terraform md automation?

Callum Robertson avatar
Callum Robertson

Sorry @Andriy Knysh (Cloud Posse) if that was confusing, wondering if you had any way of automatically generating terraform inputs/outputs in an md format

roth.andy avatar
roth.andy
segmentio/terraform-docs

Generate documentation from Terraform modules in various output formats - segmentio/terraform-docs

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Yes we use it as well. In build-harness we have a Make target, make readme, that generates md files from terraform

Rhawnk avatar
Rhawnk

Greetings all, We have a number of existing terraform modules that we are looking to expand into a multi-region/multi-env deployment process. Just curious if anyone has any recommendations or instructions on repo configuration/setup for modules that would be deployed concurrently? Thanks in Advance.

Rhawnk avatar
Rhawnk

for now, we are mostly concerned with autoscaling groups, albs, rt53 records

sype avatar

Hello to every one from France

4
Zachary Loeber avatar
Zachary Loeber

good evening

George Platon avatar
George Platon

hi guys, I’m using terraform-aws-elastic-beanstalk-environment which I link to an already created VPC, and I keep getting into the problem that it tries to create a security group twice (see the error below). `

Error creating Security Group: InvalidGroup.Duplicate: The security group 'xxxx' already exists for VPC 'vpc-xxxxxxxxxxx'
	status code: 400
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@George Platon did you check in the AWS console if a SG with that name already exists?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-elastic-beanstalk-environment

Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment

George Platon avatar
George Platon

Yes, it does not exists. I do delete everything, and then its gone

George Platon avatar
George Platon

I am also using a rds instance from cloudposse, which is in the same VPC

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

the example above gets deployed automatically by terratest

George Platon avatar
George Platon

Ill try to run the complete example, although my parameters are pretty similar

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

was working last time we updated the module

George Platon avatar
George Platon

I’m using something of this kind.

George Platon avatar
George Platon
module "vpc" {
  source     = "git::<https://github.com/cloudposse/terraform-aws-vpc.git?ref=tags/0.7.0>"
  //  General
  namespace  = var.namespace
  stage      = var.stage
  name       = var.name
  tags       = var.tags
  //  Network
  cidr_block = var.vpc_cidr_block
}

module "subnets" {
  source               = "git::<https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=tags/0.16.0>"
  //  General
  availability_zones   = var.availability_zones
  namespace            = var.namespace
  stage                = var.stage
  name                 = var.name
  tags                 = var.tags
  //  Network
  vpc_id               = module.vpc.vpc_id
  igw_id               = module.vpc.igw_id
  cidr_block           = module.vpc.vpc_cidr_block
  nat_gateway_enabled  = var.vpc_gateway_enabled
  nat_instance_enabled = false
}

// Database

module "rds_instance" {
  // possibly put this into a separate vpc without access to outside
  // make sure we use a version - e.g ?ref=tags/0.9.3
  source              = "git::<https://github.com/cloudposse/terraform-aws-rds.git>"
  //  General
  namespace           = var.namespace
  stage               = var.stage
  name                = var.name
  tags                = var.tags
  multi_az            = var.multi_az

  //  Network
  vpc_id              = module.vpc.vpc_id
  associate_security_group_ids = [module.vpc.vpc_default_security_group_id]
  security_group_ids  = [module.vpc.vpc_default_security_group_id]
  subnet_ids          = module.subnets.private_subnet_ids
  //  Rds specific
  database_name       = var.rds_db_name
  database_user       = var.rds_db_user
  database_password   = var.rds_db_password
  database_port       = var.rds_db_port
  storage_type        = var.rds_storage_type
  storage_encrypted   = var.rds_storage_encrypted
  allocated_storage   = var.rds_allocated_storage
  engine              = var.rds_engine
  engine_version      = var.rds_engine_version
  major_engine_version = var.rds_major_engine_version
  instance_class      = var.rds_instance_class
  db_parameter_group  = var.rds_db_parameter_group
  publicly_accessible = var.rds_publicly_accessible
  apply_immediately   = var.rds_apply_immediately
  deletion_protection = var.rds_deletion_protection

  db_parameter = [
    {
      name         = "myisam_sort_buffer_size"
      value        = "1048576"
      apply_method = "immediate"
    },
    {
      name         = "sort_buffer_size"
      value        = "2097152"
      apply_method = "immediate"
    }
  ]
}

// ElasticBeanStalk

module "elastic_beanstalk_application" {
  source      = "git::<https://github.com/cloudposse/terraform-aws-elastic-beanstalk-application.git?ref=tags/0.4.0>"
  //  General
  namespace           = var.namespace
  stage               = var.stage
  name                = var.name
  tags                = var.tags
  description         = "Elastic_beanstalk_application"
}

module "elastic_beanstalk_environment" {
  source                             = "git::<https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment.git?ref=tags/0.17.0>"
  //  General
  namespace           = var.namespace
  stage               = var.stage
  name                = var.name
  tags                = var.tags
  region                             = var.region
  description                        = "Elastic_beanstalk_environment"
  availability_zone_selector         = "Any 2"

  // Configuration
  // see <https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment> for more details
  dns_zone_id                        = var.eb_dns_zone_id
  elastic_beanstalk_application_name = module.elastic_beanstalk_application.elastic_beanstalk_application_name

  instance_type           = var.eb_instance_type
  autoscale_min           = var.eb_autoscale_min
  autoscale_max           = var.eb_autoscale_max
  updating_min_in_service = var.eb_updating_min_in_service
  updating_max_batch      = var.eb_updating_max_batch

  environment_type        = var.eb_environment_type
  loadbalancer_type       = var.eb_loadbalancer_type
  vpc_id                  = module.vpc.vpc_id
  loadbalancer_subnets    = module.subnets.public_subnet_ids
  loadbalancer_security_groups = [module.vpc.vpc_default_security_group_id]
//  loadbalancer_managed_security_group = [module.vpc.vpc_default_security_group_id]
  application_subnets     = module.subnets.private_subnet_ids
  allowed_security_groups = [module.vpc.vpc_default_security_group_id]
  enable_stream_logs      = var.eb_enable_stream_logs
  keypair                 = var.eb_sshkeypair


  solution_stack_name = var.eb_solution_stack_name

  env_vars = {
    db_arg = module.rds_instance.instance_endpoint
    db_host = module.rds_instance.instance_endpoint }

  
}
George Platon avatar
George Platon
George Platon avatar
George Platon

Ill try to run the complete example

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

oh ok, you are creating RDS together with EB

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and provide the same namespace, stage and name

George Platon avatar
George Platon

yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

both prob create the same SG (don’t remeber)

George Platon avatar
George Platon

can be the case, yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

try to add some attributes to any of those

George Platon avatar
George Platon

a quick fix would be to give different names

George Platon avatar
George Platon

ok, will add now

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

e.g. attributes = [“rds”] to RDS

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

or whatever name for the attribute you like

George Platon avatar
George Platon

sure, doing it now

George Platon avatar
George Platon

it is a bad practice to have similar name for multiple resources ?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we name all resources namespace-stage-name-attributes, which is perfectly fine

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

the issue you encountered is what we noticed after the modules were created. Some modules create resources like IAM Roles and Security Groups using the same pattern

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and those collide with same resources created by other modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so we started to add attributes to Roles and SGs inside modules, but not of them have been updated yet

George Platon avatar
George Platon

ok then will try to destroy them and run them again with the new attributes, and then check manually if rds SG has the correct naming, and also the ElasticBeanstalk

George Platon avatar
George Platon

it worked all fine

George Platon avatar
George Platon

thanks a lot @Andriy Knysh (Cloud Posse)

George Platon avatar
George Platon

I owe you one

:--1:1

2020-01-07

2020-01-06

Cloud Posse avatar
Cloud Posse
05:01:23 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Jan 15, 2020 11:30AM.
Register for Webinar
slack #office-hours (our channel)

Brij S avatar
Brij S

Hey all, Ive got a warning i’m not sure how to get past. Was hoping some of the experts here can point me in the right direction

Warning: Interpolation-only expressions are deprecated

  on ../main.tf line 95, in resource "aws_lambda_function" "publisher":
  95:   source_code_hash = "${filebase64sha256("${path.module}/publisher.zip")}"

Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.

(and 5 more similar warnings elsewhere)

I’m not really sure how to stop the warning, I checked the online documentation and everything seems correct

slaughtr avatar
slaughtr
To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

So I belive you just need: source_code_hash = filebase64sha256("${path.module}/publisher.zip")

:--1:2
Naseem avatar
Naseem

Anyone have good links to TF directory structures?

so far I’ve compiled up https://www.2ndwatch.com/blog/how-we-organize-terraform-code-at-2nd-watch/ , https://www.oreilly.com/library/view/terraform-up-and/9781491977071/ch04.html and http://saurabh-hirani.github.io/writing/2017/08/02/terraform-makefile

Going through a major refactor of TF configs currently and would like to get it right the first time around…

Adam Blackwell avatar
Adam Blackwell

Boston DevOps had a long conversation on the topic, but it’s sadly now lost in Slack.

https://www.reddit.com/r/Terraform/comments/bskqbg/advice_for_folder_structure/ may have some useful insights though.

:--1:1
Naseem avatar
Naseem

Thanks @Adam Blackwell

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also, check this out https://archive.sweetops.com/

SweetOps Slack Archive

SweetOps is a collaborative DevOps community. We welcome engineers from around the world of all skill levels, backgrounds, and experience to join us! This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build sweet infrastructure.

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you search for “folder structure” there will be some past discussions

:--1:1
Naseem avatar
Naseem

Great, thanks!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ah yes! forgot they have those excellent docs now….

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks for posting back!

:--1:1
Mateusz Kamiński avatar
Mateusz Kamiński

Hey, in https://github.com/cloudposse/terraform-aws-rds you do not have solution to update ca_cert (parameter ca_cert_identifier was added to aws_db_instance recently). This needs to be added, or do you have some other solution?

cloudposse/terraform-aws-rds

Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Mateusz Kamiński if it was added recently aws_db_instance, it needs to be implemented. PRs are always welcome, thanks

cloudposse/terraform-aws-rds

Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.

2020-01-03

NVMeÐÐi avatar
NVMeÐÐi

Hello, I am trying to solve terraform drift and I ran into an error Error: module “my_rds_resource”: “performance_insights_enabled” is not a valid argument For this particular resource, it’s in 0.11.14 using the terraform-aws-rds-aurora 1.21.0 release. I see that performance_insights_enabled has been supported since 1.0.0 release, any idea why my module kicks back this error?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
terraform-aws-modules/terraform-aws-rds-aurora

Terraform module which creates RDS Aurora resources on AWS - terraform-aws-modules/terraform-aws-rds-aurora

2020-01-02

Laurynas avatar
Laurynas

Hi, does anyone know why terraform sees too many changes in the task definition updates?


\# [module.ecs_app_service.module.ecs_task_definition.aws_ecs_task_definition.app](http://module\.ecs_app_service\.module\.ecs_task_definition\.aws_ecs_task_definition\.app) must be replaced
+/- resource "aws_ecs_task_definition" "app" {
      ~ arn                      = "arn:aws:ecs:eu-west-1:xxxx:task-definition/test:216" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [ # forces replacement
              ~ {
                    cpu              = 256
                  + entrypoint       = null
                  ~ environment      = [
                      - {
                          - name  = "AWS_REGION"
                          - value = "eu-west-1"
                        },
                        {
                            name  = "APP_ENV"
                            value = "prod"
                        },
                      ~ {
                          ~ name  = "AWS_USER_POOL_ID" -> "APP_DEBUG"
                          ~ value = "us-east-jgjjh" -> "0"
                        },
                      + {
                          + name  = "AWS_REGION"
                          + value = "eu-west-1"
                        },
                    ]

is it because I use Jsondecode and it changes the ordering off the elements?

loren avatar
loren

Anyone know of a way to have terraform init check a central location for providers and download them to that location if they are missing? I know there is -plugin-dir for the first part of that question, but it explicitly disables the second part. I don’t really understand the need to have the same version of the same provider in .terraform in the config working directory for every config… it’s a lot of space and a lot of downloads

loren avatar
loren

Or a different tool that can retrieve providers the way terraform init does and place them in a specified directory…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Have you tried TF_PLUGIN_CACHE_DIR?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I could have sworn this does not disable automatic fetching

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We set this to a shared location in #geodesic to speed up init process

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
hashicorp/terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is an open source tool that codifies APIs into declarative configuration files that can be shared amon…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But I am not quite sure if that meets your requirements

loren avatar
loren

i haven’t, i was under the impression that was just the env-equivalent of -plugin-dir , which the docs at least claim disable the auto-download functionality…

  -plugin-dir          Directory containing plugin binaries. This overrides all
                       default search paths for plugins, and prevents the
                       automatic installation of plugins. This flag can be used
                       multiple times.
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Fwiw, we have this set and we don’t download any plugins manually

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
du -sh ~/.terraform.d/plugins/
1.2G	/home/erik.osterman/.terraform.d/plugins/
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think it’s working…

loren avatar
loren

interesting, ok, i’ll give it a try, thanks!

loren avatar
loren

works! brilliant!

loren avatar
loren

i mean, it still copies the binary into .terraform/plugins in the config working directory, which seems unnecessary, but it gets them from the plugin cache dir, so at least i can save on some data when tethering

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Interesting - didn’t realized it double-copied them

Brij S avatar
Brij S

is anyone able to provide more insight into the following descriptions? https://github.com/cloudposse/terraform-null-label/blob/master/outputs.tf#L3 (Could you elaborate on what this is disambiguated from? Is it regarding the AWS Name tag?)

https://github.com/cloudposse/terraform-null-label/blob/master/outputs.tf#L8 (how do all of these fields get ‘normalized’? It would be nice if this was a bit more clearer)

cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

disambiguated is prob not a good name for it

cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it is constructed from the inputs you provide

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and is prob disambiguated because it’s supposed to be consistent (you use the same pattern for everything) and globally unique across all AWS accounts and environments even for the global AWS resources like S3 buckets

Brij S avatar
Brij S

I see, what about the normalized references

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

in almost all cases you don’t see/need the ‘normalization’ when you provide simple strings for namespace, stage, name etc.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

the module converts the inputs to lower-case

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

vvsp avatar

Hi There… Quick query pls:

Does EKS cluster creation using terraform creates kubeconfig at ~/.kube by default ? or do we have to configure it manually every time we create the cluster as some of the fields are cluster specific ?

maarten avatar
maarten

Have you done a POC yet with creating an EKS cluster using Terraform ?

vvsp avatar

Not yet … in the process of doing it; hence arrived at this step.

vvsp avatar

Have you guys got any info on POC for EKS with terraform ?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@vvsp here are the terraform modules that we have for EKS:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-workers

Terraform module to provision an AWS AutoScaling Group, IAM Role, and Security Group for EKS Workers - cloudposse/terraform-aws-eks-workers

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-node-group

Terraform module to provision an EKS Node Group. Contribute to cloudposse/terraform-aws-eks-node-group development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-fargate-profile

Terraform module to provision an EKS Fargate Profile - cloudposse/terraform-aws-eks-fargate-profile

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

kubeconfig is already in the cluster after you create it, you just need to read it from there

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

aws eks update-kubeconfig reads it from the cluster and saves on the file system

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and Terratest that deploys the example on AWS and checks for correct outputs https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/test/src/examples_complete_test.go

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

also, take a look at this comment from @Erik Osterman (Cloud Posse) https://github.com/terraform-aws-modules/terraform-aws-eks/issues/635#issuecomment-567691445 (regarding another EKS module form #terraform-aws-modules) describing why we have 4 different EKS modules instead of just one

Is the complexity of this module getting too high? · Issue #635 · terraform-aws-modules/terraform-aws-eks

A general question for users and contributors of this module My feeling is that complexity getting too high and quality is suffering somewhat. We are squeezing a lot of features in a single modul…

2020-01-01

Zachary Loeber avatar
Zachary Loeber

For complex terrform modules spanning multiple environments, why do I almost always regret using modules? Is there some kind of rule of thumb about module complexity that should be followed?

Zachary Loeber avatar
Zachary Loeber

like, rule 1: if you are spanning multiple providers maybe modules aren’t a good idea (or something like that)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we never manage more than one environment/account in one terraform plan/apply

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in our case, with #geodesic , we actually have a one-to-one correlation between AWS accounts and git repos.

chinedu2424 avatar
chinedu2424

What’s the best learning for newbie Engineers on terraform

Eamon Keane avatar
Eamon Keane

@chinedu2424 check out terraform up and running second edition <http://shop.oreilly.com/product/0636920225010.do> Then choose something you know how to deploy without terraform, open a free account on Terraform cloud and start iterating with a personal AWS/GCP account (you can prob find a community module to get you started).

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Zachary Loeber avatar
Zachary Loeber

Or you can inherit a 10 thousand line complex terraform manifest like I did and learn ‘under pressure’

3
vvsp avatar

Need an inputs pls:

I have a VPC: 10.0.0.0/16 Subnets: 2 pub + 2 pvt spanned across 2 AZs in one single region. pub sub1-> 10.0.0.0/24 pvt sub1 -> 10.0.1.0/24

pub sub2->10.0.2.0/24 pvt sub2 -> 10.0.3.0/24

Pvt route table 1 has two routes: local and a route for 10.0.1.0/24 to a NAT GW.

I am stuck with the below error when associating Private route table 1 to private subnet 1. wondering whats actually happening under the hood and why the issue ? any inputs will be of a great help.

API error message Route table contains unsupported route destination. The unsupported route destination is more specific or equal specific than VPC local CIDR.

maarten avatar
maarten

HI @vvsp it seems you route the private subnet itself to the nat GW, that won’t work, you need to route public routable nets to the nat gw, Common is to route 0.0.0.0/0 to the nat gw.

vvsp avatar

@maarten Thanks and that was the thing; appreciate your response;

    keyboard_arrow_up