#terraform-aws-modules (2019-06)

terraform Terraform Modules

Discussions related to https://github.com/terraform-aws-modules

Archive: https://archive.sweetops.com/terraform-aws-modules/

2019-06-02

Mike Nock avatar
Mike Nock

Hey all, does anyone have some recommendations on a terraform module setup for an ecs fargate cluster setup? It needs to support 4 services per cluster, alb, possible nginx side car container integration. Thoughts? I had one working for a while, but the tags broke.

Steven avatar

I’m in the process of upgrading mine for fargate support. Need to finish it this week. Have too many things that need it

Mike Nock avatar
Mike Nock

There was an awesome module by azavea but the tags broke on it for TFE, so I gotta find a new module. The last one was perfect, created the ALB, the ECS cluster, multiple services.

Mike Nock avatar
Mike Nock

autoscaling policies, etc. Preferably I could find another one like it, but if I have to use multiple, oh well. What are you using in your setup?

Steven avatar

Mine handles ECS service on ECS2, alb, , tags, and things. It is for a service. So, you can create as many services as needed. Both container and task definitions can be overridden for more complex setups (like sidecars, proxies). Partial fargate support is there. But not awsvpc networking yet

Steven avatar
appzen-oss/terraform-aws-ecs-service

Contribute to appzen-oss/terraform-aws-ecs-service development by creating an account on GitHub.

Steven avatar

Also, this conversation should be in #terraform

2019-06-04

johncblandii avatar
johncblandii
cloudposse/terraform-aws-ecs-alb-service-task

Terraform module which implements an ECS service which exposes a web service via ALB. - cloudposse/terraform-aws-ecs-alb-service-task

1

2019-06-05

Mike Nock avatar
Mike Nock

That’s a pretty nice module. Is there a way to output the ALB DNS name for R53 records?

David Latham avatar
David Latham

Woah! just hit my first v0.12 compatibility gotcha. We have extensive use of the null-label module and are running v.11. So this hit us… https://github.com/cloudposse/terraform-null-label/commit/e2b880b2bad3cd8ceaa87c56176c15d92e28775d Anyway - a quick sed -i "s/.../.../" solved it.

Terraform 0.12 support (#63) · cloudposse/terraform-null-label@e2b880b
  • Terraform 0.12 support * add tests scenario * Update README
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@David Latham please make sure you never pin to master in any of your infrastructure code

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(it’s not possible to maintain backwards compatibility between releases and pinning to a release is the only way to ensure stability in a global ecosystem of terraform modules)

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

antonbabenko avatar
antonbabenko

I like the minimalistic usage of terratest there.

cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

use 0.11.1 or earlier

David Latham avatar
David Latham

yup - I guess I would need to thank our consultant for that. Luckily it’s not pinned to master everywhere - just this one project and a simple enough PR to fix.

ian avatar

really really stupid question but I’m so lost: i’m trying to provision a cross-region rds replica but I can’t figure out how to create KMS keys for two different regions (us-east-2 for the source db and us-west-2 for the replica db). Do I resource two aws_kms_key and specify the region somehow?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@ian are you using one of these modules: https://github.com/terraform-aws-modules/

Terraform AWS modules

Collection of Terraform AWS modules supported by the community - Terraform AWS modules

ian avatar

no a senior dev has rolled his own setup but it’s similar to those modules and I can use them as a reference

2019-06-06

renaldrozario avatar
renaldrozario

Has anyone tested provisioner during terraform destroy? Destroy-Time Provisioners <https://www.terraform.io/docs/provisioners/index.html>

renaldrozario avatar
renaldrozario

We are currently testing it and destroy handle doesn’t seem to pick up…. If someone has tested, please advise?

Mike Nock avatar
Mike Nock

Hey guys, using this module: https://github.com/cloudposse/terraform-aws-s3-log-storage?ref=master and running into an issue where the ALBs I’m creating don’t have access to the bucket. Is that an attribute input I need to set?

cloudposse/terraform-aws-s3-log-storage

This module creates an S3 bucket suitable for receiving logs from other AWS services such as S3, CloudFront, and CloudTrail - cloudposse/terraform-aws-s3-log-storage

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Terraform AWS modules

Collection of Terraform AWS modules supported by the community - Terraform AWS modules

Mike Nock avatar
Mike Nock

Ah my apologies. I’ll use terraform channel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thx!

2019-06-11

cabrinha avatar
cabrinha

anyone have an AWS AppMesh terraform module they’re working on?

Steven avatar

Not yet. Will probably be on my project list next month

2019-06-12

cabrinha avatar
cabrinha
bitflight-public/terraform-aws-app-mesh

Terraform module for creating the app mesh resources - bitflight-public/terraform-aws-app-mesh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@jamie

jamie avatar
jamie
08:43:31 PM

@jamie has joined the channel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@cabrinha meet @jamie

cabrinha avatar
cabrinha

jamie avatar

hi

cabrinha avatar
cabrinha

hello

cabrinha avatar
cabrinha

let me know if you need any help on the app mesh module, it looks great so far

jamie avatar

@cabrinha when did you need it by? I am starting by making an mvp using terraform 0.12, and I’m not going to include all of the node features like logging and health checks at the start

cabrinha avatar
cabrinha

i’d like to use terraform 0.11

cabrinha avatar
cabrinha

and need? well the sooner the better

jamie avatar

So once the mvp is done, you can do pr’s to add in other features and such

cabrinha avatar
cabrinha

Error downloading modules: Error loading modules: module virtual_router_label: duplicated. module names must be unique

cabrinha avatar
cabrinha

this is the error I get as it sits now

jamie avatar

Lol. Yeah it’s not ready yet I’ve only been writing it for about 3.5 hours

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(that means Jamie will have a few PRs open in ~30m)

jamie avatar

Ha.

jamie avatar

It’s 7pm for me and I was about to have a beer. I am gonna keep going tomorrow:)

cabrinha avatar
cabrinha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cabrinha avatar
cabrinha

jamie avatar

Btw @cabrinha are you using an existing module to create your services and ecs cluster?

jamie avatar

Like airship?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
1
cabrinha avatar
cabrinha

nope, we use our own in-house module

jamie avatar

Ok. So to make app mesh work the task definitions have to have an extra two sidecar containers running in them

cabrinha avatar
cabrinha

sure, envoy and xray

jamie avatar

Yeah.

cabrinha avatar
cabrinha

thats easy, i’ve got my task-definition sorted out already

jamie avatar

So what I’ll do it create a template wrapper for generating the task def

jamie avatar

That adds the sidecars as well

jamie avatar

So you can pass in your container def, and ports, and app mesh name, and it outputs the full task def

jamie avatar

With the ports, proxy, and labels in place

cabrinha avatar
cabrinha

hm ok that sounds cool

jamie avatar

I’ll need to make it for when I do the example anyway

jamie avatar

Anyway, I’ll update it further tomorrow

cabrinha avatar
cabrinha

2019-06-14

Blaise Pabon avatar
Blaise Pabon

(I think I will need to watch a video or adjust my meds, because I didn’t understand that announcement…)

Blaise Pabon avatar
Blaise Pabon

(oh, boy, I just noticed the video is from AWS Summit Anaheim and I was at that show …I guess I must not have been paying attention.

2019-06-18

jamie avatar

@cabrinha this app mesh module’s example is getting bigger than the actual module

jamie avatar

haha

cabrinha avatar
cabrinha

lel

cabrinha avatar
cabrinha

yeah thats expected when you’re making on ECS cluster, VPC and all that

jamie avatar

I have a few things still to do today:

  • Complete the codepipeline build and deploy to ecr from s3 source
  • Complete the task wrapper
  • Complete the IAM roles required for running the tasks
  • Complete the ECS Service Update pipeline
1

2019-06-19

jamie avatar

@Callum Robertson this channel will help you.

Blaise Pabon avatar
Blaise Pabon

yes @Callum Robertson, the light side of the force is strong in this channel.

1
Callum Robertson avatar
Callum Robertson
02:12:45 PM

@Callum Robertson has joined the channel

2019-06-21

cabrinha avatar
cabrinha

Looks like I’m having some issue with the EKS module https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/4.0.2

cabrinha avatar
cabrinha

kubectl get svc

cabrinha avatar
cabrinha
NAME                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
exiled-marmot-mysql   ClusterIP   172.20.29.176   <none>        3306/TCP   2d
kubernetes            ClusterIP   172.20.0.1      <none>        443/TCP    3d
cabrinha avatar
cabrinha

is there no way to get VPC IPs now?

cabrinha avatar
cabrinha

hm, maybe its just the subnet tagging that I was missing

cabrinha avatar
cabrinha
Tutorial: Creating a VPC with Public and Private Subnets for Your Amazon EKS Cluster - Amazon EKS

This tutorial guides you through creating a VPC with two public subnets and two private subnets, which are provided with internet access through a NAT gateway. You can use this VPC for your Amazon EKS cluster. We recommend a network architecture that uses private subnets for your worker nodes, and public subnets for Kubernetes to create public load balancers within.

jamie avatar

@cabrinha the latest version of the dynamic subnet module handles the public private subnet tagging correctly.

cabrinha avatar
cabrinha

nice thanks

cabrinha avatar
cabrinha

are there tags needed?

cabrinha avatar
cabrinha

never saw any docs on that

jamie avatar

It depends. Many of the kubernetes services that need to access and discover aws resources require that those resources are tagged in a way that relates to the kubernetes cluster

jamie avatar

The dynamic subnets module allows you to specify a tag and a value

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i preferred the dynamic sinners module myself

jamie avatar

Ha

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

https://github.com/cloudposse/terraform-aws-dynamic-subnets was converted to TF 0.12, real tests added (using bats and terratest), the tests create real resources in AWS and check them

cloudposse/terraform-aws-dynamic-subnets

Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@cabrinha regarding what @jamie said, there are two variables https://github.com/cloudposse/terraform-aws-dynamic-subnets/blob/master/variables.tf#L1 and https://github.com/cloudposse/terraform-aws-dynamic-subnets/blob/master/variables.tf#L7 which you can use to implement any type of tagging for the subnets

cloudposse/terraform-aws-dynamic-subnets

Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets

cloudposse/terraform-aws-dynamic-subnets

Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets

cabrinha avatar
cabrinha

yeah we’re just starting to get into EKS now, most of our AppMesh stack is for use in ECS

2019-06-24

Phu Bar avatar
Phu Bar

Hi guys, I’m having trouble accessing the kibana web interface via a browser after running the terraform module.

I’m connect to a VPN and allowed the VPN cidr block to access the es cluster yet when I hit the kibana link, I get {"Message":"User: anonymous is not authorized to perform: es:ESHttpGet"}.

I tried adding the cidr block to the access policy but I get UpdateElasticsearchDomainConfig: {"message":"You can’t attach an IP-based policy to a domain that has a VPC endpoint. Instead, use a security group to control IP-based access."}

I can use the domain template to not require signing request with iam credential, which does allow me to access kibana but this is not my ideal way to set it up.

    keyboard_arrow_up