#terraform (2019-01)

terraform Discussions related to Terraform or Terraform Modules

Archive: https://archive.sweetops.com/terraform/

2019-01-31

kritonas.prod avatar
kritonas.prod

@Tobias Hoellrich Look into TF’s overrides, I believe this might give you the most elegant solution. Just put the ES module part into a separate file and name it [override.tf> or postfix it with <http://_override.tf|_override.tf](http://override\.tf) so that it’s merged last, after everything else is created. https://www.terraform.io/docs/configuration/override.html

Overrides - Terraform by HashiCorp

Terraform loads all configuration files within a directory and appends them together. Terraform also has a concept of overrides, a way to create files that are loaded last and merged into your configuration, rather than appended.

Tobias Hoellrich avatar
Tobias Hoellrich

Excellent - thanks for the replies, gentlemen. :–1:

:--1:1
Tobias Hoellrich avatar
Tobias Hoellrich

And for the records:

  • using a plain resource "aws_security_group" did not work and caused the same 'count' cannot be computed error
  • [override.tf](http://override\.tf) also did not work and caused the same error
kritonas.prod avatar
kritonas.prod

This is during the planning phase, right?

Tobias Hoellrich avatar
Tobias Hoellrich

I ended up moving a bunch of security groups into a separate module which was applied before the ES module. Tks again.

Alec avatar

Morning! I was trying to use terraform-aws-alb-target-group-cloudwatch-sns-alarms and got a bunch of errors that were due to me copy/pasting the example straight from the page, which defaulted to an older release with issues, do you normally update examples with versions?

joshmyers avatar
joshmyers

@Alec We strive to keep the examples up to date but alas, there are too many modules to keep on top of

joshmyers avatar
joshmyers

PRs welcome, or open an issue and we can have a look

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-ecs-web-app

Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more. - cloudposse/terraform-aws-ecs-web-app

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Here’s a working example

Alec avatar

Thanks @Erik Osterman (Cloud Posse) – I’m actually going to use that for another project, and I did successfully get it working once I noticed the 0.1.0 tag in the git ref – I’ll submit a PR shortly. I’m still learning how this is all structured, but it’s becoming much more familiar. Appreciate all your guys work.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @Alec

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also, here’s a complete example from our “Service Catalog” that we actively maintain and deploy: https://github.com/cloudposse/terraform-root-modules/tree/master/aws/ecs

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

:--1:1
Bogdan avatar
Bogdan
05:02:51 PM

how do you guys handle the ordered_placement_strategy in a ecs service module, from an input perspective (passing a list of maps or map) when passing several strategies? I couldn’t find an example in cloudposse ecs service modules

rbadillo avatar
rbadillo
terraform-providers/terraform-provider-aws

Terraform AWS provider. Contribute to terraform-providers/terraform-provider-aws development by creating an account on GitHub.

loren avatar
loren

pr’s been open for a while, they haven’t been merging much in the way of new features lately, https://github.com/terraform-providers/terraform-provider-aws/pull/6574

Add Route 53 Resolver endpoint resource by ewbankkit · Pull Request #6574 · terraform-providers/terraform-provider-aws

Fixes #6563. Includes: #6549 #6554 Acceptance tests (so far): $ make testacc TEST=./aws/ TESTARGS=&#39;-run=TestAccAwsRoute53ResolverEndpoint_&#39; ==> Checking that code complies with gofmt r…

loren avatar
loren

just docs, bugs, cleanup, and 0.12 prep mostly

rbadillo avatar
rbadillo

make sense

rbadillo avatar
rbadillo

thanks @loren

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

I’m seeing the ecs-web-app flag with a basic image (nginx:latest) on Fargate. Any thoughts here?

I don’t see any logs for it, but it does show:

Stopped reason Task failed ELB health checks in (target-group arn:aws:elasticloadbalancing:us-west-2:496386341798:targetgroup/sparkle-view-default-test/8bc8ccfff5c546df)

module "default_backend_web_app" {
  source    = "git::<https://github.com/cloudposse/terraform-aws-ecs-web-app.git?ref=tags/0.10.0>"
  namespace = "${local.application_name}"
  stage     = "default"
  name      = "${var.stage}"
  vpc_id    = "${module.stack.vpc_id}"

  container_image  = "nginx:latest"
  container_cpu    = "256"
  container_memory = "512"
  container_port   = "80"

  #launch_type                 = "FARGATE"
  listener_arns                 = "${module.alb.listener_arns}"
  listener_arns_count           = "1"
  aws_logs_region               = "${data.aws_region.current.name}"
  ecs_cluster_arn               = "${aws_ecs_cluster.this.arn}"
  ecs_cluster_name              = "${aws_ecs_cluster.this.name}"
  ecs_security_group_ids        = ["${module.stack.vpc_default_security_group_id}"]
  ecs_private_subnet_ids        = ["${module.stack.vpc_private_subnets}"]
  alb_ingress_healthcheck_path  = "/"
  alb_ingress_paths             = ["/*"]
  alb_ingress_listener_priority = 100

  codepipeline_enabled = "false"
  ecs_alarms_enabled   = "true"
  autoscaling_enabled  = "false"

  alb_name                                        = "${module.alb.alb_name}"
  alb_arn_suffix                                  = "${module.alb.alb_arn_suffix}"
  alb_target_group_alarms_enabled                 = "true"
  alb_target_group_alarms_3xx_threshold           = "25"
  alb_target_group_alarms_4xx_threshold           = "25"
  alb_target_group_alarms_5xx_threshold           = "25"
  alb_target_group_alarms_response_time_threshold = "0.5"
  alb_target_group_alarms_period                  = "300"
  alb_target_group_alarms_evaluation_periods      = "1"

  environment = [
    {
      name  = "PORT"
      value = "80"
    },
  ]
}
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@johncblandii (Cloud Posse) what do you mean by flag?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

flap?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

does / return 200?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

flap…yes

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

got some progress. let me check back on the default

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

yeah, it is INACTIVE now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, check ports

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

is nginx on 80 or 8080?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

yeah, just nginx:latest in use

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

checking ports

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

missing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
port_mappings = [{
    "containerPort" = "${var.atlantis_port}"
    "hostPort"      = "${var.atlantis_port}"
    "protocol"      = "tcp"
  }]
johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

target group is port 80

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ahh…on the default. 1 sec

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
  port_mappings = [
    {
      "containerPort" = "80"
      "hostPort"      = "80"
      "protocol"      = "tcp"
    },
    {
      "containerPort" = "80"
      "hostPort"      = "443"
      "protocol"      = "tcp"
    }
  ]
johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

without ssl on the image, 443 to 80 would still pass, right?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so on fargate, I tihn kthe hostPort must equal the container port

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ah, right. i think i saw that in the docs

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

def’ understand these ALBs more now

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

pretty quality

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, they are

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i love that they added the support for auth

maarten avatar
maarten

just cognito right, or did they add basic auth finally ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no, cognito

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what you added

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

that might be useful for another project we’re using

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@maarten go back under your palmtree and enjoy your vacay

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

maarten avatar
maarten

yeah, enough slack for today

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

just one with hosts for our domains all mapped through TF. (awyeah)

maarten avatar
maarten

@johncblandii (Cloud Posse) what’s also nice is the http->https redirect

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

yeah, played w/ that as well (not in the tf setup yet)

2019-01-30

praveen avatar
praveen

Hi, am working on adding windows domain administrator group to a server using DSC as an extension for terraform config. As part of terraform DSC config I am passing the argument as following

praveen avatar
praveen

“access_group”: “${local.accessgroup}”

praveen avatar
praveen

with local as accessgroup = “${replace(var.accessgroup, “_”, “\\”)}”

praveen avatar
praveen

and passing the value as accessgroup = “domain_groupname”

praveen avatar
praveen

but it fails with the following error

praveen avatar
praveen

"DSC Configuration ‘WindowsStaplesDefault’ completed with error(s). Following are the first few: PowerShell DSC resource MSFT_GroupResource failed to execute Test-TargetResource functionality with error message: Could not find a principal with the provided name [quilldevecomweb\sg-quillecmweb-quilldotcom-support] The SendConfigurationApply function did not succeed. LCM failed to start desired state configuration manually.".”

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen is it for Azure?

praveen avatar
praveen

yes @Andriy Knysh (Cloud Posse)

praveen avatar
praveen

it is for Azure platform

integratorz avatar
integratorz

Hello, I am currently working on spinning up ec2 instances in AWS. I am wondering if there is a good way to attach an encrypted EBS volume to windows server’s C drive?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(cloudposse has no windows experience =)

integratorz avatar
integratorz

Hey @Erik Osterman (Cloud Posse) I ended up figuring it out if you ever need it!

:100:1
integratorz avatar
integratorz

Not that you guys deal with windows probably ever lol

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

glad you got it running

Tobias Hoellrich avatar
Tobias Hoellrich

Good evening. My tf uses the terraform-aws-modules/security-group/aws module to setup a default security group. Right after I cloudposse/elasticsearch/aws:0.2.0 to define an Elasticsearch cluster and make use of the security group. Roughly it looks like this:

module "securitygroup" {
  source = "terraform-aws-modules/security-group/aws//modules/elasticsearch"

  name                = "${var.aws_env}_elasticsearch_default"
  vpc_id              = "${local.opsworks_vpc_id}"
  ingress_cidr_blocks = ["${local.opsworks_vpc_cidr_block}"]
}

module "elasticsearch" {
  source  = "cloudposse/elasticsearch/aws"
  version = "0.2.0"

  security_groups = ["${module.securitygroup.this_security_group_id}"]
Tobias Hoellrich avatar
Tobias Hoellrich

During plan/apply I get the error: Error: Error running plan: 1 error(s) occurred: * module.elasticsearch.aws_security_group_rule.ingress_security_groups: aws_security_group_rule.ingress_security_groups: value of 'count' cannot be computed

Tobias Hoellrich avatar
Tobias Hoellrich

Somehow the ES module does not detect the dependency and does not know it has to wait for the outcome of the SG creation. I used the same sequence in other places without issues. Do you know a way around?

Tobias Hoellrich avatar
Tobias Hoellrich

I can use the terraform ... -target module.securitygroup option to create the SG in a first run and then call apply a 2nd time, which creates Elasticsearch without issues. And alternatively I can put the SG creation in a separate module, but I like the idea of keeping these together. Thanks for any ideas.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes, 2-phased apply is basically your only option

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Tobias Hoellrich TF has troubles calculating counts between modules. Try to just create a SG using https://www.terraform.io/docs/providers/aws/r/security_group.html and provide it to module "elasticsearch" - it could work

AWS: aws_security_group - Terraform by HashiCorp

Provides a security group resource.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

another approach, if you use a consistent naming (e.g. by using https://github.com/cloudposse/terraform-null-label), you know the names and IDs of all resources in advance. So instead of providing the ID of the SG to the elasticsearch module, you can provide the same ID from the label module (e.g. https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/main.tf#L97)

cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

2019-01-29

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@squidfunk have you met @maarten? He’s the author behind #airship , another well “terraform module as a product”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@maarten is also in Berlin

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@squidfunk is maintaining this module: https://github.com/squidfunk/terraform-aws-cognito-auth

squidfunk/terraform-aws-cognito-auth

Serverless Authentication as a Service (AaaS) provider built on top of AWS Cognito - squidfunk/terraform-aws-cognito-auth

5
antonbabenko avatar
antonbabenko

@squidfunk Your module looks like very solid piece of code! Maybe I will need it in the future.

:100:1
squidfunk avatar
squidfunk

Thanks, glad to hear!

squidfunk avatar
squidfunk

Haven’t met @maarten yet though I’ve just started abstracting services into Terraform modules. @antonbabenko feel free to drop me a note if you run into any problems or have feedback!

fidget_spinner1
Maciek Strömich avatar
Maciek Strömich

@squidfunk props for the additional headers in web app tightening security

squidfunk avatar
squidfunk

@Maciek Strömich thanks - absolutely necessary in my opinion, otherwise the door to XSS/CSRF is wide open.

maarten avatar
maarten

Hi @squidfunk I’m in the tropics and not touching code by rule of law here. Happy to take a look at your stuff when I’m back, I currently have a simple Cognito implementation with the ALB here: https://airship.tf/guide/ecs_service/load_balancing.html#application-lb-cognito-authentication

Load balancing | Airship Modules

Flexible Terraform templates help setting up your Docker Orchestration platform, resources 100% supported by Amazon

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

where are you vacationing?

Load balancing | Airship Modules

Flexible Terraform templates help setting up your Docker Orchestration platform, resources 100% supported by Amazon

maarten avatar
maarten

Ko Pha-ngan

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Niiiiiice

1
squidfunk avatar
squidfunk

@maarten I’ll also take a look at your stuff Maybe I can draw some inspiration. Enjoy your vacation!

joshmyers avatar
joshmyers

hey @squidfunk I note that you aren’t using a aws_cognito_identity_provider in your module. You pass a name around for one. Is it assumed you have already set on up?

squidfunk avatar
squidfunk

I assume you mean the variable cognito_identity_pool_provider? It’s used as the developer_provider_name of the identity pool, see: https://github.com/squidfunk/terraform-aws-cognito-auth/blob/b5cf938b41b1bcb338f55639c2d44b8c76f299cc/modules/identity/main.tf#L140-L157

squidfunk/terraform-aws-cognito-auth

Serverless Authentication as a Service (AaaS) provider built on top of AWS Cognito - squidfunk/terraform-aws-cognito-auth

squidfunk avatar
squidfunk

Maybe it’s not the best name for this variable. If you find any room for improvement, I’m very happy to discuss it on the issue tracker!

joshmyers avatar
joshmyers

Ah, grand. did you look at implementing MFA at all? Seems several open issues with cognito resources to do with this

squidfunk avatar
squidfunk

Nope, not yet. Currently the sole use case is SPA. However, I assume we could totally integrate that.

squidfunk avatar
squidfunk

Thinking about it - we probably only would have to extend the API to handle auth challenges and integrate that into the frontend.

btai avatar

anyone using EKS with the kubernetes provider by providing the necessary config from your EKS resource?

btai avatar

i believe the EKS resource is missing client_cert and client_key as outputs?

provider "kubernetes" {
  host = "<https://104.196.242.174>"

  client_certificate     = "${file("~/.kube/client-cert.pem")}"
  client_key             = "${file("~/.kube/client-key.pem")}"
  cluster_ca_certificate = "${file("~/.kube/cluster-ca-cert.pem")}"
}
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we did not use kubernetes provider for the EKS module, used these resources https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/kubectl.tf

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

all those attributes for kubernetes provider are optional (client_certificate, client_key, etc.)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-kops-iam-authenticator-config

Terraform module to create and apply a Kubernetes ConfigMap for aws-iam-authenticator to be used with Kops to map IAM principals to Kubernetes users - cloudposse/terraform-aws-kops-iam-authentica…

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

btai avatar

@Andriy Knysh (Cloud Posse) i wanted to avoid using config_path because it would require exporting the kubeconfig to my local path for the new cluster

btai avatar

I guess i want it to work independently of whats running in my local (someone who doesnt have kubectl can still run terraform apply)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you use aws-iam-authenticator, the kubeconfig still needs to include this command https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/kubeconfig.tpl#L24

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

also, the resource does not export anything else for auth except certificate_authority https://www.terraform.io/docs/providers/aws/r/eks_cluster.html#attributes-reference

btai avatar

i see, so ill need to use a null_resource to export the cluster config

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

once you export kubeconfig, then you can use either kubernetes provider or https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/kubectl.tf#L30 to apply the config map

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

btai avatar

annoying thing is when I run aws eks update-kubeconfig --name {cluster_name} --profile {aws_profile}, that kube config user needs

env:
- name: AWS_PROFILE
  value: {aws_profile}

appended to it

btai avatar

ah one of the newest aws cli versions have the fix

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

anyone use the ecs cli to deploy via docker-compose files? I’m just curious if it is worth exploring

2019-01-28

btai avatar

hey my eks pods (aws-node-*) are constantly in a crashloopbackoff state and when I try to get the logs, I get

Error from server: Get https://<private_ip>:10250/containerLogs/kube-system/aws-node-tp9jn/aws-node?follow=true: dial tcp <private_ip>:10250: i/o timeout
btai avatar

any ideas?

btai avatar

i suppose i can add a bastion node

sarkis avatar
sarkis

@btai without more context to me that looks like a port not accepting traffic - not too comfortable with EKS yet but i’d assume some security group issue

btai avatar

i’ve gotten into the EKS worker node w/ a bastion server

btai avatar

this is the error log im getting for aws-node pods

{"log":"ERROR: logging before flag.Parse: W0128 21:05:18.233802      13 client_config.go:533] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.\n","stream":"stderr","time":"2019-01-28T21:05:18.23463547Z"}
{"log":"Failed to communicate with K8S Server. Please check instance security groups or http proxy setting","stream":"stdout","time":"2019-01-28T21:05:48.268072298Z"}
btai avatar

@Andriy Knysh (Cloud Posse) i noticed in your eks example you’re spinning up your eks workers in public subnets, I’ve spun them up in private subnets im wondering if thats causing the issue

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

any suggestions here on how we should handle this? https://github.com/cloudposse/terraform-aws-dynamic-subnets/pull/31

Make module more idempotent when working with peering connections, fixes #30 by agrrh · Pull Request #31 · cloudposse/terraform-aws-dynamic-subnets

This PR prevents module to wipe peering connection out when applying it to existing infrastructure. It just ignores changes for whole route part. Sadly, it looks like we can&#39;t state ignore_chan…

loren avatar
loren

Use aws_route instead of the inline route schema, make the route optional, be sure to output the route table ids, let the user add routes in a separate module?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

good suggestion. we’ll try that.

loren avatar
loren

The inline route is in public.tf, but not present in private

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, we try to never do this, but this one slipped by.

loren avatar
loren

The issue with deleting the route might be related to this, https://github.com/terraform-providers/terraform-provider-aws/issues/5631

Actual routes deleted by importing aws_route_table followed by apply · Issue #5631 · terraform-providers/terraform-provider-aws

Community Note Please vote on this issue by adding a :–1: reaction to the original issue to help the community and maintainers prioritize this request Please do not leave &quot;+1&quot; or &quot;me to…

loren avatar
loren

We’ve submitted the linked pr to fix that one, but it’s blocked pending the 2.0 release, due to a change in behavior (for backwards compatibility in 1.x)

2019-01-27

kritonas.prod avatar
kritonas.prod

I’m looking into your iam-user module and I was wondering if there’s a way to loop this and create a batch of users from a list. As far as I can understand I cant use count with a TF module, and for/for each loops are not available until TF 0.12.. Any advice is welcome!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

You can technically use count, but we don’t recommend it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If the count changes, terraform will destroy/create all resources which is ugly.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We didn’t add support for a list of users for this reason.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Our strategy is to invoke the module once for each user. We create a file per user.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This strategy works really well with #atlantis. Since each user add is a PR.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Then to remove a user from the environment, we just revert the PR.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

kritonas.prod avatar
kritonas.prod

Got it, this is probably what I’m doing as well, as a list would indeed be risky in case of an error. Great, and you’re sending out an email with the output.

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea, that sends a transactional email. Though I think there’s a bug with that whereby it runs during plan as well because it uses the external data provider.

kritonas.prod avatar
kritonas.prod

So count can be used with modules as well? I stumbled upon a years old TF ticket requesting this very thing, and it’s still open

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

No, count cannot be used with modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but you can pass a list of users to a module and have the iam resource generate N of them

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we didn’t implement that in the module for the aforementioned reasons

kritonas.prod avatar
kritonas.prod

Got it, I misinterpreted what you initially said. Thanks for the help!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

No prob! We’re here to unblock you.

2019-01-26

integratorz avatar
integratorz

Hello Everyone! I am currently using terraform to launch infrastructure into an already existing vpc.

integratorz avatar
integratorz

I am having trouble figuring out how to use an existing vpc id to deploy a resource. Any suggestions?

Steven avatar
Steven

You can do a data lookup for the VPC id

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@integratorz if you deploy into an existing VPC, you should know its ID, or at least some tags to do data lookup as @Steven mentioned

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

integratorz avatar
integratorz

If I know the vpc id how would I go about specifying it in the config?

integratorz avatar
integratorz

for instance I can’t just put vpc_id = "vpc-xxxxxx"

kritonas.prod avatar
kritonas.prod

@integratorz What’s the context? What are you trying to do exactly?

integratorz avatar
integratorz

@kritonas.prod I am working on automating deployments of new servers into an existing VPC. I was actually able to get that working but am now having trouble getting the host to join to the AWS Hosted AD Domain.

kritonas.prod avatar
kritonas.prod

@integratorz Good stuff. Are you getting any errors?

integratorz avatar
integratorz

Well when I

terraform apply
integratorz avatar
integratorz

Everything gets provisioned then I get a timeout error at the end after about 5 minutes

integratorz avatar
integratorz

Wow wouldn’t you know it

integratorz avatar
integratorz

I had a file prevision that was causing terraform to hang

integratorz avatar
integratorz

so wasn’t actually “not” joining the domain

integratorz avatar
integratorz

was just never getting to that point

integratorz avatar
integratorz
07:55:26 AM
integratorz avatar
integratorz

me right now^

4

2019-01-25

Igor Rodionov avatar
Igor Rodionov

Check this up. @antonbabenko created a list with terraform best practice. https://www.terraform-best-practices.com/

7
:--1:2
pecigonzalo avatar
pecigonzalo

sidetracking but nice find gitbooks

pecigonzalo avatar
pecigonzalo

Any tips for a tool to generate docs out of markdown? We want to have a website for our terraform modules repo markdown and other tools

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

We use a pre-commit hook generated by: https://github.com/segmentio/terraform-docs

segmentio/terraform-docs

Generate documentation from Terraform modules in various output formats - segmentio/terraform-docs

:--1:1
pecigonzalo avatar
pecigonzalo

Yeah we use that for the markdowns in the repo, but wanted to make a website out of it

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

Gotcha

pecigonzalo avatar
pecigonzalo

looking for something simpel

Steven avatar
Steven

@pecigonzalo There are programs to convert markdown to other formats. But are you looking for something that would collect the doc for all your modules?

pecigonzalo avatar
pecigonzalo

Yeah, more like live collect from a branch

pecigonzalo avatar
pecigonzalo

I guess, similar to what gitbooks does

pecigonzalo avatar
pecigonzalo

or github pages

loren avatar
loren

i’ve used both mkdocs and sphinx, both are pretty simple

loren avatar
loren

well, calling sphinx “simple” is probably not accurate i guess. but once past the initial learning curve, to where you have CI generating the html for the static site, actually maintaining the docs is quite simple

loren avatar
loren

there are all sorts of static site generators, many have plugins for IDEs, so you can see right in your editor how it will render

loren avatar
loren

mkdocs, sphinx, hugo, jekyll, etc etc

loren avatar
loren

pelican

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use hugo for docs.cloudposse.com

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

Nice. I’m going to push soon for a dev portal internally as well so will give Hugo a look

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
stakater/Forecastle

Forecastle is a control panel which dynamically discovers and provides a launchpad to access applications deployed on Kubernetes – [✩Star] if you’re using it! - stakater/Forecastle

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i want to try this out

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for the dev portal

pecigonzalo avatar
pecigonzalo

Ill give them a look, thanks for the suggestions

loren avatar
loren

you can even take a gitlab/github project, disable the source code feature, and it’ll default the home page to the wiki

pecigonzalo avatar
pecigonzalo

I did not know that, nice

joshmyers avatar
joshmyers

mkdocs is great for general kinda runbook/index sites

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@maarten vuepress for static sites… checkout airship.tf

btai avatar

@Andriy Knysh (Cloud Posse) have you seen this before?

* module.eks_workers.data.aws_ami.eks_worker: data.aws_ami.eks_worker: UnauthorizedOperation: You are not authorized to perform this operation.
btai avatar

Trying to get the latest EKS ami from the Amazon account

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

no, i did not

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

are you using our example?

btai avatar
data "aws_ami" "eks_worker" {
  filter {
    name   = "name"
    values = ["${var.eks_worker_ami_name_filter}"]
  }

  most_recent = true
  owners      = ["602401143452"] # Amazon
}
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

under what user are you provisioning the cluster?

btai avatar

an admin user

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hmm, maybe something changed on AWS side

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you don’t resolve it, I’ll take a look later today

btai avatar

so I’m assuming a role (to switch AWS accounts) and im wondering if thats the reason why?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

What permissions does the role have?

btai avatar

so the master account has no permissions

btai avatar

but im assuming a role into another account (where the provisioning actually takes place)

btai avatar

and that role has admin

btai avatar

I set the profile for the backend

btai avatar

hmm

btai avatar

i wonder if i need to set hte profile for the provider too

btai avatar

@Andriy Knysh (Cloud Posse) that was it

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

how hard would it be to override the label format with cp modules?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

namespace-stage-product means:

blah-dev-product1 blah-dev-product2 blah-test-product1 blah-test-product2

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

you do not see the products together. we generally use:

product-stage

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

They would never be in the same account, right?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

prod is solo

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

dev, qa, uat are together

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

prod+support are together

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So, there’s the environment field, but it’s not been integrated into all of our modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you want to add it as needed, I think we’d be okay with that. @Andriy Knysh (Cloud Posse)?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

how’s it format things?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The alternative is to take this into the stage name.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or the namespace

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

namespace cp-blog

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

true. that’s a fair work-around

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

Note to self: do not delete your state when testing TF before destroying resources.

btai avatar

@Erik Osterman (Cloud Posse) we need a route table per nat gateway correct?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So looking at the code yesterday, I couldn’t see where that requirement was implied or required. The underlying requirement is one subnet per AZ and therefore one NGW per AZ.

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

anyone hit this when starting up a Fargate container? (using cp modules)
CannotStartContainerError: API error (500): failed to initialize logging driver: failed to create Cloudwatch log stream: ResourceNotFoundException: The specified log group does not exist. status code:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and you create the log group?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

nah. thought the module would, but guess i should add that into the TF?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it depends which modules you use

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-ecs-web-app

Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more. - cloudposse/terraform-aws-ecs-web-app

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

makes sense

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

our “web app” module is more opinionated and creates it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

many people choose not to use the web app module and go around it

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

yeah…going around it and seemed to have left that part out.

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

but it tries to connect to one which i didn’t provide so that was odd

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

thought it’d just ignore it

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ahh….i see.

variable "log_options" {
  type        = "map"
  description = "The configuration options to send to the `log_driver`"

  default = {
    "awslogs-region" = "us-west-2"

    "awslogs-group" = "default"

    "awslogs-stream-prefix" = "default"
  }
}
btai avatar

@Andriy Knysh (Cloud Posse) how are you installing aws-iam-authenticator on your EKS masters?

btai avatar

using the terraform kubernetes provider ?

jeffrey avatar
jeffrey

hey guys, i’m running into an issue where my backend seems like it’s being executed after another role has been assumed. i have a setup where an admin account can assume roles into multiple environment accounts. while setting up dynamodb locking for the remote state file, i came across this:

terraform {
  required_version = ">= 0.11.3"
  backend "s3" {
    bucket                  = "<redacted>"
    dynamodb_table          = "terraform_locks"
    encrypt                 = true
    key                     = "infra/dev/vpc/terraform.tfstate"
    region                  = "us-east-1"
    shared_credentials_file = "~/.aws/admin-credentials"
  }
}

provider "aws" {
  region                  = "us-east-1"
  shared_credentials_file = "~/.aws/admin-credentials"
  assume_role {
    role_arn     = "<redacted>"
    session_name = "Terraform"
  }
}

i have a dynamodb table existing in the admin account, but NOT the dev account (this is the environment in my example). however i get an error about required resources not being found, but once i create the same dynamodb table in the dev account, everything works fine and i see the lock being created. anyone have an idea what’s going on here?

jeffrey avatar
jeffrey

the terraform docs state: Due to the assume_role setting in the AWS provider configuration, any management operations for AWS resources will be performed via the configured role in the appropriate environment AWS account. The backend operations, such as reading and writing the state from S3, will be performed directly as the administrator's own user within the administrative account.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so fwiw, there’s a role_arn supported by the backend configuration

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s what we use

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we explicitly specify the role_arn for both the provider and for the backend

jeffrey avatar
jeffrey

So if you plan on executing the backend in the admin account where you’re running terraform from, you should specify that account as the role?

jeffrey avatar
jeffrey

Appreciate the quick response!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in our case, we have one bucket per account (“share nothing”)

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-tfstate-backend

Terraform module that provision an S3 bucket to store the terraform.tfstate file and a DynamoDB table to lock the state file to prevent concurrent modifications and state corruption. - cloudposse…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we sometimes do cross-account to lookup outputs from other modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@btai re: how are you installing aws-iam-authenticator on your EKS masters?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you don’t

btai avatar

Oh?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

EKS comes with it already installed

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you need to do two things:

btai avatar

I installed it on my local machine

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Install it locally to be able to use kubectl with it
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Update kubecfg to use it
btai avatar

Ohh I think I need to do #2

btai avatar

Thanks

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

cloudposse/packages

Cloud Posse installer and distribution of native apps, binaries and alpine packages - cloudposse/packages

:1000:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

btai avatar

Thanks!

2019-01-24

Samuli avatar
Samuli

Could someone enlighten me on what’s the difference between terraform-terraform-label and terraform-null-label?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Samuli for simple use-cases, there are no real difference, they will do the same things… except for:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. terraform-null-label uses null_data_source to do additional things like additional tags and context (if you need those)
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. terraform-terraform-label is much simpler implementation (terraform-terraform-label was forked from terraform-null-label before a lot of additional functionality was added to -null-label)
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so if you use basic label to uniquely name resources, both will do the same

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you want context, additional params like environment, additional tags (needed for tagging some AWS resources like Autoscaling Groups, which have diff tag format), then use null-label

:--1:1
johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

@Erik Osterman (Cloud Posse) can we get your blessing on @wbrown43’s PR? https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/pull/73

We need it released for some other work

wbrown43 avatar
wbrown43
05:55:23 PM

@wbrown43 has joined the channel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

one sec

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

released 0.9.0

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sorry for the delay! always ping me here if blocked

:--1:1
johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
05:59:35 PM
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Allow setting elb_scheme for choosing internal or public LB by CGreenPayTM · Pull Request #72 · cloudposse/terraform-aws-elastic-beanstalk-environment

Have tested with load balanced environment, but not SingleInstance. Terraform wants to update the settings on the environment every time, even with no changes (and even using the module from master…

1
johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

I’ve increased the vpc limit and the interface endpoints. Any ideas what else I need to increase to get things rolling?

Error: Error applying plan:
2 error(s) occurred:
* module.stack.module.vpc.module.vpc.aws_vpc_endpoint.dynamodb: 1 error(s) occurred:
* aws_vpc_endpoint.dynamodb: Error creating VPC Endpoint: VpcEndpointLimitExceeded: The maximum number of VPC endpoints has been reached.

* module.stack.module.vpc.module.vpc.aws_vpc_endpoint.s3: 1 error(s) occurred:
* aws_vpc_endpoint.s3: Error creating VPC Endpoint: VpcEndpointLimitExceeded: The maximum number of VPC endpoints has been reached.
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

is it working after the increase?

loren avatar
loren

maybe it’s cuz dynamodb and s3 vpc endpoints are not interface endpoints, they are gateway endpoints?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

no, not working after

loren avatar
loren
Amazon VPC Limits - Amazon Virtual Private Cloud

Request increases to the following default Amazon VPC component limits as needed.

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
08:54:11 PM

am I just missing it?

loren avatar
loren

heh. you and me both! might have to just open a more generic support request

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

going w/ a general request to see what I can find out. I think their service limits are missing an option for sure

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

The follow-up response from Support. It is as suspected, but I have more questions since their suggestion is not a valid option in the dropdown.
Hi
>
Thank you for contacting AWS premium Support. My name is Tendai and I will be happy to assist you on this case.
>
I understand you are having the VpcEndpointLimitExceeded error. Upon checking your recent changes I can see that your requested an increase was for VPC interface endpoint but the actual increase which caters for your case is gateway endpoints .Ref [1].
>
Please note that Dynamo DB and s3 uses Gateway VPC Endpoints instead of VPC interface endpoints .

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

nice to know that @johncblandii (Cloud Posse)

:--1:1
Neha avatar

Hello all. question : does terraform allow enabling aws cloudtrail in another sub account while being in the main billing account?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Hey @Neha

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes it’s possible, here is an example

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

foqal avatar
foqal
09:02:59 PM

@Andriy Knysh (Cloud Posse)’s question was answered by <@Foqal>

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks @foqal

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then in other accounts (prod, staging, etc.) provision cloudtrails and point them to the bucket in audit https://github.com/cloudposse/terraform-root-modules/blob/master/aws/cloudtrail/main.tf#L33

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we separate audit from the main billing account (to allow only a few people to access it), but they could be the same

Neha avatar

gotcha. audit is the other subaccount where we want to enable cloudtrail

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

where we store all the logs in the bucket

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for security and compliance reasons

Neha avatar

that reminds me- can we do the same with GuardDuty? enabling it in another account while being in the billing account?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we did not work with GuardDuty. What resources does it require?

Neha avatar

I guess what I’m asking is, is that process only for cloudtrail? or it could be applied to other aws resources?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so, we usually separate ALL resources into different AWS account per environment -> prod, staging, dev, testing are in separate acccounts

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Neha avatar

GuardDuty just requires enabling it. but we have cloudwatch set up that is targetted to an sns rule and we’re forwarded any/all alerts that come in to guardDuty

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

sounds like you want just one GuardDuty and all alerts coming to it from different accounts

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

not sure how it (GuardDuty) works, but probably will require setting some cross-account permissions

loren avatar
loren

No permissions, it’s more like vpc peering really… You send an invite from the guardduty “master” to the “member” account, and from the member account you accept the invite.

loren avatar
loren

Currently in terraform, you can only send the invite… Acceptance is still an open PR

loren avatar
loren
[WIP] New Resource: aws_guardduty_invite_accepter by bflad · Pull Request #4610 · terraform-providers/terraform-provider-aws

Work In Progress The provider acceptance test framework needs additional updates to support a second provider configuration with additional credentials. The initial implementation in this pull requ…

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

looks like the EFS module is borked.

* module.efs.output.mount_target_ids: At column 3, line 1: conditional operator cannot be used with list values in:

${local.enabled ? aws_efs_mount_target.default.*.id : list("") }
* module.efs.output.mount_target_dns_names: At column 3, line 1: conditional operator cannot be used with list values in:

${local.enabled ? aws_efs_mount_target.default.*.dns_name : list("") }
* module.efs.output.network_interface_ids: At column 3, line 1: conditional operator cannot be used with list values in:

${local.enabled ? aws_efs_mount_target.default.*.network_interface_id : list("") }
* module.efs.output.mount_target_ips: At column 3, line 1: conditional operator cannot be used with list values in:

${local.enabled ? aws_efs_mount_target.default.*.ip_address : list("") }
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

crap use the previous release

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@joshmyers was just adding some things

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

yup

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ll get that fixed tonight

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

0.7.1 works

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Add enabled variable and update docs by joshmyers · Pull Request #21 · cloudposse/terraform-aws-efs

what Add enabled variable and update docs why So we can boolean creation of these resources from the caller module

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(easy fix)

btai avatar
cloudposse/terraform-aws-multi-az-subnets

Terraform module for multi-AZ public and private subnets provisioning - cloudposse/terraform-aws-multi-az-subnets

btai avatar

is there a benefit to having a route table per az subnet?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I cannot recall if there was a technical reason or just trying to keep it symmetrical

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

looking at it right now, it seems like we could get by with a single route table for public and a single route table for private

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the key thing is we need to have one NGW per AZ

btai avatar

why is that @Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for HA

btai avatar

how does that work?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

let’s say we deploy one NGW

btai avatar

theres a possibility a nat gateway can become unavailable?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we have instances in us-west-2a, us-west-2b and us-west-2c

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and we have one ngw in us-west-2a

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

all azs can use that ngw

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but if 2a goes off line, then 2b and 2c are all affected

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so from a networking perspective, we want to maintain HA.

btai avatar

i see

btai avatar

that makes sense

btai avatar

since the nat gateway lives inside the subnet?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, sorta. So a subnet cannot span AZs. and then the NGW is attached to a subnet.

btai avatar

do you need a nat gateway for public subnets?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

only private ones need to do NAT

btai avatar

why does your tf have it for public subnets?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think we have an IGW for public ones

btai avatar
cloudposse/terraform-aws-multi-az-subnets

Terraform module for multi-AZ public and private subnets provisioning - cloudposse/terraform-aws-multi-az-subnets

btai avatar

fyi

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

aha, so you provision the NGW on a public subnet

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then private subnets route through it

btai avatar

ah

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if the NGW was on a private network, it would make no difference

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I was going to say, you could use a NGW on a private subnet if you wanted traffic from that subnet to egress to a particular network on a specific IP

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but I am not sure if that’s techncially possible since NGWs require an EIP.

btai avatar

i see

btai avatar

yeah

btai avatar

thanks, that helps alot

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

There are a lot of subnet strategies. Especially as the organization gets larger, there will be stricter subnet allocations

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Our modules are great in a “cold start” scenario where there aren’t a lot of other conflicting subnets.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I know @Jan is working on a different subnet module strategy.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also, we have a few subnet modules for different kinds of allocations.

btai avatar

im actually going single k8s cluster in a vpc

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

What we’re missing is a preallocated/manual subnet strategy, where users specify their own subnet CIDRs.

btai avatar

so the vpc i just have split out to half public/half private subnets

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

btai avatar

pretty simple, and dependent services will just vpc peer

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Have you seen the example by @Andriy Knysh (Cloud Posse) ?

btai avatar

pretty nice actually

btai avatar

have you guys not had any problems

btai avatar

with module dependencies?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think this works well from a cold-start

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no “count of” issues

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but I could be mistaken.

btai avatar

meaning

btai avatar
module "eks_cluster" {
  source                  = "git::<https://github.com/cloudposse/terraform-aws-eks-cluster.git?ref=tags/0.1.1>"
  namespace               = "${var.namespace}"
  stage                   = "${var.stage}"
  name                    = "${var.name}"
  attributes              = "${var.attributes}"
  tags                    = "${var.tags}"
  vpc_id                  = "${module.vpc.vpc_id}"
  subnet_ids              = ["${module.subnets.public_subnet_ids}"]
  allowed_security_groups = ["${distinct(compact(concat(var.allowed_security_groups_cluster, list(module.eks_workers.security_group_id))))}"]
  allowed_cidr_blocks     = ["${var.allowed_cidr_blocks_cluster}"]
  enabled                 = "${var.enabled}"
}
btai avatar

if we take a look at this module

btai avatar

it uses module.vpc.vpc_id

btai avatar

so on initial terraform apply

btai avatar

have you guys not ran into the problem where the eks_cluster module will spin up concurrently with the vpc module

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I haven’t used this one personally

btai avatar

ah

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We mostly using kops

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

some gke

btai avatar

because i know you can’t use depends_on within a module

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea… we know all too well

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha

btai avatar

ill give this a shot though

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) knows it very well. ping him if you get stuck.

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

man

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

that depends_on is on my last nerve

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

so wish .12 would drop!

btai avatar

soon i think

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

so they keep saying

btai avatar

i know

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

I’ve resigned to “eventual”

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

right

btai avatar

yeah so like i was saying, this is as clean as it gets when writing terraform: https://github.com/cloudposse/terraform-root-modules/blob/master/aws/eks/eks.tf

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

:--1:1
btai avatar

but my first thought is if it will provision everything in the correct order without depends_on working for modules

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

literally using that one right now

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

so long as it isn’t computing, references to other modules seem to work decently

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sooooo terraform seems to infer order pretty well

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
  cluster_name                       = "${module.eks_cluster.eks_cluster_id}"
johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

^ that

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

toss something in a count and

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

because it does count before generating resources

btai avatar

yeah

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@johncblandii (Cloud Posse) did you end up using our our EKS modules?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

sure did

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

editing the cluster right now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

tied efs into it too

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cool (we’ll get that EFS bug fixed soon)

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

vpc -> subnets -> eks -> workers

^ cp stack ftw

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

< np

btai avatar

nested modules havent been an issue?

btai avatar

for you guys?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what I love about our EKS module is how easily you can create multiple types of node pools with full customization

btai avatar

you just output output

btai avatar

when you says multiple node pools

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

yeah

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

gets verbose until .12 where you can output the module itself

btai avatar

you mean private subnet instances , public subnet instances?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no, so you could create a GPU node pool, a high compute node pool, high memory node pool, spot instance node pool, etc

btai avatar

ah

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

all part of the same cluster

btai avatar

can i do public/private too?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, i don’t see why not (but haven’t tested that)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

….and generally wouldn’t recommend public anything except for ALBs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

AWS supports NLBs now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and k8s as well

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@btai the EKS example works good

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

No explicit depends on

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Terraform does it automatically

btai avatar

@Andriy Knysh (Cloud Posse) i figured

btai avatar

how would you do multiple node pools?

btai avatar

just multiple module eks workers?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

And add its security groups to the cluster security group

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
cloudposse/terraform-aws-eks-workers

Terraform module to provision an AWS AutoScaling Group, IAM Role, and Security Group for EKS Workers - cloudposse/terraform-aws-eks-workers

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

just imagine that configured differently for different types

btai avatar

how do you deploy your pods to the correct node pools?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

btai avatar

say a pod needs to go on a GPU instance?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

(disappear)

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

I’m a k8s noob; struggling this minute to be exact, lol

btai avatar

i am as well, just a few months

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

weeks for me

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

definitely dazed; (maybe a month)

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

trying to rebuild jenkins in here now (for efs vs ebs)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Probably by tagging the nodes, then k8s has labels and annotations to select the required nodes for pods

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ahhhh…yes. I forgot about tagging

btai avatar

thats right

btai avatar

@Andriy Knysh (Cloud Posse) is there a list of resources that need to have the [kubernetes.io/cluster/cluster_name](http://kubernetes\.io/cluster/cluster_name) tag?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
02:00:03 AM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is freaking amazing. lets give to @Andriy Knysh (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

246 code reviews i the past 90 days

2
1
johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
02:00:39 AM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

with an average 1 hour turnaround!!

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ridiculous!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and that’s well over 500 CRs in the past 90 days

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re a small company. this astounds me.

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

indeed

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks guys

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@btai as far as I know, these two tags are required for EKS:

btai avatar

thanks!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-workers

Terraform module to provision an AWS AutoScaling Group, IAM Role, and Security Group for EKS Workers - cloudposse/terraform-aws-eks-workers

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

as was described in the docs

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-efs

Terraform Module to define an EFS Filesystem (aka NFS) - cloudposse/terraform-aws-efs

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
03:11:16 AM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-vpn-connection

Terraform module to provision a site-to-site VPN connection between a VPC and an on-premises network - cloudposse/terraform-aws-vpn-connection

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Create site-to-site VPN connections

btai avatar
cloudposse/terraform-aws-eks-workers

Terraform module to provision an AWS AutoScaling Group, IAM Role, and Security Group for EKS Workers - cloudposse/terraform-aws-eks-workers

btai avatar

what is the purpose of the join in role = "${join("", aws_iam_role.default.*.name)}"

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s when dealing with counts for enablement/disablement of a resource

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if a count=0, aws_iam_role.default.0.name would error

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the splat operator * is required

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we call this the “join-splat” pattern

btai avatar

yeah i realized you guys are putting count enables everywhere

btai avatar

makes sense

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, makes it easier to enable/disable functionality

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

when composing modules in other modules

2019-01-23

Samuli avatar
Samuli

@Erik Osterman (Cloud Posse) could you merge https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/pull/17 if its ok and you have the time.

Make volumes configurable by ssaarinen · Pull Request #17 · cloudposse/terraform-aws-ecs-alb-service-task

Allows to add volume definitions to task like this module &quot;alb_service_task&quot; { source = &quot;../terraform-aws-ecs-alb-service-task&quot; … volumes = [ { …

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) can take a look

Make volumes configurable by ssaarinen · Pull Request #17 · cloudposse/terraform-aws-ecs-alb-service-task

Allows to add volume definitions to task like this module &quot;alb_service_task&quot; { source = &quot;../terraform-aws-ecs-alb-service-task&quot; … volumes = [ { …

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Samuli thanks! Please see the comments

Samuli avatar
Samuli

done..

joshmyers avatar
joshmyers

@Samuli That could be squashed into a single commit IMO

nutellinoit avatar
nutellinoit

?

joshmyers avatar
joshmyers

Sorry wrong person!

nutellinoit avatar
nutellinoit

praveen avatar
praveen

#terraform , am trying to add a user group using terraform config and it does not accept backslash

praveen avatar
praveen

variable “accessgroup” { default = “domain\user1” }

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

did you try \\

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the \u says to escape the u

praveen avatar
praveen

yes, I tried 4 back slashes, 2,3 and also 1 forward slashes

praveen avatar
praveen

with no luck

praveen avatar
praveen

just trued it once again with no luck

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what’s the error message?

praveen avatar
praveen
  • azurerm_virtual_machine_scale_set.vmss: Code=”VMExtensionProvisioningError” Message=”VM has reported a failure when processing extension ‘dsc-wqwdwebfe’. Error message: "The DSC Extension received an incorrect input: Compilation errors occurred while processing configuration ‘WindowsStaplesDefault’. Please review the errors reported in error stream and modify your configuration code appropriately. A Using variable cannot be retrieved. A Using variable can be used only with Invoke-Command, Start-Job, or InlineScript in the script workflow. When it is used with Invoke-Command, the Using variable is valid only if the script block is invoked on a remote computer. Exception calling "InvokeWithContext" with "2" argument(s): "A Using variable cannot be retrieved. A Using variable can be used only with Invoke-Command, Start-Job, or InlineScript in the script workflow. When it is used with Invoke-Command, the Using variable is valid only if the script block is invoked on a remote computer." A Using variable cannot be retrieved. A Using variable can be used only with Invoke-Command, Start-Job, or InlineScript in the script workflow. When it is used with Invoke-Command, the Using variable is valid only if the script block is invoked on a remote computer.\n\nAnother common error is to specify parameters of type PSCredential without an explicit type. Please be sure to use a typed parameter in DSC Configuration, for example:\n\n configuration Example {\n param([PSCredential] $UserAccount)\n …\n }.\nPlease correct the input and retry executing the extension.".”
joshmyers avatar
joshmyers

Please use code blocks when posting things like this into Slack, helpful for organisation

praveen avatar
praveen

sure, can you give me link to code blocks

joshmyers avatar
joshmyers
02:13:07 PM
praveen avatar
praveen

do we have an example for allowing backslash in terraform

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
Interpolation Syntax - Terraform by HashiCorp

Embedded within strings in Terraform, whether you’re using the Terraform syntax or JSON syntax, you can interpolate other values into strings. These interpolations are wrapped in ${}, such as ${var.foo}.

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

See the yellow box

2019-01-22

bentrankille avatar
bentrankille

Hi everyone, I took a look on your terraform modules regarding bucket S3 for log storage, cloudtrail and cloudtrail-s3-bucket (to attach a policy to a bucket) I wonder how I can use those modules in my case. I would like to create a global S3 to store all events coming from other AWS accounts. The problem I need to solve is about the Policy. To allow multiple AWS accounts to write in a bucket located on one account, we need to add resources in the aws_iam_policy_document (https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-set-bucket-policy-for-multiple-accounts.html)

To do so, I would like to get a list of AWS accounts as input (variable) and add them in the policy. Do you know how I can handle this ? Maybe there is a better way to do that…

Setting Bucket Policy for Multiple Accounts - AWS CloudTrail

Modify the policy on an Amazon S3 bucket, so that CloudTrail log files can be written to the bucket from multiple accounts.

pericdaniel avatar
pericdaniel

Is anyone using RDS database migration service to create a read replica of the main DB in another region?

Gabe avatar

Doesn’t RDS just support cross region read replica built in? https://aws.amazon.com/blogs/aws/cross-region-read-replicas-for-amazon-rds-for-mysql/

Cross-Region Read Replicas for Amazon RDS for MySQL | Amazon Web Services attachment image

You can now create cross-region read replicas for Amazon RDS database instances! This feature builds upon our existing support for read replicas that reside within the same region as the source database instance. You can now create up to five in-region and cross-region replicas per source with a single API call or a couple of […]

1
pericdaniel avatar
pericdaniel

Yes it does!

Gabe avatar

Sorry I was catching up on the channel and realized after I messaged you someone posted this later on lol

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@bentrankille maybe this will help, here we have a CloudTrail bucket module with policies https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket/blob/master/main.tf

cloudposse/terraform-aws-cloudtrail-s3-bucket

S3 bucket with built in IAM policy to allow CloudTrail logs - cloudposse/terraform-aws-cloudtrail-s3-bucket

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-cloudtrail

Terraform module to provision an AWS CloudTrail and an encrypted S3 bucket with versioning to store CloudTrail logs - cloudposse/terraform-aws-cloudtrail

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

here we create the bucket and CloudTrail in the audit account (which collects all logs from all other accounts like prod, staging, etc. https://github.com/cloudposse/terraform-root-modules/blob/master/aws/audit-cloudtrail/main.tf

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and here we create CloudTrails for all other accounts pointing to the bucket in the audit account https://github.com/cloudposse/terraform-root-modules/blob/master/aws/cloudtrail/main.tf

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Replicating Amazon Aurora MySQL DB Clusters Across AWS Regions - Amazon Aurora

You can create an Amazon Aurora MySQL DB cluster as a Read Replica in a different AWS Region than the source DB cluster. Taking this approach can improve your disaster recovery capabilities, let you scale read operations into an AWS Region that is closer to your users, and make it easier to migrate from one AWS Region to another.

bentrankille avatar
bentrankille

@Andriy Knysh (Cloud Posse) Thank you, I didn’t see the terraform-root-modules repo. I wonder how it works when you add a new trail to your audit account ? How the policy is modified ? In the terraform-aws-cloudtrail-s3-bucket module, it takes only one resource:

    resources = [
      "arn:aws:s3:::${module.label.id}/*",
]
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you mean a new bucket?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it takes the bucket

bentrankille avatar
bentrankille

If I know the name of your bucket, I can write my cloudtrail events to your bucket with this policy, right ?

bentrankille avatar
bentrankille

of course I wouldn’t have access to read it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

users who have access to the audit account would be able to read the logs and access the bucket

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

that’s the main purpose of having a separate audit account, to restrict access

bentrankille avatar
bentrankille

I get it about the read access, however, about the write access. Everyone can write to this bucket if the bucket name is known ?

Do you have only one resource in your policy like this ? “Resource”: “arnawss3:::bucket_name/AWSLogs/*”

In my mind, I would prefer have several resources to be sure who can write in my global bucket. Like this for example:

“Resource”:

  • “arnawss3:::bucket_name/AWSLogs/111111111111/*”,
  • “arnawss3:::bucket_name/AWSLogs/22222222222/*”

And one day, if I have a third account to add, I would have to add the following:

  • “arnawss3:::bucket_name/AWSLogs/33333333333/*”
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then you’ll need to add all of that to the list

resources = [
      "arn::....../.......",
]
bentrankille avatar
bentrankille

Yes, I understand that. I just want to be sure I understand well the policy in your module. If you give me your bucket name, I can write my trail events in your bucket without modifying the policy. The folders will be: your_bucket_name/AWSLogs/My_account_ID/Cloudtrail/

Am I right ?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes should work

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

since we have * at the end

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

arn::${module.label.id}/*

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(you ned to test it. If any issues, open a PR or issue, we’ll review)

bentrankille avatar
bentrankille

Thank you, I test it in my infrastructure and it works. I think this is a little weird for security reason. Every AWS accounts knowing the bucket name can write its events in this bucket.

However I don’t know how to handle it easily with terraform modules (modify the policy each time a new aws account is created)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, you know the bucket, you add the bucket ARN to

resources = [
      "arn:s3:::${module.label.id}/*",
]
foqal avatar
foqal
03:09:10 PM

Helpful question stored to @:

Hi everyone,...
pericdaniel avatar
pericdaniel

Does Terraform have an issue with pulling vpc IDs from 2 different regions?

pericdaniel avatar
pericdaniel

When I try to pull from us-west-2

pericdaniel avatar
pericdaniel

There’s no.problem

pericdaniel avatar
pericdaniel

But eu-central-1 gives me the vpc does not exist

Steven avatar
Steven

@pericdaniel You need to setup 2 providers. Since AWS providers are region specific

pericdaniel avatar
pericdaniel

@Steven so do I then pull that data within that Terraform file?

Steven avatar
Steven

@pericdaniel You setup 2 aws providers (default one and an alias) for the 2 regions, then you can do 2 VPC data lookups (1 with each provider). Then use the results however you want

pericdaniel avatar
pericdaniel

Yeah I got the first part. Not sure how to do the data lookup for the second vpc

Steven avatar
Steven

@pericdaniel This is assuming you need info from both regions in a single terraform run. Many times you don’t and can just have separate terraform runs with different provider definitions

pericdaniel avatar
pericdaniel

Yeah I’m trying to set up database migration service

pericdaniel avatar
pericdaniel

And what to do it all in one

pericdaniel avatar
pericdaniel

And just pull the subnets and vpc id

pericdaniel avatar
pericdaniel

Even though I’m passing it as bsrs

pericdaniel avatar
pericdaniel

Vara

pericdaniel avatar
pericdaniel

Vars

Steven avatar
Steven

Example VPC lookup specifing provider

Steven avatar
Steven

data “aws_vpc” “vpc” { provider = “aws.member”

filter { name = “tag:Name” values = [”${var.account_name}-mgmt”] }

Steven avatar
Steven

In this case there is an AWS provider with the alias ‘member’

pericdaniel avatar
pericdaniel

That’s what I was missing! Now I have the provider alias must he defined by module

pericdaniel avatar
pericdaniel

I got it

pericdaniel avatar
pericdaniel

Provider = AWS.alias

Steven avatar
Steven

If this is inside a module, then You define a provider construct in the module and have resources reference that. When the module is called you pass a a fully defined provider to the alias defined in the module

Steven avatar
Steven

Time to head to work. There is a doc page on this, if you need more details

joshmyers avatar
joshmyers

If you are using Organizations, you can now have a master cloudtail for the whole org

bentrankille avatar
bentrankille

I’m not sure I can use it in my case. I am a part of a bigger organization.. So unless we can use “sub-organization” I can’t use this feature.

joshmyers avatar
joshmyers

@bentrankille sorry a bit late there

:--1:2
pericdaniel avatar
pericdaniel

@Steven thank you so much for your help!

pericdaniel avatar
pericdaniel

If I’m doing multiple resource s3 bucket objects writing to the same bucket… Do they overwrite?

pericdaniel avatar
pericdaniel

I want 3 different files stored in s3 in the same bucket

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

same “path” to the file? yes, it’ll overwrite

pericdaniel avatar
pericdaniel

Same bucket

2019-01-21

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/tfmask

Terraform utility to mask select output from terraform plan - cloudposse/tfmask

3
Jan avatar

very nice addition!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

dirty little hack for now

Jan avatar

still

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Feel like the terraform core needs something to combat this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Jan avatar

yep

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
12:33:13 PM

tfmask output

joshmyers avatar
joshmyers

Anyone hit https://github.com/hashicorp/terraform/issues/17048 and have a working workaround ?

Splat syntax + count + nested map attributes · Issue #17048 · hashicorp/terraform

Related issue & conversation terraform-providers/terraform-provider-google#912 We opened this issue in the google provider, but @danawillow recommended we open it here as this is more of a core…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

how do we yak this?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

remove enabled?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(maybe share the module you’re working on for activemq)

joshmyers avatar
joshmyers

@Erik Osterman (Cloud Posse) Likely as it essentially fails when disabled (on first run anyway). If it was enabled and state written, and later enabled = "false" I suspect that may work

joshmyers avatar
joshmyers
Remove output splat syntax on nested map attributes by joshmyers · Pull Request #4 · cloudposse/terraform-aws-mq-broker

When initially building this module, it was done piece meal adding the outputs once the initial apply had been done. apply/plan etc all looked good after the initial apply. However, there is a bug …

Maciek Strömich avatar
Maciek Strömich

Hi Terraformers, @Erik Osterman (Cloud Posse) pointed me here because maybe someone is doing SES click tracking with custom email link tags? Long story short I’m trying to get graphed CW metrics for different links from an email using provided by aws cloudwatch destination. Had anyone any success with more fine grained configuration besides general open/click metrics? I know that I could get stats I need by running either firehose -> s3 -> athena or sns -> lambda -> cloudwatch but maybe there’s an easier way going with provided cloudwatch iintegration but the SES monitoring docs suck big time and it’s trial and error for me currently to figure that out.

2019-01-18

pecigonzalo avatar
pecigonzalo

But what is the challenge with the task? we have the service and task defined in terraform and it works without issues

pecigonzalo avatar
pecigonzalo

in any case, terraform is still POS for deploying this type of tasks imho

:100:1
Jan avatar

fargate is also expensive

Jan avatar

and the sizing is somewhat inflexible (in comparison to k8s)

:--1:1
pecigonzalo avatar
pecigonzalo

They did that 50% price reduction not long ago

pecigonzalo avatar
pecigonzalo

I guess it has its usecase, I just dislike awsvpc from it, as it puts there some “magic” that does not exist on the docker universe and complicates things

:100:1
pecigonzalo avatar
pecigonzalo

everyone was asking for overlay network support from them, and thye added awsvpc…. thanks AWS

Jan avatar

Yea they did the same initially with aks

Jan avatar

Was my first big issue with it

Jan avatar

Because a very practical limit on the number of nodes and pods yiu can run

Jan avatar

I managed to exhaust a test cluster during the aws summit when they announced it

pecigonzalo avatar
pecigonzalo

did they remove that now?

Jan avatar

they support other cni’s now as I understand

Jan avatar

Im keen to see what the k8s control plane as a service looks like in a few months

Jan avatar

alicloud will be adding kops native support mid this year (so says a birdie I know)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thats a big vote for kops

:--1:1
Jan avatar

at which point there are 5 large k8s control plane as a service providers that if they get to the same version / feature set it would be awesome

pecigonzalo avatar
pecigonzalo

I would love more lover for docker-swarm

2019-01-17

antonbabenko avatar
antonbabenko

https://github.com/terraformdns - should give some ideas how 0.12 will work in reality.

Terraform DNS Modules

Re-usable Terraform modules that aim to abstract over different DNS hosting services - Terraform DNS Modules

:100:3
:--1:2
pecigonzalo avatar
pecigonzalo

Nice

joshmyers avatar
joshmyers

@antonbabenko the big question is going to be how much we all have to undo our Terraform hacks to date

1
antonbabenko avatar
antonbabenko

Not sure about everyone, but my hacks are usually around counts which are inside reusable modules. I don’t use complex types, or recursions in chains of modules. Users of modules will not see too many changes, because they will continue to operate with key/values in most cases, while infrastructure teams work on modules and internals. It will be pretty nice to get rid of jsonnet and finally make 100% featurefull s3 bucket resource module

joshmyers avatar
joshmyers
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

joshmyers avatar
joshmyers

It works and is super useful, but pretty sure if something is gonna break, it be that.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Sir @jamie will you come to our rescue once 0.12 GA?

jamie avatar
jamie

Oui!

jamie avatar
jamie

I tried to make it using all built in interpolation- instead of hacks that use blocks/maps etc

jamie avatar
jamie

So I actually think it will survive the 0.12ga

1
jamie avatar
jamie

But I also think I can finesse it once it is released

sarkis avatar
sarkis

Serious question, do you ever consider not using terraform modules sometimes because they may overcomplicate the task at hand?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

For prototyping yes….

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but I kind of like modules b/c they provide a way to document a capability/requirement

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I mean, a module of (1) resource doesn’t make that much sense

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

however, if you use that (1) resource in a lot of places and there are a lot of default settings, that you want to set, then I think it still makes sense.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think “complicate” is a loaded term. I use it all the time. We as an “engineering community” use it all the time to justify one decision or another. Again, I do this. My point is more, “complicate” needs a qualifier. What about it is complicated?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I was just in another slack community, where I suggested bundling the tool chain in a docker container. They said they didn’t want to complicate things. IMO docker is the easiest thing in the world to use to bundle tools, easier than native

pecigonzalo avatar
pecigonzalo

Sure

pecigonzalo avatar
pecigonzalo

Simple stuff, sometimes is done without a module for us (the company I work at)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@sarkis one of the things I’ve started doing more of is “localized modules”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that is, sometimes it doesn’t make sense to have one-module-per-repo

sarkis avatar
sarkis

so localized modules are basically submodules and only ever used by the module it is contained in?

:100:1
sarkis avatar
sarkis

i do like this, i think what i am struggling with is the whole 1:1 module to repo approach is turning into a maintenance nightmare

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think the 1:1 module/repo strategy is best suited for companies like cloudposse that need to share modules across organizations

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think the monorepo strategy is better suited for inside organizations

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terraform-root-modules is our “monorepo” strategy

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

which contains modules and submodules therein

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

also, if you haven’t looked into terraform init -from-module=... do so!

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I this pattern

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(terragrunt uses it too)

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We are using cloudposse/tfenv to accomplish it in an alternative way using strictly environment variables

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

especially for highly esoteric business logic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

stuff that will never leave your org

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

pecigonzalo avatar
pecigonzalo

We do that as well, to ensure the code is “composable” anyway or supports intrepolations for environments

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

there’s a localized ns module.

pecigonzalo avatar
pecigonzalo

Similarly we have a repo of default-dns which setups SPF CNAMEs and other default stuff for all our registred domains

pecigonzalo avatar
pecigonzalo

some that are empty, but just so people dont use them to spam email

pecigonzalo avatar
pecigonzalo

it is a module so we can just easily spwan new ones (hopefully even easier on terraform 0.12) but it would not make sense to make that a “public” module outside of that repo, as it has no use

pecigonzalo avatar
pecigonzalo

I guess it really depends if you have a use for DRY or ensuring it meets some standard (interolation of environments, safe defaults, etc)

pecigonzalo avatar
pecigonzalo

we use data blocks all the time outside of modules, we try to avoid a bit more resources blocks

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We use this and have started to use more SSM

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

LOVE IT for passing dynamic values across modules

:--1:2
pecigonzalo avatar
pecigonzalo

Yeah, agreed

pecigonzalo avatar
pecigonzalo

The only thing that im a bit hesitant by using SSM for terraform with data providers is that now, you always need SSM to deploy. We use SSM, but with chamber and either chamber exec -- and TF_VAR variables or chamber export -f dotenv -o chamber.auto.tfvars. This way if I wan to run without SSM, we just fill the vars or set the env vars

pecigonzalo avatar
pecigonzalo

I did a PR for this reason to chamber not long ago, to add support for multi depth paths (/this/that/foo/bar)

Ryan Ryke avatar
Ryan Ryke

hey dudes, hope you are all well. running into a weird issue that i havent seen.

resource "aws_vpc_peering_conection" "staging_peer" {
  peer_vpc_id = "vpc-7476xxx"
  vpc_id      = "${module.vpc.vpc_id}"
  auto_accept = "true"
}

gives me Error: aws_vpc_peering_conection.staging_peer: Provider doesn't support resource: aws_vpc_peering_conection

pecigonzalo avatar
pecigonzalo

Missing an N

2
pecigonzalo avatar
pecigonzalo

In connection

Ryan Ryke avatar
Ryan Ryke

ahhhhh hahaha

Ryan Ryke avatar
Ryan Ryke

what a putz

pecigonzalo avatar
pecigonzalo

Ryan Ryke avatar
Ryan Ryke

im not too proud…

pecigonzalo avatar
pecigonzalo

Hey sometimes we need some fresh eyes

1
1
Ryan Ryke avatar
Ryan Ryke

that was bad

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

anyone around for some fargate chat? I can’t seem to get it to start using the cp modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so oooooooo we finally have an e2e example

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
02:08:45 AM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is deploying atlantis

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

on fargate

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

using our modules

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ok, great. checking

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

key thing…. make sure your repo has a buildspec.yaml

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we provide one in the README.md of that example

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)
cloudposse/terraform-aws-ecs-alb-service-task

Terraform module which implements an ECS service which exposes a web service via ALB. - cloudposse/terraform-aws-ecs-alb-service-task

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ah…so using terraform-aws-ecs-web-app

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, but terraform-aws-ecs-web-app is not required

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

however, it shows you how to piece it all together

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so if you don’t want to use the ecs-web-app module, you can start with your own module

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

i have the service integrated w/ the config generating

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

using terraform-aws-ecs-alb-service-task

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Initial implementation of Atlantis ECS module by joshmyers · Pull Request #1 · cloudposse/terraform-aws-ecs-atlantis

what Terraform module to deploy Atlantis to an ECS fargate cluster. why So we can run Atlantis outside of Kops

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s how we are refactoring atlantis into a standalone ecs-task-as-a-module

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so a module is like a helm chart

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(also, a lot of PRs in the past week related to ECS)

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

make sure you’re on the latest versions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

tied up some lose ends

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

added support for github webhooks

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(also, i’m actively deploying this as we speak on a customer site! so recently “tested” )

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

lol

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

deploying atlantis?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yep

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but our flavor

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

yeah

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

when a container definition changes, does it trigger a new deploy?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

or an update?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ok

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we need to ignore changes on the container definition

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

i changed ports and it didn’t show TF changes

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

interesting

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so terraform is great for deploying static infrastructure

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it is a p.o.s. for managing the SDLC of apps ontop of that infra

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

our strategy is this…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

use CI/CD for all changes there after using the patch technique

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s what the imagedefinitions.json is doing in the buildspec.yaml

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that said, before you go TOO far down this path, checkout #airship

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

they take a different approach to ours which might suit certain use-cases better

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s run by @maarten and I know @jamie is a big contributor

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

coolio

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
taint/atlantis:
        terraform taint -module atlantis_web_app.ecs_alb_service_task aws_ecs_task_definition.default
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so if I need to redeploy the definition, I have this

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

ahh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the constant challenge/strife with terraform and ECS is the task definition

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we can use the data provider to look up the current defintion, but then we break the cold start

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or we can ignore changes, but then we cannot easily update the task definition

maarten avatar
maarten

There are a few ways though, and one I haven’t implemented yet

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if we taint, and redeploy, we might revert the image that’s current deployed.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for our immediate use-case, this taint approach is working good enough without yak

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you figure out some clever way around this, would love contributionss

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. using SSM parameter store somehow

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

well fargate is the new direction so I’ll definitely hit this wall and kick some tires on solutions

1

2019-01-16

praveen avatar
praveen

#terraform , am working on creating azure virtual machine module. for which I am not able to make “azurerm_network_interface_backend_address_pool_association” optional. The intention is to make it optional is to source vm module for provisioning standalone VM, as well as for vm’s behind load balancer

praveen avatar
praveen

resource “azurerm_network_interface_backend_address_pool_association” “nic” { count = “${local.enabled}” network_interface_id = “${element(azurerm_network_interface.nic.*.id, count.index)}” ip_configuration_name = “${var.app_code}${var.app_name}IP${count.index}” backend_address_pool_id = “${var.lb_bepool_IDs}” }

praveen avatar
praveen

with locals as locals { enabled = “${var.lb_enabled == “false” ? false : true }”

praveen avatar
praveen

am i doing something wrong here

Nikola Velkovski avatar
Nikola Velkovski

what’s the error you are getting ?

praveen avatar
praveen

if am sourcing the module for deploying standalone instance the error is “module.vm.azurerm_network_interface_backend_address_pool_association.nic: Can not parse “backend_address_pool_id” as a resource id: Cannot parse Azure ID: parse : empty url”

praveen avatar
praveen

for instance behind lb the error is “module.vm.var.lb_bepool_IDs: variable lb_bepool_IDs in module vm should be type string, got list”

Nikola Velkovski avatar
Nikola Velkovski

can you show me the definition for the var lb_bepool_IDs ?

Nikola Velkovski avatar
Nikola Velkovski

variable “” {} etc..

praveen avatar
praveen

I have DM you the comeplete code

praveen avatar
praveen

variable “lb_bepool_IDs” { type = “string” }

Taco avatar

Hello, I just found CloudPosse and joined the workspace. I am looking for a Terraform module for a a VPC that lets me start with a Direct Connect based VPC and Internet Gateway. Our IT department created these two assets and so I need to start with them, not a VPC of my own making and not the Default VPC. Does such a module exist?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We don’t have a module ready for this yet. @Andriy Knysh (Cloud Posse) has started one, but not been able to finish it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

As @Jan mentions, he has something in the works but it’s not yet contributed.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we are working on VPN + Virtual gateway + Customer gateway module, not Direct Connect

Jan avatar

hahaha

Jan avatar

I just created that

Jan avatar

Its a touch rough and I need to contribute it back

Jan avatar

but can happily talk you through it

Taco avatar

Nice! I’d like that very much

Jan avatar

Are you on a tight deadline?

Jan avatar

I have a few deadlines of my own to get out before I can clean that up and push it upstream

Taco avatar

I have until the end of the month, so don’t let me come between you and your deadlines.

Jan avatar

oki cool

Jan avatar

lemme talk you through it

Jan avatar

I have a set cidr that was agreed upon with corp ITS

Taco avatar

Sounds like me; I have a VPC with a CIDR block in four separate accounts, for Dev, Test, Stage and Prod

Jan avatar

yea

Jan avatar

so my tf module is like this

Jan avatar
[vpc-accountname.division.cloud.company.com](http://vpc\-accountname\.division\.cloud\.company\.com)
├── [init.tf](http://init\.tf)
├── [outputs.tf](http://outputs\.tf)
├── [peering.tf](http://peering\.tf)
├── [valiables.tf](http://valiables\.tf)
└── [vpc.tf](http://vpc\.tf)
Jan avatar

this is 100% the stock vpc module with the addition of a few things

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen so count = "${local.enabled}" is not working because

locals {
 enabled = "${var.lb_enabled == "false" ? false : true }" }
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

is wrong

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

should be enabled = "${var.lb_enabled == "false" ? 0 : 1 }"

praveen avatar
praveen

let me change it

praveen avatar
praveen

is it ok if I have the variable as follows

praveen avatar
praveen

variable “lb_enabled” { description = “Set to false to prevent the module from creating any resources” default = “false” }

Taco avatar

@Jan - When you say the “stock vpc module” do you mean the CloudPosse module here: https://github.com/cloudposse/terraform-aws-vpc?

cloudposse/terraform-aws-vpc

Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc

Jan avatar
cloudposse/terraform-aws-dynamic-subnets

Terraform module for public and private subnets provisioning in existing VPC - cloudposse/terraform-aws-dynamic-subnets

cloudposse/terraform-terraform-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-terraform-label

Taco avatar

Ah, OK

Jan avatar
Jan
03:04:43 PM
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

welcome @Taco

Jan avatar

then in the peering, which I still need to clean up and make redundant gw/vpn’s

Taco avatar

Thanks @Andriy Knysh (Cloud Posse)

Taco avatar

OK, great, thanks! I look forward to the finished bits

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we did not use Direct Connect before, so @Jan would be of much help here

Jan avatar
Jan
03:07:58 PM
Taco avatar

I didn’t even know about it until I came to my current position, so I’m still getting used to the idea of not creating a new VPC for each application/project

Jan avatar

I need to extend it a bit

Jan avatar

andf have it to redundant cgw’s

Jan avatar

and vpn’s

Jan avatar

but thats the bassics to all of it

Jan avatar

I will make some time to commit it this week

Taco avatar

Sounds like you’ve got your hands full

Jan avatar

haha yea

Taco avatar

Thanks for your time, I’ll keep checking in and I will explore your repositories; there’s a lot of stuff…

Jan avatar

just remind me

Jan avatar

I will contribute it to cloudposse

Taco avatar

Will do

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) as per your recommendations I did made the changes with no luck. Now I made the following changes but it it still not working

praveen avatar
praveen

resource “azurerm_network_interface_backend_address_pool_association” “nic” { count = “${var.lb_enabled == “false” ? “${var.lb_enabled}” : “${length(azurerm_network_interface.nic..id)}”}” network_interface_id = “${element(azurerm_network_interface.nic..id, count.index)}” ip_configuration_name = “${var.app_code}${var.app_name}IP${count.index}” backend_address_pool_id = “${var.lb_bepool_IDs}” }

praveen avatar
praveen

variable “lb_bepool_IDs” { type = “string” default = “” }

variable “lb_enabled” { type = “string” description = “Set to false to prevent the module from creating any resources” default = “false” }

praveen avatar
praveen

for instance without LB it throws error as “ Can not parse “backend_address_pool_id” as a resource id: Cannot parse Azure ID: parse : empty url”

praveen avatar
praveen

for instance with LB it throws error as “ module.vm.azurerm_network_interface_backend_address_pool_association.nic: azurerm_network_interface_backend_address_pool_association.nic: value of ‘count’ cannot be computed”

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
count                   = "${var.lb_enabled == "false" ? 0 : length(azurerm_network_interface.nic.*.id)}"
praveen avatar
praveen

@Andriy Knysh (Cloud Posse) no luck. Same errors for both with LB & without LB as posted above

praveen avatar
praveen

DM you the complete code for your ref

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

will be back soon and take a look @praveen

praveen avatar
praveen

sure @Andriy Knysh (Cloud Posse)

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) were you able to check my code

praveen avatar
praveen

@Andriy Knysh (Cloud Posse), just to be made aware that I see there is a bug with using count and making optional for resource azurerm_network_interface_backend_address_pool_association

praveen avatar
praveen

I got it fixed using deprecated argument load_balancer_backend_address_pool_ids

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
add new resource random_password by jcarrothers-sap · Pull Request #52 · terraform-providers/terraform-provider-random

As a final solution to #17/#47, add a new resource random_password which is identical to random_string except that it treats the result as Sensitive data and avoids writing it to the console. The …

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

excited for this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(posted 1 day ago)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

crazy how often that happens to me. when I need something, someone has already opened the pr.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

last week, I was about to go add build_timeout to our terraform-aws-codebuild module, but noticed Strech96 contributed it 2 weeks ago.

2019-01-15

Jan avatar
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Jan avatar

what are all these services used for?

pecigonzalo avatar
pecigonzalo

bootstraping the root accounts

pecigonzalo avatar
pecigonzalo
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Jan avatar

what I am looking for is what are the services used for

Jan avatar

rd psql + mysql + elasticache + replicas

pecigonzalo avatar
pecigonzalo

AFAIK for KOPS

Jan avatar

nope

Jan avatar

kops does not use rds not redis / memcache

Jan avatar

not by itself any how

pecigonzalo avatar
pecigonzalo

but some operators or etc might

pecigonzalo avatar
pecigonzalo
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Jan avatar

I think its probably more for atlantis or something

pecigonzalo avatar
pecigonzalo

has references to KOPS

pecigonzalo avatar
pecigonzalo

and AFAIK it does not use elasticsearch either

Jan avatar

yea

Jan avatar

hence my asking

Jan avatar
vpc_cidr_block = "${backing_services_cidr}"
zone_name = "${domain_name}"
region = "${aws_region}"
postgres_cluster_enabled = "false"
kops_metadata_enabled = "false"
rds_cluster_replica_enabled = "false"
rds_cluster_replica_cluster_identifier = "${namespace}-${stage}-postgres"
Jan avatar

in templates/conf/backing-services/terraform.tfvars

Jan avatar

mmmmm

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan those backing services are for applications (running on k8s for example), they not for the infra

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

those are just examples

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you can provision all or some of them as needed

pecigonzalo avatar
pecigonzalo

that sounds odd, as you already have CP modules for those

praveen avatar
praveen

#terraform In Azure am sourcing virtual machine module to create multiple windows VM’s. As part of the virtual machine module I also have virtual machine extensions which does not get attach to multiple instances created while sourcing base module

praveen avatar
praveen

here is the virtual machine base module code resource “azurerm_virtual_machine” “vm” { count = “${length(var.system)}” name = “${var.app_code}${var.app_name}${local.az_env}_${count.index}” location = “${var.location}” resource_group_name = “${var.resource_group_name}” network_interface_ids = [”${element(azurerm_network_interface.nic.*.id, count.index)}”] vm_size = “${var.vm_type}” delete_os_disk_on_termination = true delete_data_disks_on_termination = true

praveen avatar
praveen

and the extension as follows

praveen avatar
praveen

resource “azurerm_virtual_machine_extension” “set_execution_policy” { name = “setexecutionpolicy-${lookup(var.system[count.index], “hostname”)}” location = “${var.location}” resource_group_name = “${var.resource_group_name}” virtual_machine_name = “${element(azurerm_virtual_machine.vm.*.name, count.index)}” depends_on = [“azurerm_virtual_machine.vm”]

publisher = “Microsoft.Compute” type = “CustomScriptExtension” type_handler_version = “1.9”

settings = <<SETTINGS { “commandToExecute”: “powershell.exe Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Force” } SETTINGS }

praveen avatar
praveen

I knoew am doing something wrong with virtual_machine_name = “${element(azurerm_virtual_machine.vm.*.name, count.index)}” in extension

praveen avatar
praveen

is there a way we can attach extensions to multiple instances created using base virtual machine module

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen we’ll take a look at ^

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan @pecigonzalo to give you more details on having backing-services in the root modules:

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) should I provide you the complete code or the snippet will suffice ?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. root-modules is the resource catalog - everything you deploy in your environment(s) goes in there. CloudPosse root modules is just an example. Every company gets its own copy with only the resources they need
Jan avatar

On my side currently im building modules for private dns zones

Jan avatar

already done as such

Jan avatar

then a terraform rendered k8s cluster spec

Jan avatar

changing a modul for kops backing state

Jan avatar

as I aklready have domain

Jan avatar

var.domain_enabled

Jan avatar

that actually helps

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Those invocations of RDS, Redis, Elasticsearch in root-modules are more than just instantiating those modules. They are connected to the other resources and networks. For example, RDS/Redis/Elasticsearch security groups are connected to the kops/EKS/ECS security groups. kops VPC is used to peer to the backing services VPC
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and all of that (and more) is codified in the root-modules, so those backing services are not just CP modules’ invocation, it’s more about connecting them together (network, DNS, SGs, etc) and provisioning the entire infra

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen you can send the code, we’ll take a look

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) I just sent you the code

Jan avatar
Jan
02:51:20 PM
:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hah, I’ve always struggled to explain this in slack without it interpreting my markdown. Using a raw snippet is a good idea!

Jan avatar

haha I have been on that ride before

Jan avatar
hello world
Jan avatar
Jan
02:52:40 PM
Jan avatar

or hit the + to the left of the text input and create a code snippet

i5okie avatar
i5okie

hi

i5okie avatar
i5okie

which files, and pushed by what?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you create a log bucket, assign it to the EB environment, and then beanstalk will push all logs from all instances (including your app) to the bucket

i5okie avatar
i5okie

doesnt work tho.

i5okie avatar
i5okie
cloudposse/terraform-aws-elastic-beanstalk-environment

Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment

i5okie avatar
i5okie

terraform created all the s3 buckets for each environment. but they are empty. and ELB logs are pushed to the AWS default of elb-logs-< your aws account number>

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i’m not sure, but maybe ELB logs get pushed just to the default bucket (don’t remember)

i5okie avatar
i5okie

in practice that is what is hapenning. im not really concerned about that tho. lol im just curious about that one setting.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you find any issues, please open a PR

i5okie avatar
i5okie

thats assuming that I know enough to make changes lol

i5okie avatar
i5okie
06:45:15 PM

was just digging around in cli.. eb config is showing these settings here

i5okie avatar
i5okie

i wonder if thats related

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(cloudposse is not actively deploying any beanstalk clusters… we wrote these in 2016 for a few customers)

SweetOps avatar
SweetOps
08:05:39 PM
Are you using some of our [terraform-modules> in your projects? Maybe you could <https://cpco.io/leave-testimonial leave us a testimonial](https://cpco.io/terraform-modules)! It means a lot to us to hear from people like you.
praveen avatar
praveen

#terraform am trying to make the following resource optional as part of azure virtual machine base module

praveen avatar
praveen

resource “azurerm_network_interface_backend_address_pool_association” “nic” { count = “${var.lb_bepool_IDs == “0” ? 0 : 1}” network_interface_id = “${element(azurerm_network_interface.nic.*.id, count.index)}” ip_configuration_name = “${var.app_code}${var.app_name}IP${count.index}” backend_address_pool_id = “${var.lb_bepool_IDs}” }

praveen avatar
praveen

with lb_bepool_IDs is defined in end code as lb_bepool_IDs = “${module.lb.backend_address_pool_id}”

praveen avatar
praveen

it throws the following error value of ‘count’ cannot be computed

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
1
praveen avatar
praveen

should I change the count decision to true or false?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen there is no easy answer to that error. You need to play with [module.lb](http://module\.lb) because it’s a dependency for "azurerm_network_interface_backend_address_pool_association" "nic"

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you can share here the code for the module so we’d take a look if anything could be resolved fast

i5okie avatar
i5okie

oh looks like terraform-aws-rds-cluster module outputs don’t work. in envvars of a beanstalk module.. i’m trying this:

i5okie avatar
i5okie
11:55:11 PM
i5okie avatar
i5okie

and in plan or apply, its not wanting to create rds_db_name because ^ module.rds_cluster_…. is empty

i5okie avatar
i5okie

when i do terraform state show… i can see these things under “module.rds_cluster_aurora_postgres.aws_rds_cluster.default”

i5okie avatar
i5okie

i have no idea how to troubleshoot this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

is the enabled flag set to "true"

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for the module.rds_cluster_aurora_postgres

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-rds-cluster

Terraform module to provision an RDS Aurora cluster for MySQL or Postgres - cloudposse/terraform-aws-rds-cluster

i5okie avatar
i5okie

yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and when you apply it, it should show all these outputs https://github.com/cloudposse/terraform-aws-rds-cluster/blob/master/outputs.tf

cloudposse/terraform-aws-rds-cluster

Terraform module to provision an RDS Aurora cluster for MySQL or Postgres - cloudposse/terraform-aws-rds-cluster

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you see them?

i5okie avatar
i5okie

under module.rds_cluster_aurora_postgres.aws_rds_cluster.default yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-elastic-beanstalk-environment

Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-elastic-beanstalk-environment

Terraform module to provision an AWS Elastic Beanstalk Environment - cloudposse/terraform-aws-elastic-beanstalk-environment

i5okie avatar
i5okie

indeed

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Did you check the beanstalk UI under environment settings if you have those values there?

i5okie avatar
i5okie

they didn’t get added, and deployed failed, thats how i found out.

i5okie avatar
i5okie

i’ve been using the normal postgres rds previously. trying aurora first time here.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Ok so the problem is the failed deployment?

i5okie avatar
i5okie

no the problem is that terraform doesn’t understand what “RDS_DB_NAME”, “${module.rds_cluster_aurora_postgres.name}“, is

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

It should be a map

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Variable

i5okie avatar
i5okie

and @Erik Osterman (Cloud Posse) i didn’t notice your question earlier. yes. it created the environment.

i5okie avatar
i5okie

and it is

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Let me show an example of that

i5okie avatar
i5okie

i just didn’t copy the []

i5okie avatar
i5okie

i’ve got a bunch of envvars in there already. i just copied the one line as example here.

i5okie avatar
i5okie

terraform created all the other envvars except ones referring to ${module.rds_cluster_aurora_postgres. …}

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-jenkins

Terraform module to build Docker image with Jenkins, save it to an ECR repo, and deploy to Elastic Beanstalk running Docker stack - cloudposse/terraform-aws-jenkins

i5okie avatar
i5okie

I’ll try it again tomorrow.

foqal avatar
foqal
05:37:23 AM

Helpful question stored to @:

oh looks like terraform-aws-rds-cluster module outputs don’t work.  in envvars of a beanstalk module.. i’m trying this:...
i5okie avatar
i5okie

@Andriy Knysh (Cloud Posse) i don’t think map is the problem here.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ok, was the RDS cluster created?

i5okie avatar
i5okie
05:41:01 AM
i5okie avatar
i5okie

yes i created the cluster first, then tried to apply targetting this one beanstalk module.

i5okie avatar
i5okie

i’ve created the environment with manually entering the envvar values. i’ll try changing them to the module outputs now and try to see if terraform will want to change envvars or not brb

i5okie avatar
i5okie

hmm

i5okie avatar
i5okie

maybe its fine

i5okie avatar
i5okie

i’ll have to try again tomorrow.

i5okie avatar
i5okie

i just remembered that the first time i tried, when i ran into this issue.. it finished created the rds cluster, but never finished adding the aurora instance.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea, maybe there were some issues with creating the RDS instances

i5okie avatar
i5okie

yep, I inherited this terraform thing, and was writing documentation on deploying a new stack.. missed some important steps assuming terraform will create the resource if its referenced. #hamberdwrong

i5okie avatar
i5okie

fixed

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

2019-01-14

maarten avatar
maarten

Hi Everyone, I am trying to fix a “value of ‘count’ cannot be computed” but I cannot pintpoint the problem. When the module is applied together with a new ALB, I get a “‘count’ cannot be computed” which doesn’t make sense to me as the count condition itself is not dependent of the alb listener resource.

https://github.com/blinkist/terraform-aws-airship-ecs-service/blob/a304a850ef734c80cb96e6ebe81f0bf3e0eec912/modules/alb_handling/main.tf#L118

Everything within the count condition is not the result of another resource. It is however a parameter coming from a merged map, but that should actually just work. @Andriy Knysh (Cloud Posse)

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

maarten avatar
maarten

I think I know the culprit. I populate a map with values dependent of another resources, in the same map are boolean values which are used in create conditionals. This map is merged with default values. I think the merge doesn’t happy before all values are populated and voila.

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Yes probably the dependency on the other resource and the merge are the culprit

Nikola Velkovski avatar
Nikola Velkovski

click

Nikola Velkovski avatar
Nikola Velkovski

we have it constatly

Nikola Velkovski avatar
Nikola Velkovski

weird usually this happens with locals

Nikola Velkovski avatar
Nikola Velkovski

and I all see is vars

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@maarten counts with maps are not handled correctly by TF https://docs.cloudposse.com/troubleshooting/terraform-value-of-count-cannot-be-computed/

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

in your module, you provide a lookup map of all possible record types and then provide the desired record type in var.route53_record_type)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

a quick fix (not ideal, but should work) would be

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  count = "${var.create && var.load_balancing_type == "application" && ! var.redirect_http_to_https && var.route53_record_type != "NONE" ? 1 : 0 }"
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it will skip the test for allowed types, but you do it in many other places anyway

Jan avatar

What module should I be using to deploy an atlantis setup

Jan avatar

I am using the geodesic ref arch if that makes a difference

joshmyers avatar
joshmyers

@Jan That is being worked on at the moment

Jan avatar
cloudposse/geodesic-aws-atlantis

Geodesic module for managing Atlantis with ECS Fargate - cloudposse/geodesic-aws-atlantis

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, this will be archived

cloudposse/geodesic-aws-atlantis

Geodesic module for managing Atlantis with ECS Fargate - cloudposse/geodesic-aws-atlantis

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re redoing it. @joshmyers is putting the finishing touches.

Jan avatar

awesomeness

Jan avatar

Will hold off for now

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ha, thanks @foqal

foqal avatar
foqal
03:17:42 PM

Helpful question stored to @:

Hi Everyone, I am trying to fix a "value of 'count' cannot be computed" but I cannot pintpoint the problem...
joshmyers avatar
joshmyers

@Andriy Knysh (Cloud Posse) is a Terraform ninja

1
joshmyers avatar
joshmyers

but still ongoing work

joshmyers avatar
joshmyers

If you have bigger fish to fry for now, I’d leave the Atlantis stuff until it properly lands (hopefully a few days)

Jan avatar

I wanted to get the k8s stuff out

Jan avatar

butwaiting on our parent zone to get delegated

praveen avatar
praveen

Hi, Just checking if we have module for creating app service environment using terraform on Azure platform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We (cloudposse) have not built out any mdoules for Azure

praveen avatar
praveen

Hi, Is there any opportunity to create app service environment in Azure platform using terraform?

praveen avatar
praveen

#terraform Hi, Is there any opportunity to create app service environment in Azure platform using terraform?

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) is app service plan & app service environment both the same?

pecigonzalo avatar
pecigonzalo

anyone know what is required to get ` provisioner “local-exec” ` to output to stdout while doing the action?

pecigonzalo avatar
pecigonzalo

I saw some resources do it, but I have a python script that is not showing anything

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Azure Resource Manager: azurerm_app_service - Terraform by HashiCorp

Manages an App Service (within an App Service Plan).

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

anyone here ever tried to get a local-exec script output to a terraform variable?

pecigonzalo avatar
pecigonzalo

Hey thanks! but not exactly what im looking for

pecigonzalo avatar
pecigonzalo

Im trying to make it so my script in a null resource outputs it output to stdout douring run

pecigonzalo avatar
pecigonzalo

so we can see the progress

pecigonzalo avatar
pecigonzalo

We dont need to capture it

pecigonzalo avatar
pecigonzalo

we are toying with the idea of a asg-roll module which basically has script to roll the instances of an ASG group

pecigonzalo avatar
pecigonzalo

but we want to see the progress

maarten avatar
maarten
does tee /dev/stdout work ?
pecigonzalo avatar
pecigonzalo

instead of getting it all at the end

pecigonzalo avatar
pecigonzalo

the “when” terraform shows output seems pretty random, as sometimes does only after it finishes

pecigonzalo avatar
pecigonzalo

sometimes is doing it after some minute ¯_(ツ)_/¯

maarten avatar
maarten
or tee /dev/stderr
pecigonzalo avatar
pecigonzalo

Ill try that, seems hacky but hacky that works is always awesome

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the terraform gods have heard our prayers and answered.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
dmlittle/scenery

A Terraform plan output prettifier. Contribute to dmlittle/scenery development by creating an account on GitHub.

:--1:5
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve added scenery to our cloudposse/packages alpine apk distribution

dmlittle/scenery

A Terraform plan output prettifier. Contribute to dmlittle/scenery development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

“A Terraform plan output prettifier” in go

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(goodbye terraform-landscape hello scenery)

:100:2
bananadance2
Gabe avatar

ya’ll ever run into issues where cloning modules is slow from github?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

especially if you have A LOT of modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i mean, it can take minutes

Gabe avatar

yeah it’s really painful in some of our directories… was contemplating uploading modules to S3 to see if that gets better performance

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are you using EFS/NFS by any chance?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’ve noticed that’s exponentially slower due to all the stat operations.

loren avatar
loren

be glad if you aren’t using codecommit github is soooo much faster, just about always

pecigonzalo avatar
pecigonzalo

I wish terraform cached the modules as it does with the plugins

:100:1
pecigonzalo avatar
pecigonzalo

I dont think it would be that hard to do, but had not had time to look at the codebase

pecigonzalo avatar
pecigonzalo

it would make dev a lot faster at least

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Work on terraform provider for DocumentDB is progressing nicely: https://github.com/terraform-providers/terraform-provider-aws/issues/7077

2019-01-13

dalekurt avatar
dalekurt

@Erik Osterman (Cloud Posse) I have a question, it may seem obvious but why Terraform over CloudFormation, if you’re planning to use AWS to host your infrastructure and services. Let me give some context, our CTO has mandated that the DevOps team to design, deploy and manage the SA using CloudFormation, the DevOps team is in opposition of this in favor of Terraform. We have to justify that decision.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I have A LOT of thoughts on this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but I can’t discuss right now. can you hit me up again tomorrow.

Steven avatar
Steven

@dalekurt I’ve had 2 main reasons: 1) Agnostic. You’ll say that you’re only going to use AWS so that doesn’t matter. But that is not reality. In addition to AWS you’ll use a notification system, version control, monitoring, etc.. By having a single tool that can manage many of these (can be extended for the ones it doesn’t), you greatly reduce the amount of tooling and learning required. I’ve used Terraform for Pagerduty, Datadog, InfluxDB, Grafana, Github, and have plans to use it for other services. 2) Neither Terraform or CloudFormation supports all of AWS or new features as they are released. But I’ve found that generally Terraform supports features sooner. Anyone can add support for a feature or request one. And if CloudFormation supports something that Terraform doesn’t, Terraform can run CloudFormation

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

TL;DR: cloudformation is one piece of the puzzle. You still need to automate everything else.

dalekurt avatar
dalekurt

Thank you @Steven you’ve mirrored my thoughts exactly, I appreciate that.

:--1:1
dalekurt avatar
dalekurt

I’m on the right track.

Jan avatar

Terraform is extensible

Jan avatar
Providers - Terraform by HashiCorp

Terraform is used to create, manage, and manipulate infrastructure resources. Examples of resources include physical machines, VMs, network switches, containers, etc. Almost any infrastructure noun can be represented as a resource in Terraform.

Jan avatar

Even if yiu only plan to use a single cloud provider there are loads of options for other services

Jan avatar

And modular

Jan avatar

Cf has also only just added config drift management

Jan avatar

I have converted every Cf org I worked with to terraform

Jan avatar

Always with lengthy deep dives into the pros and cons

dalekurt avatar
dalekurt

@Jan Did you do that manually, or with the import feature?

Jan avatar

I have used import often but almost always write the tf from scratch

Jan avatar

Whilst building the provisioning layer you have the best perspective to find flaws

Jan avatar

With production systems often its a case of import and then run plan

Jan avatar

If you see a planned change yiu have either missed something or what you are meant to have had diverged from the Cf code base

dalekurt avatar
dalekurt

Nice!

maarten avatar
maarten

@dalekurt https://www.reddit.com/r/Terraform/comments/af4lsb/why_terraform_and_not_just_shell_scripts/edvflu9 This was written for Terraform opposed to shell, but also fully captures why Terraform is such a good tool.

r/Terraform - Why terraform and not just shell scripts?

4 votes and 9 comments so far on Reddit

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ever wish you could initialize terraform state backends using environment variables? (and without #terragrunt or other wrappers)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/tfenv

Transform environment variables for use with Terraform (e.g. HOSTNAMETF_VAR_hostname) - cloudposse/tfenv

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Enhancement: Configure backend through environment variables similar to tfvars · Issue #19300 · hashicorp/terraform

Current Terraform Version 0.11.10 Use-cases In a CI pipeline, configuring the terraform commands through environment variables rather than explicit arguments is a highly useful feature. We currentl…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=my-bucket
source <(tfenv)
terraform init
:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this maps to TF_CLI_ARGS_init=-backend-config=my-bucket

The TF_CLI_ARGS* are natively supported by terraform.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

TF_CLI_PLAN_REFRESH=true maps to TF_CLI_ARGS_plan=-refresh=true

tamsky avatar
tamsky

oh snap this is cool

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Combining tfenv with direnv and a task runner (e.g. make) gives ultimate flexibility without lock-in to one particular tool.

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

2019-01-12

foqal avatar
foqal
02:09:42 AM

@foqal was added to this conversation

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’ve added Foqal as a test to see how it goes over the next few weeks.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s only in this channel right now.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Open Communities attachment image

Did you know that  over 50%  of your users questions are left unanswered? Did you also know that  60%  of the time, your contributors are answering support questions instead of making your open…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you have any feedback (Good/bad), do let me know.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or ping @vlad

vlad avatar
vlad
02:19:16 AM

@vlad has joined the channel

vlad avatar

yup im here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hi @vlad

vlad avatar

hey

vlad avatar

@Andriy Knysh (Cloud Posse)

2019-01-11

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

our best practice is never to use any of the default resources in an account

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are you piggy backing on an existing infrastructure or starting fresh?

james avatar
james

Adopting an existing infrastructure

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

aha ..

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

well, one option is to define the resources and use terraform import

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just make sure the setting match

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

run terraform plan until they sync up perfectly. then you have a baseline to affect change.

mrwacky avatar
mrwacky

except terraform import is lossy

mrwacky avatar
mrwacky

eg - you can import EC2 instances, but it might not notice or complain if you don’t set all the parameters (like terminiation_protection).. So terraform import is good, but not perfect!

sarkis avatar
sarkis

this is true - i treat terraform import in the current state as “get me started” and then follow up with more changes it missed (what i think @Erik Osterman (Cloud Posse) was also eluding to above)

:--1:1
mrwacky avatar
mrwacky

maybe he was alluding @sarkis? haha!

sarkis avatar
sarkis

that too

james avatar
james

that’s what we’re doing, just running into this ambiguity with aws_default_route_table and its ilk. I can import it with aws_route_table and get the plans to match up, just wondering if that will come back to bite me one day

james avatar
james

I guess it’s probably not a big deal, nothing is permanent after all

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
chanzuckerberg/terraform-provider-bless

Terraform provider to automate the creation of BLESS deployments - chanzuckerberg/terraform-provider-bless

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This looks amazing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Keep secrets out of terraform state store

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh I missed the gist of this but still cool

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This is related to Netflix bless.

2019-01-10

javier avatar
javier

Hi All!

1
javier avatar
javier

I have a tfstate on a existant AWS S3 bucket created manually, and now I can implement a module used to create a S3 bucket and DynamoDB to lock state….

javier avatar
javier

the question is: how I can migrate the tfstate from the original bucket to the new bucket? Or how I can apply changes into the original bucket manually created (to enable versioning and encrypt for example) without last the tfstate (because I have a production environment running)

javier avatar
javier

I’m trying this using terraform import, but I cant have good results…

maarten avatar
maarten

@javier if you remove the remote state directive in your terraform file, and run terraform init, your state should be available locally again.

maarten avatar
maarten

then you can add the directive with your new bucket configured and run init again and it should be on there. I’m not sure if it’s possible in one step, but I’m sure 2 steps will work.

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-tfstate-backend

Provision an S3 bucket to store terraform.tfstate file and a DynamoDB table to lock the state file to prevent concurrent modifications and state corruption - cloudposse/terraform-aws-tfstate-backend

javier avatar
javier

@maarten If I removed the backend directive do not lost the actual state? is not necessary copy the state from the bucket to my local?

maarten avatar
maarten

you should always make a backup of your state just in case, but if you remove the remote state directive the state should be available again as local terraform.tfstate

javier avatar
javier

perfect! Thanks @maarten @Andriy Knysh (Cloud Posse) I will try the module and removing the backend directive

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i think you do terraform init on the S3 state first, then comment out the remote state, then do terraform init again - TF should offer you to import the state

maarten avatar
maarten

exactly

javier avatar
javier

I’m trying to import the actual s3 BUCKET to my tfstate using ‘terraform import’ and apply the changes if the state for these bucket was different to the config

javier avatar
javier

but I cant have good results… so I will try the module and removing the backend directive

javier avatar
javier

thanks again

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

take a look here, maybe will help https://docs.cloudposse.com/reference-architectures/cold-start/#provision-tfstate-backend-project-for-root (it’s tricky to provision the remote state and store the state in the remote state when you don’t have the remote state yet )

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

javier avatar
javier

Good! Thanks for your help

antonbabenko avatar
antonbabenko

You can use local-exec provisioner to execute sed, which will uncomment desired remote state section in tf file after null_resource is being executed for the first time, which does nothing… troll

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That’s an interesting idea

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

self mutilating code

antonbabenko avatar
antonbabenko

Yes, I did it couple times, mostly for fun. I also tried to solve limitations of 0.11 when trying to emulate dynamic/null-able arguments in resources.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think this could be where something like pulumi could shine

antonbabenko avatar
antonbabenko

Maybe, but I am still not convinced in giving it another try. I don’t have problems which can’t be solved in Terraform. Some of solutions are not very nice, but still…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re all betting on the terraform 0.12 “hail mary”

antonbabenko avatar
antonbabenko

ohh yeah. Maybe finally I will be able to make AWS S3 module which is as flexible as my ec2-security-group one.

antonbabenko avatar
antonbabenko

hehe, @Andriy Knysh (Cloud Posse) I see you are doing similar with just sed in the docs.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea, we used sed, but now have scripts that do all the steps automatically https://github.com/cloudposse/terraform-root-modules/tree/master/aws/tfstate-backend/scripts

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

:--1:1
nathan avatar
nathan

i’m having trouble using terraform to upgrade the engine version of an RDS instance that has a read replica

nathan avatar
nathan

a straight-up terraform apply fails because the read replica needs to be updated before the primary can be updated:

nathan avatar
nathan
[[staging.gladly.qa>] backing-services](http://staging\.gladly\.qa) terraform apply
...
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ module.database.aws_db_instance.default
      engine_version: "9.6.8" => "9.6.9"

  ~ module.database.aws_db_instance.read_replica
      engine_version: "9.6.8" => "9.6.9"


Plan: 0 to add, 2 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.database.aws_db_instance.default: Modifying... (ID: staging)
  engine_version: "9.6.8" => "9.6.9"

Error: Error applying plan:

1 error(s) occurred:

* module.database.aws_db_instance.default: 1 error(s) occurred:

* aws_db_instance.default: Error modifying DB Instance staging: DBUpgradeDependencyFailure: One or more of the DB Instance's read replicas need to be upgraded: staging-read-replica-1
	status code: 400, request id: f18110e4-1b27-4d1e-bb07-3e9591d1ddbf
nathan avatar
nathan

some googling indicated that i should target the read replica and upgrade it first before updating the primary, but when i try targetting just the read replica, it picks up the primary too:

nathan avatar
nathan
[[staging.gladly.qa>] backing-services](http://staging\.gladly\.qa) terraform apply -target=module.database.aws_db_instance.read_replica
...
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ module.database.aws_db_instance.default
      engine_version: "9.6.8" => "9.6.9"

  ~ module.database.aws_db_instance.read_replica
      engine_version: "9.6.8" => "9.6.9"


Plan: 0 to add, 2 to change, 0 to destroy.
nathan avatar
nathan

anyone know how to get around this?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

can use terraform plan/apply -target=...

Nikola Velkovski avatar
Nikola Velkovski

the only thing that I can think of is doing it manually via click ops. And then applying. I haven’t had problems like this before so maybe it is something in the modules that is causing this ?

1
nathan avatar
nathan

thanks, @Andriy Knysh (Cloud Posse), but when i try to -target the replica, it picks up the primary too

nathan avatar
nathan

thanks, @Nikola Velkovski i’m probably going to end up doing click ops for this

Nikola Velkovski avatar
Nikola Velkovski

also -target sometimes decides which other resources are chained together

Nikola Velkovski avatar
Nikola Velkovski

I mean it’s not -target per se but amazon’s api

nathan avatar
nathan

also, i like the term “click ops”… have never encountered it before, TIL!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ClickOps is a good addition to DevOps

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Command: taint - Terraform by HashiCorp

The terraform taint command manually marks a Terraform-managed resource as tainted, forcing it to be destroyed and recreated on the next apply.

Nikola Velkovski avatar
Nikola Velkovski

well, my life would have been better without knowing it

nathan avatar
nathan

it looks like taint will result in the replica db being destroyed and recreated which frightens me

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

but it’s just a replica, isn’t it?

Nikola Velkovski avatar
Nikola Velkovski

chaosops FTW

nathan avatar
nathan

it’s a read replica that our application actually uses for some read-only operations unfortunately

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

why not update both as the last TF plan shows?

nathan avatar
nathan

updating both fails with “Error modifying DB Instance staging: DBUpgradeDependencyFailure: One or more of the DB Instance’s read replicas need to be upgraded:…”

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ah i see sorry

nathan avatar
nathan

no problem - i appreciate everyone’s feedback and help!

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-ssm-tls-ssh-key-pair

Terraform module that provisions an SSH TLS Key pair and writes it to SSM Parameter Store - cloudposse/terraform-aws-ssm-tls-ssh-key-pair

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(this is useful for bot accounts)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

chamber read -q atlantis atlantis_ssh_key > /dev/shm/id_rsa

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

easily rotate SSH secrets by just tainting the resource

james avatar
james

what are the options for dealing with “default” resources, like a default route table and default network ACL?

james avatar
james

my understanding is that the terraform module aws_default_route_table makes it explicit that the resource can’t be destroyed, but otherwise using the regular aws_route_table would be fine

james avatar
james

is there some trap I’m setting up for myself if I take the latter option?

2019-01-09

Lukasz German avatar
Lukasz German

Hi guys, I’m new here. First of all I want to say ‘thanks’ for such a great collection of templates that you share. I’m building a stack for couple of small backend apps written in Django. My plan is to use this: https://github.com/cloudposse/terraform-aws-ecs-web-app but I want to reduce costs by not using public/private subnets and NAT gateways. Here are the questions:

  1. Is it possible to use this template with only 1 public subnet per app (I know that it’s not recommended, but in this case costs are much more important than security)?
  2. What other resources should I define? Can you provide an example on how to use this template?
cloudposse/terraform-aws-ecs-web-app

Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more. - cloudposse/terraform-aws-ecs-web-app

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hey @Lukasz German welcome

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-ecs-web-app

Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more. - cloudposse/terraform-aws-ecs-web-app

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-ecs-web-app

Terraform module that implements a web app on ECS and supports autoscaling, CI/CD, monitoring, ALB integration, and much more. - cloudposse/terraform-aws-ecs-web-app

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it says private_subnet_ids, but it’s just a name, you can definitely provide just one public subnet Id, for example:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

private_subnet_ids = ["subnet-XXXXXX"]

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-vpc

Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

to create subnets, you can use any of our modules https://github.com/cloudposse?utf8=%E2%9C%93&q=-subnets&type=&language=

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(it’s for EKS, but will be similar for ECS)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you can set this to false to disable NAT gateways and save on cost https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/main.tf#L58

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Lukasz German avatar
Lukasz German

Thanks @Andriy Knysh (Cloud Posse), that was very helpful. I was able to prepare some config and most of the things work, but I cannot enable CoePipeline. I think it might be related with the fact that ECS service is not created by default-backend-web-app. Moreover, when I try to execute terraform apply this error occurs: aws_security_group.ecs_service: Error creating Security Group: InvalidGroup.Duplicate: The security group 'test-prod-template' already exists for VPC 'vpc-0e35f5dde1e0df651' status code: 400, request id: 02565b24-ffd0-4f31-91d9-cdac82ba072e

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Lukasz German avatar
Lukasz German
12:05:37 PM

Here is the configuration that I’m trying to apply.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’m also working on an example as we speak

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Subnets are free so you can just create them in each AZ and use as needed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for ECS in terraform-root-modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(will be ready by EOW)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

NAT gateways are not free

midacts avatar
midacts

Looking at your .travis.yml files, how does travis know what to do for these steps? I am unable to find where these steps processes are defined: make terraform/install, make terraform/get-plugins, etc

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-vpc

Terraform Module that defines a VPC with public/private subnets across multiple AZs with Internet Gateways - cloudposse/terraform-aws-vpc

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

which uses build-harness which has all of that stuff

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/build-harness

Collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more - cloudposse/build-harness

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/build-harness

Collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more - cloudposse/build-harness

midacts avatar
midacts

I see. that makes more sense. Thanks for the information.

Gabe avatar

something for you guys to consider for the null label, two tags that i think are useful that i added to our implementation are last_updated_by and repo so we can see who the last person to apply terraform was and which repo the code lives in that maintains a resource

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s nice!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’ve also seen where people add a commit sha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(or could be a module version)

pecigonzalo avatar
pecigonzalo

The only downside with this is that you will always have terraform changes this way, which can get quite noisy

2019-01-08

Jan avatar
cloudposse/terraform-aws-iam-user

Terraform Module to provision a basic IAM user suitable for humans. - cloudposse/terraform-aws-iam-user

cloudposse/terraform-aws-ses-lambda-forwarder

This is a terraform module that creates an email forwarder using a combination of AWS SES and Lambda running the aws-lambda-ses-forwarder NPM module. - cloudposse/terraform-aws-ses-lambda-forwarder

Jan avatar

what is the currently supported method for creating IAM users?

joshmyers avatar
joshmyers

@Jan Don’t think I follow, supported method?

Jan avatar

how do I add IAM users

Jan avatar

that have keybase setup already

Jan avatar

been a long two days and im tired so im probably missing something

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Jan avatar

brilliant, thank you!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(Note that directory contains templates)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-null-smtp-mail

Terraform module to send transactional emails via an SMTP server (e.g. mailgun) - cloudposse/terraform-null-smtp-mail

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Gabe avatar

anyone aware of or use any tools to allow someone to require any version of a module as long as it’s not a major breaking change for terraform? for example in node i can require a version “^1.0.0” which will give me any 1.x.x version of a package but not 2.0.0

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh that would be nice

Gabe avatar

i think it might be hard to do because source strings can’t be variables and i’ve seen on their issues this will most likely never change

Gabe avatar

one thing we were thinking of was using tags similar to 1.x.x and 1.4.x in github and move them with each release

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhhhhhhhh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s a clever hack.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I don’t think I personally want to maintain it, but I like the thinking.

sarkis avatar
sarkis

i’ve been trying to motivate myself to write an app that solves the terraform module versioning issue - unfortunately, i’m not working with terraform much at the moment at the day job so it’s making it hard – tl;dr i’ve done a ton of searching and don’t know of any existing tool or workflow for this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

also want terraform landscape rewritten to go =P so I get a single binary

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I haven’t seen any tools for that.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The other tool I want is one for vendoring modules.

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and it would rewrite all source definitions to something like: ./vendor/github.com/cloudposse/terraform-aws-vpc

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(like go)

Gabe avatar

doesn’t it do something similar now with just very ugly names? by downloading all modules in .terraform/modules/sha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, more or less.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But I would never commit .terraform/modules to git

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(because it’s so ugly) haha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ugh. that’s a good point though. terraform get -backend=false is really a lot like go get

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

with terraform being written in go and their familiarity with go, I wonder why they chose an entirely different pattern from go, java, node, etc

antonbabenko avatar
antonbabenko

Hi guys! I’ve just published automated process to publish searchable PDFs for terraform core and all official providers, so we can work while flying and offline Here we go - https://github.com/antonbabenko/terraform-docs-as-pdf . Tooling may contain bugs, so please let me know by opening an issue, or submit PRs.

antonbabenko/terraform-docs-as-pdf

Complete Terraform documentation (core + all official providers) as PDF files. Updating nightly. - antonbabenko/terraform-docs-as-pdf

3
:--1:3
fiesta_parrot1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’d like to do this eventually for cloudposse/docs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s a nice readable format

antonbabenko avatar
antonbabenko

I can now update my linkedin profile and add ghostscript and wkhtmltopdf. Both are terribly slow and demand a lot of resources to run.

maarten avatar
maarten

wkhtmltopdf will for sure attract another batch of automated recruiters

:100:2
antonbabenko avatar
antonbabenko

I need moooore spam from recruiters

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Makes TF_DATA_DIR configurable via file. by RussellRollins · Pull Request #19066 · hashicorp/terraform

Addresses issue #3503 . This request was already possible using the TF_DATA_DIR environment variable, adds the ability to do so with the standard config file as well.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Upvote

:--1:1

2019-01-07

pecigonzalo avatar
pecigonzalo

Hello! I was just looking at the updates: https://github.com/cloudposse/terraform-root-modules/pull/83/files and I believe https://github.com/cloudposse/terraform-root-modules/pull/83/files#diff-25b6b1e862ed7056e75cc43421b112c0R123 should be SecureString instead of String, but I dont konw if there is some other intention behind not making it a SecureString

[backing-services] set defaults to support disablement by osterman · Pull Request #83 · cloudposse/terraform-root-modules

what Set defaults for all envs why Without defaults, it’s pretty moot to support enabled flags since you still need to define a lot of useless envs

[backing-services] set defaults to support disablement by osterman · Pull Request #83 · cloudposse/terraform-root-modules

what Set defaults for all envs why Without defaults, it’s pretty moot to support enabled flags since you still need to define a lot of useless envs

pecigonzalo avatar
pecigonzalo

BTW, I thought you used chamber to manage the secrets, do you just bootstrap them via terraform?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

nice catch @pecigonzalo, that should be SecureString

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

in the last releases, we added code to write secrets (and other settings) to SSM from all modules via Terraform so they could be used from other modules as needed - simplifies the bootstrap process

pecigonzalo avatar
pecigonzalo

Yeah I noticed that, I think its a great idea, and in most cases is not sensitive information, but I was wondering in the cases where it is, you also leave that info in the state, which I guess is fine since its encrypted in S3 (in most cases). Just wondering if that is your workflow, as we use a similar pattern and also use chamber

pecigonzalo avatar
pecigonzalo

but we write secrets only with chamber, to keep them out of TF

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea, it’s not ideal. but as you said, it’s encrypted in S3

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and it’s simplifies the cold-start/bootstrap process a lot

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

once all the secrets are written to SSM from TF, we also use chamber to read them when we deploy k8s services

:--1:1
praveen avatar
praveen

Hi

praveen avatar
praveen

I have a doubt on terraform configuration on azure platform

praveen avatar
praveen

can we use condition like count to make the argument extension as optional in vmss on azure platform using terraform configuration

praveen avatar
praveen

#terraform like the example as given below

praveen avatar
praveen

extension { count = “${var.enabled == “true” ?}” name = “setexecutionpolicy-${local.vm_name_prefix}” publisher = “Microsoft.Compute” type = “CustomScriptExtension” type_handler_version = “1.9”

settings = &lt;&lt;SETTINGS
  {
      "commandToExecute": "powershell.exe Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Force"
  }
SETTINGS   }
praveen avatar
praveen

#terraform can you use count to enable or disable the extension depending upon our requirement?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen i think not, count could be used only in resources count = "${var.enabled == "true" ? 1 : 0}"

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for what you want to achieve, you can use the slice pattern (as we call it) to add different values to a setting depending on diff conditions

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) got it. I am trying to create base module for windows vmss. For which I will have to include optional for enabling IIS on the base module. Is there a way I can do enable IIS optional when required

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you share some code, we can point you in the right direction

praveen avatar
praveen

I have azurerm_virtual_machine_extension which I will have to make it optional for vmss base module. As I understand I cannot use azurerm_virtual_machine_extension for azure vmss. for which I will have to use extension as an argument . Just wanted to know how can I make this configuration as optional (enable/disable)

praveen avatar
praveen

resource “azurerm_virtual_machine_extension” “iis” { count = “${var.iis_enabled == “true” ? var.av_set_size : 0}” name = “iis_${format(“${var.vm_name}%02d”,count.index+1)}” location = “${var.az_location}” resource_group_name = “${data.azurerm_resource_group.rg.name}” virtual_machine_name = “${format(“${var.vm_name}%02d”,count.index+1)}” depends_on = [“azurerm_virtual_machine.vm”] publisher = “Microsoft.Compute” tags = “${local.tags}” type = “CustomScriptExtension” type_handler_version = “1.9”

settings = <<SETTINGS { “commandToExecute”:”PowerShell -Command " Install-WindowsFeature -name web-server,Web-Default-Doc,Web-Http-Errors,Web-Http-Tracing,Web-Static-Content,Web-Http-Redirect,Web-Http-Logging,Web-Stat-Compression,Web-Dyn-Compression,Web-Filtering,Web-Net-Ext45,Web-Asp-Net45,Web-ISAPI-Ext,Web-ISAPI-Filter,Web-Mgmt-Console,Web-Scripting-Tools";PowerShell -Command " Remove-WindowsFeature -name Web-Dir-Browsing"; PowerShell -Command " Start-Sleep -Seconds 60; Get-Website | Remove-Website; Start-Sleep -Seconds 30; Get-IISAppPool | Remove-WebAppPool; Set-WebConfigurationProperty -Filter ‘/system.applicationHost/sites/siteDefaults/limits’ -PSPath IIS: -Name connectionTimeout -Value (New-TimeSpan -sec 30); Set-WebConfigurationProperty -Filter ‘/system.applicationHost/applicationPools/applicationPoolDefaults’ -PSPath IIS: -Name startMode -Value ‘AlwaysRunning’; Set-WebConfigurationProperty -Filter ‘/system.applicationHost/applicationPools/applicationPoolDefaults/processModel’ -PSPath IIS: -Name idleTimeout -Value ‘00:00:00’"; exit 0” } SETTINGS

praveen avatar
praveen

here is my extension which I use in windows vm configuration making it as optional using count

imiltchman avatar
imiltchman

How about having an extension that has an empty commandToExecute?

praveen avatar
praveen

and how would I injust the command to execute while sourcing the module

praveen avatar
praveen

do you have any example/reference for it

imiltchman avatar
imiltchman

define a local commandToExecute as ${var.enabled ? 'PowerShell -Command...' : 'echo DoNothing'}. Then use settings = ${local.commandToExecute}"

praveen avatar
praveen

May I know if you have an example or reference of this scenario

imiltchman avatar
imiltchman

I’m just thinking of a possible workaround. I don’t have any examples.

imiltchman avatar
imiltchman

Looks like the slice pattern example that @Andriy Knysh (Cloud Posse) provided is a similar idea

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen do you want to enable/disable the whole extension, or just change the command inside settings?

praveen avatar
praveen

whole extension

imiltchman avatar
imiltchman

Except as a block within https://www.terraform.io/docs/providers/azurerm/r/virtual_machine_scale_set.html, not as azurerm_virtual_machine_extension, if I understood correctly

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

in what code do you want to do it? (the second snippet above has it already resource "azurerm_virtual_machine_extension" "iis")

Steven avatar
Steven

Here’s something similar as an example: https://github.com/devops-workflow/terraform-aws-autoscaling/blob/master/main.tf Look at launch_configuration. It is defined inside the module, but allow it to be passed in instead.

devops-workflow/terraform-aws-autoscaling

Terraform module which creates Auto Scaling resources on AWS - devops-workflow/terraform-aws-autoscaling

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes this is a common pattern - if it’s not provided in the variables, then create a new one

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so this is not a problem. I thought @praveen wanted to add different config to extension { in the first code snippet depending on some condition

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) “azurerm_virtual_machine_extension” is a resource which I used for creating a standalone virtual machine. I understand that we cannot use “azurerm_virtual_machine_extension” resource for vmss . for which we may have to use argument as extension to configure any vm extensions on vmss

praveen avatar
praveen

as I will have to create generic module for windows vmss. am not finding opportunity to make iss enable optional for windows vmss module

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

share the code where you use this block

extension {
   count                = "${var.enabled == "true" ?}"
   name                 = "setexecutionpolicy-${local.vm_name_prefix}"
   publisher            = "Microsoft.Compute"
   type                 = "CustomScriptExtension"
   type_handler_version = "1.9"

   settings = <<SETTINGS
     {
         "commandToExecute": "powershell.exe Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Force"
     }
   SETTINGS
 }
praveen avatar
praveen

can you provide me your email address so that I can share the code with you

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen DM me

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) DM the code

praveen avatar
praveen

if not without using extension or with any other approach can we optionally enable ISS while sourcing windows vmss module

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i’ll take a look @praveen

praveen avatar
praveen

sure

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(the slice pattern is what could be used here)

:100:1
praveen avatar
praveen

can you give me example or snippet of slice pattern

imiltchman avatar
imiltchman

It’s in the Reddit post above @praveen

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Give me a few minutes, I’ll adapt it to your example

praveen avatar
praveen

sure @Andriy Knysh (Cloud Posse)

praveen avatar
praveen

checking Reddit post @imiltchman

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen here is a simplified version of using the slice pattern adjusted for your code

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
locals {
  extensions = [
      {
        name = "StaplesBGInfo"
      },
      {
        name = "setexecutionpolicy-${local.vm_name_prefix}"
      },
      {
        name = "IIS-${local.vm_name_prefix}"
      }
  ]

  # <https://www.terraform.io/docs/configuration/interpolation.html#slice-list-from-to->
  to_index         = "${var.iss_extention_enabled == "true" ? 3 : 2}"
  extensions_final = "${slice(local.extensions, 0, local.to_index)}"
}

resource "azurerm_virtual_machine_scale_set" "vmss" {
  extension = ["${local.extensions_final}"]
}
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if var.iss_extention_enabled == "true", then extensions_final will contain all three extensions

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if var.iss_extention_enabled == "false", then extensions_final will contain the first two extensions

praveen avatar
praveen

I didn;t understand this line # https://www.terraform.io/docs/configuration/interpolation.html#slice-list-from-to- to_index = “${var.iss_extention_enabled == “true” ? 3 : 2}” extensions_final = “${slice(local.extensions, 0, local.to_index)}” }

Interpolation Syntax - Terraform by HashiCorp

Embedded within strings in Terraform, whether you’re using the Terraform syntax or JSON syntax, you can interpolate other values into strings. These interpolations are wrapped in ${}, such as ${var.foo}.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it’s a comment

praveen avatar
praveen

Is it possible to update the code which I sent you so that I can better understand

praveen avatar
praveen

by making only IIS optional

praveen avatar
praveen

should my locals look like this?

praveen avatar
praveen

locals { count = “${var.create == “0” ? 0 : 1}” asgcount = “${length(var.asg_ids) > 0 ? 1 : 0}” vm_name_prefix = “${replace(var.resource_prefix,”/[_]/”,”-“)}” extensions = [ { name = “StaplesBGInfo” }, { name = “setexecutionpolicy-${local.vm_name_prefix}” }, { name = “IIS-${local.vm_name_prefix}” } ] to_index = “${var.iss_extention_enabled == “true” ? 3 : 2}” extensions_final = “${slice(local.extensions, 0, local.to_index)}” } }

Rob avatar

Hi folks. I have a question about cloudposse/terraform-aws-cloudfront-s3-cdn. Does anyone have experience with that?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

what’s the question @Rob?

Rob avatar

Hi @Andriy Knysh (Cloud Posse), I was successful in created an S3 bucket and a CloudFront distro using this module. However, I need to set some attributes such as making the bucket private and restricting the CF distro to only accept signed URLs. I cannot find anywhere how to do that.

Rob avatar

And my Google searches are not helping

imiltchman avatar
imiltchman

@Andriy Knysh (Cloud Posse) terraform blocks are just list variables? that’s so useful… and I can’t believe I didn’t pick up on that earlier.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes they are, very useful

:--1:1
imiltchman avatar
imiltchman

For resources that can be either created within parent blocks or as separate resources, is there one method that’s better than the other? For example ingress egress rules on aws_security_group

imiltchman avatar
imiltchman

Maybe the aws_security_group example isn’t a good one, as it looks like the two methods cannot be used together. Not sure if that holds true for other similar examples.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we did both, but ended up creating separate resources (not inline) especially for SGs

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for a few reasons

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. They are managed separately and could be changed separately
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. They could be provided from outside of the module, e.g. add an additional rule to the SG - TF will not try to delete/update the SG
imiltchman avatar
imiltchman

Thanks, that makes sense

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. They could be managed separately with a count (e.g. enable/disable) - useful in many cases, and not possible in inline blocks - this is probably the most important reason to use separate resources
loren avatar
loren

on the other hand, a reason to use inline rules is that they are enforced exclusively, meaning if someone goes and adds a new rule to the security group outside of terraform, the inline rule config will detect that and create a diff to force it back to exactly what is in the tf config…

loren avatar
loren

that behavior can be desirable, or not, depending on your use case

:100:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@praveen sent you example for your code

imiltchman avatar
imiltchman

(you’d think tags would have given it away, ha)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

can’t help you with Azure questions since I’m not familiar with it, so you have to take a look at the code and figure out the remaining issues (if any) by yourself

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

sorry @Rob, too much traffic here

Rob avatar

no worries

praveen avatar
praveen

@Andriy Knysh (Cloud Posse) Thank you very much . am going to test it now and let you know

:--1:1
Rob avatar

@Andriy Knysh (Cloud Posse), I have to get lunch so I’ll ping you later when I get back. Thanks for the offer of help.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Rob the bucket is already private https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/blob/master/main.tf#L56 (if that’s was your question)

cloudposse/terraform-aws-cloudfront-s3-cdn

Terraform module to easily provision CloudFront CDN backed by an S3 origin - cloudposse/terraform-aws-cloudfront-s3-cdn

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

signed URLs are currently not supported by the module. I guess this needs to be added https://www.terraform.io/docs/providers/aws/r/cloudfront_distribution.html#trusted_signers

AWS: cloudfront_distribution - Terraform by HashiCorp

Provides a CloudFront web distribution resource.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Rob if you test it and open a PR, we’ll review promptly

Rob avatar

thanks @Andriy Knysh (Cloud Posse), I’ll give it try shortly.

Rob avatar

@Andriy Knysh (Cloud Posse) I just checked the bucket that was created and the 4 permissions, including ACLs, are marked as False. Going to delete the CF distro and bucket and try again. Will let you know. Note this is in the Permissions tab and the Public access settings button.

Rob avatar

or are you suggesting that I modify the code and submit a PR? Maybe I didn’t understand.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Rob yes, for trusted_signers, would be great if you could test it and open a PR

Rob avatar

will do. thx

Rob avatar

@Andriy Knysh (Cloud Posse), waiting for the terraform destory to complete. The CF distros take a while to complete. I’ll get it done asap and if it works, submit a PR. Thanks for your help.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea, CF is painfully slow.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

especially when having used things like CloudFlare or Fastly

Rob avatar

yeah! it finished!

Rob avatar

do I need to create a fork in order to create a PR?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, fork it, then create a new branch, add your code, open a PR

Rob avatar

The code change worked and the distro was created with a trusted_signer of “self”

Rob avatar

thx

jober avatar
jober

Hi there! I am new to the community, is this the correct place to ask questions about the terraform modules?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yep, mostly about the cloudposse/terraform-* modules

jober avatar
jober

ok!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Terraform AWS modules

Collection of Terraform AWS modules supported by the community - Terraform AWS modules

jober avatar
jober

Maybe you have a quick insight for me.

jober avatar
jober
cloudposse/terraform-aws-elasticache-redis

Terraform module to provision an ElastiCache Redis Cluster - cloudposse/terraform-aws-elasticache-redis

jober avatar
jober

And I keep getting:

Error: module.redis.aws_elasticache_replication_group.default: "replication_group_id" must contain from 1 to 20 alphanumeric characters or hyphens
jober avatar
jober

I have tried manually applying a replication_group_id but it still throws

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrm….

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

can you share the group id you manually set?

jober avatar
jober

replication_group_id = "grizzly-redis-staging"

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

too long

jober avatar
jober

OO man

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
echo  -n grizzly-redis-staging | wc
       0       1      21
jober avatar
jober

i guess reading the error would help

jober avatar
jober

wow sorry

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(i know, it doesn’t look that long!!!)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

honestly, this just came up for someone else last week

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so that’s why it was fresh in my memory

jober avatar
jober

Thank you so much!!

jober avatar
jober

Awesome thank you so much!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

np! that’s what we’re here for…

jober avatar
jober

and thanks so much for all of the great modules!!

1
:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks! means a lot to hear it…

jober avatar
jober

The amount of time saved from these modules is unreal, it is very much appreciated!!!

Rob avatar

@Andriy Knysh (Cloud Posse) I created a PR.

Rob avatar

thanks again for your help

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

btw, if you haven’t already, please give us a ★ on any of our github projects that you’re using It helps us a lot!

jober avatar
jober

starred everyone I have used

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks so much!!

Rob avatar

will do. great stuff guys. Keep up the good work. Saves us a lot of time.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @Rob!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Rob can you add 'self' is acceptable. to the variable description and then rebuild README by executing these commands:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
make init
make readme/deps
make readme
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
(we don’t directly modify README.md, we modify README.yaml and [varaibles.tf> and <http://outputs.tf outputs.tf](http://varaibles.tf)` - then the README generator builds README.md)
Rob avatar

sure thing

Rob avatar

though I didn’t see anything in the yaml file to update. Am i missing something?

Rob avatar

done and pushed

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks @Rob

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if the list is empty, does it still work?

Rob avatar

I guess I should test that. oops. shame on me Will test and notify you when complete

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks

Rob avatar

@Andriy Knysh (Cloud Posse), yes, it works if you don’t specify trusted_signers and uses the default which is []

Rob avatar

gotta love terraform

1
Rob avatar

ah, I didn’t mean that as a slight against CP.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh haha emoji fail

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I just mean that terraform breaks my heart sometimes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for me, the biggest gripe is “count of cannot be computed”

Rob avatar

hahaha. that is annoying

Rob avatar

nothing is perfect

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

has anyone here taken a serious stab at using pulumi?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The way I get it, Pulumi is to Terraform what SASS is to CSS.

Rob avatar

interesting concept. First I’ve heard of it. I like the purple website lol

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-cloudfront-s3-cdn

Terraform module to easily provision CloudFront CDN backed by an S3 origin - cloudposse/terraform-aws-cloudfront-s3-cdn

Rob avatar

thanks @Andriy Knysh (Cloud Posse)

2019-01-04