#terraform (2020-06)

@RB For using ping I would assume.

ya but why would you need ping

its 2020. what use does this have ?

a monitor or something like that

remember you still use TCP/IP

FTP is still a thing

It should definitely be optional, but it is still a debugging tool.

Also the average joe is going to lean on ‘ping’ for “is this thing up”

What’s the error you’ve encountered?

Not really what you’re asking for, but just reminded me of @antonbabenko project to turn docs into PDFs https://github.com/antonbabenko/terraform-docs-as-pdf

ahh, that IS the project i was thinking of though

dang, too bad it’s not per-version

I’m late coming here, but I’ve also been asking for versioned docs since I’ve started here. Apparently it’s on the way, but that doesn’t help today. Unfortunately checking out the version in VCS is the best way to get to the version you want. Sorry.

thanks @Jake Lundberg (HashiCorp)

oh great! why did it ever be this way in the first place - it’s always been such a pain

sooooo many tools have problems with aws credentials, regardless of the sdk/language, i don’t really blame them

i’m just excited it’s been getting some good attention today

Had similar issue as well while trying to create config-map using kubectl command for aws-auth after eks cluster deployment . As you said, even I worked it around with couple of mins sleep using null_resource and looking for better solution. One thing I noticed is eks nodes (EC2 instance) which are getting deployed on VPC are not still green on status check. If it reaches green I could see that kubeconfig starts to work..maybe we need to find a way to verify the status check before firing kubectl command. Not able to get status check info of EC2 instance in Terraform or at least I dont know how to get it.

Replace

"Resource": [
                "arn:aws:s3:::artifacts/*",
                 "arn:aws:s3:::artifacts""
            ],

yeah, tried that, thanks. That ain’t it.

this is the OU id on organizations ?

anyhow you can’t do cp with this

you need Put* or PutObject

Hey, @jose.amengual. That’s the root (principal) org ID, redacted. I don’t use the OU id.

Here’s the really odd thing: I set up a policy that sets the Principal to the AWS account ARN of my test account. I gave it s3:* . At that point, I can do an ls on objects in the bucket, but cp does not work for read or write.

When I remove that account from Resources , i lose the ability to do an ls, which indicates that that policy is in effect for this role (BTW, I’m using an assumed role from from that same master account. I’m setting up an artifacts bucket that each account should be able to pull from.

is the bucket encrypted using KMS by any chance ?

yep. That occurred to me yesterday at some point…that i’d need to assign the same access to the key. that where you’re going?

or just disable that encryption…different type?

AES256 is safe enough and simpler

if you goal is encryption at rest

otherwise you will need to share CMKs

but with AES256, it will be transparent, with no policy fiddling?

if there is any object in the bucket and KMS is enabled using a aws managed kms key you will have the problems described

correct, it is transparent

it is server side

That’s great. Thanks so much!

np

Thanks again for the help. It works, after switching to AES256 and using this policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowOrgAccountsReadOnly",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:List*"
            ],
            "Resource": [
                "arn:aws:s3:::my-artifacts",
                "arn:aws:s3:::my-artifacts/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "o-xxxxxx"
                }
            }
        }
    ]
}

Oh that’s an interesting change

roll back fixed it? if so that’s probably still a bug

yeah, roll back fixed.

0.13 still beta, lets see how it behaves when the final version is released

We have all in one git repo (sre, providing a platform etc). We use layers of directories to limit blast radius, tfstate file per directory. This enables easy symlink for Makefiles and comparisons across environments or regions. All has same controls from git perspective. Simplifies CI.

Modules separate repo.

It depends… I’ve used a monorepo for all environment and module code, modules in different repos, modules vendored into the repo, a separate repo for everything with Terragrunt / Terraform.

In terms of blast radius I’ve worked with orgs that classed the environment as the blast radius in Terraform and orgs that take a more modular approach (layers).

I wouldn’t say that there is a magic repo layout that fits ALL situations. It depends on complexity, how much you share code across teams and the scope of the infrastructure you’re delivering.

modules in seperate repos and all different implementations in a single repo each in their own subdirectory. I use terraform workspace (per environment) so the monorepo is only nested one level deep directory-wise.

wow… quite a diversity of layouts!

I’m somewhat apprehensive of a monorepo… but I can see making things simpler and having tfstate per directory seems good. But the complexity of maintaining that seems high

I’m thinking from the perspective of an organization that is new to TF and IaaC in general… so I think separate repos per AWS account would be cleaner. This is assuming separate AWS account per app/environment. Also think using TF-workspaces would be key here too.

Have your seen Charity Majors blog post about separate tfstate? I feel those in this thread might be a little ahead of that but it’s a good read about the journey. Whilst we have separate tfstate per directory, it’s a given that it’s using remote backend like S3. For us it’s 3 layers (of directories).

1. AWS Account (or other)
2. Region (we also have a global region)
3. Services boundary (base, k8s, other ec2, rds etc, …) Variable and auto tfvars files at each level symlinked as necessary into 3.

Separate repo’s useful if merge master access control or CI differs significantly, else it’ll be a pain IMHO. I can’t speak to workspaces as haven’t used.

Hi @OliverS, what is it you want to achieve with this kind of tagging ?

I want to be able to look at a resource in AWS and know who created it and when, who updated it last and when, and what was git commit of the repo on which terraform apply was run. I rather not depend on aws Cloudtrail for that.

Hi @OliverS, did you try formatdate and https://www.terraform.io/docs/configuration/functions/timestamp.html ?

Problem with timestamp is that it will get a different value every time you terraform apply. This will update all resources that have var.tags not just the ones that are changing from the apply. You can’t configure tf to ignore tags because sometimes you really want to update them. In fact you want to update one of the tags on every resource that terraform is changing.

I want to be able to look at a resource in AWS and know who created it and when, who updated it last and when, and what was git commit of the repo on which terraform apply was run. I rather not depend on aws Cloudtrail for that. But why ? Things don’t break when resources are created but when they are deleted and this scenario is not captured ? If you want to have more control then have a CI/CD with git review setup.

data "null_data_source" "start_time" {
  inputs = {
    timestamp = timestamp()
  }
}

then reference with:

data.null_data_source.start_time.inputs.timestamp

parsing examples:

date_ymd = "${substr(data.null_data_source.start_time.inputs.timestamp, 0, 4)}${substr(data.null_data_source.start_time.inputs.timestamp, 5, 2)}${substr(data.null_data_source.start_time.inputs.timestamp, 8, 2)}" #equivalent of $(date +'%Y%m%d')
date_hm = "${substr(data.null_data_source.start_time.inputs.timestamp, 11, 2)}${substr(data.null_data_source.start_time.inputs.timestamp, 14, 2)}"                                                                     #equivalent of $(date +'%H%M')

one is a resource and the other is a module

read a bit more about modules and you will see the difference

Modules encapsulate not just one resource, usually modules create all resources necessary to run something, in the case of beanstalk there is a bunch of configs to get one app running

if you look at the module code and the tf files you will see many resources being created

oh okay, thanks, that clears up a lot

you have to perform the comparison for each value, when taking this approach…

name = var.branch == "master" || var.branch == "sandbox" || var.branch == "demo" ? "service-prod" : (var.branch == "staging" ? "service-staging" : "service-dev")

but you can also use contains() for your use case:

name = contains(["master", "sandbox", "demo"], var.branch) ? "service-prod" : (var.branch == "staging" ? "service-staging" : "service-dev")

or you can just use a map for a more declarative approach:

locals {
  name = {
    master  = "service-prod"
    sandbox = "service-prod"
    demo    = "service-prod"
    staging = "service-dev"
  }
}

name = local.name[var.branch]

oh that’s neat

Haven’t heard any reports of it

Yes you could construct a string with the efs_id and create an object in s3 with the information. That said, we have used EFS for years and never had to do that, so I think this might be a case of http://xyproblem.info/

There’s an open PR already https://github.com/cloudposse/terraform-aws-elasticsearch/pull/54

This is very exciting. We have many places where being able to loop modules will dramatically simplify our code.

ya totally

no kidding. can’t wait to see how moving loops out of the modules changes the patterns we use to address the “cannot be computed”/”value not known until apply” style of errors

googled “service catalog cloudformation terraform” and came up with this… https://aws-service-catalog-factory.readthedocs.io/en/latest/factory/terraform_support.html

also this, https://github.com/aws-samples/aws-service-catalog-terraform-reference-architecture

ahah, i was actually looking for this, remembered reading an aws blog… https://aws.amazon.com/blogs/apn/using-terraform-to-manage-aws-programmable-infrastructures/

https://archive.sweetops.com/terraform-0_12/2019/06/#b5370128-c12d-46ee-aff6-68d5a5258818

I was trying to find that exact chat

awesome thanks

but is there a preference over one or the other ?

that conversation does not really go over that much

Sounds like null-label is preferred again from that conversation and the terraform-label module is now just ‘supported’

Ya null label is back in favor - though I don’t think there’s anything null about it anymore

We should probably archive the other one to reduce confusion

Thanks guys, so my guess for Terraform 0.13.0 we will have something similar

something similar?

terraform-terraform-label was the first to be 0.12 compatible and terraform-null-label was not upgraded until other modules were upgraded, so the upgrade path for 0.13 I guess will be similar, a new terraform-terraform-label 0.13 will be created and then terraform-null will be upgraded etc…

that is history I have in my head about this

ohhhh I see now, I reread the old post , well I guess I have a lot of imagination

for some reason I thought that the upgrade to 0.12 was the reason

it was mostly about HCL1 -> HCL2 (not terraform 0.11 to 0.12)

so 0.13 is still HCL2, so I don’t imagine too much pain

I see ok, thanks for clarifying

using the wrong module mate https://github.com/cloudposse/terraform-aws-eks-cluster

lol

not the answer i was lookingfor but very well

was very happy with intelliJ extension, but recent update(s) really screwed the .tf files auto-formatting =(

Same, I’ve been using pycharm for Terraform development for about a year now

you can use the terraform output command to display the output key value available in the state

please provide more context

what kind of block?

Terragrunt generate can do this for you :)

but so can a for_each

below I would like to set site config = true only when tier = Free and Size = F1.

resource "azurerm_app_service_plan" "app_service_plan" {
  name                = "${var.prefix}-${local.environment}-service-plan"
  location            = azurerm_resource_group.web_app_services_rg.location
  resource_group_name = azurerm_resource_group.web_app_services_rg.name

  sku {
    tier = local.sku_tier
    size = local.sku_size
  }
  site_config {
    always_on = true
  }
}

haha, I just found out how to do it with a map. May be you have a better or more creative way to do it ? here i switch the variable value and not the block ..

variable "always_on_enabled" {
  type = map(string)
  default = {
    stage = true
    prod  = false
  }
}

locals {
  always_on   = "${lookup(var.always_on_enabled, var.env, "stage")}"
}

and having

  site_config {
    always_on = local.always_on
  }

Hmmm… That doesn’t look sensible to me… Sure it works but couldn’t it just be:

always_on = local.sku_tier == "Free" && local.sku_size == "F1" ? true : false

Which makes it clear that:
I would like to set site config = true only when tier = Free and Size = F1.

ha nice ! it looks far better like this thanks @Tim Birkett

give this a try? https://github.com/antonbabenko/terraform-cost-estimation

Yeah.. Saw that .. But not sure if that is accurate . Anyone here used it ?

it’s aws pricing. will never be accurate.

Also it does not cover all resources .. Only 5 resources according to docs..It does not cover RDS for example

Neat… live example here: https://hieven.github.io/terraform-visual/examples/aws-s3

@rahulm4444 do you mean you need to add another permission to the policy besides what we have here https://github.com/cloudposse/terraform-aws-eks-node-group/pull/13/files?file-filters%5B%5D=.md&file-filters%5B%5D=.tf

@Andriy Knysh (Cloud Posse) I am now able to add extra policies to EC2 role by using existing_workers_role_policy_arns = [“my policy”] existing_workers_role_policy_arns_count = 1 So this is working now..

One more help needed is to incorporate “asg lifecycle hooks”

Only after node group creation I am getting the asg name, so not able to incorporate asg lifecycle hook with this script.

Is there a way to predefine asg name during the time of node group creation?

we use the label module to define all the names for all resources

so the name should be known in advance

also, you can open a PR to add a variable to specify the ASG name

if the variable is not set (null), then fall back to the old functionality

PR created to add asg_name as variable https://github.com/cloudposse/terraform-aws-eks-node-group/pull/18

Please have a look

One more thing to note is, we can have predefined name for the node group created by this terraform. Issue is with naming of backend asg of that node group

That backend asg is not a resource, it is created as a part of node group

thanks for the PR @rahulm4444

I see you defined variable "asg_name" but it’s never used

Yes, I was trying that to incorporate with main.tf but I find out asg is not created as a resource. Node group is created as a resource and asg was created in its backend. And that asg is not defined anywhere in terraform..

Not sure this is possible I am now trying to use eks-workers terraform module

ah, yes

Managed Node Group creates the ASG for you

If it’s not possible u can close that PR I will raise a new PR against eks-workers terraform module for the same purpose..

(closed the PR, thanks)

I have created a PR to have custom asg name

https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/pull/26

Could you please have a look

you check issues and prs? https://github.com/hashicorp/vscode-terraform/issues?q=sort%3Aupdated-desc+

Yeah, had a look, but didn’t see much aside from people complaining. No official statement so far as I know

their announcement of the extension just says to file an issue

doesn’t mention any other avenue for discussions… maybe their discourse site tho

here’s the language server repo… https://github.com/hashicorp/terraform-ls/issues?q=sort%3Aupdated-desc+

no discussions on discourse… https://discuss.hashicorp.com/c/terraform-core/terraform-editor-integrations/46

there was some discussion about this in the hangops terraform channel ~3 days ago. nothing promising though: https://hangops.slack.com/archives/C0Z93TPFX/p1591973636201800

Surprised I wasn’t in that channel. Ah well, at least Hashicorp has to have heard somewhere that things are not in good shape

hello, on Vscode it seems that the plugin don’t find the terraform.exe file and giving the exact PATH don’t work

https://sweetops.slack.com/archives/CB6GHNLG0/p1592211857186400

It’s broken for me from the last update.

I had to switch back to using intellij , the tf plugin in vscode wasn’t working too well for me.

I remove new version and manually installed older one

on Vscode I am using the language support written by “Anton Kulikov” version V0.1.10 which support well 0.12 extension ID: 4ops.terraform

That’s intentional. Previously we ran an OOOOOOOOLD version terraform-docs and an awk script to rewrite HCLv2 to HCLv1 (this is before they added support). In the past week we fixed that.

See here for more context.

Ah good to know, thanks!

pretty sure a “computed” value is not known until after the apply… you can add an output for the value, then apply, then you’ll see the output…

@RB

no need for “create/enable” variables, and no need for lists of complex objects in order to create multiples of things. remove all associated junk from the module, and just count/for_each on the module itself when you call it. so easy!

plus the tests are simpler, because all that logic is gone and there are fewer edge cases of user input

Yes that is something i would love. However, 0.12 was a mess when it was released and 0.13 is not even out of beta test so I’m going to wait on this one

I have played with it and liked it so far. Maybe this release will be much better than 0.12

Yeah, same. I just was updating a module today to be less opinionated and create arbitrary numbers of things… Generally easy enough with for_each, but this module has a nested community module that doesn’t support multiples, and I didn’t want to rewrite that also, with module-level support around the corner! So, decided to play with 0.13 and get some experience with new patterns. Now, I’m super excited for the release!

This work will just sit in a branch until 0.13 is released and we decide to make a big push to convert things

Context. I am trying to replace a python version that is being used. However I have read some varying approaches. The workflow is. New image is tagged in ECR > then a new Task Definition is created > Services is then deployed with update Task definition.

Your workflow looks good, what’s your concerns or issues?

If you deploy from CI/CD pipelines, you could use aws cli, or third party tools like https://github.com/fabfuel/ecs-deploy or https://github.com/silinternational/ecs-deploy aws-cli and the first one from third-party are written in python so may not work for you and not very CI friendly as requires python runtime. The second is pure shell.

That is the general pattern. Our CI builds the images and pushes them to ECR. Then I have a Jenkins job that someone can run with the desired tag. It then gets the current task def, changes the image tag, creates new task def, and updates the service

Does your Terraform script do this @Steven or is that through aws cli?

Jenkin pipeline. Mix of AWS cli and groovy)

Has anyone used Terraform to do the updates?

Terraform manages the services, but not image deploys. I’ve had terraform do the deploys at a different job. That can work, but it is much slower

Good to know. Thanks @Steven

Just depends what your environment and release process is like. Where I am now, we deploy many services at once and do a number of environments around the same time. So speed was important

Speed is critical to us as well. It doesn’t sound like Terraform for deployments is the way to go then. Except to create the infra.

Solved it by cloning the repo and removing the module.

I had worked through this previously with output, but my notes don’t seem to be accurate with respect to a procedure for running within .module, with proper envvars set

Hello David, you need to use the depends_on to create resource sequentially https://www.terraform.io/docs/configuration/resources.html#depends_on-explicit-resource-dependencies

to get information on the eip created, the documentation chapter “attributes references” list the variables that are exposed after the resource is created

https://www.terraform.io/docs/providers/aws/d/eip.html

How and where does the local exec get called?

local exec run from your machine remote exec run on the server

My understanding is this timeline:

1. create instance
2. remote exec - install python
3. local exec - run ansible playbook
4. create and attach eip With the above time, when the ansible playbook is executed the EIP does not exist.

I am considering using a load balancer which will get created first so the ansible playbook can know about it.

(also consider using the terraform-ansible-provider )

also I have found this video helpfull specially at minutes 12:56 which is on your topic : https://www.hashicorp.com/resources/ansible-terraform-better-together

You can download it from here

https://releases.hashicorp.com/terraform/0.13.0-beta2/

that’s what i did, though beta2 seems to have introduced some regressions in the init code. i went back to beta1

seems something to do with the new provider/registry logic… https://github.com/hashicorp/terraform/issues/25282

TL;DR I have some state in a S3 backend, and some local state saved alongside .tf files. How do I separate for now then merge later ?

Background: I inherited a TF setup that was done a year or so ago. Its setup to use a S3 backend Currently it contains a bunch of company wide stuff and some random S3 buckets etc.. also in there is the tf for a couple fargate clusters, and a redshift DB that are all unrelated to each other.

Additionally I have a new fargate cluster setup along with a couple S3 hosted websites, all with local terraform.tfstate files.

Plans: I would like to keep all of the company wide things in its own github repo, and terraform thats related to all these apps inside the repo’s of the respective projects they belong to.. generally in a .terraform directory.

Step 1. break out all the stuff thats co-mingled in the S3 backend and put it into the project repos it belongs to. and get it all up to data nd working with local terraform.tfstate files

Step 2. merge them back in a way with either a S3 backend, or terraform cloud so I can only push changes and make updates to project level things

This is so that a developer can make changes to a projects resources, but not see other projects or have the ability to change other projects.

would appreciate and tips / suggestions !

Our approach with has been to centralize all terraform code into a single repository that we (Ops/DevOps) control. This allows us to have a single source of truth for all deployed infrastructure.

To limit a developer’s ability to modify the overall infrastructure, you could look into Terraform Cloud. We use it and the paid version has the ability to require approvals before changes are deployed.

Is spreading out the across the projects where it belongs an anti-pattern ?

Perhaps… I think there are pros and cons both ways.

But having all infrastructure code in one place makes management of the infrastructure simpler.

You can allow folks to make changes to the TF code, but require code review/approval in TF Cloud

I really like the idea of keeping it localized inside the projects repo. we have very little “overarching” code that all projects use.

plus it allows devs of that project to see ( and modify ) the resources for only that project, and not get bogged down wading thru thousands of lines of other stuff for other projects.

Yeah… I can see the appeal

We have a small devops team (me) and almost no one else knows how to use TF.

Most of our apps are just dockerized and deployed via a TF module…so each app is encapsulated by 1 file generally in the main TF repo

so right now in the S3 backend I have code that provisions a redshift DB. How can I yank that out of that backend and put it into local tfstate files ?

while leaving the other stuff in there in tact

I would try to import the resources

import into local tfstate, then how to remove it from the other backend without deleting it ?

humm I see the issue

so you want to move the resources to a new TF repo without deleting them?

Maybe try terraform state rm and import into the other repo. https://www.terraform.io/docs/commands/state/rm.html#<i class="em em-~"</i>text=The%20terraform%20state%20rm%20command,%2C%20entire%20modules%2C%20and%20more>.

Sounds dreadfully manual

But probably the only way to do it

yeah, its a bit of a cleanup project.

so yeah, this worked pretty well.. installed tfenv for switching back and forth between versions, did all the imports on the new one first, then state rm them from the old backend and its all sorted out.

to answer a point about terraform project. I use multi repos and a remote tfstate for each . then I use data terraform_remote_state to have include state from needed repos.

The pros are:

• access and change are limited to the cloned repos

• state are splitted so user using different repos can run terraform plan in parallel The cons are:

• take care to output required value in each repos

• smaller tfstate, quicker terraform plan execution For exampel I have repos per project: network , core, kubernetes cluster, web … ( one per “product” or perimeter )

also if you have remote backend, switching to local backend will copy it .. then you can change things and push on an other backend.

awesome ! thanks @Pierre-Yves this sounds exactly how I want to organize everything.

You are absolutely right! Thanks! How did I miss that?

Ya, we have that issue too

Kind of given up on finding a fix. Closest we’ve come is using symlinks

Mmh, not cool

Thanks for the quick answer

Ya, it’s annoying that terraform output doesn’t behave like all the other commands

for this reason, we had to give up on using terraform init -from-module=... directly, but using something like #terragrunt will work around it

Will give a look to terragrunt, thanks!

Right :smile: Had no time to look more into this from now.. Best I can say is using sparse=0 , which is definitively not ideal

ok - ya, seems more like a git problem

@Jeremy G (Cloud Posse)

If one of you find time to look into this I’d be glad to merge a fix

@Erik Osterman (Cloud Posse) Not sure what you are asking. Between the problem statement and the further details in the comments, the Git issue explains pretty much everything.

More of an FYI

@zadkiel We are having another problem with helm-git when we run helmfile with helm v2, we get

Error: unknown shorthand flag: 'c' in -c

Which most likely comes from helmfile auto-detection of helm3. Do you have a workaround for that?

Works with helm-git version 0.4.2 but not 0.7.0

Looks like the problem is https://github.com/ms32035/helm-git/blob/fd7f37634d3d1d0d964a37653a9ef7f2ad92ecd2/helm-git-plugin.sh#L30

@mumoshu :point_up: This is an interaction among helmfile, helm diff, and helm-git. Seems something sets HELM_BIN to helm diff

Can you have a try with the last release?

https://github.com/aslafy-z/helm-git/releases/tag/v0.8.0

@Jeremy G (Cloud Posse)

@zadkiel This is going to be problematic for us as we use helm2 and helm3 for Helm binaries so we can have both versions installed at the same time.

I suggest you look for an environment variable called HELM_GIT_HELM_BIN and use that if set:

1. If HELM_GIT_HELM_BIN set HELM_BIN="$HELM_GIT_HELM_BIN"
2. Check HELM_BIN for sanity, use it if it passes tests. Include in sanity tests that helm version -c returns without an error, although that could be a bit tricky because you will want a portable way to enforce a timeout on that command, since it appears the terraform provider causes a hang.
3. Set HELM_BIN to “helm” if the sanity check fails

We’ve been using Terraform Cloud (free-tier) linked to GitLab. Works well for us.

nice! Seems they both have encryption, locking, and remote execution, wondering what the upside of GitLab managed is – just one less 3rd party service integration to maintain?

I think the argument for using GitLab would be total control. In our case, we were looking to have a more managed service that let us get up and going quickly. GitLab option would be more work upfront but may give you greater ability to really fine tune things.

I’ve been using it for a few projects hosted in GitLab that were previously using TF Cloud and S3 backends. It works fine with no fuss. You have to use a PAT to interact with the backend, though.

This is neat - didn’t know gitlab offered this. Fits nicely into their devops positioning.

From the looks of it, seems you’d need to create individual gitlab projects/subprojects for each TF state file you’d init? (Not simply a matter of specifying a backend “key”)?

I believe you’re correct, @vicken - one state file per project. If your project structure aligns to that, it should work well. If you’re doing a monorepo style of coding, though, it may not be the right solution.

thanks for clarifying!

Actually, I’m wrong. The projects do support multiple state files. The documentation is just behind on that. You just change the name of the file path after /terraform/state/ Issue: https://gitlab.com/gitlab-org/gitlab/-/issues/220559 Docs MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34797

Ah nice find! cutting edge stuff

Gitlab says the state is encrypted and I imagine the durability is good - does anyone have concerns with storing Terraform State in Gitlab as opposed to say s3 + dynamodb?

Does the typical “TACOS” store the state files / provide options to store in your cloud provider account of choice?

Some questions came up regarding the loss or compromise of the statefile by the provider within our team regarding storing statefiles with Gitlab - I reached out to our TAM to get more clarification on durability and security guarantees.

Thoughts @Erik Osterman (Cloud Posse) ?

eek. I think we have a few template_file references

https://github.com/search?l=HCL&q=org%3Acloudposse+template_file&type=Code

no bueno

i expect the provider will continue to be available, but i’d definitely not use it in new work, and move away from it in anything that i’m touching…

worst case, keep a copy of the binary handy and drop it alongside the terraform binary when you need it?

yeah; was just saying to myself that this is one of most fast moving GH issues I’ve ever seen!

We’ve started pinning providers as part of upgrading everything to HCL2.

https://github.com/cloudposse/terraform-aws-s3-bucket/blob/master/versions.tf#L4-L8

example

but we’re using ~> so it’s not too strict.

Guess that would help in this situation.

~> is not a pin, it is a pessimistic that has an upper bound

FWIW, I use ~> too

I wouldn’t make this issue the reason to start pinning the AWS provider. It is generally extremely stable. The recommended practice is to use the pessimistic so that you never jump to another major version.

provider "aws" {
  version = "~> 2.67.0"
}

Is functionally identical to version = ">= 2.67.0, < 3.0.0"

we also use ~>, but we also use terraform-bundle to pre-stage the providers, so we only use versions we’ve actually run through tests with the bundle

@Robert Horrox just pointed out to me something that’s going to be a big problem we need to address ASAP in anticipation of 0.13.
All cloud posse mods will fail to init
Makes testing 0.13 really hard

Here I was thinking 0.13 was no big deal.

what ?

What’s going to cause them to fail?

In the versions.tf there is a constraint for 0.12

~> 0.12

Ah. That’s an intentional thing though right? Same as they were from 11 to 12. Each one needs to be tested to see if it causes any problems on 13, and if not the constraint would just be updated

I dont see that as a problem, just a step that needs to be taken in order to support 13.

Yeah, seems legit. Luckily, if we do we want to upgrade that across all the repos it’d be easy to automate that.

It makes perfect sense, unfortunately there is no way to override the version constraint. I tried override.tf and it didn’t work

there comes the 200 PRs…..

@Erik Osterman (Cloud Posse) maybe you can cut a 0.13 branch for testing on the repos

Or I have a bad idea that no one would like

Not even I like

ya, maybe we need to do the 0.13/master branches like we did for 0.11/master

did you try to use two $$ ?

I haven’t yet @Andriy Knysh (Cloud Posse), how would I do that?

$$LATEST - maybe try that

Ahh ok. thanks @Andriy Knysh (Cloud Posse) I will give it a go. Because CloudFront needs a version number e.g. 1 when I reference the lambda in cloudfront config it errors.

The other workaround is to create an alias with a specific version number.

But that is still manual if I want to bump the version

let me know if it fixes it. We had something similar in other resources (not related to CF and Lamnda@Edge), but something to check

Managed to find a solution. I needed to have the publish attribute on my lambda_resource which then would publish the new lambda and give it a version.

Is there a difference between the role that it creates and the AWS provided role ?

If there isn’t a difference, we could omit the creation of that role, use a data source to retrieve the AWS one, and save ourselves a service role per task.

ticket: https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/issues/60

I think (not sure) that AWS only creates the AWSServiceRoleForECS role when you create your first cluster. So it wouldn’t be guaranteer to be around when the modules tries to associate it.

ah interesting

i also realized that if youre using fargate this service is no longer created

cause fargate requires awsvpc network mode

(this goes beyond my knowledge of ECS - so maybe double check in #terraform or other trusted contributor) =0

but this does not work

since it produces

an error

so I tried join

join(", ", aws_iam_role.AuroraIamAuth[*].arn)

but that does not work because it will look like this

This is a guess… have you tried: concat([aws_iam_role.AuroraAccessToDataBuckets.arn], aws_iam_role.AuroraIamAuth.*.arn)

mmmm let me try that

that did it

thanks a lot

Guesses FTW!

Use the aws ami data source

Thanks.

It depends… You can try Terraforms path.root, path.cwd etc… https://www.terraform.io/docs/configuration/expressions.html#references-to-named-values

If you want to detect the dir where the terragrunt.hcl is, you’ve got Terragrunts get_terragrunt_dir() https://terragrunt.gruntwork.io/docs/reference/built-in-functions/#get_terragrunt_dir

No I want to detect dir where terragrunt is run

like:

/
/projects/project1/terragrunt.hcl
/templates/terragrunt.hcl

, and I am running terragrunt from root path with --terragrunt-working-dir /projects/project1/terragrunt.hcl

and terragrunt.hcl should contain something like include = $pwd/templates/terragrunt.hcl

interesting… Have you tried anything yet?

it looks I have to convert a terraform template to json before parsing it terraform show -json /tmp/plan.out > /tmp/plan.out.js then I can run terraform-compliance -f compliance -p /tmp/plan.out.js and failed with any version with an error

which tool do you use to check your terraform code ?

Sorry just saw this. terraform-compliance doesn’t use any terraform unless you are using the docker image. Apart from that, if you already have a terraform executable within your disk, you can use -t parameter to pass that terraform executable.

Apart from those, the only reason why it needs terraform is to convert the plan.out to a plan.out.json . If you already convert that somehow, you can directly use the plan.out.json file with -p which won’t check for any terraform executable.

Currently terraform-compliance only support 0.12.*, we didn’t test with 0.13 yet, whenever its released, we will also add support that version asap.

I need to check again, terraform-compliance :)

it’s the name of the branch

see this example https://github.com/cloudposse/terraform-aws-jenkins/blob/master/examples/complete/fixtures.us-east-2.tfvars#L45

ty!

thread start

im trying to create a nice checklist so i can follow a plan that’s been vetted. so far ive been thinking of following the cloudposse method. except we dont use github actions.

plan so far

• .github for pull request templates • docs for module docs • examples for full examples • test for tests that run using bats • .precommit to use anton’s precommit hooks • then the standard terraform files main, outputs, variables, and versions

Does this change if I am using AWS Organizations features together with these services? E.g., if I create aws_guardduty_organization_configuration { auto_enable = true } in every region in the guardduty master-account, can I skip creating aws_guardduty_detector in all of the member accounts?

for sensitive random values please use random_password.
Identical to random_string with the exception that the result is treated as sensitive and, thus, not displayed in console output.

There’s a sensitive flag on outputs, but I don’t think that will prevent it from showing up in a plan, so I’m not sure what use it is.

just found the random_password

this was not an issue until we started working with Atlantis

thanks guys

@David Medinets Instead of input variables, which accept just static input, please use locals for “run-time” variables: https://www.terraform.io/docs/configuration/locals.html

Its stored in the state, you reference by the name of the resource

ie in the docs it shows `

  password = random_string.password.result

If you mean why isn’t there a ‘data’ lookup for it, its because this isn’t backed by some external resource such as AWS SecretsManager.

Thanks. I did not read carefully enough to see the ..result was the password.

did you set the var name correctly in the template, centos_password vs centos_user_password?

also, i would recommend using templatefile() instead of template_file, since the latter has been archived and at some point will probably be deprecated, https://github.com/hashicorp/terraform-provider-template/issues/85

That’s just embarrassing. Thanks.

eh, it’s easy to miss. it gets easier to spot when you’ve done the exact same thing yourself before, mebbe several times…….

I watched this day of cause the title got me. Honestly, didn’t think it gave too much concrete information. Just their philosophy. Which was cool, but not sure if it was what I was looking for haha.

I’m not familiar with ansible, so I don’t know what the password_hash function does, but there is a sha512 function… https://www.terraform.io/docs/configuration/functions/sha512.html

How come, it is only after I ask the questions that I realize the answer? Push the clear password into ansible and hash the password in the playbook. The terraform side only needs the clear text. password: "{{ centos_user_password | password_hash('sha512') }}"

i like the idea, but maybe a “control variable” or even the entire “control expression” instead of specifically enable/disable. not sure how to differentiate whether it is a single enable/disable variable, or a more complex expression consisting of multiple variables/locals that influence the count

this is always a good module for getting a feel for how complex a count expression can be… https://github.com/terraform-aws-modules/terraform-aws-vpc/blob/master/main.tf#L932

Perhaps if it has a count expression would be enough to presume that it may have logic to enable it

Hey @David J. M. Karlsen — Contributor team is in talks about supporting 0.13 still but no final plan yet right now. I think it’ll likely be tackled at some point in the coming weeks across all the repos as we’ve discussed just updating the terraform_version constraint to be >= 0.12 && <= 0.14, but haven’t confirmed how we’re going to accomplish it.

I’d say submit a PR that updates that constraint and we can chat through it there if you’re only looking to do this for one module.

I think you want this: https://github.com/cloudposse/terraform-example-module/blob/master/versions.tf#L2

thanks, https://github.com/cloudposse/terraform-aws-ecr/pull/52

@Josh Duffney woot woot

I remember seeing those functions but haven’t tried them yet. Thanks for the reminder.

Also why after there so many spelling errors in that post. Even tho it’s a great point to use these functions, the errors distract from this point.

great thanks @Matt Gowie for sharing your work , I was looking for a way to do variables validation !

hello @Haroon Rasheed, have you look at https://www.terraform.io/docs/providers/time/r/sleep.html ?

Looks interesting let me check it out..Thanks mate!!

It expects a list, you gave it a map

Just add [ ] around the map and I think you’re ok

Watch out on that selector, if you have tons of activities with S3 you co be looking at a big bill for cloud trail

I did this last year and we had to pay 3.5k for one month of cloud trail

@Zach did u meant it like this…?

No it needs to be a list with that map in it

@jose.amengual Thanks for the heads up!

presumably the module is using this as a quick selector where if the list is empty it doesn’t create that block on the resource

ahh Okay… Got it Thanks!! @Zach

Just one more thing… How do u manage Insights events using terraform while creating cloudtrail

any particular question? we do it, but we have to run dependabot with a custom patch with support for tf 0.12

easy enough with a github action… https://github.com/patrickjahns/dependabot-terraform-action

We use https://github.com/renovatebot/renovate to update our TF dependencies

@loren where do you store your modules…? Ours are in TFE… To my understanding, the private module registries are not (yet at least) supported - only (public?) GitHub repository module sources…

@Tyrone Meijn any issues…? Are you able to handle all of them, like, core version bumps, provider bumps and module bumps with Renovatebot?

using dependabot via the github action should work for private modules also

Ok thanks @loren - I’ll test this

Very frustrating - that was deprecated in one release, on a minor bump

we’re playing wackamole trying to keep tests stable

Yeah, understood. Apologies - I think one of the original breakages was my change. But this is a bit disappointing for the official provider, IMO

I was just looking before you posted this, it is pretty disappointing and it was not documented in the provider docs

Not even in the changelog

nope

FYI, I tried submitting an upstream issue with TF, but I can’t find the magic formula to let me submit the issue (submit is disabled).

This issue is definitely doing its best to make me feel dumb

lol

their variable descriptions look similar

only difference that i can see is that the environment is shown before the stage according to the label order

just choose one and leave the other undefined

exactly, very similar. hence my confusion!

A label follows the following convention: {namespace}-{environment}-{stage}-{name}-{attributes}. The delimiter (e.g. -) is interchangeable. The label items are all optional. So if you prefer the term stage to environment you can exclude environment and the label id will look like {namespace}-{stage}-{name}-{attributes}. If attributes are excluded but stage and environment are included, id will look like {namespace}-{environment}-{stage}-{name}

from the readme

looks like stage came first and then environment

but yea, looks like you can provide either or so if you dont like one, keep it undefined

good catch!

thanks!

ya, it was really to appease more people

our usage of stage is pretty opinionated. so adding environment appeals to more! now, IMO a stage might have multiple environments, so this is a nice way to disambiguate.

note, not all of our modules have been updated to support it. if you find it lacking, PR it and we’ll promptly approve and merge.

ooo thats good to know. i havent adopted null label but if/when i do, i’ll use stage to prevent limitations

only trough the console…I wonder how that will work

what’s the best way for us to monitor limits? have you seen a lambda for this? the one by AWS labs requires trusted advisor

I’ve used trusted advisor and cloud custodian for this

We recently moved to aws service limit checker which does this in a smoother way

this is the lambda by aws labs?

Yes

haven’t tried it but it looks useful. id love to use it but i think CICD is a prerequisite for this so it would be interesting to see it with atlantis.

just found this: https://marcyoung.us/post/atlantis-opa/

https://marcyoung.us/post/atlantis-opa/

Ah, yup, doing something similar