#terraform (2020-08)

Added PR: https://github.com/cloudposse/terraform-aws-rds-cluster/pull/74

Our plan is:

1. version in tf - no surprises
2. disable auto updates - no surprises - PR added
3. apply immediately false - do when ready via maintenance window/console/target

we do this manually and the we update terraform so it matches the version

I have not found a clean way to do this without taking down the cluster cleanly or getting a timeout

released #74 as 0.29.0

Thanks PePe. Thanks Erik for the quick turnaround!

Indeed. So my latest attempt following this: https://github.com/terraform-providers/terraform-provider-aws/issues/9401#issuecomment-551350474

With this change to instance in module:

# engine_version = "" # commented out, leave versioning in cluster only

lifecycle {
    create_before_destroy = true
    ignore_changes        = [engine_version]
  }

Then version bump in TF then terraform apply with:

1. apply_immediately = false « resulted in no change, or even pending maintenance in Console
2. apply_immediately = true (cluster only, not instance) « resulted in no change I think?
3. apply_immediately = true (cluster & instances) << See below…

Test results:

1. upgrade to cluster happening straight away in place
2. upgrade to instances happening straight away in place without creating new instances
3. ~23m taken to complete according to Terraform
4. ~10s shutdown and restart events in Console. << presumably hard outage
5. No failover events in Console - total cluster outage?
6. versioning in Terraform matching AWS Console

To be confirmed:

1. actual app downtime during this - reboots are at unknown/unscheduled time, didn’t have any apps connected to RDS.
2. whether this is still worth doing rather than just updating manually in Console or splitting TF module

Overall I think this might be sanest way to do in place upgrades with module as is.

Curious whether we:

1. We PR this, or
2. Give up and we split out of using module and try upgrading instances one at a time. We already need to blue green from Aurora MySQL 5.6 to 5.7 as that can’t be done in place. However I’d like to have a plan about future version bumps.

Hey team, we’ve reached a consensus that internally we are going to continue with the RDS module modified per above so we can do in-place minor version upgrades of our Aurora MySQL clusters.

Question I have is whether this makes sense PR’d back to the upstream module or not.

• Because regular RDS and Postgres support major version upgrades and it’s only Aurora MySQL that (currently) doesn’t the changes may not make sense.

• I don’t have capacity to test such changes with the other database types. Can only confirm good for Aurora MySQL and thus doing related logic in the module seems overly complex.

I guess ie better to leave the complexity out of the module

but maybe adding something to the docs maybe a good idea?

@Andriy Knysh (Cloud Posse) can you evaluate this when you have a chance? No rush.

Thanks everyone. It’s only a small diff and we’ll still be rebasing on and contrib upstream as go.

thanks for undestanding… 0.13 is taking all our spare time right now

You’re welcome. Very grateful to you and team with all the modules. Cheers

what AWS resources are you terraforming?

Hello @Andriy Knysh (Cloud Posse), Sorry for the late reply! EC2 Instances resource

take a look at these modules, they all related to EC2 https://github.com/cloudposse?q=ec2&type=&language=

each module has a working example in the examples/complete folder https://github.com/cloudposse/terraform-aws-ec2-instance/tree/master/examples/complete

https://github.com/cloudposse/terraform-aws-ec2-instance-group/tree/master/examples/complete

Thanks @Andriy Knysh (Cloud Posse) Very helpful!

i don’t think there is yet much in the way of an api for aws sso, for terraform to interact with… https://docs.aws.amazon.com/singlesignon/latest/PortalAPIReference/API_GetRoleCredentials.html

this issue appears to discuss the same problem… https://github.com/terraform-providers/terraform-provider-aws/issues/13755

i had tried to use aws sso previously, but found permission sets far too limiting, compared to managing policies on roles directly… posted a couple times to the aws sso forum, but haven’t come up with a viable approach yet…

• https://forums.aws.amazon.com/thread.jspa?threadID=312303&tstart=0

• https://forums.aws.amazon.com/thread.jspa?threadID=282793&tstart=0

@loren Awesome — you confirmed my findings / fear. Thanks man!

I see this is NOT yet possible: https://github.com/aws/containers-roadmap/issues/89

i’ve often seen folks use the exact <region> for the directory name, as it’s more of a unique key in the hierarchy… e.g. eu-central-1, because there are multiple aws regions in europe. of course, you can also get around that with multiple providers in a main.tf…

yep ok, makes sense, thanks

If helps, also I would agree with @loren

im currently creating a similar project.. however gone down the terragrunt route.. this is how my repo is current structured:

├── non-prod
│   ├── account.hcl
│   └── amer
│       └── us-west1
│           ├── mgmt
│           │   ├── env.hcl
│           │   ├── networking
│           │   │   ├── firewall_rules
│           │   │   │   └── ingress
│           │   │   │       └── terragrunt.hcl
│           │   │   ├── subnetworks
│           │   │   │   ├── subnet_data
│           │   │   │   │   └── terragrunt.hcl
│           │   │   │   ├── subnet_dev
│           │   │   │   │   └── terragrunt.hcl
│           │   │   │   ├── subnet_internaldmz
│           │   │   │   │   └── terragrunt.hcl
│           │   │   │   ├── subnet_services
│           │   │   │   │   └── terragrunt.hcl
│           │   │   │   └── subnet_stage
│           │   │   │       └── terragrunt.hcl
│           │   │   └── vpc
│           │   │       └── terragrunt.hcl
│           │   └── security
│           └── region.hcl
└── terragrunt.hcl

because of the way vars are passed in, if there is a requirement to go into other regions will simply be a case of copying the directory and updating a few vars.. as it follows the DRY code practices helps reduced repeated code.

this then pulls in the required modules

this is obviously at the start of the project… 2 days into the first sprint so will grow out over the next week or so with additional resources…

aws eks update-kubeconfig

https://docs.aws.amazon.com/cli/latest/reference/eks/update-kubeconfig.html

oh wow, nice

I have manually solved with:

terragrunt import --terragrunt-iam-role "arn:..." module.eks_cluster.kubernetes_config_map.aws_auth_ignore_changes[0] kube-system/aws-auth

Then running terragrunt apply again o/

This should not happen on a clean EKS deployment (from scratch). If you’re upgrading an existing EKS cluster from an older version of the module, I would expect that to happen.

Our automated tests run on this module all the time and do not encounter this error.

Will let you know on the case this happens again. But basically it was a clean EKS deployment.

@Andriy Knysh (Cloud Posse) any thoughts on what’s going on?

will deploy this module again in a few to see if will explode

yes I know, 1 sec

@ayr-ton you need to add the same data source as here https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/main.tf#L71

and cluster name should be from that data source https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/main.tf#L91

ahh, that’s interesting because I did this yesterday when I was removing the fargate profile

that will eliminate a race condition

and then I didn’t saw this error

We’re in syntony, nice.

Thanks!

same thing for fargate profile https://github.com/cloudposse/terraform-aws-eks-fargate-profile/blob/master/examples/complete/main.tf#L98

Sounds like a conversation I was having this morning with a pal of mine - he was talking about using null_resources with triggers to handle that, maybe? https://www.terraform.io/docs/provisioners/null_resource.html

But it used to work before ?

Or at least the leafover replicaset was downscaled to zero

Ah - probably not then!

Now the containers stays up

https://github.com/hashicorp/terraform-provider-kubernetes/issues/944

Over the past few weeks, i’ve had to describe pods or namespaces, update the JSON to remove the finalizers, and push that change back. not sure if a) this is asking for trouble at some point, or b) if there’s a better way…or if it’s directly related to your issue, but it’s helped me out quite a few times, recently.

Ya, this was the state of it last year when we looked into it. The next best thing we found was this: https://github.com/eeg3/workspaces-portal

(CFT)

….that could be deployed with terraform

loads for me

Load perfectly for me, try clearing your cookies, as there could be a stale one

Thanks Tom & RB, still cannot figure out the issue, I have tried clearing cookies and tried in private browsers still no use and weirdly even i cant access using mobile network . and got this message when seen in Chrome developer tools

Failed to find a valid digest in the 'integrity' attribute for resource '<https://registry.terraform.io/assets/terraform-registry-3c3897b8880537ab9759d2e91a1a39c5.js>' with computed SHA-256 integrity 'NL3YiUcpnfSMYU99vstLHDhVzYi63JZfABW7NIQVZmQ='. The resource has been blocked.

le me know if i miss anything

• try firefox or another browser

• create a new profile in chrome and try that

if you’re on a vpn, try toggling it

I am not in VPN, will try firefox.

Its still the same .

can you try a vpn if you have one? if not, you can use free protonvpn. see if that works.

brew cask install protonvpn

if that works, then I’d run a traceroute to the link on and off the vpn and diff them side by side

see where the connection is failing. it’s possible that an ISP has blocked your IP address.

will try in VPN and about IP Address Block – i wonder how they managed to even block my mobile network.

I can access from Safari Now! .

<<EOF ? i’ve only seen it with no space

or <<-EOF to allow indentation of the heredoc

also, I recommend avoiding all forms of HEREDOC and sticking it in a file and reading that file in.

e.g. I rather edit a shell script that I can run locally during development, rather than editing a shell script inside of terraform that I have to deploy with every change.

definitely find it easier to manage userdata as a file, though if vars need to be passed from terraform, then you’re probably using templatefile() and running that locally probably won’t work anyway

haha, possibly

but still, yeah, set the template values at the top of script, then at least can swap them out easily-ish

i hope this helps

resource "aws_emr_instance_group" "task" {
  cluster_id     = aws_emr_cluster.this.id
  instance_count = var.task_instance_count
  bid_price      = var.bid_price
  instance_type  = var.task_instance_type
  name           = "${var.cluster_name}-task-grp"

  ebs_config {
    size                 = var.core_volume_size
    type                 = var.core_volume_type
    volumes_per_instance = var.volumes_per_instance
  }

  autoscaling_policy = data.template_file.task_autoscaling_policy.rendered

}

data "template_file" "task_autoscaling_policy" {
  template = file("${path.module}/templates/autoscaling_policy.json.tpl")

  vars = {
    min_capacity = var.task_instance_count_min
    max_capacity = var.task_instance_count_max
  }
}

done it for emr

I’ll check it out. Thanks!

maybe you ran out of space on your laptop?

Yes unknown issue that sorted with system restart , but not out of space. thanks

what terraform version are you using? according to this it looks like it might be fixed with the latest 0.12

https://github.com/terraform-providers/terraform-provider-aws/issues/4709#issuecomment-453554068

% terraform version
Terraform v0.12.26

Thanks for Info on the BUG RB

Thanks!

I’ve got a ticket to update my tags to allow for both a standard set of tags as well as resource-specific set. Goes something like this, using @RB’s local.tags, above:

    tags = merge(local.tags,
                 {"Name" = "Resource Name}
                )

See recording: https://cloudposse.wistia.com/medias/9d4ase4qjy

I see self_link in the docs, not self_url?

Sorry if I wasn’t clear. Neither self_url, nor self_link work. Pycharm doesn’t see any of the documented attributes for auto-completion, either. It feels like i’m using a different kubernetes_namespace, but i don’t think that’s even possible. My kubernetes provider is pinned to “~> 1.12”.

if the attribute isn’t there, all i can think of is that it is null due to a count/for_each issue… the error there looks to be from an output, and outputs do not support count/for_each. so it is not recommended to use indexing of an “optional” resource… instead use the old “join” trick

output = join("", kubernetes_namespace.borrower.*.self_link)

I created an output of kubernetes_namespace.datadog and got this, which shows that self_link is an attribute of the metadata attribute, not a direct attribute of the namesapce:

test = {
  "id" = "datadog"
  "metadata" = [
    {
      "annotations" = {}
      "generate_name" = ""
      "generation" = 0
      "labels" = {}
      "name" = "datadog"
      "resource_version" = "637"
      "self_link" = "/api/v1/namespaces/datadog"
      "uid" = "1c61de6b-9dc3-4ee7-9b71-b85a78a54655"
    },
  ]
}

In any case, self_link doesn’t give me what I wanted.

the question remains, how do i get the URL of the ELB of the EKS clusters’ nginx-ingress

shoot, was hoping by resolving the tf issue that would get you there. i don’t know EKS well enough to help with that part… maybe ask in #kubernetes ?

I don’t think there’s anyplace to get that URL, other than using a data block, like this:

data "kubernetes_service" "nginx_ingress" {...

and then referencing it like this:

data "aws_elb" "nginx-ingress" {
  name = split("-", data.kubernetes_service.nginx_ingress.load_balancer_ingress[0].hostname)[0]
}

The only problem with this way – the way we’ve been doing it - is that – and I recall someone else getting unclean plans, because TF was reading some data blocks, even though nothing had actually changed.

So, pivoting from finding another way to get the ELB for nginx-ingress running on an EKS cluster, to finding out how to do this, so that you get clean plans, rather than the following, is the goal:

Understanding that it has to run, before it gets the info for the load balancer, how do i set this up to get a clean plan? Every time, it recreates the aws_route_53 records for those API servers and the aws_lb_ssl_negotiation_policy. I need this to create a clean plan.

i presume the elb is created from a different terraform config? can you just feed in the elb name as a variable, instead of a data source?

Hi @Almog Cohen if you’re looking at it in detail with the intention of using it, it’s a good opportunity for a PR!

If the infrastructure is not already in terraform, then using terraform to “patch” things in AWS won’t really work well. E.g. terraform cannot remove something it didn’t provision.

Should I import the managed policy first in each IAM role?

Yea you would have to first import what’s there into state using terraform import and then change it

That process should be run carefully and you should run a plan until it’s a noop and then make your changes

humn..I will remove that policy manually and add the correct ones using TF

Yep that’s def the easier option

cloudposse has a module for creating roles https://github.com/cloudposse/terraform-aws-iam-role

thanks!

Look at Cloudsplaining to help fix this.

https://github.com/salesforce/cloudsplaining

cool! didn’t know that!

then either bridgecrew/airIAM or duo-labs/Parliament from duo-labs to fix your IAM configs going forward

cool never heard of parliament, Im a check that out

this comes in handy https://kapeli.com/dash

terraform docset does exist even tho it’s not listed on their website

`nice

I feel that. I’ve been using the beta / rc versions on a few projects pointed at test regions that deploy in parallel with production. Haven’t hit any stumbling blocks yet, but bake time is always good.

I gave it a try and was getting some really weird errors and rolled back haha

locals {
  services = ["a", "b", "c"]
}

resource "aws_instance" "this" {
  for_each = toset(local.services)
  ...
}

This will create aws_instance.this["a"] aws_instance.this["b"] aws_instance.this["c"]

thank for your reply. unfortunately, it’s not the case I want. It’s not replicated resources. Each resources is different depend case. like iam policy, each service will have it own. And the condition to trigger creation follow what I described.

maybe you are using the wrong type of variable, try to use a hash not a list

Checking it out now.

@David J. M. Karlsen Shipped! https://github.com/cloudposse/terraform-aws-ecr/releases/tag/0.23.0

awesome - thanks!

Thanks @Matt Gowie

Seems we need https://github.com/cloudposse/terraform-aws-iam-user/issues/6 too

is this module still maintained?

I guess it stopped at https://github.com/cloudposse/terraform-aws-iam-user/pull/3

Ah yeah @David J. M. Karlsen… that module is a good bit outdated. 12.x branch but no tests which is why it isn’t merged. We should get that back into the fold as I’ve definitely used the terraform-aws-modules equivalent in the past instead of using this. To provide consistency to the community and my own future tf codebases I’d like to fix that.

Seems like it’d be an easy modules to add tests to honestly. Unless you’re interested in tackling it, I can add that to my queue for when I have a spare hour or so.

I switched to vanilla aws_* resources in the mean time, which seems to have me covered, I need some tweaking anyways. Thanks for responding though!

I think I saw this in other modules and we wrapped element on coalesense?

this is not going to work

even if the module is set to false it will try to create the resource

PR comming

I assume it might be related to a size thing, that is size of terraform repos, or amount of resources under management

I don’t think there’s a black/white rule on this. Off the cuff, here’s my recommendation:

• Use SSM for sharing values across toolchains (e.g. #terraform and helmfile) or where you need to access the values outside of terraform

• Use remote state between terraform projects. E.g. everything you provision that you are in control over.

• Use data sources for things which you might not have control over but depend on.

Thanks @Erik Osterman (Cloud Posse), this pretty much aligns with at least my personal opinion. Though i never thought about the first bullet, that is a really cool idea

a side benefit of using SSM is the permissions can be controlled with more granularity than tfstate, when needing to control access to specific values

I’m literally poc’ing them this week

I’ve had success with combo of cpu/mem/request count for the container metrics, and leveraging cps for the instance scaling

Granted all against a crappy node/express app I mocked up with faker

And I just crush it with artillery

I’m just started playing with it but it complains that I have no instances in my capacity group

New or existing cluster?

It doesn’t play well with an existing cluster, but after cycling instances in the asg, it worked

I deleted the ask and service, I will play with it a bit more

So it looks like you can NOT create a capacity provider on a service

oh, apologies i misunderstood, no the cp is on the cluster

you need to create it and attach it to the cluster and then attach it to the ecs service

that tracks the capacity cpu/mem of the instances in the asg

no worries, the consolke showed some weird empty selector and then I was wondering

so I just added in TF to the cluster and it worked

funny that TF let’s you do it and shows broken in the console

well the aws API let you do it

could you put the pem in an s3 bucket and use a data source to bring it down in order to feed it into the module ?

Yeah definitely. Sadly I was just trying to make it work so that I didn’t need to pre-generate it or set up something like that.

I tried:

# If keypair's public key exists then no need to generate the key again.
  generate_ssh_key = fileexists("../pub_keys/${var.project}-${var.environment}-keypair.pub") ? false : true

But then it ends up deleting the pub key on the 2nd apply which I don’t want.

Going to put it down for now unless somebody shouts in here: “Here’s the way to do it!”

Parameter Store

we use that for keys , licenses etc

my Social insurance number etc

I lol’d at that dude, well done.

I created and documented a manual process for now. My team can deal.

Use our SSM module instead

https://github.com/cloudposse/terraform-aws-ssm-tls-ssh-key-pair

Unfortunately, this is not the popular module but it’s the superior one.

Hahah I’m the last contributor

Didn’t even know about this. Did the most recent release as part of the ChatOps mass-update.

hey…you are the last contributor on all the modules mister microplane

next time use my username

Ya, @Matt Gowie is now probably the #1 contributor in terms of PRs

haha

I need more dark green in my github activity graph

Yeah, I don’t mind my “74 contributions on Aug. 5th 2020” day.

Anyway — Thanks for pointing that out @Erik Osterman (Cloud Posse). Will definitely check that!

I do not care

Haha @jose.amengual you can do the microplane update for bumping the version of terraform-null-label across all the repos. That’ll leave you with some nice dark green in your graph!

HAHAHAHAHA

no worries , if someone ask who is this @Matt Gowie I will say is a machine bot user

Look, I even documented it for you!

https://docs.cloudposse.com/community/contributor-tips/

following it now for my own stuff at work

Ok. Got it. I could have just referenced the larger data structure everywhere in the module call, using each.value as the key, but this worked nicely and got rid of a lot of text.

Given a map of definitions (local.monitor_defs), keyed on name, and a list of keys (local.monitor_def_keys), the following gives just the entries from the map that correspond to the specified keys:

  md = {
    for key, value in local.monitor_defs: key => value if contains(local.monitor_def_keys, key)
  }

@Eric Berg, what version of TF is that?

that was 0.12.29, but we’ve upgraded most things to 0.13.2.

Can you link a reference you used to figure out the above? I can’t quite tell how you solved the issue – what is the structure of monitor_def_keys ?

I don’t have the refs I used to figure this out anymore, @Jaeson. monitor_key_defs is just a list of strings, representing keys in the monitor_defs map. Here, I’m returning (key, value) for each entry in monitor_defs for which there is an entry in monitor_def_keys.

https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html

plus proper use of the Resource: block

I find those so confusing … its an IAM policy that governs another IAM policy that governs a role/user

right?

permission boundaries set the absolute maximum limit of what the entity is able to do. Even if someone screws up and gives them every permission under the sun the permission boundary will still block anything that isn’t allowed in it

They’re a big pain IMO, but yeah permission boundaries is the way I’ve dealt with this in the past as well.

Ok thanks for confirming I find this aspect of IAM very confusing but my junior guy says he’s got a good grasp of it

Also use branch protection and required reviews, and when running in a pull request use a credential that is limited to read-only. plan will work, but not anything that actually makes changes… Gives you the ability to inspect and approve, before doing anything crazy

and on prem runners are now supported

(announced yesterday)

Whoah

iirc, due to cloudfront limitations, the cert has to be created in us-east-1. If your provider is defaulted to another region, you have to create another provider and use that one specifically for us-east-1, and I don’t think cloudposse repos generally pin providers?

cert has to be created and verified before cloudfront will work

i created mine outside of the module using a targeted apply, then verified the record, then applied the module which takes the resource acm as an input argument

if you reuse an acm cert, you can use a data source instead

This works if you want to do it as part of one pipeline, you can just depend on both of the validations:

resource "aws_route53_zone" "delegate" {
  name = var.domain
}

resource "aws_acm_certificate" "cert" {
  provider = aws.us-east-1 # Forced to use us-east-1 due to cloudfront limitations

  domain_name       = var.domain
  subject_alternative_names = ["*.${var.domain}"]
  validation_method = "DNS"

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_route53_record" "cert_validation" {
  # It took me TWO HOURS to figure out that Terraform converted this from a list to a set in 0.13, grrrr!
  name    = element(tolist(aws_acm_certificate.cert.domain_validation_options), 0).resource_record_name
  type    = element(tolist(aws_acm_certificate.cert.domain_validation_options), 0).resource_record_type
  zone_id = aws_route53_zone.delegate.zone_id
  records = [element(tolist(aws_acm_certificate.cert.domain_validation_options), 0).resource_record_value]
  ttl     = 60
}

resource "aws_acm_certificate_validation" "cert" {
  count = 0
  provider = aws.us-east-1

  certificate_arn         = aws_acm_certificate.cert.arn
  validation_record_fqdns = [aws_route53_record.cert_validation.fqdn]
}

(note specifically the provider us-east-1 alias)

The “grrrr” comment is

Haha, came across that one today, can you tell?

Omg I ran into same issue probably took me about the same amount of time to figure out

I finally looked over the change log and found it

It makes like, literally no difference operationally, it’s JUST there to break random tf files

My coworker said to pin aws version at 2 but I was determined to upgrade to 3

ah i was not aware of the aws_acm_certificate_validation resource. very cool

and good to know abotu the aws 3.x provider with that breaking change. we’re not currently pinning but will be on the lookout for that same issue

I ended up figuring it out by looking at the state file, and nestled in between a bunch of lists was a lonely set…

I’m still pessimistically pinning the AWS provider to avoid a 3.0 surprise.

iirc, changing to a set was necessary because the aws api returns it as a set, which meant the order was constantly changing. order matters in a list, so when using multiple SANs and dealing with multiple validation records as a result, the constantly changing order would cause perpetual diffs in a plan

Ah, that’s a fairly good reason, I rescind my comment about it being pointless then

And also, a record with the domain and its wildcard subdomain use the same validation record… With a list, there would be a duplicate entry, but with a set all the duplicates get removed automatically

Ran into some of these “fun” problems working this module… https://www.github.com/plus3it/terraform-aws-tardigrade-acm/tree/master/main.tf

here’s the reference to the change in the upgrade guide, https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-3-upgrade#domain_validation_options-changed-from-list-to-set

Thanks for the advice . Got it all Terraformed now (with Terragrunt). Also noticed Erik has an issue to update the docs to use their ACM module: https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/issues/26

I guess you could do something like run the test twice in parallel stages, once for each version?

But at some point they diverge. What happens if it passes 0.13 and fails 0.12?

If it’s a contributor, so we push back and ask them to fix it? Make it work for both? What if it can’t work for both?

Well that’s the decision, right? To use tf 0.13 features, or not

The ability to do for_each and count on modules is likely going to simplify and reduce a lot of code in the cloudposse repos. I’d really prefer not to disallow 0.13

Personally I’d just cutoff 0.12 and go all in on 0.13

The cloudposse repos all need to work on 0.13 before that’s possible though

I’d enforce a minimum version known to be required based on features used, e.g. >= not ~=. Enforcing the upper bound is too much, too hard

The pinned versions for commonly used modules is a big issue tbh. For example, almost every single repo is currently broken (on 0.13) due to being pinned at <=0.16.0 of null-label, which is pinned to tf 0.12

@loren I think you’re right. mea culpa on this one. At least for for the terraform core version, I think we should only use a minimnum version because upgrading across minor versions is basically impossible in the current setup. the other challenge is our tests for examples/complete usually pull in many other modules to bring up a stack. so even simple modules with only 1 dependency, can have many dependencies in the examples.

@Makeshift (Connor Bell) so we’re using git ref sources, so it’s technically ==

I think it’s time to switch our modules away from git ref sources too and use module registry sources

@Andriy Knysh (Cloud Posse)

@Matt Gowie

i’m still wary of moving away from git refs, just because they are more portable and easier to override… maybe with the new registry features i’ll be able to set aside that concern, but i’d need to get some experience with it first

Hm, how do releases work on the registry? Do they just copy directly from github releases?

you register the module with the tf registry, and then any git tags are published as a version in the registry

all automatic at that point, no further interaction required

I guess the only difference is the syntax looks slightly cleaner then?

you also can use all the version constraints on the version field…

https://www.terraform.io/docs/configuration/modules.html#module-versions

Something to keep in mind is that large-reaching changes eg. microplane may be (slightly) more difficult using that method, since the sed find/replace will be a slightly more pain in the backside multiline

Huh, switching to the terraform registry would be interesting. I could see the benefit for hurdles like this where git refs are causing us to do manual updates and using the module version input would provide us a cleaner way to say “use any upcoming release”. But that might mean that our always increasing minor version could break module dependencies out in the wild pretty easily. Or are we thinking we would want to only allow patch version increases for module deps?

@Erik Osterman (Cloud Posse) I wonder if we would want to do a test run or two with a couple popular modules to see how that would turn out?

Test infra has been updated. Unfortunately, the scope of this change is only now apparent to me. See the thread here: https://sweetops.slack.com/archives/CB6GHNLG0/p1597383937062900?thread_ts=1597346608.048900&cid=CB6GHNLG0

I’ll put a plan together tomorrow, but we’re likely going to need some help opening PRs to expedite this upgrade. Not sure how much of this we can automate.

I’m not sure if this is neccessarily a good idea, but Terragrunt can generate remote state configuration files when running. It may be possible to make branch-specific remote states.

Sounds like it would work… Let me throw that to my lead and hear what thinks. Thank you

Alternatively, if this is for testing anyway, I would spin up a new stack per developer when needed with a completely fresh state. This does assume your TF is modularised enough to be able to spin up test instances without costing a fortune though

You could use a different s3 key backend

Or you could merge the branches and both of you can work off the same branch

Merging the branches is super unlikely because we work at different paces, different timezones…

I like the idea of having branch-specific state files better

I wonder if it would be possible to merge state files like you merge branches

That sounds… unpleasant

What? The different paces of work or merging state files?

Merging of state files. I think you’d be better off doing development in entirely separate states, merging the TF changes into a master branch, then deploying that to the master state file

Ya different states make more sense to me too. Seems cleaner. If you ever want to combine, you can always reimport resources from another state and then remove that state.

thread

what i’ve done is separate all the in-line to separate aws_route resources and then imported each aws_route from the aws_route_table

I use chamber but keep it in AWS only.

@Andy I’ve done purely storing in PStore before. I’m also more recently using Mozilla Sops + the Sops provider alongside that.

I’m actually in the middle of doing this as well and interested to find out what people recommend. Right now because my network team oversees more than one client/envioronments… I created a repo infra-networking that is a single workspace vs one per environment - dev,stage,prod because the same core team would likely be in control of all of that and it doesn’t change as often as the application or operations code does

if you have the dns staff infra-networking repo managed by the network team, what you can do is to use data source resources to query that resources and use them in your environments

How did your project go with this? Any lessons learned you would mind sharing with me?

We had to put this on hold to setup some azure k8’s clusters using tf. But if anyone else has success please share.

The entire cloud posse ecosystem of modules uses the terraform-null-label module to enforce consistency

that module normalizes everything

thanks for letting me ill follow the convention

Are you sure it’s TRACE-level output and not tf dumping state?

It might be, indeed. So initially we faced an issue with TF locks during terraform plan, but there couldn’t be any other pipeline or person triggering the same terraform apply. The error was:

2020/08/18 22:12:14 [TRACE] backend/local: requesting state manager for workspace "my-workspace"
2020/08/18 22:12:15 [TRACE] backend/local: requesting state lock for workspace "my-workspace"
o:Acquiring state lock. This may take a few moments...
e:
Error: Error locking state: Error acquiring the state lock: writing "<gs://my-bucket/terraform/my-workspace.tflock>" failed: googleapi: Error 412: Precondition Failed, conditionNotMet
Lock Info:
  ID:        1597788628081747
  Path:      <gs://my-bucket/terraform/my-workspace.tflock>
  Operation: OperationTypePlan
  Who:       runner@runner-urx-q8js-project-102-concurrent-0nbfb2
  Version:   0.12.24
  Created:   2020-08-18 22:10:27.940237341 +0000 UTC
  Info:      
Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"
flag, but this is not recommended.

Note the TRACE output, there are other TRACE, DEBUG and INFO lines prior to this.

If we do terraform plan -lock=false it shows tons of debug output like this:

...
2020-08-18T17:35:05.431Z [DEBUG] plugin.terraform-provider-helmfile:   labels:
2020-08-18T17:35:05.431Z [DEBUG] plugin.terraform-provider-helmfile:     app: prometheus-operator-prometheus
2020-08-18T17:35:05.431Z [DEBUG] plugin.terraform-provider-helmfile:     
2020-08-18T17:35:05.431Z [DEBUG] plugin.terraform-provider-helmfile:     chart: prometheus-operator-8.7.0
2020-08-18T17:35:05.431Z [DEBUG] plugin.terraform-provider-helmfile:     release: "prom"
2020-08-18T17:35:05.431Z [DEBUG] plugin.terraform-provider-helmfile:     heritage: "Helm"
2020-08-18T17:35:05.432Z [DEBUG] plugin.terraform-provider-helmfile: roleRef:
...

And it takes ages to proceed and usually fails afterwards without noticeable error.

But sometimes, quite rarerly now, it works ok, it shows normal output. We can’t figure out what was the reason. We started getting the same thing on another environment where there were no modification of tf resources, no tools were updated or something.

What a cryptic case.

Filed an issue in helmfile-provider repo

Thanks!

Sorry! missed this. will add to agenda next week.

I wouldn’t add it There really is no better way. But hey, it might lead to an interesting discusion

oh, haha thought I was in the #office-hours channel

What I ended up doing is using templatefile() + AWS SSM and overriding the entrypoint to the container.

I’d rather not change the app or modify the code / image so this was the easiest way.

secrets = [
  # Hacky hack to get the config file in the Fargate container
  #  see <https://github.com/aws/containers-roadmap/issues/56>
  # TODO: move this to Secrets Manager for extra 2KB size
  {
    name      = "ENCODED_CONFIG"
    valueFrom = aws_ssm_parameter.config.arn
  },
  {
    name      = "ENCODED_RULES"
    valueFrom = aws_ssm_parameter.rules.arn
  },
]

entrypoint = [
  "bash",
  "-c",
  "set -ueo pipefail; unset AWS_CONTAINER_CREDENTIALS_RELATIVE_URI; unset AWS_EXECUTION_ENV; mkdir /etc/samproxy; echo $ENCODED_CONFIG | base64 -d > /etc/samproxy/samproxy.toml; echo $ENCODED_RULES | base64 -d > /etc/samproxy/rules.toml; /usr/bin/samproxy"
]

Published: https://github.com/Vlaaaaaaad/terraform-aws-fargate-samproxy/blob/f38420934172f59dfac58a0b8a00150e288c9e91/samproxy.tf#L49-L78

Posting the link here in case any poor souls find this reference

Added to list for next week

oh, haha thought I was in the #office-hours channel

I don’t know about actual limits but I think people tend to keep their state files small to reduce the blast radius and make it not unbearably slow to run (big state file = lots of resources = lots of API calls = slow).

Agree with @randomy - but will also bring up next week

oh, haha thought I was in the #office-hours channel

On the other hand, it is pretty nice when you can run terraform and it handles everything. Fewer chicken-and-egg situations, no need to update various dependent stacks afterwards.

@randomy ya i’m torn on it

Multiple projects is ideal for reducing blast radius, but it complicates cold-starts and moves the responsibility of managing the complete DAG to you.

Using a single project is great because it’s all handled for you, but plans/apply can take hours. Using -target is not ideal either because you need to know which things to target or risk inconsistent state as well.

I’ve been working with CodePipeline lately to deploy to multiple AWS accounts. It’s pretty tempting to make a single TF stack that creates resources in every target environment/account + pipeline resources. It’s got a fairly small blast radius but still mixes nonprod with prod. I’m writing the module example this way and mostly prefer it over the actual implementation that involves separate stacks per account/environment using remote states etc.

To add context, this is an auto scaling group in multiple environments + 2 pipelines to deploy new AMIs and app artifacts to them. So it would be a stack per “service” but it covers all environments. It is pretty quick to run TF so the dilemma is mostly around blast radius/separation. In my current case the ASG is stateless (web servers) so I’m comfortable with it. Would be less comfortable otherwise.

Bringing this back to the original question, I struggle to decide on how to slice and dice state files but it’s always a long way off hitting 4mb.

The terraform-aws-ecs-web-app is designed to be an opinionated module that shows how to use all the other modules together. Since we write very small composable modules, this lets you pick and choose the best pieces. If the web-app module doesn’t fit that use-case, check out the main.tf file and see how it uses the other modules. Rip out what you don’t need.

But as @RB points out, there might be an easier alternative.

This looks like a good conclusion.

i tried at the time (also stated it in the original message), that did not work

setting the repo_owner is not the same as setting the codepipeline_repo_owner

I only see one reference to repo_owner in the example main.tf

repo_owner                           = var.codepipeline_repo_owner

https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/master/variables.tf#L645

have you tried setting repo_owner = "hashicorp" without codepipeline ?

hm i see

ok, i’ll try it out

yeah, it is not working

Warning: Value for undeclared variable

The root module does not declare a variable named "repo_owner" but a value was
found in file "eng/gp-view.tfvars". To use this value, add a "variable" block
to the configuration.

Using a variables file to set an undeclared variable is deprecated and will
become an error in a future release. If you wish to provide certain "global"
settings to all configurations in your organization, use TF_VAR_...
environment variables to set these instead.


Error: If `anonymous` is false, `token` is required.

  on .terraform/modules/ecs_web_app.ecs_codepipeline.github_webhooks/main.tf line 7, in provider "github":
   7: provider "github" {

I went to the [main.tf](http://main.tf) in master, and that corresponds to

https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/master/main.tf#L175

which is the same in the example as

https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/master/examples/complete/main.tf#L138

My source is terraform-aws-ecs-web-app.git?ref=tags/0.39.1 btw

try planning this entire block

module "ecs_web_app" {
  source = "git::<https://github.com/cloudposse/terraform-aws-ecs-web-app.git?ref=tags/0.36.0>"

  name                                            = local.name
  vpc_id                                          = local.vpc_id
  alb_ingress_unauthenticated_listener_arns       = [data.aws_lb_listener.selected.arn]
  alb_ingress_unauthenticated_listener_arns_count = 1
  aws_logs_region                                 = "us-east-2"
  region                                          = "us-east-2"
  ecs_cluster_arn                                 = data.aws_ecs_cluster.selected.arn
  ecs_cluster_name                                = data.aws_ecs_cluster.selected.cluster_name
  ecs_security_group_ids                          = [data.aws_security_group.selected.id]
  ecs_private_subnet_ids                          = local.subnet_ids
  alb_ingress_healthcheck_path                    = "/healthz"
  alb_ingress_unauthenticated_paths               = ["/*"]
  codepipeline_enabled                            = false
  cloudwatch_log_group_enabled                    = false
  webhook_enabled                                 = false
  alb_security_group                              = "sg-11112222"
  repo_owner                                      = "hashicorp"
}

see if this block works for you and then modify it to your liking

still no go

I replaced the ecs_web_app block from the complete example with yours, and replaced some variables

module "ecs_web_app" {
  source = "git::<https://github.com/cloudposse/terraform-aws-ecs-web-app.git?ref=tags/0.36.0>"
  name                                            = "test-web-app"
  vpc_id                                          = module.vpc.vpc_id
  alb_ingress_unauthenticated_listener_arns       = [module.alb.alb_arn]
  alb_ingress_unauthenticated_listener_arns_count = 1
  aws_logs_region                                 = "us-west-2"
  region                                          = "us-west-2"
  ecs_cluster_arn                                 = aws_ecs_cluster.default.arn
  ecs_cluster_name                                = aws_ecs_cluster.default.name
  ecs_security_group_ids                          = [module.vpc.vpc_default_security_group_id]
  ecs_private_subnet_ids                          = module.subnets.private_subnet_ids
  alb_ingress_healthcheck_path                    = "/healthz"
  alb_ingress_unauthenticated_paths               = ["/*"]
  codepipeline_enabled                            = false
  cloudwatch_log_group_enabled                    = false
  webhook_enabled                                 = false
  alb_security_group                              = "sg-11112222"
  repo_owner                                      = "hashicorp"
}

also made sure to terraform init

thats really weird

it looks like youre using an old version

can you try using 0.39.1 which is the latest tag in the repo

id also try a terraform init -upgrade

or better yet, wipe rm -rf .terraform/ && terraform init -upgrade

if that doesnt work then idk

link?

https://github.com/hashicorp/terraform/milestones

If you’re using public subnets, you’ll have to also set the public ip variable to true

If your elb is publicly facing, make sure to allow only the GitHub and your office ipcidrs

Otherwise anyone in the world will be able to hit it

correct is how I have it right now, open but restricted to my office and the gitlab server

I am using both public and private basically I am following this https://github.com/terraform-aws-modules/terraform-aws-atlantis#run-atlantis-as-a-terraform-module

I am going to set ecs_service_assign_public_ip=true as you mentioned

I think you either want a nat gateway or a nat instance, not both.

yes, either one, not both

we use nat gateways in staging and prod

we use nat instances (micro) in dev and test and sandbox to save some money (EC2 instances are cheaper than NAT Gateways)

@Andriy Knysh (Cloud Posse) btw, is there any special reason to create EKS workers in public subnet? Isn’t secure to put in private subnet by default? at https://github.com/cloudposse/terraform-aws-eks-cluster examples

  module "eks_workers" {
...
    subnet_ids                         = module.subnets.public_subnet_ids

thanks for the advice guys, the comments make it clear, at the moment I’m just testing around but the advice to use one instead of both worked

@ismail yenigul in production you should put the worker nodes into private subnets

Yes, that would be great to update example to make it safer

the cluster itself should be given all subnets, private and public. k8s will use the public subnets to create load balancers in

also it could be better to add subnet tagging with elb and internal-elb by default

Ah where is that? I was looking at https://github.com/cloudposse/terraform-aws-eks-cluster/blob/master/examples/complete/main.tf

the example is outdated and was not updated to use the subnet tags

we will update it asap when have time

I can send PR if you happy for that changes but it seems README.md is auto generated

we have GitHub action for that, don’t worry, we’ll run the command on your PR

https://github.com/cloudposse/terraform-aws-eks-cluster/pull/73 here you go

Is there a way to convert between tf and hcl and back?

What’s the use case

I was looking at https://github.com/minamijoyo/hcledit and wondering how to programmatically remove a resource from terraform code while maintaining file structure and comments

.tf is just a file extension. The syntax of the contents is HCL

Looks like hcledit should be able to handle that

Nifty project, nice find

it’s like policy_sentry querying of iam perms but instead it queries arguments for terraform resources

this guy minamijoyo is killing it with the tf tools

https://github.com/minamijoyo

This is cool!

Ya. Minamijoyo has some great tools. Check out tfschema too

https://github.com/minamijoyo/tfupdate

Niiiiiice!

whoah hold on. The quotes aren’t required on the resource type and name, for terraform?

Nope, maybe once upon a time they were, but not since hcl2 and tf 0.12 iirc

omg

If you are having your hands first time dirty with Terraform, for starters this book is a must:

https://www.terraformupandrunning.com/

P.S. I am not related to the book or authors in any way.

you can output the entire resource:

output "dns_record" {
  value = aws_route53_record.cloudfront
}

or a map of specific attributes:

output "dns_record" {
  value = { for key, resource in aws_route53_record.cloudfront : key => { fqdn = resource.fqdn } }
}

or a list of a a single attribute:

output "dns_record" {
  value = [ for resource in aws_route53_record.cloudfront : resource.fqdn ]
}

docs on for expressions: https://www.terraform.io/docs/configuration/expressions.html#for-expressions

Can also do aws_route53_record.cloudfront.*.fqdn iirc

Or does that only work on counts?

That syntax only works on count… When using for_each the object is a map, not a list

Ah, my apologies

https://github.com/terraform-aws-modules/terraform-aws-notify-slack

Usually, notifications are part of the CI/CD pipeline if you are using it for infra delivery.

Thanks for the ping. Will this send plan and/or destroy output to slack? Is there an example slack msg I can look at to see if that’s what I want?

Nevermind, it seems to be doing it now…

In general though, where we’ve had to control these sorts of events we use the null_resource with triggers

Ref: original issue: https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/issues/51

Ref: original pr: https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/pull/52/files

i believe the awsvpc netowrk mode is required for fargate

so the code is basically saying, if the network mode is not awsvpc (not for fargate), then disable creation of the ecs service role, the service policy, the role policy, and remove the iam role from the ecs service

could you create an issue ticket of what is breaking, how it’s breaking, and what your use case is ?

Our Terraform plan is attempting to disable this role for our fargate instances..

I’ll create a ticket

fyi, for now, can you pin to the tag version prior to that pr ?

please also include your arguments for the terraform module like the var.network_mode

yeah, we’re pinning to a tagged version. that’s why we’re just now seeing an issue

oh ok cool so at least 0.25.0 works for now until we can come up with a solution upstream

Sure thing! thanks for your help

then it will be easier to move the pin forward

of course, np!

i was the one that put the change in so i should probably fix the mistake wherever it may be

yes, it uses the provider region

from the v3 upgrade guide: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/version-3-upgrade#region-attribute-is-now-read-only

kewl i see

There is a need for a nice template system I think. I’d be much more interested if it was a Go cross platform app and easy as git town or similar tool.

Yeah, I mentally took points off because it was written in Ruby as well. I’m no great Go programmer by any means, but any ops / infra related tooling creator should realize by now that you need to write your tool in Go for the community to get behind you.

Lol. Maybe not required to be in Go but saying no windows support is kinda dropping a huge gapping hole in coverage. I work on macOS but any tooling I use for team needs cross platform support or at least be a docker workflow at the minimum. Has to be super easy to get going or no chance of adoption in such a busy world imo.

We are releasing a library built around variant2 by @mumoshu that extends this kind of functionality for terraform and helmfile. The rad thing about variant2 is it cross compiles to multiple platforms and is written in go. But all the workflows are written in native HCL2.

The ETA is “any day” now, but there are a couple of bug fixes we need before it works as a remote module. Btw, variant2 supports remote variants just like terraform supports remote modules. So it’s an insanely reusable cli workflow.

https://github.com/mumoshu/variant2

Stoked

Hah. Uses Ruby/Thor. Looks just like my opinionated Puppet tool from ages ago: https://github.com/tehmaspc/puppet-magnum

Agree - wouldn’t want to use Ruby anymore myself either.

we use a data source to find the vpc using an application tag

we have application=oregon for a single vpc so a data source uses that to get that vpc.

then we split that vpc up in public and private subnets

so we then use a data source using the vpc id from the first vpc data source and look for public=true and we get a list of public subnets

And then you just make sure you leverage all those subnet ids across the AZs; do you do anything more? We’re doing the same with respect to looking up the subnets via tags.

I’m thinking now maybe my devs haven’t been iterating through the list of subnets and that’s probably why we’re exhausting our little address we do have in one of the 4 subnets for which we’re having a current issue

@RB - appreciate it; needed to spitball out ideas. Cheers. Think we’ve found some culprits.

ah yes that makes sense. yes we leverage all those subnet ids across the azs so amazon will balance between them

we dont do anything more but we use /18s and /20s so we havent run out of address space in our vpcs

we do have a balancing issue tho. we recently added a new public and private subnet in the 2d az and now we have to re-apply the terraform that uses the dtaa sourcrs in order to take advantage of the new subnet

Isn’t that the whole idea of having a public vs. private load balancer?

Oh, i think i need to use the targets property on aws_ssm_association

now with the [context.tf](http://context.tf) we can plop that into every module so we keep our variables consistent. as many of you have probably realized, we inconsistently support things like environment, label_order, etc. because it was a manual, error prone process of copying distributing them to all the modules.

Here’s an example of the file: https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf

You can start using this today in your projects.

Note, this also adds a this module

so you can pass module.this.context between modules.

looks like it’s happening to be on version 17.0 and 16.0 as well. I probably did something wrong

These error messages are very cryptic. I ran into something similar with using dynamic blocks for copy_action for AWS Backup service. In my case, the issue was that the AWS provider 2.58 had been updated to include support for it and even though my code was using 2.70 , I was seeing this error , until I pinned the AWS provider version in code explicitly to 2.70 instead of saying >2.11 etc.

The reason I am telling you this is that it could be your code is correct but maybe one of the provider version etc is breaking it. I would be very interested in knowing how you fix this error for your case.

also, I saw this error with TF versions :12.19 12.23 and 12.29

very interesting, I currently have my AWS provider set to v2.28.1

try setting it to something latest..2.70 for example

the provider changelog typically indicates when support for a particular feature was introduced. But testing it like this will save the trouble of going through the versions.

also I found this -> https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/issues/20

weird, I keep getting an error trying to terraform init --upgrade with using any of the newer providers.. I wonder if my Terraform version needs to get bumped up. Even if I put it back to the way I had it or just completely remove the requirement Terraform still errors out. :confused:

Error: no suitable version is available

and my ~/.terraform.d/ directory only has a checkpoint_cache and checkpoint_signature file in it

No provider “aws” plugins meet the constraint “ < 4.0,>= 3.0,~> 2.0,~t; 2.0,~> 2.0,~t; 2.0”.

The version constraint is derived from the “version” argument within the provider “aws” block in configuration. Child modules may also apply provider version constraints. To view the provider versions requested by each module in the current configuration, run “terraform providers”.

To proceed, the version constraints for this provider must be relaxed by either adjusting or removing the “version” argument in the provider blocks throughout the configuration.

Error: no suitable version is available

hmm looks to be only for this one repo. Terraform init still works in my other repos.

fixed it.

hey..awesome ! what was the issue ?

I ended up pulling in a different TF module for S3 and bumped the version to the latest which sourced in the AWS 3.0 provider. So when I ran terraform providers I saw the 3.0 being sourced from that S3 module.

still haven’t fixed the initial issue I had with the original module

terraform-aws-helm-xxxx?

is that the terraform recommendation? or a suggestion?

not sure there is an official recommendation

there is no “rule” here, just guidelines. When I do something like this I do use the terraform-prov1-prov2-xxx syntax

The recommending naming that you’re looking at, is a requirement for releasing on terraform registry. If you don’t plan on doing that then naming is completely up to you. example: we use that naming for public modules, but for internal modules we use tf-<thing being managed>. Helps people know what is public and what is not

yeah it would be going into a registry (terraform enterprise…or is it called cloud now ) which is why im trying to figure out

Good to know thank’s @Steven, I’ve never published one on the registry so that was news to me

right, I shouldve mentioned that this was for a registry

You could use local_file and make sure the directory that the file gets added to is set up in your KUBECONFIG environment variable

@Tom Howarth This is how I’m doing this on a project:

resource "null_resource" "eks_kubeconfig" {
  triggers = {
    eks_endpoint = module.eks_cluster.eks_cluster_id
  }

  provisioner "local-exec" {
    command = "aws eks --region ${var.region} update-kubeconfig --name ${module.eks_cluster.eks_cluster_id}"
  }
}

That works too, though I prefer to keep different files when possible so $HOME/.kube/config file doesn’t get super cluttered

aws eks update-kubeconfig has a --kubeconfig flag to specify the file that gets modified/created

@roth.andy Ah TIL. I like that idea. This is my first major k8s project, so I haven’t hit that level of .kube/config clutter, but I can see that being annoying with a bunch of projects / environments.

https://github.com/antonbabenko/pre-commit-terraform

is what I use frequently. includes tflint

Very cool!!!! I just ran in cloudposse template I used and it flagged pinned at master recommending better practice. Love it. Going to check out this repo too

first time i setup pre-commit successfully. Love it. Terraform markdown docs + formatting + whitespace trim now enabled.

I’m a fan of trying to do this in github actions when possile to eliminate any local dependency on someone but i imagine it would be really easy to run the same triggers in github

tflint and checkov are very useful

caught a lot of things before committing