#terragrunt (2021-12)

terragrunt

Terragrunt discussions Archive: https://archive.sweetops.com/terragrunt/

2021-12-31

2021-12-29

Maxime Thébault avatar
Maxime Thébault

Hi there! We’re considering some refactoring on our terraform repo and I would have loved some feedback/insight from you before making any decision.

Currently, our file structure looks like this:

terraform
├── live
│   ├── terragrunt.hcl
│   └── aws_account_name
│       ├── account.hcl
│       └── eu-west-1
│           ├── region.hcl
│           ├── prod
│           │   ├── env.hcl
│           │   └── appA
│           │       └── terragrunt.hcl
│           └── dev
│               ├── env.hcl
│               └── appA
│                   └── terragrunt.hcl
└── modules
	└── appA
		├── ec2.tf
		└── route53.tf

In this example, the appA module defines all the AWS resources needed for the app, and accepts a few input variables for the things that should change between the dev and the prod environment (e.g. instance types). Files in the “live” folder are really lightweight, whereas modules can get bigger in size depending on the application complexity.

We plan to host all of this in a menorepo. We plan to use Atlantis + terragrunt-atlantis-config for the CI/CD part.

Now, here are some of my questions:

  1. Does this design scale well?
  2. Reading online documentation, it seems like everyone agrees module versioning is a best-practice. So that would mean I should version the appA module and use explicit version in the “live” folder, instead of just terraform { source = "../../../../../..//modules/appA" }. To me, it feels like something that would be extremely painful, even more so in a monorepo context. What’s your take on this?
  3. I read about terraform-related testing: module unit testing with tools such as terratest, and more serious testing through infrastructure promotion to production. In your experience, is all of this really necessary? Isn’t a simple look at the plan enough to decide if the changes are OK?
  4. Do you guys stick to just using a master branch with MR for changes, or do you use more than one branch? If so, what for? I would be particularly interested in hearing your thoughts on all of this @David, I’ve carefully read your Transcend blog post and your feedbacks on this Slack, but there are still things I’m uncertain about. I’m fine with concise answers, unlike what it seems like with this long message :D

Thanks!

loren avatar
loren

if you haven’t seen it already, the gruntwork team recently published a good article on using the latest features to reduce duplicate info even more… https://gruntwork.io/guides/upgrades/how-to-dry-your-ref-arch/

How to DRY your Reference Architecture with Terragrunt Multi-include

Learn how to DRY your Reference Architecture with Terragrunt Multi-include

Maxime Thébault avatar
Maxime Thébault

Thanks! I didn’t see this specific article, but I saw some alternative way on their Github: https://github.com/gruntwork-io/terragrunt/issues/1451#issuecomment-737155756

Identical .hcl files in too many places. how to make .hcl files DRY? · Issue #1451 · gruntwork-io/terragrunt attachment image

We have a repo structure like so: my-account - dev - eks - main.yaml - terragrunt.hcl - rvw - eks - main.yaml - terragrunt.hcl - stg - eks - main.yaml - terragrunt.hcl - prd - eks - main.yaml - ter…

mikesew avatar
mikesew

thanks for the links - any tips for those of us who arent using terragrunt?

for submodules within the monorepo, we dont personally bump version numbers. for exsternal modules (ie. in another repo), yes we do

loren avatar
loren

That’s basically how I do it also. I version the monorepo as a whole, but not the internal references

1
1

2021-12-23

2021-12-22

2021-12-21

Frank avatar
Frank

Hi! Is there anyone here who is running terragrunt inside CI/CD pipelines? If so, how are you dealing with things like dependencies?

David avatar
David

I use Atlantis (https://www.runatlantis.io/) to deploy our Terragrunt code, and use https://github.com/transcend-io/terragrunt-atlantis-config to automatically ensure dependencies are always up to date.

Disclaimer: I did write https://github.com/transcend-io/terragrunt-atlantis-config (though it’s all open source and free)

Terraform Pull Request Automation | Atlantis

Atlantis: Terraform Pull Request Automation

GitHub - transcend-io/terragrunt-atlantis-config: Generate Atlantis config for Terragrunt projects. attachment image

Generate Atlantis config for Terragrunt projects. Contribute to transcend-io/terragrunt-atlantis-config development by creating an account on GitHub.

Frank avatar
Frank

Ah cool, I did come across your repo last week while researching the topic

David avatar
David

let me know if you have any troubles with it if you decide to try it

Frank avatar
Frank

Currently I have setup CI/CD pipelines in Gitlab which runs plan and apply but those dependencies are rather problematic..

Frank avatar
Frank

Right now it would require me to make multiple subsequent MR’s to get things in there

1
Frank avatar
Frank

You want a Bastion? You need a VPC for that. plan + apply vpc -> plan/apply bastion. Feels really impractical. Hopefully Terragrunt and this config can deal with that more practically

David avatar
David

That’s a tricky problem, I think https://github.com/runatlantis/atlantis/issues/972#issuecomment-651360022 is a decent bet for making those sorts of cases easier (or at least something like it), though it hasn’t been commented on by maintainers

tbh I’ve been moving more recently to use something like terragrunt-atlantis-config that generates a dynamic CI workflow (see: https://circleci.com/blog/introducing-dynamic-config-via-setup-workflows/) that will apply the terragrunt/pulumi modules we have in order, just for those modules affected by a given PR.

Atlantis could definitely add something like that, but it’s an important enough feature at my company where we aren’t all-atlantis anymore

Frank avatar
Frank

I’m curious as to how terragrunt-atlantis-config would fit in with CircleCI? Does CircleCI trigger atlantis via webhooks?

2021-12-16

2021-12-09

muhaha avatar
muhaha

Is possible to predownload provider ( not in tf registry ) from github and install it before terraform invocation ? Thanks

Tim Gourley avatar
Tim Gourley

Its been a while but you can add it to the user provider cache directory and it will be picked up from there instead of downloaded. We have a custom provider which we handle that way. it was tricky sorting the directory layout if I recall.

    keyboard_arrow_up