#terragrunt (2021-02)
Terragrunt discussions
Archive: https://archive.sweetops.com/terragrunt/
2021-02-09
Do we have AWS Profile option using Terragrunt?
I am ware that using Terraform, we can use provider block:
provider "aws" {
profile = "default"
region = "us-west-2"
}
Where we can specify profile name.
How I can change that in the terragrunt.hcl?
Is there any way we can override it during terragrunt apply or any run time operation?
remote_state {
backend = "s3"
config = {
bucket = "foo-bar-tfstate"
key = "<tfstatename>.tfstate"
region = "us-east-1"
encrypt = true
profile = "<aws-profile-name>"
}
}
Something like this ?
Got it Thank you I was able to use given suggestion
2021-02-10
Am new to Terragrunt. Trying to run Cloudrail as part of my Terragrunt flow to test plans before they are applied. I have three modules - item1, item2, item3. Cloudrail needs to take in the plan.out out as its input, and spits out some stdout content as well as an exit code. Based on this exit code, I want to move forward to apply.
So am running this:
terragrunt run-all plan -out plan.out
terragrunt run-all apply -auto-approve plan.out
And I’ve added this after_hook:
terraform {
after_hook "cloudrail_after_hook" {
commands = ["plan"]
execute = [
"docker",
"run",
"--rm",
"-v", "${get_env("PWD", "")}:/data",
"indeni/cloudrail-cli",
"run",
"-d", "${path_relative_to_include()}",
"--tf-plan", "${path_relative_to_include()}/plan.out",
"--origin", "ci",
"--build-link", "<https://somelink>",
"--execution-source-identifier", "somebuildnumber - tg module ${path_relative_to_include()}",
"--api-key", "${get_env("CLOUDRAIL_API_KEY", "")}",
"--auto-approve"
]
}
}
But:
- The execution of the Cloudrail container is all over the place. It’s hard to know which module the Cloudrail output belongs to without using
--terragrunt-log-level info. - The whole concept of passing to apply a plan I created in a previous step doesn’t seem to be working. Am I missing something?
@Mohammed Yahya help!!!
the -all commands are a hot mess. it’s a nice idea, but seems to only work in very restrictive use cases
So if you have multiple modules, you run each plan separately?
personally, yes, i manage that execution/orchestration outside terragrunt.
In that case, a simple plan -out plan.out (with after_hook) and apply plan.out, just done multiple times.
yes, exactly
i also agree on the terragrunt output being interleaved and difficult to understand when using -all… i have an open issue for addressing it… https://github.com/gruntwork-io/terragrunt/issues/1194
Ah! I was looking for the same thing. This interleaving was messing with my mind.
i got so sick of it, i had to write a custom command runner to inject the paths for our own usage i’d try to fix it in terragrunt, but golang makes my head hurt
python asyncio ftw!
Amen to that
semi-related, absolute paths are too noisy… https://github.com/gruntwork-io/terragrunt/issues/1515
Also, the change adding –terragrunt-log-level is very new, and there is a new pr making “info” the default, basically reverting the output back to before they had log levels… Basically, output of -all is a work in progress
One thing you can try, is to redirect the cloudrail output to a known location, with a filename based on the terragrunt-working-dir, so you can review it afterwards
Oh yeah that could work too. There’s the --output-file and --output-format options for Cloudrail.
Don’t use plan-all / apply-all
not production safe
@Yoni Leitersdorf (Indeni Cloudrail) first make it more simple by creating a function alias in .bashrc or .zshrc as this example:
slscan() {
docker run --rm -e "WORKSPACE=$(pwd)" -e GITHUB_TOKEN -v "$(pwd):/app" shiftleft/scan scan $*
}
add there what ever logic you need,
then cal slscan in your terragrunt
IMHO I’m using a Taskfile (modern Makefile alternative) for all the logic I need before calling Terraform or Terragrunt, example:
task init
task plan # this will write plan.out
task cloudrail # this will read/scan plan.out
task apply # this will apply plan.out
task inframap # this will draw plan.out
Why wrap Terrragrunt (a wrapper) in another wrapper?
Good one, but when you work so deep with IaC you need your own wrapper
Not true
I don’t use Terragrunt anymore, I’m using my own 
you reach a point where you have complex logic and need your own tool
I agree with you but the point is all of these tools are just wrapper around Terraform, so used what ever you prefer, and match your use case
Need a tool, don’t need own tool….
Using Terragrunt/Atlantis without issues on a large codebase, multi region, multi account …
I’d rather take something sane off the shelf, than build your own and have to write fancy bespoke adapters for everything
Thank you for the feedback everyone. I feel much better now (I thought it was me!).
atlantis is nice, but you still need to run it and maintain it. it’s not actually free. a cli wrapper can run in any existing CI or local system, and doesn’t consume resources when not in use. most of our implementations are in codecommit, with event hooks to trigger codebuild
so pros and cons both ways
@Yoni Leitersdorf (Indeni Cloudrail) might be worth revisiting the -all workflow… terragrunt has published a few versions already that are trying to address some of your pain….
Yeah I noticed
Did you give it a shot?
i have not… my workflows tend to avoid -all…
@Mohammed Yahya has joined the channel
2021-02-11
Hi Everyone Our team runs terragrunt modules locally , what are the best solutions/ practice to run modules in more unified pattern Note we have s3bucket + dynamodb for locking state Thanks in Advance
hi @Asis - you can give env0 a try, we support deployments and running terragrunt and terraform modules in a unified solution. You can read mode here - https://www.env0.com/
- Disclaimer - I am the co-founder and CTO of env0.
2021-02-17
2021-02-26
Hi All, I’ve been looking at Atlantis for automation with our Terragrunt configurations. The recommended approach seems to be to plan individual projects. Atlantis can do some parallel operations but it is real fuzzy to me if that actually works (workspaces get pulled into the conversation etc) I have a test configuration with about 24 modules and the Atlantis plan takes ~30 minutes since it processes each model in sequence. I did get some parallel operations but then there were lock contentions, and the workspace topic freaks me out as I understood that is not something you do when using Terragrunt.
So, what am I missing, am I using a hammer to drive a screw? Why is plan-all not discussed? I do understand the shortcomings of unapplied modules affecting dependent modules in the plan but I think this is one of the choices we make when we move to Terragrunt. Any suggestions on a more appropriate automation tooling for Terragrunt managed envs?
TIA, Tim
Check out env0 support of Terragrunt - https://www.env0.com/blog/terragrunt-release (disclaimer - I am co-founder and CEO of env0)
You can also still use Atlantis and just use https://github.com/transcend-io/terragrunt-atlantis-config to automatically make atlantis work for terragrunt with parallelization and automatic atlantis.yaml generation so that all affected modules are planned
I tried the terragrunt-atlantis-config and it did not seem help much. Isn’t atlantis still going to process modules individually and sequentially?
If you use the --parallel and ``–create-workspace` flags it will process in parallel. I have about 600 terragrunt modules that can all be planned in about 15 minutes
There is a concept of Atlantis workspaces, and there is a concept of terraform workspaces, but they are entirely unrelated, so --create-workspace is safe to use. It only refers to creating Atlantis workspaces
Thanks for the responses! I’ll give this another go…
2021-02-28
How do you guys structure your folders when dealing with iam users in terragrunt? One terragrunt.hcl file per iam user?
I have one module that makes all users, and then I have separate IAM role modules that assign users to roles