#terragrunt (2020-06)
Terragrunt discussions
Archive: https://archive.sweetops.com/terragrunt/
2020-06-01
does anyone know how one is able to run cli commands using after_hook
? Im trying the following
after_hook "after_hook" {
commands = ["apply"]
execute = ["aws", "s3 ls"]
}
but I get the following error
[terragrunt] 2020/06/01 10:03:21 Detected 1 Hooks
[terragrunt] 2020/06/01 10:03:21 Executing hook: after_hook
[terragrunt] 2020/06/01 10:03:21 Running command: aws s3 ls
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument command: Invalid choice, valid choices are:
seems its not recognizing the command
because you ran aws "s3 ls"
and not aws s3 ls
…
after_hook "after_hook" {
commands = ["apply"]
execute = ["aws", "s3", "ls"]
}
:sweat_smile: ah, yeah thats it - one more question for you if you dont mind! I’m trying to run the following cli command to tag the state bucket thats created
execute = ["aws", "s3api", "put-bucket-tagging", "--bucket", "my-bucket", "--tagging", "TagSet=[ for key, val in local.tags: { \"Key\" = key, \"Value\" = val } ]"]
but it seems I cant do the interpolation there :thinking_face: am I missing anything here or can I not use local.tags
here
create the tag-string in another local
then use
execute = ["aws", "s3api", "put-bucket-tagging", "--bucket", "my-bucket", "--tagging", "TagSet=${local.tag_set}"]
I added
tag_set = [ for key, val in local.tags: { "Key" = key, "Value" = val } ]
as a local and ran again with your suggestion but got
Invalid template interpolation value; Cannot include the given value in a string template: string required.
because you didn’t create a string, you created a list of maps
oh, right - the awscli docs requires a list of maps https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-tagging.html
well and it requires json not hcl
try to jsonencode it…
execute = ["aws", "s3api", "put-bucket-tagging", "--bucket", "my-bucket", "--tagging", "TagSet=${jsonencode(local.tag_set)}"]
ah was just about to try that
ahh so close! looks like jsonencode
turns =
into :
!
Error parsing parameter '--tagging': Expected: '=', received: '"' for input:
TagSet=[{"Key":"abc:account:accountid","Value":"123456789012"},{"Key":"abc:business:cost","Value":"0000"}]
any ideas on how to preserve the =
Don’t. Just use the json syntax. The TagSet key needs to be part of the json dictionary… https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-tagging.html
what do you mean?
my string looks exactly like the example
TagSet=[{Key=string,Value=string},{Key=string,Value=string}]
Look at the json syntax
locals {
tag_set = {
TagSet = [ for key, val in local.tags: { "Key" = key, "Value" = val } ]
}
}
execute = ["aws", "s3api", "put-bucket-tagging", "--bucket", "my-bucket", "--tagging", jsonencode(local.tag_set)]
oh man! that did it!
thank you
2020-06-04
Someone else with this issue? https://github.com/gruntwork-io/terragrunt/issues/1035#issuecomment-638837106
I am using Terragrunt v0.21.11 If I say: terraform { extra_arguments "common_vars" { commands = get_terraform_commands_that_need_vars() arguments = [ "-var", foo=bar", ] } …
This seems related to https://github.com/gruntwork-io/terragrunt/issues/983, which I issued a long ways back.
Terragrunt doesn’t discriminate between apply calls with vs without a planfile, and doing so seems non-trivial
At my company the problem disappeared when we switched from using local secrets files to fetching secrets from Vault
Right now, writing code such as terraform { extra_arguments "common_var" { commands = get_terraform_commands_that_need_vars() arguments = ["-var-file=${get_aws_account_id()}.tfvars&q…
Ye, I need to figure out how to get rid of var files inlcuded in every apply. I have region.tfvars and env.tfvars included in tree structure
From what I can tell on Terragrunt 0.21.11, this is not an issue. When you call apply with a plan file, it removes all arguments starting with "-var" or "-var-file". This is a problem for me because I pass ["-var", "foo=bar"] and it ends up calling terraform with just foo=bar, so I have to use the form -var=foo=bar to make it correctly remove that when the apply is called with a plan.
?
what does your terraform
block look like now?
terraform {
extra_arguments "common_vars" {
commands = get_terraform_commands_that_need_vars()
optional_var_files = [
"${get_parent_terragrunt_dir()}/terraform.tfvars",
"${get_parent_terragrunt_dir()}/common.tfvars",
"${find_in_parent_folders("region.tfvars", "skip-env-if-does-not-exist")}",
"${find_in_parent_folders("env.tfvars", "skip-env-if-does-not-exist")}",
]
}
extra_arguments "init-no-color" {
commands = ["init"]
arguments = ["-no-color"]
}
extra_arguments "disable_input" {
commands = get_terraform_commands_that_need_input()
arguments = ["-input=false"]
}
before_hook "copy_common_main_providers" {
commands = ["init-from-module"]
execute = ["cp", "${get_parent_terragrunt_dir()}/common/main_providers.tf", "."]
}
after_hook "remove_useless_copy_of_main_providers" {
commands = ["init"]
execute = ["rm", "-f", "${get_parent_terragrunt_dir()}/${path_relative_to_include()}/main_providers.tf"]
}
}
terragrunt should remove var or var-file when apply with plan
2020-06-08
Is possible to get path ( absolute ) if I am using terragrunt --terragrunt-working-dir $mydir
?
For example:
Terragrunt
is triggered from /tmp/
and is using --terragrunt-working-dir foo/bar/project
I need relative path like foo/bar/project
In fact foo/bar/project
is located in /tmp
, but on other enviroments, this path may be different
i’m confused a bit, it sounds like you are looking for a relative path, but you are asking about an absolute path
*relative, sorry
get_terragrunt_dir()
will give you the absolute path
you might be able to do it with some creative use of path_relative_to_include()
Terragrunt allows you to use built-in functions anywhere in terragrunt.hcl
, just like Terraform.
I am not using include
path_relative_to_include()
will always return .
maybe run_cmd()
?
lolol, all part of what i meant by “creative use”
sometimes you have to adjust how you have things setup, in order to use the features a tool offers…
well, its should be possible to do:
run_cmd("echo","${get_terragrunt_dir()#pwd}")
, does not work yet .. :X
Where do you put your provider settings?
provider "aws" {
region = var.aws_region
# Make it faster by skipping something
skip_get_ec2_platforms = true
skip_metadata_api_check = true
skip_region_validation = true
skip_credentials_validation = true
}
also this setting?
terraform {
backend "s3" {}
}
What the best way and how do you handle this? I’ve created terragrunt.tf in each module with both of this settings. Is this correct way? Previously I was using hook to copy main_providers.tf
provider "aws" {
region = var.aws_region
# Make it faster by skipping something
skip_get_ec2_platforms = true
skip_metadata_api_check = true
skip_region_validation = true
skip_credentials_validation = true
}
provider "aws" {
alias = "virginia"
region = "us-east-1"
}
terraform {
backend "s3" {}
required_version = ">= 0.12.0"
}
variable "aws_region" {
type = string
default = "eu-central-1"
description = "AWS region to use for all resources"
}
got it https://github.com/gruntwork-io/terragrunt-infrastructure-live-example/blob/master/terragrunt.hcl
A repo used to show examples file/folder structures you can use with Terragrunt and Terraform - gruntwork-io/terragrunt-infrastructure-live-example
Yep, that file is a good example
2020-06-16
What’s do you think about that type of approach to put this data in input section of HCL file?
custom_task_policy_statement = [
{
actions = [
"s3:GetObject",
"s3:ListBucketVersions",
"s3:ListBucket",
"s3:GetObjectTagging",
"s3:GetBucketVersioning",
"s3:GetBucketAcl",
"s3:GetBucketCORS",
"s3:GetBucketLocation",
"s3:GetObjectVersion",
"s3:PutObject",
"s3:GetAccountPublicAccessBlock",
"s3:ListAllMyBuckets",
"s3:HeadBucket"
]
effect = "Allow"
resources = [
dependency.s3.outputs.this_s3_bucket_arn,
"${dependency.s3.outputs.this_s3_bucket_arn}/*"
]
},
{
actions = [
"kms:DescribeKey",
"kms:GenerateDataKey*",
"kms:Encrypt",
"kms:ReEncrypt*",
"kms:Decrypt"
]
effect = "Allow"
resources = [dependency.kms.outputs.key_arn]
}
]
I usually either:
• ask for a string policy, which can be made in inputs with jsonencode
• ask for an array of existing IAM Policy arns Your approach here could definitely work, but I’d make sure it works with things like condition clauses. Terraform’s typing isn’t great for working with maps of semi-structured data, but it can work
I have kinda similar to this
Without the dependency usage
2020-06-18
Hey all, I was wondering if theres any way to combine tags from the child into the parent hcl? For example, I’ve got the following in the my parent hcl file
locals {
env = element(split("/", local.relative_path), 1)
base_tags = {
"accountid" = get_aws_account_id(),
"environment" = local.env,
}
}
and in my child hcl file I have
locals {
# This loads base tags from parent hcl and adds additional ones
# Ideally I want to do the opposite - define some tags here, and combine
# them in the parent hcl so that we have access to the full set of tags there
# to use for s3_bucket_tags and dynamodb_table_tags
# I'm not sure if/how that's possible
# See <https://terragrunt.gruntwork.io/docs/features/locals/>
# Their solution with yaml doesn't help us because 2 of the 3 base tags are
# dynamically computed
base_config = read_terragrunt_config(find_in_parent_folders())
tags = merge(local.base_config.locals.base_tags, {
"abc" = "0123"
"dept" = "engineering"
})
}
any ideas
So you want in your child merged tags from parent and child?
other way around
you can merge multiple values from mutliple level
locals {
base_tags = {
"accountid" = "123123123",
"environment" = "test",
}
other_tags = {
"test" = "asd"
}
}
output "tags" {
value = merge(local.base_tags, local.other_tags)
}
I understand that, what I want is to take the tags declared in a child.hcl and use them in the parent.hcl
two different files on different levels
2020-06-19
2020-06-22
2020-06-24
Guys, I need help:
/project1/terragrunt.hcl
file contains ( its main file ):
include {
path = find_in_parent_folders("templates/aws.hcl")
}
/templates/aws.hcl
file contains:
terraform {}
remote_state {
disable_init = tobool(get_env("TERRAGRUNT_DISABLE_INIT", "false"))
backend = "s3"
config = {
bucket = "${local.bucket}"
region = "${local.backend_region}"
key = "${get_env("PPATH")}/terraform.tfstate"
encrypt = true
dynamodb_table = "${local.dynamodb_table}"
}
}
locals {
bucket = "${get_env("bucket")}"
backend_region = "${get_env("backend_region")}"
dynamodb_table = "${get_env("dynamodb_table")}"
}
generate "provider" {
path = "provider.tf"
if_exists = "overwrite"
contents = <<EOF
provider "aws" {
version = "2.67.0"
region = ???
}
EOF
}
generate "backend" {
path = "backend.tf"
if_exists = "overwrite"
contents = <<EOF
terraform {
backend "s3" {}
}
EOF
How can I set region in /project1/terragrunt.hc
with minimum effort?
• its not possible to set it via locals
• maybe with env_vars ? Thanks
Yeah, the locals
block their is scoped to that terragrunt file, while the provider
block inside the generate
block is injected as a terraform file, so you can’t directly use any locals there.
I would suggest using templatefile
, and injecting in get_env("backend_region")
in the region = ???
spot
templatefile ? which one? I dont follow
No worries! I was suggesting you use this function here: https://www.terraform.io/docs/configuration/functions/templatefile.html
So I’d add a /templates/provider.tf
file with contents:
provider "aws" {
version = "2.67.0"
region = ${REGION}
}
and then in your /templates/aws.hcl
file I would change the generate
block to be:
generate "provider" {
path = "provider.tf"
if_exists = "overwrite"
contents = templatefile("./provider.tf", {
REGION = get_env("backend_region")
})
}
The other option is to just not specify region at all in your provider, and terraform will default to the AWS_REGION
env var
The templatefile function reads the file at the given path and renders its content as a template.
No, thats not a problem.. I dont want to preset region with ENV var, I just want pure gitops, but it should not be hardcoded in template, which is shared … So something like this:
/project1/terragrunt.hcl
file contains ( its main file ):
include {
path = find_in_parent_folders("templates/aws.hcl")
}
locals {
region = "eu-central-0"
}
/templates/aws.hcl
file contains:
terraform {}
remote_state {
disable_init = tobool(get_env("TERRAGRUNT_DISABLE_INIT", "false"))
backend = "s3"
config = {
bucket = "${local.bucket}"
region = "${local.backend_region}"
key = "${get_env("PPATH")}/terraform.tfstate"
encrypt = true
dynamodb_table = "${local.dynamodb_table}"
}
}
locals {
bucket = "hardcoded"
backend_region = "hardcoded"
dynamodb_table = "hardcoded"
}
generate "provider" {
path = "provider.tf"
if_exists = "overwrite"
contents = <<EOF
provider "aws" {
version = "2.67.0"
region = ${local.region}"
}
EOF
}
generate "backend" {
path = "backend.tf"
if_exists = "overwrite"
contents = <<EOF
terraform {
backend "s3" {}
}
EOF
Ah, that is not possible. Terragrunt scopes locals to just the file you are working in, unfortunately. I’m not sure if there’s a bug or not for it, but you can’t to parent -> child or child -> parent locals through include
blocks if I remember right
it feels backwards from the terragrunt paradigm to try to pass values from the child config to the parent config
you can do things like define common values in a .yml file and read them with yamldecode()
, or keep them in a “common” terragrunt config and use read_terragrunt_config()
… https://terragrunt.gruntwork.io/docs/reference/built-in-functions/#read_terragrunt_config
Terragrunt allows you to use built-in functions anywhere in terragrunt.hcl
, just like Terraform.
@loren like this https://pastebin.com/raw/Q7UzBybA ? Will it work? :X
i still think you have the paradigm backwards, and need to flip things around. what you are calling a “template” is what terragrunt refers to as a “parent”, and the relationship works the other way around
at least, for the “include” functionality
i’m not really sure how to do what you want, exactly how you want it. i’m just pointing you towards the tools that might get you to an equivalent setup. but it probably requires rethinking your approach
tools?
functions as tools, in this case
2020-06-25
I’m using terragrunt
heavily at the moment but one thing is eluding me… During development of modules I pin to a branch and sometimes branches of Terraform modules under that. Currently, I’m deleting the .terragrunt-cache
before every apply which is a bit frustrating, is there anyway to force the changes to be pulled in rather than deleting the cached modules every time I want to apply changes? Maybe there’s soe magic option I’ve missed, re-running an “init” doesn’t pull the newest changes to a branch in either
pass the flag --terragrunt-source-update
and it will automatically delete the cache for that config
I normally use https://terragrunt.gruntwork.io/docs/reference/cli-options/#terragrunt-source if I’ve actively developing a module, that way you don’t even have to commit/push to test it
Terragrunt forwards all arguments and options to Terraform. Learn more about CLI options in Terragrunt.