#terragrunt (2020-1)

Terragrunt discussions Archive: https://archive.sweetops.com/terragrunt/

2020-01-26

joshmyers

get_input isn’t a valid function.

2020-01-25

Adrian

I have some extra var files defined and want to use some variables from this files in input for example region in container task definition, is this possible?

David

It is! What have you tried so far?

Adrian
Don't convert inputs to env vars. Instead, generate terragrunt.tfvars.json and use as -var-file · Issue #752 · gruntwork-io/terragrunt

I've been reading documentation for terraform 0.12 in regards to handling env vars, and https://www.terraform.io/docs/configuration/variables.html states that: Some special rules apply to the -…

Adrian

this is what i need


\# stage/frontend-app/terragrunt.hcl
terraform {
  source = "..."
}


\# Include all settings from the root terragrunt.hcl file
include {
  path = "${find_in_parent_folders()}"
}

inputs = {
  aws_region = get_input("aws_region")
  remote_state_bucket = get_input("remote_state_bucket")

  instance_type = "t2.micro"
  instance_count = 10
}
Adrian

aws_region is defined in tfvars which is included in optional_var_files

Adrian

how can I achieve this?

Adrian
  aws_region = get_input("aws_region")

2020-01-24

joshmyers

@Brij S post what you have/what you problem is

Brij S

I assume a role using aws-okta cli and use the env vars from that. I have a provider as follows

provider "aws" {
  region              = "us-west-2"
  alias               = "dig"
  assume_role {
    role_arn = "arn:aws:iam::xxxxxxxxxx:role/tf"
  }
}

but this provider gets ignored and I get the following error in the cli:

Error: AccessDenied: User: arn:aws:sts::xxxxxxxxxxxx:assumed-role/acount-name-ReadWrite/user is not authorized to access this resource
loren

At what point in the workflow is that happening? Are you using remote tfstate in s3? How is that configured?

joshmyers

Have had similar but need exact info.

2020-01-23

Brij S

I’m having an issue where terragrunt ignores a providers assume role/iam role block when assuming a role and using the creds from that as env vars - has anyone dealt with this?

2020-01-22

Chase Ward
11:55:00 PM

@Chase Ward has joined the channel

2020-01-17

Milos Backonja

Hi Guys, do you know if I can use terragrunt command in before hooks?

    before_hook "test" {
    commands     = ["init"]
    execute      = ["terragrunt", "apply", "-target=null_resource.rule", "-auto-approve"]
  }
joshmyers

Isn’t that going to cause recursion issues?

loren

depends on when the state is locked… i think it would work. does seem fragile though. but could be an interesting approach to dealing with the “count” cannot be computed style of error

:--1:1
loren

would at least need to disable autoinit in the hook to avoid recursion, --terragrunt-no-auto-init

Milos Backonja

Thanks guys, eventually I dropped approach with null resource and use External Data Source

2020-01-11

2020-01-10

2020-01-09

Milos Backonja

guys how to safely migrate state from one s3 bucket to another with terragrunt ?

maarten

First of all you can make a backup of your state by copying it elsewhere, this way you don’t get sweaty hands. Secondly, I think that the moment you change the backend in terragrunt, terragrunt/terraform will take care of the move.

Milos Backonja

Thanks Maarten, well I synced s3 buckets and changed backend config after that. It looks ok now :)

Brij S

Ive got a terraform module that creates resources in two different aws accounts. I handle this by doing the following:

provider "aws" {
  region  = "us-west-2"
  profile = "profile1"
}

provider "aws" {
  region              = "us-west-2"
  profile             = "profile2"
  alias               = "digi"
}

And in the module itself, the different provider is picked up like this:

resource "aws_route53_record" "dig_ns" {
  provider = aws.digi

I’m trying to utilize terragrunt to deploy many modules. This becomes difficult since the above method no longer works, has anyone encountered this? If so, how have you got around this. I dont think Terragrunt supports multiple providers like this

Brij S

I dont quite understand how tg can use different providers/regions(if at all )

maarten

@Brij S

few comments for you 1 is , route53 is not region dependent so if that’s the only use-case you don’t have to worry about it.

  1. I try to make modules which are used for a single region, if I need them to be applied in a different region it will be a different apply

  2. Using a structure like the following can help you with that:

├── envs
│   ├── aws
│   │   ├── dev
│   │   │   ├── eu-central-1
│   │   │   │   ├── applications
│   │   │   │   ├── infra
│   │   │   ├── eu-west-1
Brij S

@maarten, yes r53 is not region specific, but it is account specific. I have a unique setup where I need to create a new zone and then take its NS records and insert them into a different zone in a different account. Hence the multiple providers

Brij S

all of my modules are quite generic already like you mentioned. With TG - it seems like i can only use on provider so if I need to create edge acm certs in us-east-1 and all other resources in us-west-2 then it becomes a mystery

maarten

@Brij S could you delegate the zone to the other account instead ?

Brij S

unfortunately, no

Brij S

new zone in new acccount with NS records of said zone in old account

tamsky


yes r53 is not region specific, but it is account specific
Adding to the use case list for this is AWS govcloud[1]. govcloud’s Route53 service only allows creating private zones. Account-level delegations across the govcloud/commercial partition line is also not possible.

[1] https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/govcloud-r53.html

Lists the differences for using Amazon Route 53 in the AWS GovCloud (US-West) Region compared to other AWS regions.

slaughtr

I don’t see why that wouldn’t work, unless maybe you’re on an older version?

Brij S

well, in the .hcl file you dont specify providers of any sort?

Brij S

or any profiles

slaughtr

True, but terragrunt is just applying terraform. So you should be able to leave your tf files more or less the same

Brij S

hmm, im confused

Brij S

the terragrunt docs mention only leaving .hcl files in the ‘live’ repo

slaughtr

I have only done some light testing around multiple providers in my TF but TG didn’t seem to mind

Brij S

so if my tg file looked like this

include {
  path = find_in_parent_folders()
}

terraform {
  source = "git::[email protected]:foo/modules.git//app"
}

inputs = {
  zone_name       = "[domain.com](http://domain.com)"
  comment         = "Managed by Terraform"
Brij S

where do you plugin multiple providers

slaughtr

You put the multiple providers in your .tf files like you would normally

slaughtr

Terragrunt is just a wrapper for terraform, it doesn’t need all that info

slaughtr

So if you had [module.tf](http://module.tf) with multiple providers before, just point TG at it and it should (probably) just work

Brij S

@slaughtr heres my structure

├── global
│   ├── [main.tf](http://main.tf)
│   └── terragrunt.hcl
├── terragrunt.hcl
└── us-east-1
    ├── [main.tf](http://main.tf)
    ├── terragrunt.hcl
    └── [variables.tf](http://variables.tf)
Brij S

this issue arose when, the variables file in the us-east-1 folder wasnt enough to pick up the inputs from the global folder as you helped me with earlier

Brij S

so i decided to do all .hcl, just like the examples in their docs

Brij S

the main.tf in global has an output called zone_id (in the module itself). I setup the hcl file as you assisted me with as follows in the us-east-1 folder:

include {
  path = find_in_parent_folders()
}

dependency "global" {
    config_path = "../global"

    mock_outputs = {
    zone_id = "Z3P5QSUBK4POTI"
  }
  mock_outputs_allowed_terraform_commands = ["plan", "validate"]
}

inputs = {
  zone_id = dependency.global.outputs.zone_id
}

then in the variables.tf file i created a var called zone_id but it just picks up the mock value instead of the real created one

slaughtr

The format I follow (and I guess I didn’t read/should ahve read more/something) is something like this:

├── module
│   ├── [main.tf](http://main.tf)
│   ├── somecode.js
|   └── [variables.tf](http://variables.tf)
└── us-east-1
    ├── terragrunt.hcl
    ├── module_tg
	    ├── terragrunt.hcl
slaughtr

Just a side point, so you know you can do it

slaughtr

Oh if you have a mock value it will use that, you can remove it in production

slaughtr

There might be a way to say “if not present use mock” but I’m not sure

Brij S

yeah it has the

mock_outputs_allowed_terraform_commands = ["plan", "validate"]
Brij S

but that doesnt work

Brij S

it still picked up mock value when I ran apply-all

slaughtr

Hmm, I think the -all commands might follow a slightly different set of rules, don’t quote me on that. I never use them because I find they super conflate everything, and really it’s pointless til you’ve applied each thing individually because TG kinda sucks at dependency resolution

Brij S

so how do you go about applying all the modules in seperate folders?

slaughtr

I also don’t use mock outputs so I’m not sure what’s going on there. I’d recommend commenting it out for the moment

slaughtr

I apply each module individually

slaughtr

Generally I try not to change more than one module at a time anyway

slaughtr

Back to your original query, though: you have terraform config that has multiple providers that worked in the past with just TF?

Brij S

yes

Brij S

i was trying to convert to using just .hcl, as per the docs

slaughtr

So if you point at that in your source and get your inputs and dependencies figured out it should just work

Brij S

how does tg know to pick up credentials for a different provider?

slaughtr

Can you point me to where in the docs it says that? I’ve not seen that rec before

slaughtr

TG is just wrapping around terraform, so if terraform knows it you’re good

Brij S

my module, has a resource that explicitly has a provider set for it like this

resource "aws_route53_record" "digital_ns" {
  provider = aws.digi
Brij S

in my main.tf file I have two providers, one with an alias of digi

Brij S

however, if there is no tf file and only a .hcl file - how will tg know to use that profile and that alias etc

slaughtr
So you’re still going to have your [module.tf> and <http://main.tf main.tf](http://module.tf), right?
Brij S
Keep your Terraform code DRY

Learn how to achieve DRY Terraform code and immutable infrastructure.

slaughtr

Ah ok I think you may have misunderstood.

slaughtr

In a separate repo, called, for example, live, you define the code for all of your environments, which now consists of just one terragrunt.hcl file per component

Brij S

my modules are in a different repo

Brij S

i am creating a live repo

slaughtr

So you have your .tf files like normal, you point at them in the .hcl files.

Brij S

like this

terraform {
  # Deploy version v0.0.3 in stage
  source = "git::[email protected]:foo/modules.git//app?ref=v0.0.3"
} 

?

slaughtr

Yup!

Brij S

no thats not what im getting at

Brij S

I havea module .. like the one you can find on terraform registry

slaughtr

All TG is doing here is doing inputs/outputs for TF. It doesn’t control your providers or anything.

Brij S

my root .hcl file

remote_state {
  backend = "s3"
  config = {
    bucket         = "test-terraform-state-us-west-2"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = "us-west-2"
    encrypt        = true
    dynamodb_table = "test-lock-table-us-west-2"
    profile        = "profile1"
  }
}
Brij S

tg wont automatically assume to use profile1 for everything?

Brij S

since all the child .hcl files inhert from this

slaughtr

It will, yes. But using the provider block in your tf will work as it used to. Just like tf uses whichever profile you’re using and then follows the directive of the provider block

slaughtr

If you have a child .hcl that shouldn’t use profile1 at all you can not use the path = find_in_parent_folders() bit

Brij S

but then it wont use the remote state bucket?

Brij S

also, let me try this out now, with the changes weve discussed. I think i’ll run into a problem with passing the output though - lets see

slaughtr

Here’s my root terragrunt.hcl using two accounts, if that helps

slaughtr

Though I assume you’re not looking at dev vs prod necessarily, so that might not be the most helpful

slaughtr

But it gives you an idea of conditionally changing profiles at that level

Brij S

ok let me see

slaughtr

To be clear I’m just doing dev and prod and applying the same resources to them conditionally based on an env var (which is sent in via an alias, I have tg and tgprod to be safe)

slaughtr

TG can be a bit to wrap your head around, especially since 0.19 introduced the .hcl files

Brij S

yeah its a bit confusing

Brij S

even going from TF -> TG

slaughtr

Pre 0.19 it was much less confusing. Though now that I’ve got it all setup I much prefer the current method.

Brij S

true, let me give this a go - i’ll message back on this thread tomorrow if i need any more assistance

Brij S

thank you for all the help by the way!! much appreciated

slaughtr

No problem, I know what a nightmare it can be trying to migrate. Hopefully it does the thing and you can focus on more fun stuff

slaughtr

Feel free to throw a message at me if you still need help!

Brij S

didnt work

Brij S

its complaining about multiple providers with the same name

Brij S
Error: Duplicate provider configuration

  on [provider.tf](http://provider.tf) line 1:
   1: provider "aws" {

A provider configuration for "aws" with alias "digital" was already given at
[main.tf:6,1-15](http://main.tf:6,1-15). Each configuration for the same provider must have a distinct
alias.
Brij S

even though its only once..

Brij S
.
├── global
│   ├── [main.tf](http://main.tf)
│   └── terragrunt.hcl
├── terragrunt.hcl
└── us-east-1
    ├── [main.tf](http://main.tf)
    ├── terragrunt.hcl
    └── [variables.tf](http://variables.tf)
Brij S

main.tf inside global is

provider "aws" {
  region  = "us-west-2"
  profile = "profile1"
}

provider "aws" {
  region              = "us-west-2"
  profile             = "profile2"
  alias               = "digital"
}

terraform {
  backend "s3" {}
}
slaughtr

Hmm that’s weird, that should work afaik.

slaughtr

One thing to look out for with TG is the terragrunt get -update command doesn’t seem to work very well in 0.19, so when you make changes you often need to run a command to delete cached files:

alias tg-cache-list='find . -type d -name ".terragrunt-cache"'
alias tg-cache-del='find . -type d -name ".terragrunt-cache" -prune -exec rm -rf {} \;'

That’s from the tg docs somewhere. Catches me on occasion. There’s a chance something is cached

slaughtr
Oh, uh on [provider.tf> line 1: in your error…do you have a <http://provider.tf provider.tf](http://provider.tf) file? that would cause problems if it’s also got a non-aliased provider in it
getSurreal

replying in threads would be appreciated

1
Brij S

sorry about that

2020-01-01

    keyboard_arrow_up