#terragrunt

Terragrunt discussions Archive: https://archive.sweetops.com/terragrunt/

2019-10-18

Milos Backonja

Hi Guys, Should this create tags for backend resoruces (dynamodb and s3)? s3 and dynamodb are created with terragrunt, but without tags

Todd Lyons

Possibly a dumb question: I import aws resources that were manually created into terraform configs, and generally have gotten decent at it. (It helps that the latest terraform state show prints things out in hcl format.) The concept of importing things into terragrunt seems nearly impossible to get right. Would I be off base for settling on this:

  1. New resources: use terragrunt and terraform modules for all new resources.
  2. Existing resources: use straight terraform when need to import existing infrastructure.
  3. New resources: terragrunt is acceptable when work flow is to create and destroy resources but without service disruption.

2019-10-08

2019-10-07

antonbabenko

You can use functions like list() or element() inside of inputs to modify as what you want from the outputs.

hi @antonbabenko, thank for verification. Can you express more detail how to achieve that result ? Below is detail of my case:

  • http://output.tf in vpc module ``` output “internal_ssh_security_group_id”{ value = “${aws_security_group.internal_ssh.id}” }

output “external_ssh_security_group_id”{ value = “${aws_security_group.external_ssh.id}” }

Here is working example in other input declare with the actual sg-id get from aws console:

inputs = { profile = “xxx-sg1” region = “ap-southeast-1” name = “xxx” environment = “staging” number_hosts = “1” instance_type = “t2.micro” key_name = “xxx-staging” security_groups = [“sg-id-xxx”,”sg-id-xxyy”]

I want the security_groups receive value from those 2 above output in vpc module, which means it make as a list

dependency “vpc” { config_path = “../vpc” } terraform_version_constraint = “<0.12” include { path = find_in_parent_folders() }

terraform { source = “git://[email protected]/redcrane/sandbox/terraform-modules/linux-bastion.git?ref=staging>” }

inputs = { profile = “xxx-sg1” region = “ap-southeast-1” name = “xxx” environment = “staging” number_hosts = “1” instance_type = “t2.micro” key_name = “xxx-staging” security_groups = [“dependency.vpc.outputs.external_ssh”, “dependency.vpc.outputs.external_ssh” ] } ```

antonbabenko

Remove double quotes from the values - dependency.vpc.outputs.external_ssh

hi @antonbabenko

I tried as you suggest

but when run with terragrunt init –terragrunt-source-update

It show error: Underlying error: invalid primitive type name “list”

here is what I declare in module’s http://variable.tf :

variable "sg" {
  type  = "list"
  default = []
 }

And here is in terragrunt.hcl:

inputs ={
.
.
.
sg                         =[dependency.vpc.outputs.internal_ssh_security_group_id,dependency.vpc.outputs.external_ssh_security_group_id]

outputs in module:

output "internal_ssh_security_group_id"{
  value = "${aws_security_group.internal_ssh.id}"
}

output "external_ssh_security_group_id"{
  value = "${aws_security_group.external_ssh.id}"
}

here is the result when running terraform out -json in .cacheterrafrom in ../vpc:

"external_ssh_security_group_id": {
        "sensitive": false,
        "type": "string",
        "value": "sg-0bdebf4a204c92cda"
    },
    "internal_ssh_security_group_id": {
        "sensitive": false,
        "type": "string",
        "value": "sg-02a7ed268ef320cbb"
    },
antonbabenko

Underlying error: invalid primitive type name "list" - is the problem. Terragrunt passes values correctly it seems. What versions of terraform and terragrunt are you using? Can it be that terragrunt does not work with your older version of terraform?

well I use Tf ver 0.11 due to requirement, and I use terragrunt version v0.19.27 with terraform constraint.

terraform_version_constraint = “<0.12”

2019-10-06

Hi guys

Is there anyway I can use terragrunt.hcl with a module that contains a module inside it ?

for example:

In the terragrunt.hcl


terraform_version_constraint = "<0.12"
include {
  path = find_in_parent_folders()
}

terraform {  
  source = "git:<i class="em em-<ssh"></i>//xxx/custom_ecs_cluster.git?ref=terraform_0.11>"
}

inputs = {
  name                          = "xxx"
  profile                       = "xxx"  
  region                        = "xxx"
}

The module custom_ecs_cluster http://main.tf’s content:

module "aws_ecs_cluster" {
  source = "git:<i class="em em-<ssh"></i>//xxx:aws-ecs-cluster.git?ref=terraform_0.11>"
loren

we do this all the time. hasn’t been anything special to it. it just works…

also, does anyone know how to transfer output of other dependency as list value ? as I declare the output in the module and testing with showing output successfully.But when running the terragrunt apply. it keep show the error as “ Underlying error: invalid primitive type name “list” example:

dependency "vpc" {
  config_path = "../vpc"
}
terraform_version_constraint = "<0.12"
include {
  path = find_in_parent_folders()
}

terraform {  
  source = "git:<i class="em em-<ssh"></i>//[email protected]/redcrane/sandbox/terraform-modules/linux-bastion.git?ref=staging>"
}


inputs = {
  profile                       = "xxx-sg1"  
  region                        = "ap-southeast-1"
  name                          = "xxx"
  environment             = "staging"
  number_hosts           = "1"
  instance_type            = "t2.micro"
  key_name                   = "xxx-staging"
  security_groups         [dependency.vpc.outputs.sg_id] <== this part is that allowed ? Or is there anyway that I can parse the output as list value to this   
}

2019-09-16

Sharanya

quick question

  • terragrunt –version /var/lib/jenkins/workspace/[email protected]/durable-dfb884a1/script.sh: line 1: terragrunt: command not found [Pipeline] } —- Have this error in my Jenkins build…. have any one come across this ever ?
aaratn

@Sharanya is terragrunt in your path ?

Sharanya

yes

aknysh

in the PATH or is installed

aaratn

+1

Sharanya

i dont think terragrunt in installed on the server

aaratn

You will need to install it

2019-09-11

2019-09-10

Mariano Godoy

Hi guys!, does anyone know how to solve dependencies between modules without having to run a terragrunt plan-all from the root directory? This is our current directory structure

root
├── dynamodb
│   ├── TableOneDir
│   │   └── terragrunt.hcl
│   ├── TableTwoDir
│   │   └── terragrunt.hcl
├── lambda
│   ├── LambdaOneDir
│   │   └── terragrunt.hcl
│   ├── LambdaTwoDir
│   │   └── terragrunt.hcl
├── secrets
│   └── terragrunt.hcl
├── sns
│   └── terragrunt.hcl
└── terragrunt.hcl

We need to run plan only in the LambdaOne directory and auto resolve a dependecy of a TableOneDir existing.

Also this are out hcl examples

terraform {
  source = "git:<i class="em [email protected]"></i>Company/core-of-sources.git//lambda/LambdaOneDir?ref=branch-one"
}

# Include all settings from the root terragrunt.hcl file
include {
  path = find_in_parent_folders()
}

# Dependencies
dependencies {
  paths = ["../../dynamodb/TableOneDir"]
}
loren

there are some very new features of terragrunt to support this… make sure you are using the latest version and then read these sections in the readme:

https://github.com/gruntwork-io/terragrunt#passing-outputs-between-modules

https://github.com/gruntwork-io/terragrunt#unapplied-dependency-and-mock-outputs

@cytopia

gruntwork-io/terragrunt

Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules. - gruntwork-io/terragrunt

Gustavo Adrián Crespi

Nice feature! But still we have a problem we don’t really know how to solve. Say we modify our implementation of the LambdaOne module and also some configuration on dynamo’s TableOne we need for the same feature, then we would like to run a plan for those changes. We would like only to run terragrunt plan inside of the LambdaOne folder and doing so we expect it automatically to cascade the plan to its dependencies (in this case TableOne) is that possible? We understand that running terragrunt plan on the root folder would solve this problem but since we are using Atlantis doing so would automatically lock all the directory and won’t allow us to work concurrently with other team mates.

loren

no, not using plan. but plan-all should do exactly that

loren

oh, but right, that will lock all the other directories, hrm…

loren

maybe open a feature request for a new flag --terragrunt-process-dependencies or somesuch

Gustavo Adrián Crespi

You are right! plan-all is what we need to use, but I’ve found there is a problem with the non-interactive mode and that is that while setting this option on, there is no way to turn on the option to also apply the command on the dependencies of the module. I’ve spotted the Issue here: https://github.com/gruntwork-io/terragrunt/issues/860

Setting to apply external dependencies in Non-interactive mode · Issue #860 · gruntwork-io/terragrunt

Today&#39;s behaviour defaults to not applying external dependencies in non-interactive mode as we can see in these lines of code. I would like to have some configuration (for example a cli option)…

AgustínGonzalezNicolini

@Gustavo Adrián Crespi we may also consider opening a PR and colaborating

AgustínGonzalezNicolini

2019-09-06

antonbabenko

Hi guys! @loren Have you found a solution to this https://github.com/gruntwork-io/terragrunt/issues/785 ? I wonder how people are solving odd (“new”) behavior of init hooks?

postpone check for configured backend after init hook is executed · Issue #785 · gruntwork-io/terragrunt

Change-Request: Postpone check for checkTerraformCodeDefinesBackend() after before_hooks are run. I guess it makes sense to run a before_hook before deciding how to actually execute theterraform co…

loren

solve? no… workaround, yes… it’s not pretty

loren

basically just gitignoring the copied file

loren

in terragrunt.hcl

  after_hook "provider" {
    commands = ["init-from-module"]
    execute  = ["cp", "${get_terragrunt_dir()}/../../provider.tf", "."]
  }
loren

in .gitignore

# Terragrunt files copied by hooks
dev/**/provider.tf
loren

i tried your workaround of having a hook delete the file, but turns out we actually need it there because terragrunt tracks the files it copies from the terragrunt dir to its cache working dir, and purges anything from the cache working dir that it didn’t copy

antonbabenko

Great, I ~solved~ workaround it exactly as you said. I am not sure there will be a proper fix in the near future. It starts to look a big ugly here and there with terragrunt…

loren

yeah, i think we need the workflow to be part of terragrunt proper, rather than implemented with hooks

loren

it’s not something the gruntwork folks do, so they just don’t see it as a real use case at the moment

1
antonbabenko

As another not elegant hack I can run terragrunt plan --terragrunt-source-update to get source reinitiated every time, then I can have after_hook which will remove the file.

loren

Right, yeah, that also works, but is slow

1
Adrian

add some dep between modules and its very slow

1

2019-08-14

Adrian

Can someone help with remote state references? Where to put

    data "terraform_remote_state" "vpc" {
    backend = "s3"
    config {
      bucket  = "${var.remote_state_bucket}"
      region  = "${var.aws_region}"
      key     = "${var.remote_state_key}"
      profile = "${var.aws_profile}"
    }
  }
ruan.arcega

i usually do like this, create http://data.tf file and put in there in the module i call the remote state with this way, example: ${data.terraform_remote_state.vpc.id}

Adrian
In previous releases, a reference to a vpc_id output exported by the remote state data source might have looked like this:

data.terraform_remote_state.vpc.vpc_id
This value must now be accessed via the new outputs attribute:

data.terraform_remote_state.vpc.outputs.vpc_id
ruan.arcega

oh yes, sorry i forgot *.outputs.* does it works for you ?

Adrian

yes , that one piece was missing, just put data.terraform_remote_state…. in module and its works. I’m searching for solution to keep it dry right now

Adrian

this should goes to module configuration?

2019-08-13

2019-07-31

2019-07-30

Hey all - has anyone figured out a way to use a before_hook to set environment variables that terragrunt can use?

I’m using direnv for directory based variables which works fantastic for single plan/applies, but it won’t work with an apply-all

Erik Osterman

It’s possible if you call direnv exec on the command in each directory

Erik Osterman

But as @joshmyers hinted, it’s a bit dangerous

That doesn’t seem to work with a before hook

I have .envrc files in the child directories with overrides and region/module specific config

Erik Osterman

can you share what you tried doing?

So I have a directory structure like so

account-id – nonprod\ —- test\ —— ap-southeast-2\ ——– app_name\ ———- .envrc ———- terragrunt.hcl ——– .envrc —- .envrc —- terragrunt.hcl

Each child directory can have an .envrc file that loads environment variables specific to it or it’s child directories (eg. region specific variables for a region folder and it’s children)

I tried using a before_hook to run a direnv reload - but that won’t work as it spawns a new child process

I can’t think of any way to get it to set environment variables before a terragrunt execution - so I thought about writing variables to a tfars. But this is going to block that idea: https://github.com/hashicorp/terraform/issues/19424

Terraform 0.12.0-alpha no longer allows assignments in terraform.tfvars without a corresponding variable declaration · Issue #19424 · hashicorp/terraform

Terraform Version Terraform v0.12.0-alpha2 + provider.google v1.19.1-4-gf3de5334 + provider.null v1.0.0-5-gf54ff98 Expected Behavior Terraform v0.11.10 and earlier allowed assignments to occur in t…

Erik Osterman

are you using make too?

Not for any heavy lifting - just to run some basic scripts (eg. make new account creates a new terragrunt folder structure based on a template)

joshmyers

Isn’t apply all a bit YOLO, considering you can’t plan-all and have it correctly reflect dependency changes as the apply on the former hasn’t run yet…?

1

Only looking to do it for an initial spin up

davidvasandani

@ if you figure this out, I’m interested in how you do it.

Unfortunately not

So create new aws account, run apply-all

1

2019-07-29

Sharanya

Hey Folks, Trying to find some Terraform Modules related to AWS - app stream service ( for creating fleets and stacks) any help appreciated

2019-07-09

aaratn

Is anyone having issue with terraform 0.12 and terragrunt 0.19.8 not picking up credentials parameter when using gcs as backend ?

aaratn
terragrunt] [/Users/aarat/git/terraform/variables/staging] 2019/07/09 21<i class="em em-36"></i>04 Initializing remote state for the gcs backend
[terragrunt] 2019/07/09 21<i class="em em-36"></i>04 dialing: google: could not find default credentials. See <https://developers.google.com/accounts/docs/application-default-credentials> for more information.
aaratn

I am seeing this and unable to initialize

loren

there have been a handful of changes to GCS backend in recent versions, maybe try 0.19.5? if that works, open an issue describing your config and use case

loren
Terragrunt 0.19.6 breaks existing GCP config · Issue #767 · gruntwork-io/terragrunt

A working config on 0.19.5 now fails on 0.19.6 (likely due to GCP changes) I have this as a parent remote_state { backend = &quot;gcs&quot; config = { bucket = &quot;project-factory-592341-tfstate&…

2019-06-26

Nehal

HI all, just . quick one, I need some help installing terragrunt v18 on mac os, I’ve managed to do it via brew but that installs a dependancy of terraform v12 (i need v11). I went to releases, but not sure how to install Darwin_64 on mac? Any ideas, thanks

antonbabenko

@Nehal You can remove terraform after it has been installed by terragrunt, or use tfenv to manage terraform installation.

1
Nehal

thanks..didn’t know about tfenv, look into it

antonbabenko

https://github.com/tfutils/tfenv - this is the one I am talking about

tfutils/tfenv

Terraform version manager. Contribute to tfutils/tfenv development by creating an account on GitHub.

4

2019-06-14

2019-06-13

Hi @loren Have you thought best practices yet for running pre-0.19 with post 0.19 ? Seperate tree structure for post 0.19, or having both .tfvars and .hcl in one structure.

Erik Osterman

Use direnv with envrc :-)

Erik Osterman

In a geodesic shell, of course

Hi Erik! For the binaries i see that working, but not so much for the tfvars / .hcl implementqtion

Erik Osterman

Our strategy is that a given folder should only use one version of terraform. It doesn’t make sense to have 2 versions of terraform in the same project folder

Erik Osterman

I guess the problem with terragrunt is you have it operate across folders of different versions?

Erik Osterman

A project folder for us is a root module

well with terragrunt you can have a tree model with tfvars inheritance more or less, provider, account and region specific information can be autofilled this way. But this does mean that the tree is being traversed and being searched for other specific tfvars. From 0.19 this information is not in tfvars anymore but in .hcl, so in case one wants to slowly move certain root modules in this structure to v0.19/tf 0.12 that is something to be taken care of.

1
loren

I wanted to be able to update quickly to tf 0.12 but with backwards compatible module changes, then leisurely update modules to take advantage of new syntax. Unfortunately there are just way too many, very basic (IMO) backwards incompatible changes in tf 0.12

loren

So now it’s likely to be quite a while before we update active root configs to tf 0.12 since we have to go all in to do it

loren

Also means probably won’t be using terragrunt 0.19 until then, and when we do we will also begin using it across all configs in the project simultaneously

loren

Sucks because I want to use tf 0.12 now, but the backwards incompatibility means we won’t be able to adopt it for quite some time

loren

So I guess, to answer the actual question, no, I haven’t thought about how to use tg 0.19 and tg <0.19 in the same project since at the moment I’m expecting eventually we’ll update everything at once

Thanks, sounds like you’re stuck with the same problem, and probably means we need to start with 0.12

1

2019-06-10

loren
gruntwork-io/terragrunt

Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules. - gruntwork-io/terragrunt

1

2019-06-08

loren
Terraform 0.12 support by brikis98 · Pull Request #731 · gruntwork-io/terragrunt

This PR upgrades Terragrunt to work with Terraform 0.12. Key changes: We are migrating from having the Terragrunt configuration in terraform.tfvars to terragrunt.hcl. This is necessary because Te…

1

2019-05-31

Thanks for the feedbacl & Rex. Good call on these

2019-05-08

Joe Perez

Is it me or do the terragrunt docs seem lacking? I’ve been having to dig through git issues to find out more. Any external doc references are appreciated

Erik Osterman
Gruntwork Community Forum

The Community Forum for Gruntwork customers

Joe Perez

thank you

Erik Osterman

we (cloudposse) don’t use terragrunt

Joe Perez

I really think the tool is great for what it’s trying to do, it’s just a bit frustrating

1
Jonathan Le

yeah - having used terragrunt where i’m currently working, i can’t recommend it again. it’s very opinionated in how modules are created and used with the “source=” stuff in the tfvars file.

trying to have more than 1 module in a state when not everything is copied to the “.terragrunt-cache” directory really annoyed me.

1
Joe Perez

@Jonathan Le I thought each module in terragrunt was supposed to have it’s own state file? What were you trying to accomplish with multiple modules in a single state file?

1
Jonathan Le

You are right - each state should be it’s own module. Like 1 module per app/stack.

For me, TG and this method caused me to have a lot of copy/paste between modules though.

1
Jonathan Le

Like, there’d be a basic model of what an EC2 instance was to me and it would have been great to have this as a module like I had without TG.

I had to copy/paste between the resource configuration between similar TG modules to pull them off and the tradeoff really annoyed me.

1
Jonathan Le

It was great to have cool stuff like the following to try to keep things “dry”:

1
Jonathan Le

but the very similar resource config across TG modules was super duper annoying.

1
Joe Perez

yeah that doesn’t look fun

1
Joe Perez

are you talking about sharing stuff between modules too?

1
Jonathan Le

not sure what you mean about “sharing stuff between modules”. if you mean about datasources between the different module states, that works fine with TG.

if you mean about trying to stay DRY, i didn’t feel i was able to with TG.

e.g, I have very similar looking resource config for ec2 instances across various modules:

1
Jonathan Le
1
Joe Perez

ahhhh we’re using a segregated ec2 module, we’re not embedding it into other modules as part of a full “service”

1
Joe Perez

what do you guys use?

Erik Osterman

terraform natively supports intialization of projects using the -from-module parameter

Erik Osterman

you can pass that as an environment variable too

Erik Osterman

TF_CLI_ARGS_init=-from=module=....

Erik Osterman

so we define all of our root module invocations here: https://github.com/cloudposse/terraform-root-modules

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Erik Osterman

we wrote an “env mapper” to make setting envs for terraform easier (but it’s not required)

Erik Osterman
cloudposse/tfenv

Transform environment variables for use with Terraform (e.g. HOSTNAMETF_VAR_hostname) - cloudposse/tfenv

Erik Osterman

here’s how we put it together:

Erik Osterman
cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

Erik Osterman

the .envrc indicates the module we want to use

Erik Osterman

then we just call terraform init and it automatically imports the remote module

Erik Osterman

no wrappers required

Joe Perez

damn that’s nice

Joe Perez

I was worried about what happens when terraform is updated when using terragrunt

Joe Perez

I’m gonna need to dig through all of this

Erik Osterman

yea, that’s what i like about this.

Joe Perez

what about all the state stuff that terragrunt does for you, how are you guys handling that?

Erik Osterman

terragrunt is basically a task runner

Erik Osterman

there are a lot of task runners out there.

Erik Osterman

we use make predominantly with terraform

Erik Osterman

terragrunt helps keep your code DRY, but if you decompose thee different things that terragrunt does, you can easily accomplish it other ways

Erik Osterman

we use direnv to keep project level settings

Erik Osterman

we use make for a task runner

Erik Osterman

we use tfenv to make mapping envs easier (totally optional)

Erik Osterman

we use terraform init -from-module=... (using the TF_CLI_ARGS_init=-from-module=... env), to keep things DRY.

Whats the use for -from-modules= I never understood this workflow. Could you elaborate?

Erik Osterman

This allows you to bootstrap modules. Terragrunt uses it (if you look under the hood) too.

Erik Osterman

So in our case, we have a central catalog of “root” modules. Every time we merge to master, we cut a release.

Erik Osterman

When we want to use that module for an environment (e.g. prod) we call terraform init with -from-module to download it. That way we never copy/paste the code. It also allows us to have different environments pinned to different versions.

Erik Osterman

Terragrunt does the same thing. Only it writes it to a cache folder

Thanks, that part was clear tho, but why not just

module "this" {
source = "sourcelink"
vars = "that"
}

sort of thing

what is the advantage?

Erik Osterman

the combination of these things create a generalizable pattern that works formore than terraform

Erik Osterman

for example, we use helm and helmfile

Erik Osterman

we package all the tools in geodesic, which is our cloud automation image

Erik Osterman
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

1
1
Erik Osterman

we also use variant to define cli tools specific to our organization.

1
Erik Osterman
mumoshu/variant

Wrap up your bash scripts into a modern CLI today. Graduate to a full-blown golang app tomorrow. - mumoshu/variant

Erik Osterman

We have lots of examples of variant in use here: https://github.com/cloudposse/geodesic/tree/master/rootfs/usr/local/bin

cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Joe Perez

this is all really good stuff to take back to my team

Erik Osterman

cool! happy to share more or answer any questions.

Joe Perez

this is a great start, thank you

2019-03-01

2019-02-28

Tobias Hoellrich

Good afternoon #terragrunt - I have an odd situation I can’t explain and was wondering if one of you had any insight.

Tobias Hoellrich
11:25:57 PM
loren

What happens if you run a) without –terragrunt-source?

Tobias Hoellrich

@loren - same thing happens: terragrunt wants to create all the wave resources.

loren

And the contents of wave/terraform.tfvars? And do you have a parent terraform.tfvars?

loren

At least the source line, if vars are sensitive

Tobias Hoellrich
terragrunt = {
  terraform {
    source = "git:<i class="em em-<ssh"></i>//[email protected]ithub.com/xxxxx/terraform-modules.git//wave>"
  }

  # dependencies for wave
  dependencies {
    paths = ["../vpc", "../route53", "../securitygroups"]
  }

  # Include all settings from the root terraform.tfvars file
  include = {
    path = "${find_in_parent_folders()}"
  }
}

terraform {
  backend "s3" {}
}

# -------------------------------------------------------------------------------------------------------

wave_es_cluster_name = "events"
Tobias Hoellrich

And yes, there is a parent terraform.tfvars. It sets up s3 remote states and dynamo locking.

loren

I think the terraform backend bit is off… That should be in a .tf file, not .tfvars

loren

I’ll have to take a look at my own setups when back at a computer to compare

Tobias Hoellrich

You mean this from the parent terraform.tfvars?

Tobias Hoellrich
terragrunt = {
  # Configure Terragrunt to automatically store tfstate files in an S3 bucket
  remote_state {
    backend = "s3"

    config {
      encrypt        = true
      region         = "ca-central-1"
      s3_bucket_tags {
        creator        = "terraform"
        terraform      = "true"
        purpose        = "canada"
        name           = "terraform state storage"
      }

      dynamodb_table_tags {
        creator        = "terraform"
        terraform      = "true"
        purpose        = "canada"
        name           = "terraform lock table"
      }
    }
  }
Tobias Hoellrich

(truncated)

loren

No, sorry, I can’t copy/paste easily on my phone, I mean the block with backend "s3" {} in wave/terraform.tfvars

Tobias Hoellrich

ok - i’ll look

Tobias Hoellrich

I removed it, but still see the same situation: plan-all wants to create all wave-resources; plan-all --terragrunt-include-dir wave does not want to create them; plan inside the wave directory also does not want to create them.

loren

ok, i’m not seeing anything, but have the sense it’s got to be something fundamental that is just easy for the eyes to pass over

loren

maybe open an issue on the terragrunt repo… they’re pretty decent about responding to this kind of help request

Tobias Hoellrich

tks for your help!

Tobias Hoellrich

And I’m ashamed to admit that this was a stupid mistake where I had started a new component in a parallel directory, which had the terraform.tfvars file from the wave-directory in it. Hope I did not waste too much of your time, @loren. Thanks again!

loren

Aha! That’ll do it! No worries, glad you figured it out!

Tobias Hoellrich

~Any ideas why plan-all in the root wants to create resources that already exist?~ This turned out to be a stupid mistake where I had created a new component in the live directory and it had a copy of the wave terraform.tfvars in it …

2019-02-14

Erik Osterman
05:23:04 AM

@Erik Osterman set the channel purpose: Terragrunt discussions Archive: https://archive.sweetops.com/terragrunt/

@Erik Osterman how are we suppose to use terragrunt in your new setup. I see eligible terraform.tfvars only in atlantis-repo sub-module.

Erik Osterman
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Erik Osterman

this is all you need

Erik Osterman

that said, I see no strong use-case for it anymore

Erik Osterman

I think make is a better task runner

Erik Osterman

and tfenv let’s use define module imports

Erik Osterman
cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

want to avoid tfenv, it seems over populating the environment by appending all current env variables with TF_VAR

Erik Osterman

with tfenv, we don’t need any wrappers to call terraform

Erik Osterman

fair enough

Erik Osterman
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Erik Osterman

i just don’t like this

Erik Osterman

overloading .tfvars with a non-portable terraform code that is vendor specific to terragrunt

also i am thinking may be its kind of simpler if we have a whitelisting variables rather then excluding regex

Erik Osterman

while envs are universal across apps

Erik Osterman
06:16:40 AM
Erik Osterman

TFENV_WHITELIST is supported

I saw that, I was saying more about ideology. The whitelist value is .*.

Erik Osterman

what i don’t like is terraform telling me how envs should look in the first place. just like i don’t like chamber telling me that envs must be upper case (which doesn’t work with terraform)

Erik Osterman

so we have env warfare

Should be something like geodesic whitelist, and customer whitelist. Where we convert all the values required for geodesic modules. SPECIALLY the ones which we are currently getting

Erik Osterman

I just wish terraform and terragrunt would not stipulate a convention on envs and then things would just work. so tfenv is an ambassador. it does the dirty work.

the approach makes sense in the current context, its just having too much env variables makes my life hard

Erik Osterman

just set TFENV_WHITELIST and TFENV_BLACKLIST in your environment however you want

Erik Osterman

we were so busy mapping envs from one tool to the next. that’s what we wanted to avoid.

too much work to make a module get up. Its as good as doing a export variable in a tfenv.sh

Erik Osterman

so setting too many envs makes everyones life hard

Erik Osterman

but the env always has many envs, that’s just a fact-of-linux

Erik Osterman

on OSX, I have 87 envs that i never set

Erik Osterman
export\|wc

i have total 57, out of which I know about atleast 40

Erik Osterman

so in our Dockerfile we had SOOOOOO many envs. it was unmanageable.

Erik Osterman

so we’ve gotten rid of most of them (compared to before)

Erik Osterman

then moved to .envrc (direnv) so we localize these settings

yeah regarding that, I saw below snippet in rc.d terraform

# Translate environment variables to terraform arguments
	[ -z "${TF_FROM_MODULE}" ] \|\| export TF_CLI_INIT_FROM_MODULE="${TF_FROM_MODULE}"
	[ -z "${TF_STATE_FILE}" ] \|\| export TF_CLI_INIT_BACKEND_CONFIG_KEY="${TF_BUCKET_PREFIX}/${TF_STATE_FILE}"
	[ -z "${TF_BUCKET}" ] \|\| export TF_CLI_INIT_BACKEND_CONFIG_BUCKET="${TF_BUCKET}"
	[ -z "${TF_BUCKET_REGION}" ] \|\| export TF_CLI_INIT_BACKEND_CONFIG_REGION="${TF_BUCKET_REGION}"
	[ -z "${TF_DYNAMODB_TABLE}" ] \|\| export TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE="${TF_DYNAMODB_TABLE}"
	[ -z "${AWS_PROFILE}" ] \|\| export TF_CLI_INIT_BACKEND_CONFIG_PROFILE="${AWS_PROFILE}"
	[ -z "${AWS_ROLE_ARN}" ] \|\| export TF_CLI_INIT_BACKEND_CONFIG_ROLE_ARN="${AWS_ROLE_ARN}"
Erik Osterman

yea, that’s strictly for backwards compatibility

Erik Osterman

I didn’t want to tell everyone to rewrite their envs

Erik Osterman

the TF_CLI_* convention is canonical

which i assume should be good enough if we have all variables in left hand side set

Erik Osterman

~the ones on the LHS are the legacy ones we’ve had in our docs~

Erik Osterman

yeah, my point even if we are using the legacy variables, things should work

Erik Osterman

I mean the ones in the [ ... ] conditionals

e.g TF_BUCKET_REGION

Erik Osterman

yep, so using that mapping they will continue to work

Erik Osterman

but TF_CLI_INIT_BACKEND_CONFIG_REGION is canonical

Erik Osterman

in that it maps precisely and consistently to the TF_CLI_ARGS_init=-backend-config=region=blah which is the terraform native convention

Erik Osterman

technically, even tfenv isn’t needed. it’s just a convenience.

Erik Osterman

so we can set k/v pairs as ENVs

Erik Osterman

rather than mucking with the tf flags in compacted envs

this is what I am doing currently

Erik Osterman

the compacted envs?

Erik Osterman

I don’t like terraform native envs

Erik Osterman
TF_CLI_ARGS_init="-from-module=git:<i class="em em-<https"></i>//github.com/cloudposse/terraform-root-modules.git//aws/ecs?ref=tags/0.40.0> -backend-config=region=us-west-2 -backend-config=dynamodb_table=cpco-testing-terraform-state-lock -backend-config=bucket=cpco-testing-terraform-state -backend-config=key=ecs/terraform.tfstate"
Erik Osterman

that’s what it looks like.

Erik Osterman

PIA to toggle individual fields

Erik Osterman

so we just came up with a convention to not do that. but that’s opinionated and might not suit all parties.

Erik Osterman

I like it b/c you can set envs at different levels (E.g. Dockerfile, project, parent folder, etc)

i was doing something like export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=niki-root-terraform-state

Erik Osterman

that looks good

but its still asking me for bucket name

Erik Osterman

TF_CLI_INIT_BACKEND_CONFIG_BUCKET="cpco-testing-terraform-state"

Erik Osterman

that’s what we are doing

Erik Osterman
TF_CLI_PLAN_PARALLELISM=2
TF_CLI_INIT_BACKEND_CONFIG_REGION=us-west-2
TF_CLI_INIT_FROM_MODULE=git:<i class="em em-<https"></i>//github.com/cloudposse/terraform-root-modules.git//aws/ecs?ref=tags/0.40.0>
TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=cpco-testing-terraform-state-lock
TF_CLI_INIT_BACKEND_CONFIG_BUCKET=cpco-testing-terraform-state
TF_CLI_INIT_BACKEND_CONFIG_KEY=ecs/terraform.tfstate

Erik Osterman

this definitely works with every terraform * command

Erik Osterman

if you use tfenv to cast it to the TF_CLI_ARGS_blah=....

in short with my legacy variables, use_terraform and terraform init should give me required resuly

Erik Osterman

true

Erik Osterman

# Terraform State Bucket
ENV TF_BUCKET_REGION="${AWS_REGION}"
ENV TF_BUCKET="${NAMESPACE}-${STAGE}-terraform-state"
ENV TF_DYNAMODB_TABLE="${NAMESPACE}-${STAGE}-terraform-state-lock"

i can’t use use_terraform also

Erik Osterman

is what we have in our docker file

has to source it manually

Erik Osterman
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

something like this is happening

Erik Osterman
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

 ✓  (root-admin) tfstate-backend ⨠  use_terraform 
 ✓  (root-admin) tfstate-backend ⨠  printenv \| grep TF_CLI
TF_CLI_INIT_BACKEND_CONFIG_REGION=ap-south-1
TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=niki-root-terraform-state-lock
TF_CLI_INIT_BACKEND_CONFIG_BUCKET=niki-root-terraform-state
TF_CLI_INIT_BACKEND_CONFIG_PROFILE=root-admin
TF_CLI_INIT_BACKEND_CONFIG_KEY=tfstate-backend/terraform.tfstate
 ✓  (root-admin) tfstate-backend ⨠  terraform init
Initializing modules...
- module.tfstate_backend
- module.tfstate_backend.s3_bucket_label
- module.tfstate_backend.dynamodb_table_label

Initializing the backend...
bucket
  The name of the S3 bucket

  Enter a value: 
Erik Osterman

zoom?

2019-02-05

Samuli

any best practices on how to reference between terraform dependencies on different terragrunt modules. eg. I have vpc as a module and bastion as another module (terragrunt modules both) and I need to reference the vpc-id (and subnet-ids) from vpc in bastion

antonbabenko

if you want you can try code generated by http://modules.tf - draw vpc+asg in http://cloudcraft.co and click export

antonbabenko

I wrote small shell script which does replacement using hooks

Samuli

looks interesting. Have to give them a try at some point

Erik Osterman

yes, use SSM

Erik Osterman

or use remote state provider

Erik Osterman

we have examples of both in this repo https://github.com/cloudposse/terraform-root-modules

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

joshmyers

@Samuli Are these modules being run in the same Terraform run? (do they share state)

Samuli

It would be straight forward if they did but with terragrunt modules they are not. So I went with remote_state..

2019-01-08

davidvasandani

I noticed terragrunt compatibility for the first time here: https://github.com/cloudposse/root.cloudposse.co/blob/master/conf/terraform.tfvars

Is this something you work to support in addition Geodesic?

1
Erik Osterman

yes, so I wanted to demonstrate that geodesic is a superset of tools including terragrunt

Erik Osterman

for now, we’re still not sold on adopting terragrunt (for our purposes)

Erik Osterman

but I understand people have a large investment in terragrunt (@antonbabenko) so figured we should have a story for showing how it’s done with geodesic.

2
davidvasandani

Awesome and thanks!

2
Erik Osterman
Document Cloud Posse vs Gruntworks · Issue #351 · cloudposse/docs

what Describe our differentiators Gruntworks is an awesome contributor to open source and demonstrate solid engineering skills. They have a vast, well-tested, library of proprietary terraform modul…

2018-12-22

So, I’m just starting out with Terraform. There is not a lot of complexity in our stack, but I want to get off on the right foot and keep things simple. So far, I’m not a big fan of having nearly identical backend configs everywhere, and also I don’t really feel comfortable with the notion of workspaces. Seems like Terragrunt could address these concerns, but it’s a wrapper and I don’t know if it’s a good idea for me to be starting off with it. Thoughts? Should I try to incorporate Terragrunt in to the solution, or keep to Terraform for now and come back when I have more experience and legitimate issues that need to be solved?

Erik Osterman

Good questions… so I think that it’s a good idea to first master the fundamentals of terraform before relying on terragrunt. It’s possible to keep things dry without needing to use the wrapper approach. I’m also not keen on relying on terragrunt as a wrapper, but have used it in a few circumstances.

Erik Osterman

I don’t like how the varfiles are overloaded with interpolations which make them non-portable

Erik Osterman
uber/astro

Astro is a tool for managing multiple Terraform executions as a single command - uber/astro

Erik Osterman

We’re not currently using it, but it’s an interesting alternative

Erik Osterman

We’re using mostly Makefiles if we need any more complex orchestration

Erik Osterman


I’m not a big fan of having nearly identical backend configs everywhere

Erik Osterman
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. https://slack.cloudposse.com/ - cloudposse/geodesic

Erik Osterman

that way we don’t need to hardcode any backend configs in source control

Erik Osterman

Also terragrunt it might take some time before terragrunt supports terraform 0.12 , but that might not matter

Thanks, this is helpful. There is this temptation to try to set everything up perfectly from the get-go, but ultimately it’s a process, and I feel like it’s not healthy to try to skip too many steps in the evolution.

Erik Osterman

Haha, I can relate to that.

Erik Osterman

I don’t think you can necessarily go wrong with using terragrunt.

Erik Osterman

The good thing is you’ll decompose everything nicely into lots of modules and organize things as you should.

Erik Osterman

You can remove terragrunt as well, down the road.

Erik Osterman

….that said, have you seen our strategy?

Erik Osterman

We use docker and multi-stage builds to keep things dry

Erik Osterman

run everything in containers

Erik Osterman

…that is, we run even our “infrastructure as code” in containers

Erik Osterman
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. https://slack.cloudposse.com/ - cloudposse/geodesic

Great, more late night reading material for the holidays! Haha, thanks for that, it looks interesting. And thanks for addressing my Terragrunt question; I agree - seems like the key is to get the modules in place and I’ll have flexibility to adjust my approach in the future.

Erik Osterman

exactly - that’s the most important thing

Erik Osterman

write small reusable, composable modules that build on each other

Erik Osterman

do not embed any stage or environment specific settings - those should be inputs (variables)

Erik Osterman

consider using terraform-null-label to consistently generate resource name

Erik Osterman

look into chamber for secrets

Erik Osterman

look into aws-vault for assuming roles

Erik Osterman

also, recommend studying a lot of other modules to see how they are organized and broken down

Yes, great advice, much appreciated.

2018-12-02

davidvasandani

There’s a #terragrunt channel!!

1
Erik Osterman

But of course! We don’t discriminate :P

antonbabenko

LOL

2018-11-30

Erik Osterman
11:28:34 PM

@Erik Osterman set the channel topic:

antonbabenko

The icon is Gruntwork logo, not terragrunt product ;) There is an open issue about missing logo for terragrunt

Erik Osterman

Haha yea

Erik Osterman

I guess I should rename the icon

Erik Osterman

It was a compromise

antonbabenko

Yeah, np really

antonbabenko

Do you need icon for http://modules.tf ? I will make one :))

Erik Osterman

haha, go for it

2018-11-27

joshmyers
10:32:33 AM

@joshmyers has joined the channel

jerry
12:04:53 PM

@jerry has joined the channel

2018-11-25

Hello everyone, I am trying to run terragrunt plan using circleci and I am getting the following error

 [terragrunt] [/root/project/cloudfront_frontend] 2018/11/26 03<i class="em em-21"></i>58 The non-interactive flag is set to true, so assuming 'yes' for all prompts
[terragrunt] [/root/project/cloudfront_frontend] 2018/11/26 03<i class="em em-21"></i>58 Creating S3 bucket cb-terraform-infra-state-sit
[terragrunt] 2018/11/26 03<i class="em em-21"></i>59 AccessDenied: Access Denied
	status code: 403, request id: 35E32BD7CF45BED5, host id: f3c85r+xX5QKBmRdlhW8JCQyj+AqUtKGZWwLvbuo/YkjErGKUuVJ6kAs0NIjooDC83uXs1JkjXc=
[terragrunt] 2018/11/26 03<i class="em em-21"></i>59 Unable to determine underlying exit code, so Terragrunt will exit with error code 1
Exited with code 1

I have the credentials in place ~/.aws/credentials . It works perfectly in local. Not sure how to fix it

I am using the following docker image antonbabenko/terracing:v0.0.6

It looks like permission issue but I have given fullAdminAccess to the user.

Erik Osterman

hrm… so shouldn’t need to write credentials to ~/.aws/credentials

Erik Osterman

using the standard AWS envs should be sufficient

Erik Osterman

AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

Erik Osterman

in fact, i’d try to get it working with ENVs locally (for testing)

Erik Osterman

then once you find the working combination, that’s what you’ll set in circle

2018-11-22

Bogdan
10:55:32 PM

@Bogdan has joined the channel

2018-11-21

antonbabenko

Hi guys! If you want to check how I do dynamic parameters in static terraform.tfvars with Terragrunt and tell me if you know better solution, you should read my blog post (and try it) - https://medium.com/@anton.babenko/modules-tf-convert-visual-aws-diagram-into-terraform-configurations-e61fb0574b10

modules.tf — Convert visual AWS diagram into Terraform configuration

I am excited to announce public-release of http://modules.tf — project which allows conversion of visual AWS diagrams created using http://Cloudcraft.co

2018-11-19

2018-11-18

04:23:54 PM

@ has joined the channel

2018-11-17

What is the usecase for terragrunt with workspaces module etc?

Erik Osterman

Are you asking generally speaking?

Erik Osterman

Like what is it solving?

In general, or to whoever is still using it. In my workflows it just had too much overlap, so we are not using it anymore.

antonbabenko

@Erik Osterman not yet, but should do it after AWS re:invent.

2018-11-16

loren

pass --terragrunt-source-update

loren

@Vi ^^^

loren

terragrunt uses a hash of the source url to determine whether the source has changed. if you are using tagged/versioned modules, that works great… you update the source ref, and terragrunt will detect the changed hash and automatically pull down the new version

2
loren

if you are not using a version ref in your source, then you can use --terragrunt-source-update to force terragrunt to pull down the source

2

Thank you…

2
Erik Osterman

has anyone played around with the terraform 0.12 rc with terragrunt?

loren

i would not expect terragrunt to work with tf 0.12 just yet, pending some kind of resolution to this issue, https://github.com/gruntwork-io/terragrunt/issues/466

Separate configuration file for Terragrunt? · Issue #466 · gruntwork-io/terragrunt

Hi! I&#39;m one of the engineers at HashiCorp who works on Terraform Core. As you might be aware, we&#39;ve been working for some time now on various improvements to the Terraform configuration lan…

2018-11-15

stephen
04:42:59 PM

@stephen has joined the channel

Vi
10:58:26 PM

@Vi has joined the channel

Hi, I was working with terragrunt and observed that terragrunt does not pick up the changes automatically for modules. Is there any way to sort this out?

2018-11-06

06:23:21 PM

@ has joined the channel

2018-10-30

nukepuppy
03:15:21 PM

@nukepuppy has joined the channel

2018-10-29

06:29:36 PM

@ has joined the channel

Erik Osterman
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

2018-10-28

antonbabenko

Hi guys! Thanks @Erik Osterman for making this one. It’s been a while since I have used terragrunt for real in a big projects, so my knowledge may a bit broken now

2018-10-27

03:03:28 AM

@ has joined the channel

2018-10-25

07:22:51 AM

@ has joined the channel

Raghu
08:27:38 AM

@Raghu has joined the channel

mallen
07:03:48 PM

@mallen has joined the channel

2018-10-24

Erik Osterman
06:06:06 PM

@Erik Osterman has joined the channel

Erik Osterman
06:06:06 PM

@Erik Osterman set the channel purpose: Terragrunt discussions

06:06:07 PM

@ has joined the channel

antonbabenko
06:06:07 PM

@antonbabenko has joined the channel

loren
06:10:24 PM

@loren has joined the channel

aknysh
06:10:51 PM

@aknysh has joined the channel

Erik Osterman

are any of you using extra_args for init and adding -no-color?

Erik Osterman

i’m using 0.17 and getting the -no-color arg duplicate with breaks auto-init

Erik Osterman

(if I run terragrunt init -no-color, it works)

ndobbs
06:44:54 PM

@ndobbs has joined the channel

catdevman
08:12:24 PM

@catdevman has joined the channel

loren

i used to do that for -no-color but moved away from it

Erik Osterman

what do you do now?

loren

i decided i like the color on the cli, when invoked manually

Erik Osterman

oh, i like that

Erik Osterman

but i don’t want it for CI (atlantis)

loren

for CI, i use terraform envs

Erik Osterman

ok

Erik Osterman

TF_CLI_ARGS?

loren

or set the envs in a makefile instead

Erik Osterman

ok

loren

either way

Erik Osterman

i tried the TF_CLI_ARGS and still got an error

loren

TF_CLI_ARGS is broke… need to use the per command options TF_CLI_ARGS_apply

Erik Osterman

used make to work aroundn it for now

    keyboard_arrow_up