#terragrunt (2019-05)

terragrunt

Terragrunt discussions

Archive: https://archive.sweetops.com/terragrunt/

2019-05-08

Joe Perez avatar
Joe Perez

Is it me or do the terragrunt docs seem lacking? I’ve been having to dig through git issues to find out more. Any external doc references are appreciated

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Gruntwork Community Forum

The Community Forum for Gruntwork customers

Joe Perez avatar
Joe Perez

thank you

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we (cloudposse) don’t use terragrunt

Joe Perez avatar
Joe Perez

I really think the tool is great for what it’s trying to do, it’s just a bit frustrating

1
Jonathan Le avatar
Jonathan Le

yeah - having used terragrunt where i’m currently working, i can’t recommend it again. it’s very opinionated in how modules are created and used with the “source=” stuff in the tfvars file.

trying to have more than 1 module in a state when not everything is copied to the “.terragrunt-cache” directory really annoyed me.

Joe Perez avatar
Joe Perez

@Jonathan Le I thought each module in terragrunt was supposed to have it’s own state file? What were you trying to accomplish with multiple modules in a single state file?

Jonathan Le avatar
Jonathan Le

You are right - each state should be it’s own module. Like 1 module per app/stack.

For me, TG and this method caused me to have a lot of copy/paste between modules though.

Jonathan Le avatar
Jonathan Le

Like, there’d be a basic model of what an EC2 instance was to me and it would have been great to have this as a module like I had without TG.

I had to copy/paste between the resource configuration between similar TG modules to pull them off and the tradeoff really annoyed me.

Jonathan Le avatar
Jonathan Le

It was great to have cool stuff like the following to try to keep things “dry”:

Jonathan Le avatar
Jonathan Le

but the very similar resource config across TG modules was super duper annoying.

Joe Perez avatar
Joe Perez

yeah that doesn’t look fun

Joe Perez avatar
Joe Perez

are you talking about sharing stuff between modules too?

Jonathan Le avatar
Jonathan Le

not sure what you mean about “sharing stuff between modules”. if you mean about datasources between the different module states, that works fine with TG.

if you mean about trying to stay DRY, i didn’t feel i was able to with TG.

e.g, I have very similar looking resource config for ec2 instances across various modules:

Jonathan Le avatar
Jonathan Le
Joe Perez avatar
Joe Perez

ahhhh we’re using a segregated ec2 module, we’re not embedding it into other modules as part of a full “service”

Joe Perez avatar
Joe Perez

what do you guys use?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terraform natively supports intialization of projects using the -from-module parameter

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you can pass that as an environment variable too

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

TF_CLI_ARGS_init=-from=module=....

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so we define all of our root module invocations here: https://github.com/cloudposse/terraform-root-modules

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we wrote an “env mapper” to make setting envs for terraform easier (but it’s not required)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/tfenv

Transform environment variables for use with Terraform (e.g. HOSTNAMETF_VAR_hostname) - cloudposse/tfenv

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s how we put it together:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the .envrc indicates the module we want to use

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then we just call terraform init and it automatically imports the remote module

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no wrappers required

Joe Perez avatar
Joe Perez

damn that’s nice

Joe Perez avatar
Joe Perez

I was worried about what happens when terraform is updated when using terragrunt

Joe Perez avatar
Joe Perez

I’m gonna need to dig through all of this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, that’s what i like about this.

Joe Perez avatar
Joe Perez

what about all the state stuff that terragrunt does for you, how are you guys handling that?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terragrunt is basically a task runner

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

there are a lot of task runners out there.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use make predominantly with terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terragrunt helps keep your code DRY, but if you decompose thee different things that terragrunt does, you can easily accomplish it other ways

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use direnv to keep project level settings

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use make for a task runner

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use tfenv to make mapping envs easier (totally optional)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use terraform init -from-module=... (using the TF_CLI_ARGS_init=-from-module=... env), to keep things DRY.

pecigonzalo avatar
pecigonzalo

Whats the use for -from-modules= I never understood this workflow. Could you elaborate?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This allows you to bootstrap modules. Terragrunt uses it (if you look under the hood) too.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So in our case, we have a central catalog of “root” modules. Every time we merge to master, we cut a release.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

When we want to use that module for an environment (e.g. prod) we call terraform init with -from-module to download it. That way we never copy/paste the code. It also allows us to have different environments pinned to different versions.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Terragrunt does the same thing. Only it writes it to a cache folder

pecigonzalo avatar
pecigonzalo

Thanks, that part was clear tho, but why not just

module "this" {
source = "sourcelink"
vars = "that"
}

sort of thing

pecigonzalo avatar
pecigonzalo

what is the advantage?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the combination of these things create a generalizable pattern that works formore than terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for example, we use helm and helmfile

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we package all the tools in geodesic, which is our cloud automation image

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It's the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we also use variant to define cli tools specific to our organization.

bananadance1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
mumoshu/variant

Wrap up your bash scripts into a modern CLI today. Graduate to a full-blown golang app tomorrow. - mumoshu/variant

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have lots of examples of variant in use here: https://github.com/cloudposse/geodesic/tree/master/rootfs/usr/local/bin

cloudposse/geodesic

Geodesic is a cloud automation shell. It's the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Joe Perez avatar
Joe Perez

this is all really good stuff to take back to my team

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cool! happy to share more or answer any questions.

Joe Perez avatar
Joe Perez

this is a great start, thank you

2019-05-31

Meb avatar

Thanks for the feedbacl & Rex. Good call on these

    keyboard_arrow_up