#terragrunt (2021-04)

Our root terragrunt hcl looks pseudo like

I don’t think that’s compatible with what we’re doing now. … not using hcl for locals / no ‘locals’ in terragrunt.hcl …

Where region.hcl looks like

( we could put a locals block int terragrunt.hcl, but it would need to not override what was imported via tfvars. )

Don’t think you can get the value to feed back into where you want from a tfvar file

locals are useful as you can a) Use interpolation b) pull them in from other places and extract values.

My only concern with them was that I thought there was a limited number of layers we could pull in for locals. We have these layers: account, region, env.

If I can add an hcl with just what I’m trying to import into terragrunt.hcl, and it not cause issues with the layers of tfvars, I’m good with that.

You can pull in what you want, as you can see we are pulling a hierarchy of hcl files with local blocks in them.

Ok, is there a way to pull in multiple layers of tfvars into a locals block in a single hcl file?

Not the last time I checked (if I’m understanding correctly)

Ok. Thanks for your time!

if you switch from tfvars in hcl to tfvars.json in json, then you can read the file into terragrunt with jsondecode(file('path/to/file.tfvars.json'))

https://www.terraform.io/docs/language/values/variables.html#variable-definitions-tfvars-files

Terraform also automatically loads a number of variable definitions files if they are present:
* Files named exactly terraform.tfvars or terraform.tfvars.json.

• Any files with names ending in .auto.tfvars or .auto.tfvars.json.

Thanks for the response! That sounds like it could be worth exploring. I need to access a variable in the TF with var.regional_parameters.var_name .. how would that json look? ( and how would I then access it from the terragrunt.hcl file after I read it in there? )

I found this: https://community.gruntwork.io/t/new-to-terragrunt-question-about-passing-in-aws-region-as-input-and-picked-up-by-the-root-terragrunt-hcl/465

yep, that’s saying the same thing i am

I might be able to figure it out with that. Thanks again!

And that will get me local.vars.region , in the terragrunt.hcl file, but that would be var.region, in the TF, right?

presuming you load the file into the local named vars, yes

If so, the thing left for to figure out is how to get the extra var.regional_parameters.region piece in. It’s json, so maybe: { “regional_parameters”: { “region”: “us-east-1” } } and then local.vars.regional_parameters.region in the HCL?

correct

Cool. Tysm.

@joshmyers a If that’s for astro, i see it’s not very active, so i agree.

That was an answer to the only question I could see in your statement

Agree with Astro, nice idea, wouldn’t use.

whops, sosry for not being clear.

@managedkaos i did create a multistage yaml pipeline with PR code commenting and approval gate for desired environment.

So I did solve that, but I don’t think I want to keep that long-term if I can leverage something more developed and worked on than my custom implementation, such as Atlantis.

The variables/backend stuff is a pain still. I think terragrunt does this automatically

https://sweetops.slack.com/archives/CDMJ4BBR8/p1618502121036200

@managedkaos responding to your post in this thread so the history/archive keeps the context.

Yes, I agree it was nice. I did this with PR commenting in yaml pipeline, but it’s the backend and variable stuff that’s painful and the PR commenting/workflow is only 10-20% of my main concern. I guess that’s why terragrunt seemed promising to simplify that part of it in my scenario.

Prior company was better with Terraform Cloud for sure, but looking for more flexibility now

got it

same story with taskfile, I have my own terraform stack management, more tight control than Terragunt or terraspace and just simpler, and work with vanilla terraform

I love that tool. it’s great!

Only problem I have with it is a silly one. For small adhoc tasks the bash scripting is to me far worse than powershell 7 i’m used to as it’s not even a fully featured sh. For all else it’s great!

Would love to see your terraform task runner file if you could remove anything sensitive for my education. I’ve done pretty extensive ones for docker/go, but not used with terraform.

It’s a different focus for sure, as task is just running vanilla right? I want the dynamic backend + PR commenting workflow and terragrunt approaches differently. Taskfile makes it easier to run, but doesn’t solve the DRY stuff

My current approach was similar this is what it looks like. It’s basically building the tfvars/backend based on parameters. Still not overall happy with it over perhaps something designed for this from the beginning.

When you say that you had a go with cloudposses yaml configuration, does it mean that you tried atmos?

both manually and with atmos. It’s just not the right fit for my use case, despite my for all things cloudposse

Taskflow looks more advance than taskfile, the idea to create these task runners without having any embedded secrets or sensitive data, can you share your taskflow I’m interested

here a simple version of my tf taskfile https://github.com/mhmdio/iac-taskfile-framework/blob/main/terraform/Taskfile.yml

Why not use Task? Task is simpler and easier to use than Make it still has some problems:
Requires to learn Task’s YAML sturcture and the minimalistic, cross-platform interpreter which it uses.
Debugging and testing tasks is not fun.
Harder to make some reusable tasks.
Requires to “install” the tool. taskflow leverages go run and Go Modules so that you can be sure that everyone uses the same version of taskflow.

I think after give this a shot, this is more advance use case for me now, other wise, I will be ended reinvent Terragrunt, which I don’t want to do, YAML declaration for my tasks seems valid to me, for the time being

Yeah, so it’s a different focus. For make style stuff it works really fast for exploration to use go-task. I still use it.

For building automation with native Go, then taskflow will be better. You could interact with your business apis and do tons of things that would be clunky. So basically go-task = alternative to save typing for cli commands. Taskflow –> go powered tooling.

Since I’m doing FT Go development I’m working with taskflow to improve my Go chops and have more flexibility, but for “make build” style quick tools go-task all the way

Here’s a sample of a func to validate terraform path and log success or failure.

go

func getTerraformPath(tf *taskflow.TF) (terraformPath string) {
	// terraformCmd.Dir = stack

	terraformPath, err := filepath.Abs(path.Join(toolsDir, "terraform"))
	if err != nil {
		tf.Logf(":arrows_counterclockwise: skipcannot resolve realpath of terraformPath at: [%s] -> [%v]\n", terraformPath, err)
		tf.SkipNow()
	}
	if _, err := os.Stat(terraformPath); os.IsNotExist(err) {
		tf.Logf(":exclamation: cannot find terraform at: [%s] -> [%v]\n", terraformPath, err)
		tf.SkipNow()
	}
	pterm.Success.Printf(":white_check_mark: found terraform at: [%s]\n", terraformPath)
	sb := &strings.Builder{}
	tfVersion := tf.Cmd(terraformPath, "--version", "-json")
	tfVersion.Stdout = io.MultiWriter(sb)
	if err := tfVersion.Run(); err != nil {
		pterm.Error.Printf(":exclamation: terraform --version: [%s]\n", err)
		tf.FailNow()
	}
	ver := TerraformVersion{}
	s := sb.String()
	err = json.Unmarshal([]byte(s), &ver)
	if err != nil {
		pterm.Error.Println(":exclamation: was not able to parse terraform version")
		tf.Fail()
	}
	pterm.Success.Printf(":white_check_mark: terraform version: [%s]\n", ver.TerraformVersion)

	return terraformPath
}

disclaimer: still new to go, so lots of messy code

Here’s a screenshot of my “fun” version where I was just playing around with Pterm to try and make things more visually appealing

Go task is great for simple things though! Kinda forces me to keep it simple.

I prefer powershell over bash for this type of task automation as I like structured objects over parsing text typically, so my personal favorite tool is InvokeBuild for more robust local scripted task running. Go task is kinda a test to force myself to keep it “less” powerful and limit what I put in there

cool, thanks @sheldonh i learned a lot today

@sheldonh took the next step, and look like Terragrunt is using https://github.com/urfave/cli which 15000 starts on gituhb VS Taskflow, and seems nicely to use with PTerm, I think I like this path.

nice! for the record, taskflow is a different concept. cobra and cli both are focused on building command line apps.

I think taskflow is designed to be more of a go build ./build.go tool, that basically means zero dependencies on install.

A cli tool is probably a great fit for established standard workflows, but probably not as good for easy adhoc tasks like taskflow. YMMV. Let me know what you find out as well

It differs for different versions of terragrunt. The most recent declaration that I use is:

inputs = merge(
  # Configure Terragrunt to use common vars encoded as yaml to help you keep often-repeated variables (e.g., account ID)
  # DRY. We use yamldecode to merge the maps into the inputs, as opposed to using varfiles due to a restriction in
  # Terraform >=0.12 that all vars must be defined as variable blocks in modules. Terragrunt inputs are not affected by
  # this restriction.
  yamldecode(
    file("${find_in_parent_folders("region.yaml", "empty.yaml")}"),
  ),
  yamldecode(
    file("${find_in_parent_folders("env.yaml", "empty.yaml")}"),
  ),

You prevent not matching such file by creating empty empty.yaml

ARGGH. Got you. That’s really useful. Might add that to the issue as well so no one else repeats the work. I’m going to try that now

@Marcin Brański that’s what you put in the child file right?