#terragrunt (2019-02)
Terragrunt discussions
Archive: https://archive.sweetops.com/terragrunt/
2019-02-05
any best practices on how to reference between terraform dependencies on different terragrunt modules. eg. I have vpc as a module and bastion as another module (terragrunt modules both) and I need to reference the vpc-id (and subnet-ids) from vpc in bastion
if you want you can try code generated by modules.tf - draw vpc+asg in cloudcraft.co and click export
I wrote small shell script which does replacement using hooks
looks interesting. Have to give them a try at some point
yes, use SSM
or use remote state provider
we have examples of both in this repo https://github.com/cloudposse/terraform-root-modules
@Samuli Are these modules being run in the same Terraform run? (do they share state)
It would be straight forward if they did but with terragrunt modules they are not. So I went with remote_state..
2019-02-14
@Erik Osterman (Cloud Posse) how are we suppose to use terragrunt in your new setup. I see eligible terraform.tfvars only in atlantis-repo sub-module.
this is all you need
that said, I see no strong use-case for it anymore
I think make is a better task runner
and tfenv let’s use define module imports
want to avoid tfenv, it seems over populating the environment by appending all current env variables with TF_VAR
with tfenv, we don’t need any wrappers to call terraform
fair enough
i just don’t like this
overloading .tfvars with a non-portable terraform code that is vendor specific to terragrunt
also i am thinking may be its kind of simpler if we have a whitelisting variables rather then excluding regex
while envs are universal across apps
TFENV_WHITELIST is supported
I saw that, I was saying more about ideology. The whitelist value is .*.
what i don’t like is terraform telling me how envs should look in the first place. just like i don’t like chamber telling me that envs must be upper case (which doesn’t work with terraform)
so we have env warfare
Should be something like geodesic whitelist, and customer whitelist. Where we convert all the values required for geodesic modules. SPECIALLY the ones which we are currently getting
I just wish terraform and terragrunt would not stipulate a convention on envs and then things would just work. so tfenv is an ambassador. it does the dirty work.
the approach makes sense in the current context, its just having too much env variables makes my life hard
just set TFENV_WHITELIST and TFENV_BLACKLIST in your environment however you want
we were so busy mapping envs from one tool to the next. that’s what we wanted to avoid.
too much work to make a module get up. Its as good as doing a export variable in a tfenv.sh
so setting too many envs makes everyones life hard
but the env always has many envs, that’s just a fact-of-linux
on OSX, I have 87 envs that i never set
export|wc
i have total 57, out of which I know about atleast 40
so in our Dockerfile we had SOOOOOO many envs. it was unmanageable.
so we’ve gotten rid of most of them (compared to before)
then moved to .envrc (direnv) so we localize these settings
yeah regarding that, I saw below snippet in rc.d terraform
# Translate environment variables to terraform arguments
[ -z "${TF_FROM_MODULE}" ] || export TF_CLI_INIT_FROM_MODULE="${TF_FROM_MODULE}"
[ -z "${TF_STATE_FILE}" ] || export TF_CLI_INIT_BACKEND_CONFIG_KEY="${TF_BUCKET_PREFIX}/${TF_STATE_FILE}"
[ -z "${TF_BUCKET}" ] || export TF_CLI_INIT_BACKEND_CONFIG_BUCKET="${TF_BUCKET}"
[ -z "${TF_BUCKET_REGION}" ] || export TF_CLI_INIT_BACKEND_CONFIG_REGION="${TF_BUCKET_REGION}"
[ -z "${TF_DYNAMODB_TABLE}" ] || export TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE="${TF_DYNAMODB_TABLE}"
[ -z "${AWS_PROFILE}" ] || export TF_CLI_INIT_BACKEND_CONFIG_PROFILE="${AWS_PROFILE}"
[ -z "${AWS_ROLE_ARN}" ] || export TF_CLI_INIT_BACKEND_CONFIG_ROLE_ARN="${AWS_ROLE_ARN}"
yea, that’s strictly for backwards compatibility
I didn’t want to tell everyone to rewrite their envs
the TF_CLI_* convention is canonical
which i assume should be good enough if we have all variables in left hand side set
~the ones on the LHS are the legacy ones we’ve had in our docs~
yeah, my point even if we are using the legacy variables, things should work
I mean the ones in the [ ... ] conditionals
e.g TF_BUCKET_REGION
yep, so using that mapping they will continue to work
but TF_CLI_INIT_BACKEND_CONFIG_REGION is canonical
in that it maps precisely and consistently to the TF_CLI_ARGS_init=-backend-config=region=blah which is the terraform native convention
technically, even tfenv isn’t needed. it’s just a convenience.
so we can set k/v pairs as ENVs
rather than mucking with the tf flags in compacted envs
this is what I am doing currently
the compacted envs?
I don’t like terraform native envs
TF_CLI_ARGS_init="-from-module=git::<https://github.com/cloudposse/terraform-root-modules.git//aws/ecs?ref=tags/0.40.0> -backend-config=region=us-west-2 -backend-config=dynamodb_table=cpco-testing-terraform-state-lock -backend-config=bucket=cpco-testing-terraform-state -backend-config=key=ecs/terraform.tfstate"
that’s what it looks like.
PIA to toggle individual fields
so we just came up with a convention to not do that. but that’s opinionated and might not suit all parties.
I like it b/c you can set envs at different levels (E.g. Dockerfile, project, parent folder, etc)
i was doing something like export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=niki-root-terraform-state
that looks good
but its still asking me for bucket name
TF_CLI_INIT_BACKEND_CONFIG_BUCKET="cpco-testing-terraform-state"
that’s what we are doing
TF_CLI_PLAN_PARALLELISM=2
TF_CLI_INIT_BACKEND_CONFIG_REGION=us-west-2
TF_CLI_INIT_FROM_MODULE=git::<https://github.com/cloudposse/terraform-root-modules.git//aws/ecs?ref=tags/0.40.0>
TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=cpco-testing-terraform-state-lock
TF_CLI_INIT_BACKEND_CONFIG_BUCKET=cpco-testing-terraform-state
TF_CLI_INIT_BACKEND_CONFIG_KEY=ecs/terraform.tfstate
this definitely works with every terraform * command
if you use tfenv to cast it to the TF_CLI_ARGS_blah=....
in short with my legacy variables, use_terraform and terraform init should give me required resuly
true
# Terraform State Bucket
ENV TF_BUCKET_REGION="${AWS_REGION}"
ENV TF_BUCKET="${NAMESPACE}-${STAGE}-terraform-state"
ENV TF_DYNAMODB_TABLE="${NAMESPACE}-${STAGE}-terraform-state-lock"
i can’t use use_terraform also
is what we have in our docker file
has to source it manually
something like this is happening
✓ (root-admin) tfstate-backend ⨠ use_terraform
✓ (root-admin) tfstate-backend ⨠ printenv | grep TF_CLI
TF_CLI_INIT_BACKEND_CONFIG_REGION=ap-south-1
TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=niki-root-terraform-state-lock
TF_CLI_INIT_BACKEND_CONFIG_BUCKET=niki-root-terraform-state
TF_CLI_INIT_BACKEND_CONFIG_PROFILE=root-admin
TF_CLI_INIT_BACKEND_CONFIG_KEY=tfstate-backend/terraform.tfstate
✓ (root-admin) tfstate-backend ⨠ terraform init
Initializing modules...
- module.tfstate_backend
- module.tfstate_backend.s3_bucket_label
- module.tfstate_backend.dynamodb_table_label
Initializing the backend...
bucket
The name of the S3 bucket
Enter a value:
zoom?
sure
2019-02-28
Good afternoon #terragrunt - I have an odd situation I can’t explain and was wondering if one of you had any insight.
What happens if you run a) without –terragrunt-source?
@loren - same thing happens: terragrunt wants to create all the wave resources.
And the contents of wave/terraform.tfvars? And do you have a parent terraform.tfvars?
At least the source line, if vars are sensitive
terragrunt = {
terraform {
source = "git::<ssh://[email protected]/xxxxx/terraform-modules.git//wave>"
}
# dependencies for wave
dependencies {
paths = ["../vpc", "../route53", "../securitygroups"]
}
# Include all settings from the root terraform.tfvars file
include = {
path = "${find_in_parent_folders()}"
}
}
terraform {
backend "s3" {}
}
# -------------------------------------------------------------------------------------------------------
wave_es_cluster_name = "events"
And yes, there is a parent terraform.tfvars. It sets up s3 remote states and dynamo locking.
I think the terraform backend bit is off… That should be in a .tf file, not .tfvars
I’ll have to take a look at my own setups when back at a computer to compare
You mean this from the parent terraform.tfvars?
terragrunt = {
# Configure Terragrunt to automatically store tfstate files in an S3 bucket
remote_state {
backend = "s3"
config {
encrypt = true
region = "ca-central-1"
s3_bucket_tags {
creator = "terraform"
terraform = "true"
purpose = "canada"
name = "terraform state storage"
}
dynamodb_table_tags {
creator = "terraform"
terraform = "true"
purpose = "canada"
name = "terraform lock table"
}
}
}
(truncated)
No, sorry, I can’t copy/paste easily on my phone, I mean the block with backend "s3" {} in wave/terraform.tfvars
ok - i’ll look
I removed it, but still see the same situation: plan-all wants to create all wave-resources; plan-all --terragrunt-include-dir wave does not want to create them; plan inside the wave directory also does not want to create them.
ok, i’m not seeing anything, but have the sense it’s got to be something fundamental that is just easy for the eyes to pass over
maybe open an issue on the terragrunt repo… they’re pretty decent about responding to this kind of help request
And I’m ashamed to admit that this was a stupid mistake where I had started a new component in a parallel directory, which had the terraform.tfvars file from the wave-directory in it.
Hope I did not waste too much of your time, @loren.
Thanks again!
Aha! That’ll do it! No worries, glad you figured it out!
~Any ideas why ~his turned out to be a stupid mistake where I had created a new component in the live directory and it had a copy of the wave terraform.tfvars in it … plan-all in the root wants to create resources that already exist?