#terraform (2020-04)

terraform Discussions related to Terraform or Terraform Modules

Archive: https://archive.sweetops.com/terraform/

2020-04-29

Pierre Humberdroz avatar
Pierre Humberdroz

Does someone know if I can import an existing rds instance into my terraform stack ?

Pierre Humberdroz avatar
Pierre Humberdroz
terraform-aws-modules/terraform-aws-rds

Terraform module which creates RDS resources on AWS - terraform-aws-modules/terraform-aws-rds

RB avatar

It needs to follow the import example

terraform import module.my.terraform.reference.aws_db_instance.this my-rds-name-in-aws

so maybe something like this

terraform import module.db.module.db_instance.aws_db_instance.this my-rds-name-in-aws

where my-rds-name-in-aws is your RDS instance name

terraform-aws-modules/terraform-aws-rds

Terraform module which creates RDS resources on AWS - terraform-aws-modules/terraform-aws-rds

Pierre Humberdroz avatar
Pierre Humberdroz

ouch !

Pierre Humberdroz avatar
Pierre Humberdroz

thanks !

np1
RB avatar

Anyone else run into this? If so, would appreciate upvotes

https://github.com/terraform-providers/terraform-provider-aws/issues/11801

Order is lost for data aws_iam_policy_document when applied to resource aws_kms_key · Issue #11801 · terraform-providers/terraform-provider-aws

Community Note Please vote on this issue by adding a :–1: reaction to the original issue to help the community and maintainers prioritize this request Please do not leave "+1" or other comme…

Gowiem avatar
Gowiem

@RB I have not, but obviously the ordering problem in Terraform sucks. I did just notice yesterday that CP calls this out in their kms-key module:

https://github.com/cloudposse/terraform-aws-kms-key/blob/master/variables.tf#L76


A valid KMS policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy.
So maybe you need to provide all policy key / values?

cloudposse/terraform-aws-kms-key

Terraform module to provision a KMS key with alias - cloudposse/terraform-aws-kms-key

RB avatar


In this case, please make sure you use the verbose/specific version of the policy

RB avatar

i will try this!

Gowiem avatar
Gowiem

It looks like you are doing that though after reading your issue more closely… so I’m not sure what is up there.

Gowiem avatar
Gowiem

I supply a policy like so in my module and I don’t have that issue:

https://github.com/masterpointio/terraform-aws-ssm-agent/blob/master/main.tf#L248

Maybe you need the top level version and id? That would be odd though..

masterpointio/terraform-aws-ssm-agent

Terraform module to create an autoscaled SSM Agent instance. - masterpointio/terraform-aws-ssm-agent

RB avatar

the https://www.terraform.io/docs/providers/aws/d/iam_policy_document.html data source provides the version argument for you and I’ve already set the policy_id which maps to Id in JSON

AWS: aws_iam_policy_document - Terraform by HashiCorp

Generates an IAM policy document in JSON format

RB avatar

you make a good point tho, perhaps I should try it with the raw JSON to see if it still has the issue… i did see this reordering happen on another usage of this data source for an iam policy but once applied, and modified, i no longer saw the reordering in the plan

RB avatar
RB
02:54:31 PM

¯_(ツ)_/¯

Gowiem avatar
Gowiem

Haha yeah — I dunno. The ordering issue is a PITA wherever it rears its ugly head.

RB avatar

i have a gross python script for it now as a workaround… trying to get this put into our atlantis builds so we can use it in custom workflows

sheldonh avatar
sheldonh

Does anyone know the right git repo or forum to provide feedback on terraform cloud? I have some general things as a user I want to provide feedback on regarding usability but can’t figure out the right place to get that heard.

loren avatar
loren

twitter

loren avatar
loren

or, ask on twitter for a poc. hashicorp seems pretty responsive on twitter…

Jake Lundberg (HashiCorp) avatar
Jake Lundberg (HashiCorp)

You can also use the HashiCorp community forum on Discuss: https://discuss.hashicorp.com/c/terraform-core/27

sheldonh avatar
sheldonh

For terraform cloud ? I’m specifically referring to the new website service. Some usability issues I’ve wanted to give feedback on and see if it helps.

PePe avatar

I’m going crazy!!!! No matter what I do, this thing keeps trying to create webhooks!!!!!!!!! https://github.com/cloudposse/terraform-aws-ecs-atlantis/blob/0.14.0/main.tf#L54

cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

PePe avatar
Error: GET <https://api.github.com/orgs/xxxxxx>: 401 Bad credentials []

  on .terraform/modules/atlantis.ecs_web_app.ecs_codepipeline.github_webhooks/main.tf line 1, in provider "github":
   1: provider "github" {



Error: GET <https://api.github.com/orgs/xxxx>: 401 Bad credentials []

  on .terraform/modules/atlantis.webhooks/main.tf line 1, in provider "github":
   1: provider "github" {
cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

PePe avatar

I’m trying to make it so that I created the webhooks after the fact

PePe avatar

terraform apply -var-file=“us-east-2.tfvars” -var=‘webhook_enabled=false’ even if I do that it just keep trying to enable it

PePe avatar

I must be getting old

Gowiem avatar
Gowiem

Is it just the github provider being invoked/initialized? Did you provide correct creds to that provider?

PePe avatar

Well the module is being invoked even when the enabled flag is not true

Gowiem avatar
Gowiem

I think that happens with providers unfortunately. There is no count param so you can’t turn them off.

Gowiem avatar
Gowiem

At least I think I remember running into that and it being an issue.

PePe avatar

Ohhh so they always get evaluated

Gowiem avatar
Gowiem

Yeah. So maybe passing a valid github_organization and github_token will do the trick? Regardless, that’s a pain.

PePe avatar

I will

PePe avatar

Pass it and see

PePe avatar

yep, that worked

PePe avatar

is not creating any webhooks

PePe avatar

so it is the provider that gets evaluated always first

PePe avatar

annoying

PePe avatar

I’m using https://github.com/cloudposse/terraform-github-repository-webhooks.git?ref=0.6.0 to create webhooks but once the webhook is created and the user is changed from admin to read then terraform tries to read the /hook url for that repo and gets a 403, is there a workaround for this so that terraform does not try to create the webhook every single time ?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

the bot needs to be an admin to create webhooks

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

after that, we usually switch it to Read

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

tested the module many months ago, did not see the issues with webhooks at the time

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

did you look at the tests for the atlantis module?

PePe avatar

but then if I switch the bot user to read for that repo, then it tries to create the webhook repo again

PePe avatar

is this maybe different in github enterprise ?

PePe avatar

after changing to read

PePe avatar

this will happen

PePe avatar
Error: POST <https://api.github.com/repos/xxxxxx/terraform-xxxxxxs-ecs-atlantis/hooks>: 404 Not Found []

  on .terraform/modules/atlantis.webhooks/main.tf line 6, in resource "github_repository_webhook" "default":
   6: resource "github_repository_webhook" "default" {
PePe avatar

but I can pull with that user no problem

PePe avatar

the bot/atlantis user I have is a Member

PePe avatar

in the org

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

not sure, never tested with GH enterprise

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

but yes, the bot needs to be an Admin at the time of provisioning

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

there is no way around that since to work with wenhooks you have to be admin

Gowiem avatar
Gowiem

Terraform project folks are looking for backend / provisioner maintainers: https://discuss.hashicorp.com/t/seeking-terraform-open-source-backend-maintainers/8113

Seeking Terraform Open Source Backend Maintainers

Terraform is actively seeking maintainers for some of our remote state backends. As a maintainer, you’ll participate in pull requests and issues for a given area of Terraform. For example, you could be working on the Postgres Backend. There is no expectation regarding time commitments or response frequency at this point. In addition to pull requests and issues, you’ll be helping us establish guidance and grow our contribution program. If you’d like to participate in the Terraform project, hea…

OliverS avatar
OliverS

Hi everyone I’m trying to use the eks_fargate_profile module from github on an EKS cluster in which I will define 2 namespaces, so I need 2 fargate profiles (I could use one profile with 2 selectors, but this is not really practical from an automation point of view – if I want to create/delete a namespace, I have to modify the one profile rather than create/delete a separate one – plus there is an AWS-imposed limit of 5 selectors per profile so 5 namespaces – not likely I would hit that limit but not unlikely either).

Unfortunately the fargate profile name generated by the module does not include the kubernetes_namespace value, so I end up with duplicate resources (like the IAM role):

Error: Error creating IAM Role rnd-poc-kim-fargate: EntityAlreadyExists: Role with name rnd-poc-kim-fargate already exists.
	status code: 409, request id: 12f2e0cb-9028-43ac-8791-b03bf5398f2a

  on .terraform/modules/eks_fargate_profile_default/main.tf line 35, in resource "aws_iam_role" "default":
  35: resource "aws_iam_role" "default" {



Error: Error creating IAM Role rnd-poc-kim-fargate: EntityAlreadyExists: Role with name rnd-poc-kim-fargate already exists.
	status code: 409, request id: cbe7771a-a866-45d1-a35e-b82727e84f15

  on .terraform/modules/eks_fargate_profile_staging/main.tf line 35, in resource "aws_iam_role" "default":
  35: resource "aws_iam_role" "default" {

I could of course clone the module code since it is standalone, but then I loose the benefit of fixes made by CloudPosse, so I’m wondering if there is a better way, maybe there is a workaround I’m not thinking of.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

each module has namespace, stage, name and attributes. You can add var.attributes=["something"] to one of the modules so all the generated names/IDs will be in the format namespace-stage-name-attributes which will be unique

OliverS avatar
OliverS

That’s one of the first things I tried but it did’t work. However given other errors I had it is possible I had other stuff incorrect that I have since fixed so will try again.

2020-04-28

ikar avatar

Hey there! How would you set instance_market_options of aws_launch_template only for stage environment? Define two similar launch templates with count condition?

randomy avatar
randomy

using a dynamic block

randomy avatar
randomy
Expressions - Configuration Language - Terraform by HashiCorp

The Terraform language allows the use of expressions to access data exported by resources and to transform and combine that data to produce other values.

ikar avatar

looks great! thanks

ikar avatar

well …not great, it looks ugly, but works

randomy avatar
randomy

hah, that is a verbose dummy value. that is how i’d do it except the for_each would fit on line line.

ikar avatar

thanks again! multi line is because of pycharm auto-format

RB avatar

hey all, if you had to give a talk to a bunch of ppl for 30 min to an hour on infrastructure as code (terraform) and policy as code (open policy / cloud custodian), what are some things you’d mention ?

roth.andy avatar
roth.andy

pets :cat2: vs cattle :cow2:

:100:1
roth.andy avatar
roth.andy

Continuous Integration of IaC is just as valuable as CI of app dev work

atlantis1
roth.andy avatar
roth.andy

automated testing

RB avatar

how do you do automated testing with terraform? we recently started terratest for internal modules coming from devops team but havent pipelined it

roth.andy avatar
roth.andy

terratest

RB avatar

we’re a few degrees behind testing but ill definitely add more information for testing, ty

roth.andy avatar
roth.andy

code quality/validation. Sonarqube isn’t much help but terraform has lots of linting/etc

Zachary Loeber avatar
Zachary Loeber

I’d make some kind of joke about ClickOps to start. Then I’d roll right into ‘this craze of DevOps’ and how that has taken over our industry, then I’d bounce right into the fact that infrastructure was jealous so now we have Infra As Code.

cool-doge1
Zachary Loeber avatar
Zachary Loeber

Then I’d go over how it speeds up deployments, allows for closer integration into the same tools that the developers use for version control, and why it rocks

Zachary Loeber avatar
Zachary Loeber

or something like that

Zachary Loeber avatar
Zachary Loeber

I also generally am not allowed to talk too much for a reason so take what I say with several grains of caution…

RB avatar

I love the term ClickOps. I’ll definitely use that. I have some deployment speed slides, version control, and other bennies

RB avatar

lol wuttt what did you do Zachary

Zachary Loeber avatar
Zachary Loeber

I’m way too into this stuff, it can be great for the right crowd, but scary to others

Zachary Loeber avatar
Zachary Loeber

Sounds ylike you have a fun task regardless.

Zachary Loeber avatar
Zachary Loeber

If you get it recorded or something you should share with the rest of us to live vicariously through ya

RB avatar

haha yea definitely fun! i’m trying to make it less company specific so in the future the deck can at least be released

Zachary Loeber avatar
Zachary Loeber

If you want to be a total nutcase about it you can create your deck with hugo, markdown, and reveal.js (my own example here: https://github.com/zloeber/deck.loeber.live)

zloeber/deck.loeber.live

Zachary Loeber’s Presentations. Contribute to zloeber/deck.loeber.live development by creating an account on GitHub.

Zachary Loeber avatar
Zachary Loeber
:--1:1
Zachary Loeber avatar
Zachary Loeber

not quite the same as a pptx file…

RB avatar

you should add that link to the repo description at the top

RB avatar

thats beautiful looking! i will definitely have to convert this over and put in my own repo called presentations or similar great portfolio booster too

Zachary Loeber avatar
Zachary Loeber

ha, I just haven’t created a professional powerpoint in like 20 years and found this as a geeky alternative

Zachary Loeber avatar
Zachary Loeber

Nice!

2020-04-27

Cloud Posse avatar
Cloud Posse
04:00:13 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is May 06, 2020 11:30AM.
Register for Webinar
slack #office-hours (our channel)

curious deviant avatar
curious deviant

How are folks testing their terraform code? Are there any recommended tools/frameworks out there ? Appreciate any pointers.

Gowiem avatar
Gowiem
gruntwork-io/terratest

Terratest is a Go library that makes it easier to write automated tests for your infrastructure code. - gruntwork-io/terratest

Gowiem avatar
Gowiem

Most of the recently updated CloudPosse modules have tests which are good examples.

curious deviant avatar
curious deviant

thank you..I’ll take a look

loren avatar
loren

i always highly recommend these slides, also from the gruntwork team… https://www.infoq.com/presentations/automated-testing-terraform-docker-packer/

Automated Testing for Terraform, Docker, Packer, Kubernetes, and More

Yevgeniy Brikman talks about how to write automated tests for infrastructure code, including the code written for use with tools such as Terraform, Docker, Packer, and Kubernetes. Topics covered include: unit tests, integration tests, end-to-end tests, dependency injection, test parallelism, retries and error handling, static analysis, property testing and CI / CD for infrastructure code.

Gowiem avatar
Gowiem

Yeah, Loren’s link is a good one. Here is the talk: https://www.youtube.com/watch?v=xhHOW0EF5u8&feature=emb_title

curious deviant avatar
curious deviant

Thank you everyone for your help . Really appreciate it.

btai avatar

so for some reason, when I ran a terraform destroy , terraform decided to attempt to delete my vpc module components first before my EKS cluster built on top of it causing it to error out. (terraform needs module dependency IMO ) It also deleted my kube auth config map for EKS along with it so now terraform or I cannot auth to the cluster. Anyone know of a workaround to fix this (recreating the auth config map)? Other than manually deleting all the AWS resources.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

why not run terraform plan/apply again and see what it will try to create?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

did you deploy anything to EKS/Kubernetes like cert-manager or externa-dns?`

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

before you destroy the cluster?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for example, if you deploy nginx-ingress to Kubernetes, and then will try to destroy the EKS module and the VPC, it will error out, because nginx-ingress deploys a load balancer into the public subnets

btai avatar

I just did that @Andriy Knysh (Cloud Posse). First I needed to get access back into my EKS cluster by using the user that provisioned the cluster (in my case, my terraform bot account)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you don’t delete the nginx-ingress release and try to destroy EKS/VPC, it will be stuck at deleting the subnets b/c the load balancer and ENIs will be still attached to them

btai avatar

@Andriy Knysh (Cloud Posse) that was it

btai avatar

I feel like the helm release was deleted first, but the load balancer that got provisioned might not be deleted w/it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

If anything like that happens, we just manually delete the load balancer using AWS console, and then destroy/plan/apply work again

btai avatar

that’s what I did which irritates me. But good to know that’s the best we can get sometimes.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you forget to delete Kubernetes releases and destroy the EKS cluster, then yes, you can delete the leftovers manually. It applies to nginx-ingress and a buch of other releases like cert-manager and external-dns

cabrinha avatar
cabrinha

I’ve got a question about the EMR Cluster module.

cabrinha avatar
cabrinha

I would like to create a DNS record that points to all of the master instances, for HA purposes

cabrinha avatar
cabrinha

It seems that TF doesn’t allow a way to do that, though? https://www.terraform.io/docs/providers/aws/r/emr_cluster.html#master_public_dns

AWS: aws_emr_cluster - Terraform by HashiCorp

Provides an Elastic MapReduce Cluster

cabrinha avatar
cabrinha

eventhough the master group can be 3 nodes for HA, I’m wondering how I could use TF to assign a DNS name to all three …

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

did you try to create 3? what is the value ofmaster_public_dns output in this case? It might be an array if you create 3 masters, so you can add a DNS record for all of them. (I don’t remeber if it was a single value or an array, although we did not test with 3 masters)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

also, what AWS console shows for the master DNS if you create 3? Does it show all three domains, or just 1?

cabrinha avatar
cabrinha

Just 1.

cabrinha avatar
cabrinha

master_public_dns is a CNAME pointing to a single compute.internal ip address/record

2020-04-24

sheldonh avatar
sheldonh

Thing 1: I want to improve my terraform IAM service account security. What’s the best cross platform way to ensure the credentials are encrypted in state but can be accessed by other modules, and not output to console any longer.

sheldonh avatar
sheldonh

Thing 2: I want to give a very simplistic process for updating ssm parameter store values and eventually lock down console. I already have format, and was planning on environment folders with tfvars and single resource call for parameters.

The catch is I can’t lock down drift yet. I want to make it automerge to master for this one project (will use github action/probot) after it passes all checks. I don’t want it to allow destruction of resources though without approval.

Would setting a dynamic block for lifecycle work, requiring a explit “allow destruction” in the input array? And do you think i could trigger the pull request to NOT auto apply if it detects destruction?

That might be an enhancement with terraform cloud. Autoapprove except with destroy, or autoapprove new but not changed/destroy.

2020-04-23

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Someone has graciously updated tfmask to support 0.12 output

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/tfmask

Terraform utility to mask select output from terraform plan and terraform apply - cloudposse/tfmask

:--1:3
:100:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

0.4.0 adds support.

2020-04-22

Igor Bronovskyi avatar
Igor Bronovskyi
07:25:20 AM

can help me?

Hello. I have trouble with aws_appautoscaling_policy . When left one container this alarm not disappeared. https://take.ms/5cyu8

ikar avatar

^^ this looks okay to me, the alarm definition is CPUUtil <= 10 -> therefor in ALARM state

Igor Bronovskyi avatar
Igor Bronovskyi

So. There was one container and the alarm does not disappear.

Pierre-Yves avatar
Pierre-Yves

hello, I use count = var.env == "prod" ? 1 : 0 to trigger the creation of a resource only if env = “prod”. Is there a way to do it at the module level ?

randomy avatar
randomy
module "something" {
  source = "somewhere"
  enabled = var.env == "prod"
}
randomy avatar
randomy
resource something "something" {
  count = var.enabled ? 1 : 0
}
Pierre-Yves avatar
Pierre-Yves

thanks I will try it

randomy avatar
randomy

basically, pass through an enabled (or similar) variable and use that for count. there isn’t a terraform feature for enabling/disabling a whole module.

Zach avatar

HashiCorp says they’re going to add for_each to the module level soon

Gowiem avatar
Gowiem
Announcing HashiCorp Cloud Engineering Certifications

Introducing the HashiCorp Cloud Engineering Certification program, for cloud engineers to demonstrate their knowledge, capability, and expertise working with the de facto standard …

5
sheldonh avatar
sheldonh

For those using Terraform Cloud

How do you simplify deploying the same work to multiple regions?

The way I see it is repeated code in the project or creating a workspace per region (can do through terraform provider so it’s not a pain with some initial legwork).

And no terragrunt or make files involved here… all terraform cloud please

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ya, programatic creation of workspaces

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Consider defining them all in a .yaml config and then terraforming it out.

sheldonh avatar
sheldonh

But you can’t do a for_each style approach on providers so what would yaml give you?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

creating the workspaces uses a single provider with terraform cloud, no?

loren avatar
loren

the provider config can reference variables, right? so per workspace, toggle the region variable, reference it in the provider config?

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then each workspaces is associated with a region which terraform runs against

sheldonh avatar
sheldonh

Right, that’s what I said in post, workspace per region. I was wondering if there was anything more elegant or that’s what I’d have to do.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Not that I am aware of

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@johncblandii (Cloud Posse)

loren avatar
loren

well, it’s a workspace per deployment, so it could be a matrix of region+whatever-else. half of why i avoided workspaces entirely, just too inflexible

sheldonh avatar
sheldonh

Nice. I’ve actually been wanting to pilot doing this in Azure DevOps since this would be a very very simple thing to just flip into a matrix build and then have it run plans for every region and even account all in one plan.

sheldonh avatar
sheldonh

It’s not prebuilt, have plumbing to do, but if I can convince folks to give me a chance to do it, i think its more scalable

loren avatar
loren

yeah, we basically use terragrunt to give us a lot of that flexibility, while retaining deployments in independent remote-states

loren avatar
loren

also allows us to test locally more easily

loren avatar
loren

but if you don’t want terragrunt or some other wrapper, there’s a lot more legwork IMO

sheldonh avatar
sheldonh

So it comes back to if I’m NOT using terraform cloud that terragrunt provides the most flexibility to rapidly deploy compared to doing all that in the Azure DevOps pipeline.

I’m still mixed on approach. Trying to keep it simple, but investing in a little more complication with something like terragrunt might provide initial complications while simplifying cross region/account deployments a lot

sheldonh avatar
sheldonh

Decisions decisions

loren avatar
loren

you can still use terragrunt in the pipeline

sheldonh avatar
sheldonh

To a team early in any automated pipeline adoption do you feel terragrunt will over complicate or will be reasonable to adopt as a team

loren avatar
loren

i think if you’re looking at multi-account, multi-region then the complexity will be there with any solution, and everyone will need to learn something new and dig pretty deep to understand the abstractions

sheldonh avatar
sheldonh

And terragrunt is established enough that it’s a solid approach vs trying to do it all in pipelines etc. Leverage terragrunt to simplify while using pipelines to run what already was simplified

sheldonh avatar
sheldonh

Makes sense. I tried it last year but still wasn’t 100% on cloud pipelines with terraform so it over complicated it for me. I’m pretty solid now so probably would make more sense

loren avatar
loren

latest terragrunt has gotten pretty powerful with it’s dependency and file/code generation features, and access in terragrunt.hcl to tf 0.12 features and functions (e.g. for loops and templatefile(), etc)

sheldonh avatar
sheldonh

sounds good. I’ll review for a new “root-module” repo and see how it goes then

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

#terragrunt has obviously proven very effective for this. Also, our latest project is using #variant and it’s hands down the coolest cli approach I’ve seen and it’s all written in pure HCL and compiled down to a single binary. We can define our cli exactly the way we want it to work and it’s not limited to terraform.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re using it with helmfile as well.

loren avatar
loren

i’d love to see an implementation using variant to compare to terragrunt

sheldonh avatar
sheldonh

I’m in a windows environment so didn’t look at variant more. I’ll have to rexamine.

sheldonh avatar
sheldonh

I work with PowerShell and it seemed bash focused

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ya, i want to discuss on office hours soon. just need to get approval from client.

loren avatar
loren

just use wsl+bash instead of powershell, problem solved!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

variant cross compiles (it’s in go)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

variant gives you a modern cli interface to wrap any and all commands, workflows, pipelines, etc you need.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That way you define it in variant, run it locally for development, then stick those commands in your Ci/CD pipelines for automation.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this gives you the best of both worlds. simpler CI/CD pipelines and local execution for development.

sheldonh avatar
sheldonh

Ok, so basically an alternative to like act to run your github actions locally.

sheldonh avatar
sheldonh

Instead now we remove build system and have the commands run by this executable and pipelines just run that and don’t do more logic/processing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ya, sorta. however, with variant, there’s a focus on the cli interface

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. this can be done with make, ansible, etc.

sheldonh avatar
sheldonh

I’m already doing that with InvokeBuild in powershell for most things, but would be interested in learning more on this. Trying to learn Golang a bit too so i like cross platform/single exe concept

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but you end up with ugly as sin commands

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in our case, we literally run:

mycli deploy stack
sheldonh avatar
sheldonh

Got you. I love trying new things. Just trying to see what value it brings for my use case.

In my case, I just run Invoke-Build deploy -Configuration qa and it would do the same thing. it’s just PowerShell Core tasks that can be defined

sheldonh avatar
sheldonh

I’ll have to experiment, sounds pretty cool

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

with variant, you have fully control over the positional arguments (e.g. mycli a b c) and options (e.g. mycli a b -foo=2)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it compiles down to a single binary that is easily shared

sheldonh avatar
sheldonh

And as a plus i can easily run in vscode the same commands for example

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

built-in testing framework

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

built-in slack bot

sheldonh avatar
sheldonh

That’s a plus for sure. Dependencies.

sheldonh avatar
sheldonh

Nice! So I’m just chatting, not saying my solution is better. All of those things except dependencies are easy in PowerShell Core, as I have a function for New-BuildSlackMessage and I’ve designed these functions to update not post another completion message.

Argument parsing in powershell is arguably better than any cli, but definitely isn’t a single binary

sheldonh avatar
sheldonh

After build completes it posts with the humanized timing. I hate build notification defaults as so noisy.

sheldonh avatar
sheldonh

Definitely will look at variant when I get some time as I’m loving a single binary, just have to figure out the flexibility compared to what I’ve got right now

randomy avatar
randomy

https://github.com/raymondbutcher/pretf has a lot of feature overlap with Terragrunt but with other features like support for multiple AWS credentials, and writing dynamic resources in Python if you feel the need.

raymondbutcher/pretf

Generate Terraform code with Python. Contribute to raymondbutcher/pretf development by creating an account on GitHub.

randomy avatar
randomy

In particular it’s good for structuring projects with tfvars in environment/region-specific subdirectories, then you run pretf from that there without cli arguments and it all works.

btai avatar

I used terraform CLI workspaces + terraform cloud worksapces to achieve that. my terraform cli workspaces follow this format {environment}_{region} i.e. dev_us-west-2 . Creating a new terraform CLI workspace auto-creates the TF Cloud workspace with the correct naming scheme using the name prefix in the remote backend config. The one caveat is that TF Cloud doesn’t understand TF CLI workspaces (terraform.workspace is “default” to it currently). There has been a bunch of complaints about that so I’m assuming in the future it will be fixed. The workaround I have is adding a workspace variable in TF_CLOUD and in my terraform code i just have this ternary operation that allows me to do local tf applies as well as remote tf applies:

var.workspace != "" ? var.workspace : terraform.workspace
btai avatar

How I get my region from my TF CLI workspace i.e. prod_us-east-1:

output "region" {
  description = "the environment provided in the Terraform CLI workspace (`<environment>_<region>[_<unique-name>]`)"
  value = element(
    split(
      "_",
      var.workspace != "" ? var.workspace : terraform.workspace,
    ),
    1,
  )
}
johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

I honestly do not think TFC using default is a bug they’ll fix. It is just an overuse of the word workspace where anyone using the CLI thinks of workspaces differently.

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

In this case, there is a single place (tf workspace) being used so it is default

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

but…to your point, yes…just set a var for the region. AWS_REGION (if you’re using AWS) to simply get it

btai avatar

This issue comment shows promise but i havent verified https://github.com/hashicorp/terraform/issues/22802#issuecomment-617642088

${terraform.workspace} value is default and not actual workspace name (when using the terraform cloud remote backend) · Issue #22802 · hashicorp/terraform

Terraform Version Terraform v0.12.8 Terraform Configuration Files resource &quot;aws_instance&quot; &quot;example&quot; { count = &quot;${terraform.workspace == &quot;default&quot; ? 5 : 1}&quot; #…

sheldonh avatar
sheldonh

I’m doing something very similar. I don’t do name parsing, and everything is remote. I just use 2 workspace variables in remote

• region

• stage The stage let’s me get the remote credentials appropriate for the service account rather than including in the workspace. The region allows also setting. I am trying to setup workspaces with code as much as I can but only a few projects have that fully done. If i could link the github repo when doing tf init that would be great!

Thanks for sharing this info! Cool to see different ways people are doing it. Sounds like while I’m on terraform cloud I should use just one workspace with region name like I was approaching.

In the meantime, evaluating terragrunt is promising, albeit it might be hard to build the entire workflow of plan/wait for approval and all that terraform cloud offers already. Makes me think I’ll have a harder time selling that

sheldonh avatar
sheldonh

Just learned…. New variable: TFC_WORKSPACE_NAME

party_parrot1
btai avatar

@sheldonh in TFC, that is unfortunately going to be the TFC workspace (default) as opposed to the TF CLI workspace name. Im personally curious if it is the full TFC workspace name <prefix><tf_cli_workspace> and you could prob cut the prefix out in your TF code. I haven’t tested it but this HashiCorp member said as such: https://github.com/hashicorp/terraform/issues/22802#issuecomment-618583576

${terraform.workspace} value is default and not actual workspace name (when using the terraform cloud remote backend) · Issue #22802 · hashicorp/terraform

Terraform Version Terraform v0.12.8 Terraform Configuration Files resource &quot;aws_instance&quot; &quot;example&quot; { count = &quot;${terraform.workspace == &quot;default&quot; ? 5 : 1}&quot; #…

sheldonh avatar
sheldonh

I just stopped a while back even trying to use it and instead configure based on a var if required. Thanks for clarifying that variable doesn’t do what expected

Joe Hosteny avatar
Joe Hosteny

Not sure if this is the right channel for this, but would someone at CP mind tagging a new version of https://github.com/cloudposse/terraform-github-repository-webhooks? I had a PR that got merged, but it looks like that one is not setup to tag automatically on merge to master.

cloudposse/terraform-github-repository-webhooks

Terraform module to provision webhooks on a set of GitHub repositories - cloudposse/terraform-github-repository-webhooks

Maxim Mironenko (Cloud Posse) avatar
Maxim Mironenko (Cloud Posse)

will check for it right now

cloudposse/terraform-github-repository-webhooks

Terraform module to provision webhooks on a set of GitHub repositories - cloudposse/terraform-github-repository-webhooks

Joe Hosteny avatar
Joe Hosteny

Thanks @Maxim Mironenko (Cloud Posse)! Sorry for commenting on the closed PR - there didn’t look to be an issue template I could use to open one for that.

Maxim Mironenko (Cloud Posse) avatar
Maxim Mironenko (Cloud Posse)

here it is: <https://github.com/cloudposse/terraform-github-repository-webhooks/releases/tag/0.6.0>

Maxim Mironenko (Cloud Posse) avatar
Maxim Mironenko (Cloud Posse)

thanks for your contribution

Joe Hosteny avatar
Joe Hosteny

Thanks for taking care of this, appreciate it!

Brij S avatar
Brij S

For anyone using the terraform-aws-vpc module, I noticed it doesnt create a vpc endpoint policy. Is this intentional? Does anyone have a work-around to add a policy to a vpce created by the module?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Just haven’t gotten around to it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

PR’s welcome

Brij S avatar
Brij S

oh great, i’ll take a look

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Are you working towards CIS compliance?

Brij S avatar
Brij S

no, we are in the middle of a cloud security audit and some things came up which made me think about current resources im deploying and making sure they dont show up in the audit in the future

Brij S avatar
Brij S

I realized that the vpce is created with a public policy so I wanted the option to tighten it down a bit

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ok, makes sense

sheldonh avatar
sheldonh

For terraform cloud, is there any name matching rules for auto.tfvars for loading? The docs were confusing. Was hoping to preload a few of these autos for different regions/names and just set one workspace variable that would ensure this file for default values gets loaded.

i do this in one project via yaml but trying to explore the native built in offering for merging default values from files.

2020-04-21

Pratap avatar
Pratap

Ho to all, can anyone help to to fetch the alerting details or data from Elasticsearch and send it to otrs ticketing tool using terraform. I read the aws kinesis using terraform but a confusion is still there. So I am asking here. If anyone worked with on this plz help me.

drexler avatar
drexler

Hello is there a way to reference multiple attributes for a partition key in dynamodb?

Gowiem avatar
Gowiem

Hey folks — Anybody know of a way to add tags to an existing resource? Specifically the main route table which is generated implicitly when creating a VPC?

Gowiem avatar
Gowiem

Is the answer to create a new route table with the same routes and then do a aws_main_route_table_association?

johncblandii (Cloud Posse) avatar
johncblandii (Cloud Posse)

Did you try importing the route table into your state then editing via specific TF code?

Gowiem avatar
Gowiem

I’m writing an open source module, so I don’t think would work cause it needs to be easily repeatable across projects.

:--1:1
jedineeper avatar
jedineeper

I would normally recommend avoiding the default route table and leaving it unused, but you should be able to use this resource in tf to adopt it and then tag it.

https://www.terraform.io/docs/providers/aws/r/default_route_table.html

AWS: aws_default_route_table - Terraform by HashiCorp

Provides a resource to manage a Default VPC Routing Table.

paultath81 avatar
paultath81

Hello - I’m having bit of a brain fart , hoping someone can help.

Within my resource "aws_launch_configuration" i’ve set my image_id to reference var.image_id (depending on OS type (e.g windows or Centos)

If I set var.image_id to centos how can i tell user_data to execute a bash template file oppose to setting var.image_id to windows and executing a powershell template file?

paultath81 avatar
paultath81

hope that makes sense.

paultath81 avatar
paultath81

or should i use template_cloudinit_config data source and include both bash and powershell script as multi part script?

paultath81 avatar
paultath81

i got what i’m expecting to work - seems a bit hacky.. Any feedback would be appreciated too :)

user_data                   = var.ami_name == "CentOS" ? file("${path.module}/templates/install_xxx.sh") : data.template_file.windows_userdata.rendered
jedineeper avatar
jedineeper

If there are just two options then maybe make the incoming variable a binary “linux” or “windows” and do a map lookup on that within the module?

jedineeper avatar
jedineeper

Or maybe use an ami data source to lookup the platform value and similarly map it out?

2020-04-20

Sorin avatar
Sorin

Am I right that I cannot use this module for default VPC? Because it REQUIRES me entering all the CIDR info which the Terraform “standard” module can do without, in a default VPC environment. By the way, specifying in docs which are the mandatory arguments would only help…

Sorin avatar
Sorin

Oops oky I’ve found a place where the “required” are marked as such (but not in the main page, only when I click inputs)

Sorin avatar
Sorin

And none of the examples basic/complete validate, missing required arguments or having unsupported arguments… (tf 0.12.24)

daniel274 avatar
daniel274

How would one convert a map of key=value to a string of “key=value,key=value…”

loren avatar
loren

probably using the string templating and a for loop… https://www.terraform.io/docs/configuration/expressions.html#string-templates

Expressions - Configuration Language - Terraform by HashiCorp

The Terraform language allows the use of expressions to access data exported by resources and to transform and combine that data to produce other values.

daniel274 avatar
daniel274

think ive got it

daniel274 avatar
daniel274

join(“,”, formatlist(“%s=%s”, keys(var.common_tags), values(var.common_tags)))

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’m surprised this works. Looking at it makes think it would produce a list like ['key1=key2', 'key3=key4', 'value1=value2', 'value3=value4']

daniel274 avatar
daniel274

you made me doubt myself so i went back to check it worked right:

Outputs:

trevor = bob=fred,dan=true,default_tag=default_value
daniel274 avatar
daniel274

and can confirm they go through my annotations of my ingress controller and then back as tags to the elb it sets up

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks for confirming! Interesting behavior of formatlist

Cloud Posse avatar
Cloud Posse
04:00:11 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Apr 29, 2020 11:30AM.
Register for Webinar
slack #office-hours (our channel)

Brij S avatar
Brij S

does anyone know if terraform-enterprise supports submodules? Similar to this https://registry.terraform.io/modules/terraform-aws-modules/iam/aws/0.0.4

Pierre Humberdroz avatar
Pierre Humberdroz

enterprise or cloud?

Brij S avatar
Brij S

oh, i suppose enterprise was renamed to cloud

2020-04-19

2020-04-18

Joan Hermida avatar
Joan Hermida

Hi! Someone, I’m dying here. Does Terraform has a way to create EventBridge Event Buses? I just can’t find how. Every search points to aws_cloudwatch_event_permission but that’s out of scope here. Anyone?

Joan Hermida avatar
Joan Hermida

Terraform v0.11.14

loren avatar
loren

not yet, check this issue, and the prs linked to it… https://github.com/terraform-providers/terraform-provider-aws/issues/9330

Amazon EventBridge · Issue #9330 · terraform-providers/terraform-provider-aws

Community Note Please vote on this issue by adding a :–1: reaction to the original issue to help the community and maintainers prioritize this request Please do not leave &quot;+1&quot; or &quot;me to…

1

2020-04-17

sheldonh avatar
sheldonh

So i’m trying to think of ways to simplify running in terraform cloud + later maybe some azure devops. Thinking of the terraform-root-modules repo Cloudposse has and made me think… while I need separate repos for modules, would it be a better practice to setup all my root module plans from 1 root repo in the same manner? The checks list in github would be huge after a while but would be probably easier to contribute new plans at that point, if I required no modules in there, just root plans.

I don’t use Make, but probably would setup either some InvokeBuild powershell helper for local ops, and then have a terraform plan in there that setup each folder added automatically as workspaces with version control hooks and all (already built this).

Any thoughts to the contrary?

sheldonh avatar
sheldonh

I want something like this in my slack room Please tell me they need more beta users, because I would love for a bot that gave me suggestions from confluence and previous history. The only thing I’ve seen before cost a fortune and I couldn’t get something like that through at all.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

You’ve seen #variant (variant2)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

comes with Slackbot built-in

sheldonh avatar
sheldonh

Now i just want to figure out how to make it answer questions completely wrong by putting the wrong keywords in… Any know how to stop robots from taking over the world?

sheldonh avatar
sheldonh

Maybe you didn’t all see it, but Foqal prompted me with some prior message history about make files. Pretty cool concept!

2020-04-16

nileshsharma.0311 avatar
nileshsharma.0311

@channel I am thinking of using a remote storage backend for storing terraform state files , the way I use it , it often contains sensitive info like ssh keys etc , what is a good solution, S3 or consul ? From a cost and security perspective. Thanks in advance

Abel Luck avatar
Abel Luck

We’ve always stored state encrypted in S3. Consul has too much overhead, unless perhaps you’re already using it

nileshsharma.0311 avatar
nileshsharma.0311

Thanks @Abel Luck , For encryption, do you suggest server side encryption or client side ?

Abel Luck avatar
Abel Luck

The terraform remote s3 backend only supports server side encryption (using AWS KMS keys).

nileshsharma.0311 avatar
nileshsharma.0311

Noted thanks for the help

Abel Luck avatar
Abel Luck

That’s for the storage of remote state.

However we use client side encryption for the storage of input variables. We do this with mozilla’s sops tool.

Abel Luck avatar
Abel Luck

So our inputs to terraform are committed to git, encrypted.

Abel Luck avatar
Abel Luck

There is a terraform sops provider that makes loading encrypted secrets from a sops file easy

Abel Luck avatar
Abel Luck

We find it much easier to handle then a full HA vault deployment (We’re a small team, so vault is overkill).

nileshsharma.0311 avatar
nileshsharma.0311

Got it , I was confused too whether we should use vault or no , it’s definitely an overkill for small teams , That was some good insight

Christian Roy avatar
Christian Roy

Hello, I am new here. SweetOps looks like a strong community!

I am thinking about using Terraform and I am wondering if it would easily allow me to specify that one of the resources needed is a helm chart in a private helm repo on gcs and it would create (and push) it if it does not exists, and another resource needed would be a docker image on google container registry and if it does not exists it would create (and push) it… So far I’ve seen resources being “ip address, vnc, vm” but not what I just explained. Did I miss something?

Thanks

Zachary Loeber avatar
Zachary Loeber

I’ve never used terraform to push a helm repo. The helm provider only has data sources for repositories and resources that allow you define and create releases from charts within them.

Zachary Loeber avatar
Zachary Loeber

if you have the entire chart locally that you would push for a deployment you can just refer directly to the chart location and bypass the entire repo creation process

Zachary Loeber avatar
Zachary Loeber
resource "helm_release" "local" {
  name       = "my-local-chart"
  chart      = "./charts/example"
}
Igor Bronovskyi avatar
Igor Bronovskyi

Hello. I have trouble with aws_appautoscaling_policy . When left one container this alarm not disappeared. https://take.ms/5cyu8

Igor Bronovskyi avatar
Igor Bronovskyi
resource "aws_appautoscaling_target" "target" {
  service_namespace  = "ecs"
  resource_id        = "service/${aws_ecs_cluster.main.name}/${aws_ecs_service.main.name}"
  scalable_dimension = "ecs:service:DesiredCount"
  min_capacity       = 1
  max_capacity       = 5
}

resource "aws_appautoscaling_policy" "down" {
  name               = "${var.project}_scale_down"
  service_namespace  = "ecs"
  resource_id        = "service/${aws_ecs_cluster.main.name}/${aws_ecs_service.main.name}"
  scalable_dimension = "ecs:service:DesiredCount"

  step_scaling_policy_configuration {
    adjustment_type         = "ChangeInCapacity"
    cooldown                = 60
    metric_aggregation_type = "Maximum"

    step_adjustment {
      metric_interval_lower_bound = 0
      scaling_adjustment          = -1
    }
  }

  depends_on = [aws_appautoscaling_target.target]
}

resource "aws_cloudwatch_metric_alarm" "service_cpu_low" {
  alarm_name          = "${var.project}_cpu_utilization_low"
  comparison_operator = "LessThanOrEqualToThreshold"
  evaluation_periods  = "2"
  metric_name         = "CPUUtilization"
  namespace           = "AWS/ECS"
  period              = "60"
  statistic           = "Average"
  threshold           = "10"

  dimensions = {
    ClusterName = aws_ecs_cluster.main.name
    ServiceName = aws_ecs_service.main.name
  }

  alarm_actions = [aws_appautoscaling_policy.down.arn]
}
Christian Roy avatar
Christian Roy

So, sounds like the terraform helm provider would need to have something called helm_chart which would be linked to a helm_repository and some files (local or git) that would run helm package ${DIR} and then handle the different ways to “add” a chart to a repo, which I think may have many ways. I use gcs so for me its helm gcs push ${chart}.tgz ${reponame}

Zachary Loeber avatar
Zachary Loeber

right a resource called helm_chart specifically that would run all the package and push stuff, typically if you are doing this it would be from another pipeline though (simply using the helm command). The gcs repo would be created by terraform in some other pipeline at some other level

Christian Roy avatar
Christian Roy

Thank you

2020-04-15

Abel Luck avatar
Abel Luck

When using https://github.com/cloudposse/terraform-aws-ec2-autoscale-group if I want to create an ASG of size=1 , and attach an EBS volume that persists and is always mounted onto the active instance, should I use the block_device_mappings input.. or create the ebs volume separately and use userdata to attach it on boot?

Reading the docs of aws_launch_template doesn’t clear it up for me.

If I include a config such as the following, will it create a new volume for each ASG instance created or re-use the same one?

block_device_mappings = [
      {
        device_name  = "/dev/sda1"
        virtual_name = "root"
        ebs = {
          encrypted             = true
          volume_size           = 50
          delete_on_termination = false
          volume_type           = "gp2"
        }
      }
    ]
Zach avatar


or create the ebs volume separately and use userdata to attach it on boot?
This is correct

1
1
Alex Friedrichsen avatar
Alex Friedrichsen

Hello, I am noticing issues with https://github.com/cloudposse/terraform-aws-eks-cluster whether I am adding/removing managed node groups or just making a minor change to an existing one (example: increasing desired number of nodes). In all circumstances, I am able to run an initial terraform plan/apply but any future run returns the following error

module.eks_cluster.kubernetes_config_map.aws_auth_ignore_changes[0]: Creating...

Error: configmaps "aws-auth" already exists

  on .terraform/modules/eks_cluster/auth.tf line 84, in resource "kubernetes_config_map" "aws_auth_ignore_changes":
  84: resource "kubernetes_config_map" "aws_auth_ignore_changes" {

This includes even setting the variable kubernetes_config_map_ignore_role_changes = false as recommended. Using the latest stable version of the plugin (0.22.0) as well as the latest stable terraform. Thoughts?

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Alex Friedrichsen avatar
Alex Friedrichsen

the issue happens even I run the example from the examples/complete directory verbatim. I would expect that I could run terraform apply seemingly over and over if there are no changes to state, but the 2nd run always results in the error.

cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

can’t confirm right now the problem, but the gist of the issue is a fundamental problem with how the auth config map is managed. basically, managed eks node groups create this automatically if it doesn’t exist, and update it if it does exist. the hack is to create an interstitial dependency in terraform using a null resource so we first create the map with terraform (so terraform is aware of the resource) before the node groups and then create the managed node group.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

not ruling out that the behavior might have changed as is sometimes the case.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@johncblandii (Cloud Posse) have you noticed anythign lately?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@ manually delete the config map from the cluster

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

It was created before by something I don’t know what

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

And now it’s not in the terraform state for some reason

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Or you can import it

Alex Friedrichsen avatar
Alex Friedrichsen

Thanks @Erik Osterman (Cloud Posse) and @Andriy Knysh (Cloud Posse).

Seems like the issue was in fact what was described about the configmap not being in state.

Equal parts working with an existing cluster (provisioned previously with an older version of the same module) and equal parts network timeout on my end when trying before with new clusters. Thanks again.

:--1:3
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@ if you don’t do it already, you should pin the module to a release (either the last one or the prev one if you used it before). Don’t use master b/c it could introduce breaking changes.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ahhh cool, glad you got to the bottom of it. Yes, makes sense that it would have been a migration problem from the old version of the module to the new one.

2020-04-14

Tom Howarth avatar
Tom Howarth

I am attempting to build out my first autoscaling group and as a part of the launch instance I need to install some programs and upload some config files, when I was just creating a single instance I would use host = self.public.ip to login to the externally facing IP Address and it would run its magic, however host = self.public_ip is an unsupported attribute in “aws_launch_configuration. so how will I handle this now?

Pierre Humberdroz avatar
Pierre Humberdroz

You would like to look into: https://docs.aws.amazon.com/batch/latest/userguide/launch-templates.html and Cloud-Init section of this you can provide this also via the terraform resource afaik.

Launch Template Support - AWS Batch

AWS Batch supports using Amazon EC2 launch templates with your compute environments. Launch template support allows you to modify the default configuration of your AWS Batch compute resources without requiring you to create customized AMIs.

ms16 avatar

I would use suggest looking TF data templates to pass to pass custom scripts as user data to the launch templates

Albert Domenech avatar
Albert Domenech

I’m trying to use https://registry.terraform.io/modules/cloudposse/emr-cluster/aws/0.3.0?tab=inputs|emr-cluster module but I don’t see step component defined in the aws_emr_cluster resource. The only reference I see, applies ignore_changes meta-argument on the lifecycle block. Is there any way to provision custom jar with map/reduce jobs via terraform with this setup?

curious deviant avatar
curious deviant

Hello I have a terraform modules + CircleCI situation that I am pretty sure others must have resolved for. Basically CircleCI is unable to download/clone terraform modules in a repo that references them, both hosted in our GitHub. From what I have researched looks like it may be a security feature in CircleCI. How should this be solved for ? Also let me know if there’s another channel I should post this query to.

ms16 avatar

@curious deviant If the TF modules are hosted in private repo, your CI agent will need clone access to your repository ,You may create a bot user in ur github org with read-only access to terraform modules repo and configure the access token in the CI pipeline with some sort of encryption , so when the pipeline executed the CI agent authenticates into your github org to clone TF repos

curious deviant avatar
curious deviant

thanks !

matthew.king avatar
matthew.king

if I use a module, say one of the awesome ones you guys created, can I on my local run of terraform use a terraform.tfvars file to input into the module that I’m using via git? or am I not understanding how the module creation and download works?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, you instantiate a module and provide the variables for it. The variables’ values can come from [variables.tf](http://variables.tf) or from any .tfvars files

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-eks-cluster

Terraform module for provisioning an EKS cluster. Contribute to cloudposse/terraform-aws-eks-cluster development by creating an account on GitHub.

matthew.king avatar
matthew.king

OK cool so I just put all the variables I want defined in the variables.tf and go from there. Cool thank you

Brij S avatar
Brij S

How do people here who use lambdas, deploy lambda config changes (with no code change) and get a version deployed and the alias updated? I dont think Terraform handles this well

Joe Presley avatar
Joe Presley

It’s been a while since I had to manage lambdas but my go to was the Serverless Framework. Terraform isn’t good at managing the lifecycle of a lambda function.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terraform does suck for lambdas.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

one of the best modules (or so I’ve heard) is https://github.com/claranet/terraform-aws-lambda if you do want to go this route.

claranet/terraform-aws-lambda

Terraform module for AWS Lambda functions. Contribute to claranet/terraform-aws-lambda development by creating an account on GitHub.

ms16 avatar

Im using SAM from AWS

2020-04-13

Cloud Posse avatar
Cloud Posse
04:00:50 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Apr 22, 2020 11:30AM.
Register for Webinar
slack #office-hours (our channel)

Doug Lethin avatar
Doug Lethin

hello - can someone possibly explain the difference between https://github.com/cloudposse/terraform-terraform-label and https://github.com/cloudposse/terraform-null-label. It’s ot obvious from their README. It seems the latter is more active and has more inputs and examples. Also confused as the doc says The null in the name refers to the primary Terraform null provider used, but looks like as of v0.15.0 use of null_resource has been removed, but probably the name remains for backwards compatibility. Is the former terraform-terraform-label deprecated in favor of terraform-null-label? Thanks.

cloudposse/terraform-terraform-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-terraform-label

cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@ we should have this documented, but for now here’s the quick answer

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
SweetOps Slack Archive

SweetOps is a collaborative DevOps community. We welcome engineers from around the world of all skill levels, backgrounds, and experience to join us! This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build sweet infrastructure.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(check out some of the responses there)

2020-04-12

2020-04-11

rohit avatar
rohit

did anyone use this module ? if yes, can you let me know your experienve with it

sheldonh avatar
sheldonh

I’m interested as well. I’ve heard some say for serverless just user serverless cli or Sam as terraform overcomplicates this one aspect and all toolchains are driven for cli for this. I’d like to also know if anyone has used that or equivalent and prefer it over the other methods.

rohit avatar
rohit

yeah i would like to hear about it too

2020-04-10

Abel Luck avatar
Abel Luck

Is it possible in terraform 12 to dynamically define provider blocks based on a variable?

I tried something like this:

locals {
  satellite_regions = {
    eu  = "eu-central-1"
    us1 = "us-east-2"
  }
}
provider "aws" {
  for_each = local.satellite_regions
  alias    = each.key
  region   = each.value
}

But terraform complains

Error: Reserved argument name in provider block
    on [main.tf](http://main.tf) line 17, in provider "aws":
  17:   for_each = local.satellite_regions

The provider argument name "for_each" is reserved for use by Terraform in a
future version.

I’m not fully up to speed on HCL2, so maybe my syntax is wrong? (FWIW Terraform v0.12.24)

Zach avatar

Like the error says, you can’t use for_each in some blocks yet. Hashicorps is expanding the use of for_each soon, but you can’t use it everywhere.

Abel Luck avatar
Abel Luck

is there another workaround to do achieve variable-defined providers?

Zach avatar

I’m not sure its capable of it yet

loren avatar
loren
Instantiating Multiple Providers with a loop · Issue #19932 · hashicorp/terraform

Current Terraform Version Terraform v0.11.11 Use-cases In my current situation, I am using the AWS provider so I will scope this feature request to that specific provider, although this may extend …

:--1:1
randomy avatar
randomy
raymondbutcher/pretf

Generate Terraform code with Python. Contribute to raymondbutcher/pretf development by creating an account on GitHub.

randomy avatar
randomy

Or manually define every possible region provider, then for each region call a module and pass the region-specific provider in, along with an “enabled” flag for that module based on whether that region was included in your list.

2020-04-09

Pierre-Yves avatar
Pierre-Yves

Hello, I am using Vscode and was looking for syntax highlight and linting; I just want to share that the syntax highlight extension to use is “terraform” 0.1.8” from Anton Kulikov ( 25k download ) latest commit in December 2019.

the extension “terraform” version 1.4.0 from “Mikael Olenfalk” (520k download ) is not working with the latest terraform syntax: latest commit October 2019.

Zach avatar

Yah ansiblevscode is pretty bad for terrraform now due to the lack of support on the plugin. JetBrains (IDEA or PyCharm) has a better plugin although its suffering from the same sort of lack of updates and doesn’t recognize some of the newer features in 0.12

Pierre-Yves avatar
Pierre-Yves

ansible ?

Zach avatar

err sorry, vscode I was editing ansible IN vscode when I wrote that

loren avatar
loren

i’ve been using the vscode with the language server for tf 0.12 support. it’s still a work in progress, but works for a good 90% of my use cases

Zach avatar

I kept trying the language server and it never seemed to do anything (at the time) so I eventually gave up

loren avatar
loren

been using it for months now, no problem

Zach avatar

Ok I can give it a shot again.

The nice thing with Jet Brains IDE plugin is you get full featured ‘where is this used’ and refactoring, and you can cmd/ctrl-click to jump to the definition of a var/local or into a module

loren avatar
loren

i never use that kind of feature anyway can’t keep track of the shortcuts

Zach avatar

same usually, but since this is mostly ‘click and go’ its the one I find helpful

imiltchman avatar
imiltchman

I have switched to IntelliJ because of this, and I am a big fan of their plugin

jedineeper avatar
jedineeper

Anyone know if it’s possible to use the new tf12 syntax to add optional parameters to a resource? I want to pass through a string for redrive_policy on an sqs_queue but it doesn’t take blank as an empty string. Trying to figure out if i can use a for loop to skip adding the redrive_policy parameter at all if the variable is empty?

randomy avatar
randomy

yes, you can use null for that

randomy avatar
randomy
variable "redrive_policy" {
  type    = string
  default = null
}
jedineeper avatar
jedineeper

Ah, of course. Fantastic, thanks.

:--1:1
Shawn Petersen avatar
Shawn Petersen

Does anyone know how to setup codepipeline resource for multi-region actions? The documentation is unclear, saying I need to add “region” to the “artifact_store” block for multi-region, but every time I try that, I get “Error: region cannot be set for a single-region CodePipeline”

How would I make it mutli-region?

randomy avatar
randomy

My colleague helped get this added to the AWS provider but I’m not very familiar with it myself. Did you see that it was only just released, and check the relevant PRs? https://github.com/terraform-providers/terraform-provider-aws/blob/master/CHANGELOG.md#2560-april-03-2020

Shawn Petersen avatar
Shawn Petersen

Thanks man, yeah I saw it was only recently released. I’ve updated to latest AWS provider but still doesn’t seem to work. The documentation just isn’t clear enough, or else it’s still broken?

Shawn Petersen avatar
Shawn Petersen

I wish there was an example in the PR with it working

randomy avatar
randomy

See how there is ArtifactStore and ArtifactStores - the latter has regions

randomy avatar
randomy

Oh, it looks like Terraform combines them into a single concept, somehow.

Shawn Petersen avatar
Shawn Petersen

yeah, i tried using the plural ArtifactStores instead of ArtifactStore, but it threw another error saying: Unsupported block type

randomy avatar
randomy

if you have 1 artifact_store block then it maps to ArtifactStore but if you have multiple it will map to ArtifactStores

randomy avatar
randomy

have you got multiple blocks and did you set region in them all?

Shawn Petersen avatar
Shawn Petersen

my goal is to run the pipeline with a single artifact in one region (CodeBuild) and deploy to multi regions (ECS)

Shawn Petersen avatar
Shawn Petersen

so i only need the single S3 bucket artifact in one region

randomy avatar
randomy

Let me see if I can convince my colleague to join this channel. It sounds like what he is currently working on (which is why he worked on the region feature).

Shawn Petersen avatar
Shawn Petersen

cool, great

randomy avatar
randomy

I don’t think he’s online at the moment.

Shawn Petersen avatar
Shawn Petersen

ok

Shawn Petersen avatar
Shawn Petersen

it might be that it doesn’t support my use case yet

Shawn Petersen avatar
Shawn Petersen

the alternative seems to build in each region, then deploy from there, but that just seems redundant

randomy avatar
randomy

I tried to figure it out myself but I can’t tell why multiple artifact stores are necessary. Will wait and see what he says. It might be a while though because it’s the Easter weekend.

Shawn Petersen avatar
Shawn Petersen

sure, no worries. I really appreciate you jumping in to try and help though. thanks

:--1:1
Shawn Petersen avatar
Shawn Petersen

I might be on to something. I created a single region pipeline using TF, and then manually added the other Deploy regions, and ran terraform plan again to see the differences. It looks like it might need multiple artifact_store blocks for each region, but the location isn’t the S3 bucket, it’s a codepipeline ID for that region, which I’m trying to figure out…. The diff shows this:

  - artifact_store {
      - location = "codepipeline-eu-west-1-221120437065" -&gt; null
      - region   = "eu-west-1" -&gt; null
      - type     = "S3" -&gt; null
    }
  - artifact_store {
      - location = "codepipeline-us-west-2-423457879940" -&gt; null
      - region   = "us-west-2" -&gt; null
      - type     = "S3" -&gt; null
    }
randomy avatar
randomy

are they your account ids? maybe they are real buckets

Shawn Petersen avatar
Shawn Petersen

yeah those aren’t my AWS account numbers, and those aren’t S3 buckets in my account

Shawn Petersen avatar
Shawn Petersen

looks like AWS-owned IDs?

randomy avatar
randomy

https://stelligent.com/2018/09/06/troubleshooting-aws-codepipeline-artifacts/
When you first use the CodePipeline console in a region to create a pipeline, CodePipeline automatically generates this S3 bucket in the AWS region.

randomy avatar
randomy

Shawn Petersen avatar
Shawn Petersen

oh wait, you are right.. it did create those S3 buckets for me

Shawn Petersen avatar
Shawn Petersen

this is starting to make more sense. It might be required to have an artifact_store (S3 bucket) in each region I plan to deploy to

Shawn Petersen avatar
Shawn Petersen

I have enough to move forward. Thanks again for your help Raymond!

:--1:1
Gowiem avatar
Gowiem

Is there a way for terraform to associate an instance with a launch template / ASG or wait for the ASG / launch template to create an instance? I’d like to include the launched instance’s details in the output of my Terraform module.

Gowiem avatar
Gowiem

Specifically I’m looking for a way to provision a Launch Template + ASG and then get the resulting instance IDs that are spun up from those resources.

ms16 avatar

@Gowiem not sure if TF lets you this , Im think you could you use terraform local-exec and run an aws cli script which will query the ASG created and print the instance ids.

Gowiem avatar
Gowiem

Yeah @ms16, was afraid of that. Will have to look into that and see if I can make something like that work.

randomy avatar
randomy

What do people think about Atlantis doing terraform apply before merging the pull request? They explain why on this page https://www.runatlantis.io/docs/locking.html

loren avatar
loren

my gut reaction is

randomy avatar
randomy

Same, but then I thought about it and the alternatives seem worse.

randomy avatar
randomy

Does the apply happen in a separate CI/CD system? Do you have Slack notifications to bring you into the approval page? Then another Slack notification if the apply fails? This seems messy.

loren avatar
loren

it’s messy either way i suppose. we consider pr approval as “go” and the merge then just is the ci trigger. the ci job status needs to be monitored in either case.

randomy avatar
randomy

I’m trying to build something similar to Atlantis using GitHub Actions, and having all of the human interaction in the pull request page is really tempting.

randomy avatar
randomy

apply before merge makes it so much simpler

loren avatar
loren

really? github-actions works fine on branches, right?

randomy avatar
randomy

yes they do

loren avatar
loren

we typically operate around several events… prs, branches, and tags. the ci does different things in all cases

randomy avatar
randomy

external CI system?

loren avatar
loren

usually all the same CI system, but it can be several. depends on the project

randomy avatar
randomy

i mean do you have a separate CI system or just github?

loren avatar
loren

we haven’t done much with github-actions ourselves. we’ve toyed with it here and there, but haven’t had a reason to move things actively

loren avatar
loren

we’re mostly still on travis-ci, but also codecommit, appveyor, azure pipelines, and now a little github actions

randomy avatar
randomy

i’m trying to build a terraform workflow that’s all in github

loren avatar
loren

also toyed with a github repo linked to gitlab-ci…

randomy avatar
randomy

so no logging into separate CI systems

loren avatar
loren

yeah, if you have private github access, and github actions logs are private as well, i can see that

randomy avatar
randomy

and no extra infrastructure to run

loren avatar
loren

i’m always super cautious with lettings prs have access to credentials

randomy avatar
randomy

yeah, it checks the user’s identity before running a plan, and pull requests from forks won’t work

loren avatar
loren

are the logs private? i mean, what if an authorized user is debugging something and changes the pr to log sensitive values?

randomy avatar
randomy

i may move the actual terraform apply part into codebuild to reduce the AWS access that github needs

randomy avatar
randomy

i think the logs will be as private as the repo itself, so a user could potentially reveal sensitive values that way

randomy avatar
randomy

if the terraform apply part is moved to codebuild, access to logs could be restricted to

randomy avatar
randomy

but i’m more stuck on the user interface and workflow side of it, hence the question about applying before merging

loren avatar
loren

heh

loren avatar
loren

if you want something else to compare to, here’s one of our public projects that implements the pr/branch/tag workflow using travis-ci…. https://github.com/plus3it/wrangler-watchmaker/blob/master/.travis.yml#L31-L91

plus3it/wrangler-watchmaker

Manages buckets and files needed for the public/default watchmaker configuration - plus3it/wrangler-watchmaker

loren avatar
loren

i’d like to get a plan in the pr, but exposing the credentials is a no go on the travis-ci side…

loren avatar
loren

would could use a github webhook to trigger codebuild, run the plan, and post the status check back to the commit/pr with a link to the job. we’ve done that also, for other projects

randomy avatar
randomy

what is watchmaker?

randomy avatar
randomy

the travis stuff looks probably good but i want to keep the entire workflow within github, and preferably within pull requests

randomy avatar
randomy

and i want to avoid: plan, approve, merge, apply fails, confusion

randomy avatar
randomy

it’s very early stages. still trying to figure out how i want it to work.

loren avatar
loren

Yeah, totally, was just providing another reference, maybe spark some ideas around the interface/config…

randomy avatar
randomy

Thanks, it is helpful

randomy avatar
randomy

I’ll do this iteratively. Start entirely in GitHub, then look at moving pieces into external systems (like CodeBuild). But first I need to nail the interface.

randomy avatar
randomy

Atlantis seems to have it about right, but apply-before-merge is radical.

ms16 avatar

Does anyone know why using -detailed-exitcode for terraform plan returns 2 shell exit code ?

randomy avatar
randomy
Command: plan - Terraform by HashiCorp

The terraform plan command is used to create an execution plan. Terraform performs a refresh, unless explicitly disabled, and then determines what actions are necessary to achieve the desired state specified in the configuration files. The plan can be saved using -out, and then provided to terraform apply to ensure only the pre-planned actions are executed.

ms16 avatar

Thanks @randomy I just RTD

ms16 avatar

Im just working on making our CI build fail and send Slack notification when the terraform plan stage for X env fails. Using azpipelines slack not and AzDO YAML pipeline to build CICD.

2020-04-08

sahil kamboj avatar
sahil kamboj

Hey guys, just started working with terraform can we make output of terraform store in s3 so my services fetch details from there like elb name- id etc. dont want to use aws cli

Chris Fowles avatar
Chris Fowles

for other terraform you can use this datasource to query a remote state

Chris Fowles avatar
Chris Fowles

for things other than terraform - you probably want to write values to something like parameter store or something like that

Chris Fowles avatar
Chris Fowles

if you’re really set on writing to s3 you could use this resource to write a file out to s3 as part of your terraform module https://www.terraform.io/docs/providers/aws/r/s3_bucket_object.html

AWS: aws_s3_bucket_object - Terraform by HashiCorp

Provides a S3 bucket object resource.

sahil kamboj avatar
sahil kamboj

thnx Chris

setheryops avatar
setheryops

Does anyone know of a way that you can take an output and set a CircleCI env var as that output repended with TF_VAR_ ? Other than a script possibly…im really more wondering if there is a way in CCi but thoguht id ask here since its TF related too.

github140 avatar
github140

I don’t know in CCi however I’d use terraform-docs json & jq to prepare it.

Dave Barnum avatar
Dave Barnum

I have a question about terraform-null-label; It seems like the tags support maps. However, some Azurerm resources like azurerm_kubernetes_cluster support maps while others like azuread_service_principal support only lists. Is there any way to output (or input) lists of tags from null-label?

Gowiem avatar
Gowiem

I don’t know for certain, but I guess likely not since most CP modules are AWS focused and tags in AWS are exclusively (I think) key value pairs. This would probably be a good contribution to that module though.

David avatar
David

Right now, when we add new devs to our team, we add their name, email, github handle, etc. to a terraform input file, and then we have Atlantis provision their Datadog account, github invite to our org, AWS IAM User, etc.

I am looking into Okta so that our Ops side of things can create users across all our accounts, but have some concerns where it seems like support for AWS users and some other orgs would become harder to work with (SAML seems quite annoying compared to SSO, for example, and we like having IAM Users as we already have strategies for eliminating long-lasting credentials)

For those who have faced similar issues before, how did you decide which accounts to provision through Okta/terraform? If you have a well-oiled terraform/atlantis setup, do you feel that Okta is still worth pouring some money into?

Zach avatar

I definitely do not have a well oiled setup yet, but what I do like is that Okta lets me add MFA to things that otherwise don’t really support it. And have different MFA policies depending on what application they’re accessing.

Chris Fowles avatar
Chris Fowles

The other great advantage of SAML is a single point to de-provision access quickly. Auditing login events also becomes a lot easier.

I’d look at the decision by prioritized pragmatism - are things that Okta is going to do part of the current major priority of the business? Usually I see this kind of priority come around during a compliance event, like targeting ISO 27001 or PCI DSS compliance or an IPO.

2020-04-07

caretak3r avatar
caretak3r

question, I have a dir/repo setup like this:

/repo/
-- terraform.tfvars
-- .envrc

• .envrc has:

export TF_CLI_INIT_FROM_MODULE="git::<https://github.com/***/terraform-root-modules.git//aws/tfstate-backend?ref=master>"
export TF_CLI_PLAN_PARALLELISM=2
export TF_BUCKET="devops-dev-terraform-state"
export TF_BUCKET_REGION="us-east-1"
export TF_DYNAMODB_TABLE="devops-dev-terraform-state-lock"
source <(tfenv)

• terraform.tfvars has:

namespace="devops"
region="us-east-1"
stage="dev"
force_destroy="false"
attributes=["state"]
name="terraform-tfstate-backend"

But when i run terraform init it complains about a non-empty directory. i am trying to learn this before jumping to geodesic, but i don’t know how to get the root module copied to my repo above. am i doing something incorrectly?

error:

❯ terraform init
Copying configuration from "git::<https://github.com/***/terraform-root-modules.git//aws/tfstate-backend?ref=master>"...
Error: Can't populate non-empty directory
The target directory . is not empty, so it cannot be initialized with the
-from-module=... option.
:--1:1
1
caretak3r avatar
caretak3r

^bump

androogle avatar
androogle

can you post the specific error

androogle avatar
androogle

looking at what you’ve supplied it doesn’t look like there’s anything for terraform to init

androogle avatar
androogle

do you have a main or any other kind of .tf with provider resources defined?

caretak3r avatar
caretak3r

@androogle i updated the original post

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The behavior of terraform changed in 0.12

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

0.11 used to allow initialization of a directory with dot files

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have a long thread discussing this here: https://sweetops.slack.com/archives/CB84E9V54/p1582930086018800

I’ve run into the

The target directory . is not empty, so it cannot be initialized with the
-from-module=... option.

issue trying to use terraform 12 on a geodesic container. Anyone know of a work around? For the meantime I’m going to create a temp dir and init into there and move the files the the dir I want to use.

caretak3r avatar
caretak3r

@Erik Osterman (Cloud Posse) aww, damn. i was looking for anything in the changelogs yest about this specific behavior. maybe just init into a new dir, and then copy the tfvars/envrc into the copied-module dir?

Abel Luck avatar
Abel Luck

How do you all work with a situation where you want a terraform module to spin up resources (instances in this case) in multiple regions?

Abel Luck avatar
Abel Luck

I’ve got my terraform root module (remote state in s3) and i want to create an app server in several regions, ideally in one terraform invocation.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This is a design decision that comes down to what you want to achieve with multi-region support

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

One consideration is to move your S3 state out of AWS so that it’s decoupled from AWS failures.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Otherwise, if you want to use something like the terraform-aws-s3-statebackend module, best-practice would be to have one state bucket per region to decouple state operations so they aren’t affected by regional failures.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Note that terraform cloud is not hosted on AWS, so using terraform cloud as the statebackend is an alternative.

randomy avatar
randomy

Ignoring whether that’s a good idea or not, if you want it in 1 terraform invocation and state file then you can:

• create a module with your instances

• call your module for each region, passing a region-specific aws provider into the module

randomy avatar
randomy

See the first example of https://www.terraform.io/docs/configuration/modules.html#passing-providers-explicitly where they call a module and pass a region-specific provider into the module as the default aws provider for that module

Modules - Configuration Language - Terraform by HashiCorp

Modules allow multiple resources to be grouped together and encapsulated.

Abel Luck avatar
Abel Luck

Thanks! I remember now I’ve used explicit providers before when using letsencrypt staging and production api servers at the same time

Abel Luck avatar
Abel Luck


Ignoring whether that’s a good idea or not,
I’d be happy to hear your thoughts on why it’s not a good idea.

randomy avatar
randomy

I wouldn’t say it’s not a good idea, just that there are trade offs. It’s mainly to do with “blast radius” of changes, and what happens if the main region fails. It is probably fine to do what you’ve proposed though.

sarkis avatar
sarkis
mvisonneau/tfcw

Terraform Cloud Wrapper. Contribute to mvisonneau/tfcw development by creating an account on GitHub.

1

2020-04-06

PePe avatar

Hi, Any ide why this https://github.com/cloudposse/terraform-aws-s3-bucket/blob/master/main.tf have only lifecycle rules for versioned buckets ?

cloudposse/terraform-aws-s3-bucket

Terraform module that creates an S3 bucket with an optional IAM user for external CI/CD systems - cloudposse/terraform-aws-s3-bucket

Cloud Posse avatar
Cloud Posse
04:00:06 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions on terraform and get to know others in the community on a more personal level. Next one is Apr 15, 2020 11:30AM.
Register for Webinar
slack #office-hours (our channel)

PePe avatar
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Maxim Mironenko (Cloud Posse) is back, so we can take a look at some things

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

let’s move PR requests to #pr-reviews

PePe avatar

ohh cool, sorry I did not that channel existed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s pretty new

curious deviant avatar
curious deviant

Another terraform best practices question.. i have a multi account setup wherein I create one environment type (dev, test, stage) per AWS Account. What could be a preferred strategy for storing remote state in S3 backend ? I am currently using 1 bucket /env to store state and the state bucket resides in the same account as the infrastructure being spun by terraform. Someone on the team recommended using one state bucket in the shared service account for all environments. They just want to be able to see all state files in one bucket. While it’s technically feasible, I am thinking this adds additional complexity (cross account setup) without any real benefit. What do folks think ?

androogle avatar
androogle

I don’t know if its considered best practice but I use one bucket per account and use workspaces for environments. I switch between accounts with -backend-config

androogle avatar
androogle

I’ve got a shell function to switch back and forth:

tfswitch() {
  export AWS_PROFILE=$1
  terraform init -reconfigure -backend-config=backends/$1.tf
}
androogle avatar
androogle

I guess I’d also be hesitant to have the prod s3 state bucket share a space with the dev s3 state bucket and the IAM policies for both state writing and cross-account access get complex to manage

:--1:2
curious deviant avatar
curious deviant

I agree… those are my concerns too.

androogle avatar
androogle

additionally if you’re operating as a team you’ll still probably need separate dynamodb lock tables for each env

curious deviant avatar
curious deviant

ok sure.. I haven’t used them thus far but I can take a look to see how that would work out.

Mikael Fridh avatar
Mikael Fridh

I’m using one single s3 bucket, one single dynamo table for maybe 30 or so state files (30 “stacks”)

androogle avatar
androogle

how do you use 1 dynamo table for multiple states?

androogle avatar
androogle

is it just an item per state in the same table and you specify it explicitly in the backend config?

Mikael Fridh avatar
Mikael Fridh

my backend.config is the same in ALL stack folders:

terraform {
  backend "s3" {
    profile        = "aws-profile"
    region         = "aws-region"
    bucket         = "s3-bucket"
    dynamodb_table = "my-infra-terraform-locks"
  }
}
androogle avatar
androogle

then person B making a change to state B has to wait until person A making a change to state A is finished?

androogle avatar
androogle

if they’re all using the same lockfile?

Mikael Fridh avatar
Mikael Fridh

then I have a little Makefile by which I do everything … make plan for example.

That sets the key for each folder based on the current path I’m in:

so if I’m in repo/terraform/infra/battlefield/bl-prod-mesos:

it will do:

echo key = "infra/terraform/b9d-infra/infra/battlefield/bl-prod-mesos/terraform.tfstate" > .terraform/backend.conf
Mikael Fridh avatar
Mikael Fridh

I could also have hardcoded the above in all individual folders… but so far I never did. Makefile does everything the same every time

Abel Luck avatar
Abel Luck


They just want to be able to see all state files in one bucket.
I question that.. why do you need to see the state files? Smells fishy.

We have one account per environment generally speaking, and one state bucket per account. Sometimes we end up with a second or third state bucket in account for a different client project, that’s usually just for small projects and only in the dev/test account.

Shawn Petersen avatar
Shawn Petersen

i have a syntax question. How do I use a colon ‘:’ inside a conditional expression? I want to append a second variable (var.env) to the end of either option like this value = “${var.one != “” ? var.oneenv : var.twoenv}” what am i missing?

Mikael Fridh avatar
Mikael Fridh

regular string interpolation:

  • 0.12: value = var.one != "" ? "${var.one}:${var.env}" : "${var.two}:${var.env}"
Shawn Petersen avatar
Shawn Petersen

ah perfect. thanks aton!

imiltchman avatar
imiltchman

0.11 looks off to me on first sight. I would create a local first with : and then reference it for value here

imiltchman avatar
imiltchman

something like this:

locals {
   trueValue = "${var.one}:${var.env}"
   falseValue ...
}
value = "${var.one != "" ? local.trueValue : local.falseValue}"
:--1:1
Mikael Fridh avatar
Mikael Fridh

I simply forgot 0.11 tricks altogether by now

:--1:1
Shawn Petersen avatar
Shawn Petersen

thanks guys, i used v0.12

2020-04-05

PePe avatar

is someone here working on this module ? https://github.com/bitflight-public/terraform-aws-app-mesh

bitflight-public/terraform-aws-app-mesh

Terraform module for creating the app mesh resources - bitflight-public/terraform-aws-app-mesh

Zachary Loeber avatar
Zachary Loeber

Deploying Kube Apps via the terraform provider, a quick blog I whipped up. More on the beginner side of things but with some interesting tools and a pretty comprehensive example terraform example module for a full deployment of an AKS cluster with app deployment: https://zacharyloeber.com/blog/2020/04/02/kubernetes-app-deployments-with-terraform/

Kubernetes App Deployments with Terraform attachment image

Kubernetes App Deployments with Terraform - Zachary Loeber’s Personal Site

cool-doge1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

My first introduction to finalizers were with installing rancher

Kubernetes App Deployments with Terraform attachment image

Kubernetes App Deployments with Terraform - Zachary Loeber’s Personal Site

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

OMG what a PIA to uninstall an app with dozens of finalizers

Zachary Loeber avatar
Zachary Loeber

I am so glad to get validation from someone like yourself on those stupid things

2020-04-04

Zach avatar

Anyone else using pycharm and the HCL plugin for terraform work? Is there a way to solve the “Can’t locate module locally” error when using remote sourced modules from github? The only workaround I’ve found is to declare a ‘wrapper’ module around the remote module that passes vars to it … which is pretty silly. So if I try to directly use a CloudPosse module or even one from the official terraform registry, it can’t do any code completion or validations.

Failing that, what are people using these days for ‘full featured’ terraform authoring?

Alex Siegman avatar
Alex Siegman

I know a lot of folks use IDEA, which is just a more generic PyCharm. I use VS Code, but as I move to TF12 it doesn’t handle the mix of 11 and 12 very well, and the experimental language server is not the best, but it’s been improving. I think the IntelliJ plugins are more featureful and supporting of 12. I could just have my stuff set up poorly, I haven’t spent much time on it.

Zach avatar

Yes I used VS Code originally but TF12 broke it, and they never really got a good plugin after that

Alex Siegman avatar
Alex Siegman

Yeah. Besides lack of time and just having everything already built in TF11 so path of least resistance, that’s probably the biggest thing holding me back is that it’s not well supported in my dev environment

Zach avatar

hmm so IDEA is just the java IDE from JetBrains, they make PyCharm too. I’d have to assume its the same plugin in fact.

2020-04-03

sumit parmar avatar
sumit parmar

hey guys how to update map values while using it

tags={ Department=”cabs” OS=”ms” Application=”cabs” Purpose =”app”} when i . use tags=var.tags just need to update few values such as OS = ms to OS=linux , purpose=app to purpose=db

loren avatar
loren
merge - Functions - Configuration Language - Terraform by HashiCorp

The merge function takes an arbitrary number of maps and returns a single map after merging the keys from each argument.

loren avatar
loren
tags = merge(
  var.tags,
  {
    Purpose = "baz"
  }
)
ikar avatar

Dear all, is there a way how to define locals for the file scope only? Not for the whole module?

loren avatar
loren

negatory. terraform loads from all .tf files in the directory. only option is to create separate modules

ikar avatar

okay, thanks!

chrism avatar
chrism

https://github.com/cloudposse/terraform-aws-eks-cluster/commit/162d71e2bd503d328e20e023e09564a58ecee139 removed kubeconfig_path which I was using to ensure the kubecfg was available to apply the haproxy ingress after setup. Looking at the changes to the outputs etc I can’t see a way to still get my grubby mitts on the cfg file.

Use `kubernetes` provider to apply Auth ConfigMap (#56) · cloudposse/[email protected]
  • Use kubernetes provider to apply Auth ConfigMap * Use kubernetes provider to apply Auth ConfigMap * Use kubernetes provider to apply Auth ConfigMap * Use kubernetes provider to a…
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

kubeconfig path was used only for the module’s internal use, to get kubeconfig from the cluster and apply the auth config map

Use `kubernetes` provider to apply Auth ConfigMap (#56) · cloudposse/[email protected]
  • Use kubernetes provider to apply Auth ConfigMap * Use kubernetes provider to apply Auth ConfigMap * Use kubernetes provider to apply Auth ConfigMap * Use kubernetes provider to a…
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you can use the same command aws eks update-kubeconfig as the prev version of the module was using

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

to get kubeconfig from the cluster

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

but outside of the module since it does not need it now, and using CLI commands in the module is not a good idea anyway - not portable across platforms, does not work on TF Cloud (you have to install kubectl and AWS CLI)

:--1:1
chrism avatar
chrism

Ta its all in geodesic so its been fine. I just tripped up leaving it on “rel=master” .

The .11 version had a kubectl file which applied the configmap auth; we were hooking on to that thread to bootstrap rancher.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, if you want to continue using the prev functionality, don’t pin to master

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

in fact, never pin to master, always pin to a release

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(fewer surprises since any module will evolve and introduce changes)

chrism avatar
chrism

yep yep; Totally down to me rushing to test migrating to the v.12 version. I’ll file it along side my decision to upgrade EKS to 1.15 on a friday and it murdering the cluster in the eyes of rancher (but running fine in EKS)

Tony avatar

hey guys, I created an ami from an ec2 instance I had configured and am trying to now deploy a new ec2 via terraform using that AMI while also joining a managed AD domain in AWS. For some reason when I use my AMI and not say the default win2019 amazon AMI to build this EC2 it fails to join my domain upon creation. Any thoughts? Do I need to prepare the machine in any way prior to creating an AMI out of it so that Terraform can do the domain joining?

Mike Martin avatar
Mike Martin

Need some help - we’re about to begin rolling out our modules to production and need to decide whether to break out the modules like CloudPosse has it (a module per github repo) OR just make a monolith repo that contains all of our modules (making it easier to develop against since all dependencies are in one repo). Likely using TF Cloud. I’m in the boat of break them out - my teammates are against me. Need help! lol Thoughts?

sheldonh avatar
sheldonh

Break them out!

Mike Martin avatar
Mike Martin

Agreed - now how do I defend it…

sheldonh avatar
sheldonh

Tell them it’s how terraform module versioning and integration works.

sheldonh avatar
sheldonh

You increase the reliability by seperating as you tag and release versions. This powers the terraform modules. You can’t git tag folders

sheldonh avatar
sheldonh

I’ll probably write up a blog post on this soon. I ended up building an Azure DevOps pipeline that I just copy and pasted into different terraform module projects and it automatically versions them using Gitversion.

sheldonh avatar
sheldonh

I showed this logic and my team is starting to do the same thing now not a monolith repo. The very nature of terraform modules and being able to version breaking changes and so on would get requires single repost per terraform module.

I’m sure you can get some work around but it’s definitely an anti pattern in my opinion to try and do a monolith repo for Terraform modules. And let me clarify I mean this for terraform cloud. I’m doing all of my pipelines and telephone cloud I don’t even know how it could even work reasonably with their terraform module registry if you are trying to use a monolith repo. It’s not really even a question of an uphill battle, in my opinion, it’s a question of if they’re trying to force an anti-pattern based on how they prefer to do it rather than how terraform cloud is structured to do it

sheldonh avatar
sheldonh

Lastly I’ll mention that you should create your get repos for the simple yaml file and a Terraform plan. Then you won’t have any problems setting up your and managing your repos. I have 90. I think the closest team has like 10 because they’re working on more traditional applications. small purpose repos just work better in my opinion with terraform as well as CICD tools

sheldonh avatar
sheldonh

Hope that helps

sheldonh avatar
sheldonh

Terraform docs back this up
you can publish a new module by specifying a properly formatted VCS repository (one module per repo, with an expected name and tag format; see below for details). The registry automatically detects the rest of the information it needs, including the module’s name and its available versions.

sheldonh avatar
sheldonh

Cheers

Mike Martin avatar
Mike Martin

Since we are using GitHub, they are arguing you could just pin it to a commit hash…

sheldonh avatar
sheldonh

Of course, ie tags

Mike Martin avatar
Mike Martin

I really appreciate the input! I’ll give you a summary in a bit of what happens…

sheldonh avatar
sheldonh

Ask them how they plan on ensuring

  • one version of a module in one folder is easily versioned against another that has pinned changes from a different time.
  • not download the entire repo for each module use… Which it would do by default
sheldonh avatar
sheldonh

To me it seems they need to get comfortable with the terraform recommendation and not try think more repos more problems. It makes it easier.

sheldonh avatar
sheldonh

And solve the management of them by in and hour or so write a yaml file that created all your repos and even adds GitHub hooks for slack and more. It’s actually not much extra effort at all then and your actually improving your source control system manangement too

sheldonh avatar
sheldonh
How Do I Decompose Monolithic Terraform Configurations? attachment image

Throwing everything into one unwieldy configuration can be troublesome. The solution: modules.

sheldonh avatar
sheldonh

And ig you choose to not use terraform registry then you have to start managing each jobs GitHub authorization instead of having that handled by the built by in registry oath connection. It’s really to me a lot more work to try and not use their recommendation IF you are using terraform cloud.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for some background, @Mike Martin are you practicing monorepos for other things you’re doing?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(i find the monorepo / polyrepo argument to be more of an engineering cultural holy war with everyone on both sides wounded and fighting on principle)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

also, one repo per module is not necessarily required either. we do that as large distributors of open source. there’s also a hybrid mode.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This talks about doing something like creating a terraform-networking repo and then sticking modules for that in there. Since this is for your organization it can be highly opinionated and grouped accordingly.

sheldonh avatar
sheldonh

100% agree. The folks I’m working with don’t do everything through code and so using one get repo for everything miscellaneous has been typically the approach. they get nervous about new repos cuz they’re not used to it. Making it easier to manage and create them consistently I think is a big big win that helps that move forward

Patrick M. Slattery avatar
Patrick M. Slattery

Hi, I’m trying to use a feature flag/toggle in Terraform with for_each previously I have used a count for the toggle but that does not work with for_each Does anyone know how I can do this?

resource "google_project_service" "compute_gcp_services" {
   for_each = {
    service_compute               = "[compute.googleapis.com](http://compute.googleapis.com)"                 # Compute Engine API
    service_oslogin               = "[oslogin.googleapis.com](http://oslogin.googleapis.com)"                 # Cloud OS Login API
  }
  project  = google_project.project.project_id
  # count    = "${var.compute_gcp_services_enable == "true" ? 1 : 0}"
  service  = each.value
  disable_dependent_services = true
  disable_on_destroy = false
  depends_on = [
    google_project_service.minimal_gcp_services
  ]
}
Gowiem avatar
Gowiem

@Patrick M. Slattery You need to create a condition and provide an empty value and a non-empty value. Here’s an example:

    dynamic "action" {
      for_each = var.require_deploy_approval ? [1] : []

      content {
        name      = "ProductionDeployApproval"
        category  = "Approval"
        owner     = "AWS"
        provider  = "Manual"
        version   = "1"
        run_order = 1
      }
    }

2020-04-02

sweetops avatar
sweetops

Does anyone have an example of using log_configuration with the terraform-aws-ecs-container-definition module? I’m trying to update from 0.12.0 to latest (0.23.0) and it looks like the logging configuration has changed. But I can’t find an example of how to implement it now.

androogle avatar
androogle
    "logConfiguration" = {
      "logDriver" = "awslogs",
      "options" = {
        "awslogs-region"        = var.region,
        "awslogs-group"         = "/${terraform.workspace}/service/${var.service_name}",
        "awslogs-stream-prefix" = var.service_name
      }
    }
androogle avatar
androogle

oh sorry, I didn’t see the terraform-aws-ecs-container-definition module part

androogle avatar
androogle

nm that

Gowiem avatar
Gowiem

@sweetops I’m using v0.23.0 and the following works for me:

locals {

  log_config = {
    logDriver = "awslogs"
    options = {
      awslogs-create-group  = true
      awslogs-group         = "${module.label.id}-logs",
      awslogs-region        = var.region,
      awslogs-stream-prefix = module.label.id
    }
    secretOptions = null
  }

  # ... 
}


\# ...

log_configuration            = local.log_config

You having trouble beyond that?

Brij S avatar
Brij S

Do I need to escape any characters in the following terraform?

  coredns_patch_cmd = "kubectl --kubeconfig=<(echo '${data.template_file.kubeconfig.rendered}') patch deployment coredns --namespace kube-system --type=json -p='[{"op": "remove", "path": "/spec/template/metadata/annotations", "value": "[eks.amazonaws.com/compute-type](http://eks.amazonaws.com/compute-type)"}]'"

I get the following error and I’m not sure why its asking for a new line

Error: Missing newline after argument

  on [variables.tf](http://variables.tf) line 101:
  (source code not available)

An argument definition must end with a newline.
Brij S avatar
Brij S

nevermind, got it

setheryops avatar
setheryops

Has anyone here ever had a lambda in one account that needed to be triggered by SNS in another account and used the aws_sns_topic_subscription resource?

setheryops avatar
setheryops

I keep getting an error on plan that the SNS account is not the owner of the lambda in the lambda account

curious deviant avatar
curious deviant

Maybe https://jimmythompson.co.uk/blog/sns-and-lambda/ is helpful describing the permissions policy that will need to be setup in a cross account scenario ?

Linking together Lambda and SNS across AWS accounts –Jimmy Thompson - Software Engineer

A guide on how to link together Lambda functions and SNS topics that belong in different AWS accounts.

setheryops avatar
setheryops

Thanks…ill check it out

:--1:1
Gabe avatar

Question regarding launch templates, block devices and instance types: Do you always use the same device_name for the root volume? Or change it based on instance type? For example, do you always use /dev/sda1 or /dev/xvda?

loren avatar
loren

changes based on instance type, and on OS

Zachary Loeber avatar
Zachary Loeber

What was the terraform provisioner used to collect script output into state again?

Zachary Loeber avatar
Zachary Loeber

thanks!

loren avatar
loren

github has fallen over, in case that’s not responding for you… https://www.githubstatus.com/

GitHub Status

Welcome to GitHub’s home for real-time and historical data on system performance.

Zachary Loeber avatar
Zachary Loeber

I just noticed the same, thanks for confirming I’m not the only one…

:--1:1
loren avatar
loren

that link is for an actual custom provider… there is also a module that abuses null resources terribly to get the outputs into state… but can’t find it right now cuz it’s on github

Zachary Loeber avatar
Zachary Loeber

Yeah, I know they are all dirty hacks and all. I’m not super proud to have to be using it

loren avatar
loren

or you can use the external provider, which is easy too, and pretty clean. only extra downside to me is that the stdout is not output which can make it hard to troubleshoot. i compensate by using the python logger to write to a file

Zachary Loeber avatar
Zachary Loeber

I just need one dumb airflow fernet key

loren avatar
loren

Zachary Loeber avatar
Zachary Loeber

its such a small python script you can oneline the thing

Zachary Loeber avatar
Zachary Loeber

I’ve been avoiding doing so for a while now but I’m seeing no good way to generate an airflow fernet key via tf and its such a short python script and all..

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

btw, for all you terraformers out there, I think you’ll dig what @mumoshu as been cooking up with #variant v2. He has created a declarative way to describe any “cli” tool using HCL that can be compiled down to a single binary. It means anyone on your team who can write HCL can help write the cli tool for your company (e.g. ./acme eks up ) Maybe you use #terragrunt today. Maybe you wish it did things that it doesn’t do today, but you’re not a go programmer. With #variant , that’s less the case because you can define arbitrary workflows like you would in a Makefile (only with a Makefile it doesn’t compile down to a binary for easy distribution) and it’s all in HCL so it easy for everyone to grok.

https://github.com/mumoshu/variant2

mumoshu/variant2

Turn your bash scripts into a modern, single-executable CLI app today - mumoshu/variant2

:--1:7
sheldonh avatar
sheldonh

What! That’s so cool. Going to check it out for sure. And I can figure out a way to shoehorn some power shell in there

mumoshu/variant2

Turn your bash scripts into a modern, single-executable CLI app today - mumoshu/variant2

mumoshu avatar
mumoshu
01:06:54 AM

@mumoshu has joined the channel

2020-04-01

discourse avatar
discourse
09:53:58 AM
Versioning and Deploying Secrets [Terraform]

FWIW on my small team (< 5 engineers) we use mozilla’s sops with great success.

For ages we relied purely on the PGP feature, but recently switched to KMS and it works great. We also use it in ansible using a custom plugin.

I believe our technique meets all your requirements. You can only store json or yaml data, but we get around that by wrapping blobs (pems, etc) in json/yaml and then shovelin…

:--1:2
Zachary Loeber avatar
Zachary Loeber

Curious how others are feeding output from terraform deployments into their pipeline as code

maarten avatar
maarten

You mean chaining it ? What I used was to store certain results in SSM, everything else can read from that.

:--1:1
Zachary Loeber avatar
Zachary Loeber

I know of SSM but haven’t used it, my assumption is that consul would be the non-cloud specific alternative to it right?

1
maarten avatar
maarten

correct

Zachary Loeber avatar
Zachary Loeber

Thanks @maarten, I appreciate the quick feedback

Aaron R avatar
Aaron R

Hi all. Hope you are all well during the covid outbreak. I’ve been attempting to use this great terrafrom module https://github.com/cloudposse/terraform-aws-jenkins

However my use case is slightly different to how this module has been setup. At the moment jenkins is running on elastic bean stalk with public facing load balancers. I want to these load balancers private facing only accessible by VPNing into the the specific VPC that is being run on

cloudposse/terraform-aws-jenkins

Terraform module to build Docker image with Jenkins, save it to an ECR repo, and deploy to Elastic Beanstalk running Docker stack - cloudposse/terraform-aws-jenkins

Aaron R avatar
Aaron R

i.e. place the load balancers on a private subnet per region and each instance on a private subnet per region

Aaron R avatar
Aaron R

wondering if anyone had done this before with this module and how they got it to work?

Aaron R avatar
Aaron R

I’m basing it off this

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

the complete example gets automatically provisioned on AWS when we run tests

Aaron R avatar
Aaron R

so I’m basing it off this

Aaron R avatar
Aaron R

however I presume I need to create additional private subnets to place the elastic load balancers

Aaron R avatar
Aaron R

as I also want the elastic load balancers to be private

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

loadbalancer_subnets = module.subnets.private_subnet_ids

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then place it in private

Aaron R avatar
Aaron R

can the load balancer and instance be placed in the same subnet though?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you can place it in any subnet you want

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

depends on your use-case

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you need to place LBs in separate subnets (for any reasons), you can create more private subnets, using any subnet strategy you want

Aaron R avatar
Aaron R

so i’ve used this complete main example to scale up a working jenkins instance on aws. (public facing load balancers). I then modified it to place the load balancers in the same private subnet as the instances. However when on the elasticbeanstalk console, attempting to set the application as internal it fails. (even tho everything is private). I connected a client vpn to this vpc and attempted to connect via the load balancer - couldn’t manage to do it. This led me here: https://forums.aws.amazon.com/thread.jspa?messageID=415184&#415184

Aaron R avatar
Aaron R

thanks for your help by the way

Aaron R avatar
Aaron R

I think it’s something to do with trying to set the elastic bean stalk application from public to internal

Aaron R avatar
Aaron R

as it’s set to public by default

Aaron R avatar
Aaron R

when then attempting to set it as an internal app (once terraformed up) it fails to do so

Aaron R avatar
Aaron R

therefore I cannot hit the load balancer when in a VPN attached to the VPC but I can directly hit the instance

Mikael Fridh avatar
Mikael Fridh

Anyone else with this pet peeve? When using a data source for example to get the current IP addresses of elastic network interfaces attached to an NLB… any resource which then make use of this data will always show a diff known after apply even though the data is actually the same every time … Any way around this except just not using it as data, but converting it a possibly a static variable after first creation is done? …

androogle avatar
androogle

out of curiosity, are you looking up an EIP you generated and assigned to the NLB or is the the managed IP’s that AWS assigns to the NLB?

androogle avatar
androogle

If the later, I wonder if the provider can distinguish the difference for that resource and assumes that the IP can change (similar to how it does in ALB’s) and just doesn’t keep the IP static

androogle avatar
androogle

since the resource itself is known to have changing IP’s out of band of the provision process

Mikael Fridh avatar
Mikael Fridh

I let AWS assign those IPs in this case.

The problem is the resource (aws_lb) doesn’t supply the data at all - so I have to go via the data source.

Mikael Fridh avatar
Mikael Fridh

But you are on to something.

Mikael Fridh avatar
Mikael Fridh

or hmm maybe not.

androogle avatar
androogle

yeah if you’re letting AWS assign them, there’s no gauntree they’ll remain that IP I don’t think

Mikael Fridh avatar
Mikael Fridh

The problem is the internal IPs…

Mikael Fridh avatar
Mikael Fridh

I’m not sure I can explicitly decide them?

Mikael Fridh avatar
Mikael Fridh

maybe I can create two elastic interfaces. let me check

Mikael Fridh avatar
Mikael Fridh

no, they can only be auto-assigned it seems.

Amazon creates those elastic network interfaces, and “owns” them, but they are created “in” my account

androogle avatar
androogle

yeah I think you’re right on NLB, no ability in API to specify anything beyond the subnet

Mikael Fridh avatar
Mikael Fridh

I can live with this… but anyone else is going to freak out when things show up as changes.

androogle avatar
androogle

I wonder if you can use lifecycle ignore_changes on that scope

Mikael Fridh avatar
Mikael Fridh

I wonder if maybe it’s mostly depends_on which makes this an issue…

Currently depends_on for data resources forces the read to always be deferred until apply time, meaning the results are always unknown during planning. depends_on for data resources is therefore useful only in some very specialized situations where that doesn't cause a problem, as discussed in the documentation.
Mikael Fridh avatar
Mikael Fridh

Yeah, it’s that.

And I use depends_on here for a good reason…

Because a data source cannot be “optional” I have to make sure the aws_lb resource gets created first. So data source is used second. Otherwise, the first terraform run won’t work fully. dangit.

Mikael Fridh avatar
Mikael Fridh
Data Resource Lifecycle Adjustments · Issue #17034 · hashicorp/terraform

Background Info Back in #6598 we introduced the idea of data sources, allowing us to model reading data from external sources as a first-class concept. This has generally been a successful addition…

androogle avatar
androogle

ah boo

androogle avatar
androogle

well, I normally wouldn’t suggest this but in this kind of scenario you could replace your eni/eip datasource with an external datasource. Write a botoscript or something to lookup the IP and handle it gracefully.

androogle avatar
androogle

because it seems more of an issue with the provider and resource than with the API itself

Mikael Fridh avatar
Mikael Fridh

Amazon needs to add these IPs in their API responses in order for Terraform to be able to fix it properly I guess.

Mikael Fridh avatar
Mikael Fridh

Because going via the network interface list is a bit of a kludge.

Mikael Fridh avatar
Mikael Fridh

I’ll see if I can do some coalesce trick with an external program like you said.

Mikael Fridh avatar
Mikael Fridh

I have the same pain with ecs task definitions… because they can be updated from terraform - or from CodeDeploy …

hope this data source issue on plan can be fixed soon .

androogle avatar
androogle

yeah, for that it looks like airship.tf has a nice approach with drift management via lambdas

1
Mikael Fridh avatar
Mikael Fridh

And my current way of running terraform doesn’t even support executing an external programs right off the bat … since I use AWS profiles.

But I tested it, and it works.

Mikael Fridh avatar
Mikael Fridh
output nlb_ingress_lb {
  value = merge(
    aws_lb.nlb_ingress,
    {
      proxy_ips = coalescelist(
        flatten([data.aws_network_interface.nlb_ifs.*.private_ips]),
        jsondecode(data.external.get_nlb_ips.result.private_ips),
      )
    }
  )
}
:--1:1
Mikael Fridh avatar
Mikael Fridh

airlift yeah this one had some nice things. “cron” jobs etc…

Mikael Fridh avatar
Mikael Fridh

\# lookup_type sets the type of lookup, either

\# * lambda - works during bootstrap and after bootstrap

\# * datasource - uses terraform datasources ( aws_ecs_service ) which won't work during bootstrap
variable "lookup_type" {
  default = "lambda"
}

clever indeed.

randomy avatar
randomy

CFN stacks with custom resources can help sometimes. I used one here to do ECS deployments that don’t interfere with Terraform. It’s just using ECS rolling updates, not CodeDeploy though, but we might extend it to support CodeDeploy. https://github.com/claranet/terraform-aws-fargate-service (it’s in early stages of being taken from a production system and turned into generic module)

claranet/terraform-aws-fargate-service

Terraform module for creating a Fargate service that can be updated with a Lambda function call - claranet/terraform-aws-fargate-service

randomy avatar
randomy

Also you can use 2 separate stacks and output data source values from one, use it in the other, if you want to “save” values. Not always ideal of course.

Mikael Fridh avatar
Mikael Fridh

Yep, considered the two stacks..

Mikael Fridh avatar
Mikael Fridh

I adapted my module to use the tricks from the [airlift.tf> modules now and have solved the task definition issues </i](http://airlift.tf) Now I’m building a similar lambda to solve the NLB lookup .

randomy avatar
randomy

does airlift require setting the lookup variable to one value during bootstrap, and then another afterwards?

randomy avatar
randomy

i mean airship (guess you did too?)

randomy avatar
randomy
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Joe Presley avatar
Joe Presley

Has anyone seen Terraform useful for a situation where you want about 500 non-technical users create their own prepackaged resources in the cloud? For example, everyone gets the same account setup with predefined VM instance? My instinct is that Terraform is not the best tool for this, but I’ve seen people start with the idea that Terraform could run the backend.

maarten avatar
maarten

If it’s non technical-users does it make sense for them to do the deployment themselves ? From Administrator POV Terraform would be perfect to create 500 predefined instances for example.

Joe Presley avatar
Joe Presley

We wouldn’t want them to do the deployment, but they would sign up for the package that would trigger a deployment. So think of a case where a member of the public signs up for a service that deploys a server for their use. The end user would be a data scientist. So it may be 200. It may be 500. It may be a 1000.

Joe Presley avatar
Joe Presley

I’m asking for a general pattern, but the couple of use cases I’ve seen is with an organization that wants to allow teams of data scientists from the public to use a common dataset with jupyterlab instances stood up, etc.

maarten avatar
maarten

Ah ok, in that case Terraform could for sure work out. In AWS Case I think Cloudformation would maybe be more straight forward as no tooling would be necessary.

Joe Presley avatar
Joe Presley

What’s the pattern you would use for Terraform? Would it be something like a webhook trigger on Jenkins to run a Terraform module that accepts different inputs? Is that a better option than say use a language SDK to great a gui?

Soren Martius avatar
Soren Martius

Scalr is moving in that direction

1
Joe Presley avatar
Joe Presley

I was on a demo call with Scalr a while back for a different use case. I understand they use Terraform on the backend, but I don’t see how Terraform can be used in a normal GitOps way for them. Do you know what pattern they use?

maarten avatar
maarten

@Joe Presley I’d leave Terraform out of it as it’s not an API and with many automated deployments error handling of Terraform will be an issue. Maybe you can look into using the provided IaaS for the Cloud Provider and it’s native deployment mechanisms like Cloudformation.

Joe Presley avatar
Joe Presley

That makes sense. Thanks for the feedback.

    keyboard_arrow_up