#terraform-0_12 (2019-06)

terraform

Discuss upgrading to terraform 0.12

Archive: https://archive.sweetops.com/terraform-0_12/

2019-06-02

shtrull avatar
shtrull

please someone tell me what i am doing worng

shtrull avatar
shtrull
09:14:12 AM
shtrull avatar
shtrull

and i am getting back

shtrull avatar
shtrull
09:14:31 AM
shtrull avatar
shtrull

seems like it founr the right subent id but ignores it

shtrull avatar
shtrull

when i remove the module.main.web_public_subnets[0].id and hard code the subnet it works

shtrull avatar
shtrull

and this happens also if i make module.main.web_public_subnets output to be only one var and then remove the [0]

shtrull avatar
shtrull

never mind i added .id but forgot that i took the value from an output and not the resource output

2019-06-04

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
0.12 support for terraform-docs by osterman · Pull Request #152 · cloudposse/build-harness

what Add a hack to support terraform 0.12 with terraform-docs Use awk to generate variables and outputs suitable for terraform-docs why terraform-docs does not yet support HCL2 references segm…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s dirty. it’s nasty. but if you need terraform-docs to work with 0.12, it works for us.

4
1
cytopia avatar
cytopia

Great work and thanks for that! As @antonbabenko has mentioned an issue on this PR and I am also facing this, I’ve tried to address it: https://github.com/cloudposse/build-harness/pull/155

#152 Ensure terraform-docs 0.12 multi-line default statement is parsed correctly by cytopia · Pull Request #155 · cloudposse/build-harness

terraform-docs 0.12 multi-line default parsing There currently seems to be an issue with multi-line default statement parsing as mentioned by @antonbabenko here: #152 (comment) This PR tries to add…

antonbabenko avatar
antonbabenko

Awesome, I will give it a try on Monday! Really hope it works :)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks @cytopia !

cytopia avatar
cytopia

@Erik Osterman (Cloud Posse) @antonbabenko one last issue I am currently facing is default = null. To what should this be transformed for Terraform < 0.12 docs generation?

antonbabenko avatar
antonbabenko

it is tricky… default = null in 0.12 is closest to default = "null" in 0.11. It is a hack, really, because there was no null value before 0.12.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

null == ""

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or maybe you can hack it like

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

null == “null

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

with markdown

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@antonbabenko

antonbabenko avatar
antonbabenko

Awesome! Thanks @Erik Osterman (Cloud Posse)!

2019-06-11

Nikola Velkovski avatar
Nikola Velkovski

A small appreciation to hands down one of the best modules out there

module "label" {
  source    = "git::<https://github.com/cloudposse/terraform-null-label.git?ref=0.12.0>"
  stage     = local.stage
  name      = local.name
  delimiter = "-"
  tags = {
    "kubernetes.io/cluster/${module.label.id}" = "shared"
  }
}
2
1
nyan_parrot1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrm… does this work? looks like the tags are referring to the label itself?

Nikola Velkovski avatar
Nikola Velkovski

Yes it does!

Nikola Velkovski avatar
Nikola Velkovski

Not sure if this will work in 0.11 though.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @Nikola Velkovski!! comments like these is what keeps us going!

1
jose.amengual avatar
jose.amengual

and the salary….and food too

jose.amengual avatar
jose.amengual

Nikola Velkovski avatar
Nikola Velkovski

And the coffee

1
Nikola Velkovski avatar
Nikola Velkovski

2019-06-13

maarten avatar
maarten

@Erik Osterman (Cloud Posse) what is the problem with the current terraform-terraform-label pr for 0.12.. I’d ike to push dynamodb to 0.12. Also current travis config fails for 0.12 still if im not mistaken.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thought I replied but don’t see the messages

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we need to convert this repo to use codefresh where all of our tooling is to support 0.12

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

andriy needed to take a couple days off, so project is on hold until next week

maarten avatar
maarten

All clear, thanks

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

I’ll finish it tomorrow

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) started working on it but had to take some days off

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We need to take a few steps with each module

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

First updating it to use Codefresh where we have all the 0.12 tooling in place

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I will try to rush this out though for you if it’s a blocker

niek avatar

Hi,

I just started to upgrade the gilab-runner-spot-module (for AWS) to TF 0.12. Would be great if you can help me with the following questions.

I used the tool terraform-docs to generate documentation of the variables, this tool seems not working (yet) for 0.12. Does anybody knows / uses an alternative approach?

Does anybody has a tip what is the best way to upgrade a data “aws_ami”. In my TF 0.11 module I had provided the option to inject the filter via a variable as follow:

variable "ami_filter" {
  type        = "list"

  default = [{
    name   = "name"
    values = ["amzn-ami-hvm-2018.03*-x86_64-ebs"]
  }]
}

data "aws_ami" "runner" {
  most_recent = "true"

  filter = "${var.ami_filter}"

  owners = ["${var.ami_owners}"]
}

Does anybody has a good example how I can keep the filter(s) configurable in tf 0.12?

niek avatar

Oops, already hab an answer on the filter question, map with a dynamic block

2019-06-14

2019-06-15

2019-06-18

davidvasandani avatar
davidvasandani

Is it possible to taint an entire module yet?

1

2019-06-19

rohit.verma avatar
rohit.verma

hi all, seems like terraform-null-label is not compatible with terraform 0.12. The same module being used in all modules make them kind of unusable too

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

to use it with TF 0.12, pin to the latest release

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

to use with TF 0.11, pin to 0.11.1

rohit.verma avatar
rohit.verma

then we have to change this in all the modules correct, i can send pr for the modules which we uses.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we have to update all the modules, we are working on it now

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

PRs are welcome, thanks

rohit.verma avatar
rohit.verma

or may be a better solution is https://dependabot.com/terraform/

Dependabot

Automated dependency updates for your Ruby, Python, JavaScript, PHP, .NET, Go, Elixir, Rust, Java and Elm.

rohit.verma avatar
rohit.verma

which should fix this across all repos for you

rohit.verma avatar
rohit.verma

have you tried that?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we are using dependabot

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

not sure if it’s possible to automatically convert everything to TF 0.12

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

works in simple cases

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

but usually you have to manually fix/update the code after the conversion

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

as far as I’ve seen it, the automatic conversion can’t work in some cases or breaks code in other cases

rohit.verma avatar
rohit.verma

i though you only need to update the null-label dependency across all projects.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if your modules are TF 0.11 and you point them to null-label 0.12, will it work?

rohit.verma avatar
rohit.verma

i didn’t tried null-label 0.12 yet. What I do know is you can make null-label 0.12 work with both 0.11 and 0.12.

rohit.verma avatar
rohit.verma

If its not working for any particular syntaxes, should raise a bug in terraform

rohit.verma avatar
rohit.verma

anyways you can specify that modules with tag version like 2.x only support 0.12

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

will not work with TF 0.11

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if your modules are TF 0.11

rohit.verma avatar
rohit.verma

i got it, you are saying automated null-label bump is not satisfactory since you want to address tagging along with it.

rohit.verma avatar
rohit.verma

anyways I will send pr for the modules which we are using.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

i mean when you update your modules to TF 0.12 and if the modules use the label, you will bump the label version in the new TF 0.12 code

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

automated label bump to TF 0.12 in the module that supports only 0.11will not work

rohit.verma avatar
rohit.verma

that i agree, but if we fixed the deps and ci runs auto terraform 0.12upgrade --yes we can easily check narrow down the errors. Few modules like https://github.com/cloudposse/terraform-aws-key-pair.git don’t need anything apart from label bump and upgrade.

cloudposse/terraform-aws-key-pair

Terraform Module to Automatically Generate SSH Key Pairs (Public/Private Keys) - cloudposse/terraform-aws-key-pair

rohit.verma avatar
rohit.verma

anyways there is a faster way which is clone all repos and do sed to replace version.

rohit.verma avatar
rohit.verma

so automation won’t be worth it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Dependabot is awesome but without automated tests, a sure fire way to absolutely break everything with regressions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We tried it on a handful of repos about a year ago and quickly things started breaking (aka GitHub issues started spiking)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So as part of our upgrading to 0.12 we are trying to add more tests with terratest and bats

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That way we catch regressions or at least have a way to test for them as they arise in the future

rohit.verma avatar
rohit.verma

yup ci should have better tests for automations

rohit.verma avatar
rohit.verma

But with 100+ modules we should work in that direction as well. I would love to help with something you required.

rohit.verma avatar
rohit.verma

one more thing I would recommend is to have version.tf file. The automatic upgrade does create this file and its easy to identify current terraform version for modules by just looking at the presence of file.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, we are using [versions.tf](http://versions.tf) file in all repos we convert

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@rohit.verma would massively appreciate some help

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I will type up some guidance

rohit.verma avatar
rohit.verma

today i sleep, tomorrow morning I would try an approach to bump all those modules which requires lesser tinkering with some scripts.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you have time to do a quick call this week that would be awesome too

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ok

rohit.verma avatar
rohit.verma

the part where i got lazy is fork and raise pr

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

You were working with some in house git bots, right?

rohit.verma avatar
rohit.verma

yes but they are mostly chatops related thing, and work within an org

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Did you finish rolling out test-infra used by istio?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

And k8s

rohit.verma avatar
rohit.verma

no but we did the vanilla prows one and hooked with tekton

rohit.verma avatar
rohit.verma

you can avoid tekton and can continue use cloudfresh in parallel

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Btw office hours today at 11:30 - in 1.5 hours

rohit.verma avatar
rohit.verma

i got severe cold, and its already past 10 pm for me

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh no! Definitely get some rest

rohit.verma avatar
rohit.verma

early morning person, i probably won’t be able to attend any of your office hours till I am in bangalore.

rohit.verma avatar
rohit.verma

Will catchup tomorrow morning with what I was able to do.

rohit.verma avatar
rohit.verma

Also you need some marking strategy whether the repo is done, may be presence of version.tf or a specific tag versioning method

rohit.verma avatar
rohit.verma

I would recommend version.tf

rohit.verma avatar
rohit.verma

If you agree can we add that to null label repo

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

sure

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-terraform-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-terraform-label

rohit.verma avatar
rohit.verma

Awesome

Callum Robertson avatar
Callum Robertson

That label module is the hotness

nyan_parrot1

2019-06-20

joshmyers avatar
joshmyers

wave hey folks - if you were starting a new project, thoughts on using 0.12 from the get go….? cc @antonbabenko @Erik Osterman (Cloud Posse) @Andriy Knysh (Cloud Posse)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

should be ok. Not sure about it 100% since we converted just a few modules, working on the rest now

joshmyers avatar
joshmyers

Nice, any “fun” with lists of maps/other workarounds we came up with for enabled/disabed etc?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(we did not get to those modules yet)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-null-label

Terraform Module to define a consistent naming convention by (namespace, stage, name, [attributes]) - cloudposse/terraform-null-label

joshmyers avatar
joshmyers

hah, I bet

joshmyers avatar
joshmyers

Cheers Andriy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

0.12 is definitely more enjoyable

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I would start fresh with that

2
marc avatar

are many of you guys running into:

Error: Failed to instantiate provider "aws" to obtain schema: Incompatible API version with plugin. Plugin version: 4, Client versions: [5]

I get this with 0.12.2.

the solutions I seem to find out there are to recompile tf, but wondered if people had a workaround?

loren avatar

What version of the aws provider are you using?

marc avatar
  version             = "~> 2"
marc avatar

huh, I guess

marc avatar
  version             = "~> 2.16"

works

loren avatar

Ahh, they must have updated the schema version?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
GitHub

GitHub is where people build software. More than 36 million people use GitHub to discover, fork, and contribute to over 100 million projects.

rohit.verma avatar
rohit.verma

why have terraform-terraform-label and terraform-null-label. Are we planning to deprecate anyone?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terraform-terraform-label was introduced as a simpler alternative to terraform-null-label which grew unwieldy in 0.11

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

with 0.12, terraform-null-label is once again sane, but we’ll continue supporting terraform-terraform-label

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and both have been updated to 0.12

2019-06-23

Callum Robertson avatar
Callum Robertson

Thanks @Erik Osterman (Cloud Posse)

    keyboard_arrow_up