#airship (2019-03)

airship

Home of Airship ECS Modules ( https://github.com/blinkist/terraform-aws-airship-ecs-service / https://github.com/blinkist/terraform-aws-airship-ecs-cluster )

Archive: https://archive.sweetops.com/airship/

2019-03-01

maarten avatar
maarten

hey guys, I’ll try to spend some more time again, started working somewhere new so last weeks were just flying and I needed to be away from my laptop in the evenings.

3
3

2019-03-03

2019-03-12

jaustinpage avatar
jaustinpage

Hi Guys!

I was running through the airship demo, but, got stuck on some errors on the last step, creation of the fargate service itself. I checked github issues, didnt see anything similar, checked github pr’s, didnt see anything that looked the same.

in trying to debug, i bumped to the 0.8.8 tag, instead of 0.8.6 as listed in the docs

The errors are:

* module.fargate_service.module.iam.aws_iam_role_policy_attachment.ecs_tasks_execution_role: 1 error(s) occurred:
* module.fargate_service.module.iam.aws_iam_role_policy_attachment.ecs_tasks_execution_role: Resource 'aws_iam_role.ecs_task_execution_role' not found for variable 'aws_iam_role.ecs_task_execution_role.id'
* module.fargate_service.module.iam.output.ecs_task_execution_role_arn: Resource 'aws_iam_role.ecs_task_execution_role' does not have attribute 'arn' for variable 'aws_iam_role.ecs_task_execution_role.*.arn'
* module.fargate_service.module.iam.output.lambda_ecs_task_scheduler_role_arn: Resource 'aws_iam_role.lambda_ecs_task_scheduler' does not have attribute 'arn' for variable 'aws_iam_role.lambda_ecs_task_scheduler.*.arn'
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: 1 error(s) occurred:
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: Resource 'aws_iam_role.lambda_ecs_task_scheduler' not found for variable 'aws_iam_role.lambda_ecs_task_scheduler.name'

so it would seem like: <https://github.com/blinkist/terraform-aws-airship-ecs-service/blob/master/modules/iam/main.tf#L22> isnt getting created. This is weird, because I set create=true as well as fargate_enabled=true

jaustinpage avatar
jaustinpage

i was curious if there was something simple that i missed, or a pointer of where to look next

maarten avatar
maarten

Hi @jaustinpage do you have any other errors maybe regarding the creation of IAM resources ? Also, does your user for terraform has AdministratorAccess ?

jaustinpage avatar
jaustinpage

@maarten: no, no other errors. lemme make 100% certain i have admin…

jaustinpage avatar
jaustinpage

yep, the assumerole i am using has AdministratorAccess

maarten avatar
maarten

ok, so that’s all good

jaustinpage avatar
jaustinpage

im surprised that there isnt a resource creation error, it seems like it never got to the point where it could try to create resources

maarten avatar
maarten

can you run terraform -v in your root module ?

jaustinpage avatar
jaustinpage

one other potential red herring: when i tried to create the ecs cluster, i did get an error, but then manually applied https://github.com/blinkist/terraform-aws-airship-ecs-cluster/pull/10 to fix

jaustinpage avatar
jaustinpage
terraform -v
Terraform v0.11.11
+ provider.aws v2.0.0
+ provider.null v2.1.0
+ provider.template v2.1.0

Your version of Terraform is out of date! The latest version
is 0.11.12. You can update by downloading from www.terraform.io/downloads.html
jaustinpage avatar
jaustinpage

ill update tf

maarten avatar
maarten

ok, let me go through the same loop as you, and see if the new aws provider breaks something

jaustinpage avatar
jaustinpage
terraform -v
Terraform v0.11.13
+ provider.aws v2.0.0
+ provider.null v2.1.0
+ provider.template v2.1.0
jaustinpage avatar
jaustinpage

updated terraform makes no difference, lemme double check my providers…

jaustinpage avatar
jaustinpage

i was 1 version back on the aws provider, lets see if that fixes

jaustinpage avatar
jaustinpage

ahh, gotta manually patch for the owners attribute again…

maarten avatar
maarten

no wait

maarten avatar
maarten

use 0.5.1

maarten avatar
maarten
module "ecs" {
    source  = "blinkist/airship-ecs-cluster/aws"
    version = "0.5.1"
  
jaustinpage avatar
jaustinpage

ahhh! nice

jaustinpage avatar
jaustinpage

much appreciated!

maarten avatar
maarten

ok, now I’m running ecs service adding

jaustinpage avatar
jaustinpage

yep, no luck with provider aws v2.1.0

jaustinpage avatar
jaustinpage

@maarten: thank you so much for all the help, by the way, i really appreciate it

maarten avatar
maarten

ok, i get some other errors regarding policies

jaustinpage avatar
jaustinpage

a different set of errors?

maarten avatar
maarten

* module.fargate_service.module.iam.aws_iam_role_policy.lambda_lookup_policy: 1 error(s) occurred:

* aws_iam_role_policy.lambda_lookup_policy: Error putting IAM role policy terraform-20190312184616302000000001: MalformedPolicyDocument: The policy failed legacy parsing
	status code: 400, request id: 1e46ae1b-44f7-11e9-ac46-ddd75d8a23d7
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: 1 error(s) occurred:

* aws_iam_role_policy.lambda_ecs_task_scheduler_policy: Error putting IAM role policy terraform-20190312184616313900000002: MalformedPolicyDocument: The policy failed legacy parsing
	status code: 400, request id: 1e461169-44f7-11e9-8ea3-e7746315f9c9
jaustinpage avatar
jaustinpage

hmmm, that would mean that it made it further than it did for me i think…

jaustinpage avatar
jaustinpage

i dont know if this helps, but, i ran tf on 0.8.6, then 0.8.8, im not sure if that caused some of the resources to make it further or not…

maarten avatar
maarten

it should just work, so let me fix this, not sure what’s going on

jaustinpage avatar
jaustinpage

ok, let me know what I can do to assist

Release notes from terraform-aws-airship-ecs-cluster avatar
Release notes from terraform-aws-airship-ecs-cluster
06:59:10 PM

0.5.1: Added owners attribute to ecs_ami (#10) added owners attribute to ecs_ami owner: self -> amazon

blinkist/terraform-aws-airship-ecs-cluster

Terraform module which creates an ECS Cluster with integrated instance scaling and EFS mounting capability - blinkist/terraform-aws-airship-ecs-cluster

maarten avatar
maarten

@jaustinpage what do you have at ` region = `

jaustinpage avatar
jaustinpage

region = "${data.aws_region.current.name}"'

jaustinpage avatar
jaustinpage

Line 0: data "aws_region" "current" {}

maarten avatar
maarten

ok, that’s good

jaustinpage avatar
jaustinpage

the reason for that is im creating the same resources in 2 regions. technically i am calling the airship module from a homespun “region” module, so that i can make all the regions the same.

jaustinpage avatar
jaustinpage

and, i may have doctored the above logs to hide this fact…

maarten avatar
maarten

we all have our secrets

maarten avatar
maarten

so are you using 2 different aws providers atm ?

jaustinpage avatar
jaustinpage

3, but only 2 are having airship called at the moment

maarten avatar
maarten

and you pass the provider block to both modules

jaustinpage avatar
jaustinpage

the 3rd one is handling some aws credential stuff that is happening behind the scenes

jaustinpage avatar
jaustinpage

i pass the provider block to the homespun region module

jaustinpage avatar
jaustinpage

then use default inheritance from there

maarten avatar
maarten

ok, but for me to understand

maarten avatar
maarten

you have one root module, in which you create 3 different aws providers

maarten avatar
maarten

then you create 2 ecs clusters

jaustinpage avatar
jaustinpage

let me put together some excerpts

maarten avatar
maarten

you pass both ecs clusters the provider block like:

module “example” { source = “./example” providers = { aws = “aws.usw2” } }

maarten avatar
maarten

and pass the same provider block to the relevant ecs service modules

jaustinpage avatar
jaustinpage

and then the “modules/region” module looks just like a normal demo.

jaustinpage avatar
jaustinpage

inside the modules/region module, because it was handed a single aws provider, we let the default provider inheritance in terraform do the rest of the work

maarten avatar
maarten

that should cascade down

maarten avatar
maarten

But i’m not interpolating region names in my resources. Could you interpolate the region name or a short version of it in the name of the ecs service ?

maarten avatar
maarten

and test

jaustinpage avatar
jaustinpage

sure thing, i would have hoped a name conflict to produce a better error message (which would be a tf/provider bug), but ill give it a shot

jaustinpage avatar
jaustinpage

` name = “${var.name}-${terraform.workspace}-${data.aws_region.current.name}”`

maarten avatar
maarten

how long are the names of your workspaces ?

jaustinpage avatar
jaustinpage

‘lower’ and ‘production’

maarten avatar
maarten

maybe you can substr 0,3 them for now, the ecs cluster name will also be interpolated, and that can result in some issues unfortunately

maarten avatar
maarten

lets first see if we can get it to work with short names

jaustinpage avatar
jaustinpage

hmm, i am getting the 64char limit now

jaustinpage avatar
jaustinpage

ill find another way to unique them

maarten avatar
maarten

k

jaustinpage avatar
jaustinpage

i ran into some other bugs, itl take a sec for me to fix them.

maarten avatar
maarten

sure

jaustinpage avatar
jaustinpage

yeah, got some name collisions with the change in name above, its just gonna take a sec to remove the existing resource and re-add to make sure that all the resources get recreated

jaustinpage avatar
jaustinpage

ok, back to the initial state

jaustinpage avatar
jaustinpage

of errors that were listed above

maarten avatar
maarten

Is there a way for you to simplify the setup step-by-step to be able to start ruling out stuff.

jaustinpage avatar
jaustinpage

i take it you are having difficulty reproducing?

maarten avatar
maarten

the errors I had earlier were something else.

jaustinpage avatar
jaustinpage

yep, ill simplify.

maarten avatar
maarten

and was a human error on my side, by bluntly copying the ecs_service block without modifying the region

jaustinpage avatar
jaustinpage

messing with the name of both the ecs and fargate_service modules appears to have moved this on to a new error. thanks so much for your help @maarten, i think that was it

jaustinpage avatar
jaustinpage

must have been some sort of weird naming collisions just not throwing the errors i would have expected

maarten avatar
maarten

That happens sometimes, the real errors are hidden, takes some effort to figure those out.

2019-03-13

jamie avatar

Those are the worst.

jamie avatar

It can take ages to debug resources that have maps or lists that have one wrong or missing value in them.

jamie avatar

Because the terraform parser that handles it says that the error is in a count, or in another component.

2019-03-14

2019-03-18

Release notes from terraform-aws-airship-ecs-service avatar
Release notes from terraform-aws-airship-ecs-service
09:27:49 PM

0.9.0: Load balancer map removal (#47) Added an example of create = false Load balancer map removal

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2019-03-21

johnbeans avatar
johnbeans

when i run terraform with the airship module it seems to want to try to overwrite the task definition of my service… is there anything i can do to make it skip trying to do that?

  ~ module.fargate_service.module.ecs_service.aws_ecs_service.app_with_lb_awsvpc
      task_definition: "internal-tools-admin-api:15" => "internal-tools-admin-api:12"

2019-03-22

maarten avatar
maarten

John, can you (privately) paste me your ecs module definition, and disclose a bit more of your plan. Normally the module takes the current taskdefinition unless it’s changed. If it is changed it would create a new task definition with the image of the live definition.

1

2019-03-25

Release notes from terraform-aws-airship-ecs-service avatar
Release notes from terraform-aws-airship-ecs-service
08:37:45 AM

0.9.1 Moving getting_started into this repository as well

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

maarten avatar
maarten

hm

Release notes from terraform-aws-airship-ecs-service avatar
Release notes from terraform-aws-airship-ecs-service
09:23:05 AM

0.9.1.1 fixes for the nlb examples (#56)

Fix Terraform Registry issues with examples by maartenvanderhoef · Pull Request #56 · blinkist/terraform-aws-airship-ecs-service

Registry won&#39;t publish currently. Attribute redefined: The argument &quot;filter&quot; was already set at ./examples/with_nlb/main.tf:72,3-9. Each argument may be set only once. (in main.tf on …

Release notes from terraform-aws-airship-ecs-service avatar
Release notes from terraform-aws-airship-ecs-service
09:42:47 AM

0.9.1.2 variables fix (#57)

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Release notes from terraform-aws-airship-ecs-service avatar
Release notes from terraform-aws-airship-ecs-service
10:53:06 AM

0.9.2: Travis cli install + proper config build trigger (#55) Travis install + proper config

Install travis gem only within the condition branch=master

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

    keyboard_arrow_up