#airship (2019-03)

hey guys, I’ll try to spend some more time again, started working somewhere new so last weeks were just flying and I needed to be away from my laptop in the evenings.

Hi Guys!

I was running through the airship demo, but, got stuck on some errors on the last step, creation of the fargate service itself. I checked github issues, didnt see anything similar, checked github pr’s, didnt see anything that looked the same.

in trying to debug, i bumped to the 0.8.8 tag, instead of 0.8.6 as listed in the docs

The errors are:

* module.fargate_service.module.iam.aws_iam_role_policy_attachment.ecs_tasks_execution_role: 1 error(s) occurred:
* module.fargate_service.module.iam.aws_iam_role_policy_attachment.ecs_tasks_execution_role: Resource 'aws_iam_role.ecs_task_execution_role' not found for variable 'aws_iam_role.ecs_task_execution_role.id'
* module.fargate_service.module.iam.output.ecs_task_execution_role_arn: Resource 'aws_iam_role.ecs_task_execution_role' does not have attribute 'arn' for variable 'aws_iam_role.ecs_task_execution_role.*.arn'
* module.fargate_service.module.iam.output.lambda_ecs_task_scheduler_role_arn: Resource 'aws_iam_role.lambda_ecs_task_scheduler' does not have attribute 'arn' for variable 'aws_iam_role.lambda_ecs_task_scheduler.*.arn'
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: 1 error(s) occurred:
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: Resource 'aws_iam_role.lambda_ecs_task_scheduler' not found for variable 'aws_iam_role.lambda_ecs_task_scheduler.name'

so it would seem like: <https://github.com/blinkist/terraform-aws-airship-ecs-service/blob/master/modules/iam/main.tf#L22> isnt getting created. This is weird, because I set create=true as well as fargate_enabled=true

i was curious if there was something simple that i missed, or a pointer of where to look next

Hi @jaustinpage do you have any other errors maybe regarding the creation of IAM resources ? Also, does your user for terraform has AdministratorAccess ?

@maarten: no, no other errors. lemme make 100% certain i have admin…

yep, the assumerole i am using has AdministratorAccess

ok, so that’s all good

im surprised that there isnt a resource creation error, it seems like it never got to the point where it could try to create resources

can you run terraform -v in your root module ?

one other potential red herring: when i tried to create the ecs cluster, i did get an error, but then manually applied https://github.com/blinkist/terraform-aws-airship-ecs-cluster/pull/10 to fix

ill update tf

ok, let me go through the same loop as you, and see if the new aws provider breaks something

updated terraform makes no difference, lemme double check my providers…

i was 1 version back on the aws provider, lets see if that fixes

ahh, gotta manually patch for the owners attribute again…

no wait

use 0.5.1

ahhh! nice

much appreciated!

ok, now I’m running ecs service adding

yep, no luck with provider aws v2.1.0

@maarten: thank you so much for all the help, by the way, i really appreciate it

ok, i get some other errors regarding policies

a different set of errors?

hmmm, that would mean that it made it further than it did for me i think…

i dont know if this helps, but, i ran tf on 0.8.6, then 0.8.8, im not sure if that caused some of the resources to make it further or not…

it should just work, so let me fix this, not sure what’s going on

ok, let me know what I can do to assist

0.5.1: Added owners attribute to ecs_ami (#10) added owners attribute to ecs_ami owner: self -> amazon

@jaustinpage what do you have at ` region = `

region = "${data.aws_region.current.name}"'

Line 0: data "aws_region" "current" {}

ok, that’s good

the reason for that is im creating the same resources in 2 regions. technically i am calling the airship module from a homespun “region” module, so that i can make all the regions the same.

and, i may have doctored the above logs to hide this fact…

we all have our secrets

so are you using 2 different aws providers atm ?

3, but only 2 are having airship called at the moment

and you pass the provider block to both modules

the 3rd one is handling some aws credential stuff that is happening behind the scenes

i pass the provider block to the homespun region module

then use default inheritance from there

ok, but for me to understand

you have one root module, in which you create 3 different aws providers

then you create 2 ecs clusters

let me put together some excerpts

you pass both ecs clusters the provider block like:

module “example” { source = “./example” providers = { aws = “aws.usw2” } }

and pass the same provider block to the relevant ecs service modules

and then the “modules/region” module looks just like a normal demo.

inside the modules/region module, because it was handed a single aws provider, we let the default provider inheritance in terraform do the rest of the work

that should cascade down

But i’m not interpolating region names in my resources. Could you interpolate the region name or a short version of it in the name of the ecs service ?

and test

sure thing, i would have hoped a name conflict to produce a better error message (which would be a tf/provider bug), but ill give it a shot

` name = “${var.name}-${terraform.workspace}-${data.aws_region.current.name}”`

how long are the names of your workspaces ?

‘lower’ and ‘production’

maybe you can substr 0,3 them for now, the ecs cluster name will also be interpolated, and that can result in some issues unfortunately

lets first see if we can get it to work with short names

hmm, i am getting the 64char limit now

ill find another way to unique them

k

i ran into some other bugs, itl take a sec for me to fix them.

sure

yeah, got some name collisions with the change in name above, its just gonna take a sec to remove the existing resource and re-add to make sure that all the resources get recreated

ok, back to the initial state

of errors that were listed above

Is there a way for you to simplify the setup step-by-step to be able to start ruling out stuff.

i take it you are having difficulty reproducing?

the errors I had earlier were something else.

yep, ill simplify.

and was a human error on my side, by bluntly copying the ecs_service block without modifying the region

messing with the name of both the ecs and fargate_service modules appears to have moved this on to a new error. thanks so much for your help @maarten, i think that was it

must have been some sort of weird naming collisions just not throwing the errors i would have expected

That happens sometimes, the real errors are hidden, takes some effort to figure those out.

Those are the worst.

It can take ages to debug resources that have maps or lists that have one wrong or missing value in them.

Because the terraform parser that handles it says that the error is in a count, or in another component.

0.9.0: Load balancer map removal (#47) Added an example of create = false Load balancer map removal

when i run terraform with the airship module it seems to want to try to overwrite the task definition of my service… is there anything i can do to make it skip trying to do that?

  ~ module.fargate_service.module.ecs_service.aws_ecs_service.app_with_lb_awsvpc
      task_definition: "internal-tools-admin-api:15" => "internal-tools-admin-api:12"

John, can you (privately) paste me your ecs module definition, and disclose a bit more of your plan. Normally the module takes the current taskdefinition unless it’s changed. If it is changed it would create a new task definition with the image of the live definition.

0.9.1 Moving getting_started into this repository as well

hm

0.9.1.1 fixes for the nlb examples (#56)

0.9.1.2 variables fix (#57)

0.9.2: Travis cli install + proper config build trigger (#55) Travis install + proper config

Install travis gem only within the condition branch=master