#airship (2019-01)

@Bryan has joined the channel

Welcome @Bryan what brought you here ?

hey @maarten, back in the land of the living?

yap

happy NY

qq: can you remind me why the airship scheduled tasks use lambda rather than just pure cloudwatch events?

the lambda checks for the current task definition, so after updating the ECS service it will stick with what is currently running.

normal scheduled tasks are configured with a certain task definition, so after updating the service they would refer to a non-active taskdef

have a better idea ?

right! thanks. will ponder it

@maarten so we are belatedly looking at moving everything to newer airships with the lambda lookup/task definition selector; but this means creating O(N) of all of the resources involved - aws_iam_role.lambda_lookup, aws_iam_role_policy.lambda_ecs_task_scheduler_policy (even if no task scheduling is used), aws_iam_role_policy.lambda_lookup_policy, aws_lambda_function.lambda_lookup, as well as various data sources. ideally we would have O(1), e.g. just one lambda that can be used for multiple defs with the necessary parameters passed in. any thoughts on this?

Cool.. For the purpose of speeding up I assume ? Have you thought of creating seperate env’s per application, this really works well, also limits blast radius.

1. For the lambda lookup we could create a function_arn param which overrides and disables the current one.
2. lambda_ecs_task_scheduler_policy can be fixed ..

Partly speeding up but also just general cleanliness/KISS - if we have 100 services, then we don’t really want to have 100 lambdas “just” to help with their deployment, if we can do the same thing with 1

@puru has joined the channel

So checked out Traefik a bit, looks promising and could be cool for a reference architecture. It needs a K/V store for Cluster mode, and DynamoDB support is currently not merged yet in the library it uses https://github.com/abronan/valkeyrie so that’s that.

Without cluster mode every instance of traefik would hit the aws api’s for changes, which doesn’t scale

https://sweetops.slack.com/archives/CCT1E7JJY/p1546924976035700

Thanks Erik! That got time, it is too expensive to do anything with it.

@Maciek Strömich has joined the channel

I think if Airship has Traefik with something like this, I think it can be a more complete project, what do you guys think ? https://github.com/containous/traefik/issues/4363

0.8.7: Label support (#38) Label support Restructure input blocks

Thanks @maarten for this great project and outstanding docs.

Here is a small problem that I encountered with getting started guide. When I tried to apply everything at once (including ECS cluster and service) this errors occur:

* module.fargate_service.module.alb_handling.aws_route53_record.record_alias_a: aws_route53_record.record_alias_a: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_custom_listen_host_redirect_to_https: aws_lb_listener_rule.host_based_routing_custom_listen_host_redirect_to_https: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host: aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host_cognito_auth: aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host_cognito_auth: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_custom_listen_host: aws_lb_listener_rule.host_based_routing_custom_listen_host: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_route53_record.record: aws_route53_record.record: value of 'count' cannot be computed

But if you plan and apply in exact moments that you described in the guide it works fine.

Now I’m struggling with adding CI/CD for it. I’m trying to use https://github.com/cloudposse/terraform-aws-ecs-codepipeline but have no idea how to pass variables to buildfile.yml. Have you got any clues?

Hi @Lukasz German I’ve created a ticket for that, will have a look what can be done there. I’ll have a look on Monday to see if we can combine it with terraform-aws-ecs-codepipeline

0.8.8: Add more output variables (#42) More outputs

hey all! im fairly new to terraform so i really enjoyed reading about the airship modules for setting up ECS

my only question is… im currently wanting to testdrive how the airship modules creates the ECS cluster and service however it looks like in the getting started guide i am forced to own a domain as a requirement. is there any way i can get through the getting started guide WITHOUT having to setup a route53 host and SSL?

Hi @johnbeans! It’s a bit hard without as the ALB does the ‘routing’ based on the hostname

hey @maarten is this a requirement of AWS or airship’s module? please note im still somewhat new to using AWS with ALB and ECS so pardon my ignorance if its a silly question

The getting started was made to utilize the ALB. But for the ecs module it’s not necessary.

do you want to reach the docker service from the outside ?

yes i do

It’s really much simpler to get a (sub) domain and work from there, but to have something to work without. for use with ALB you do need the ALB https listener as it’s kind of mandatory. What you can do is to supply the ALB with a self signed certificate.

Within load_balancing_properties you can set route53_record_type to none, so you don’t need to provide a route53_zone_id.

You can add the dns name of the alb to the custom_listen_hosts, this make sure a rule is made so that traffic to the dns name of the alb is forwarded to the ecs task.

thanks maarten it sounds like ill be better off just getting a domain that i can sandbox with

so i got the getting started guide working :slightly_smiling_face: very smooth!

i noticed that if i want to change the desired_capacity after initially following the getting started guide, it does not actually result in expected behaviour. instead, it creates a new empty cluster called “fargate” and with the existing service “demo-web” it just creates a copy of the existing task definition and deploys it without actually changing the number of tasks… am i doing something wrong or using the incorrect property in the module? id ultimately like to increase the desired count from 1 to 2

Hi @johnbeans set the min capacity to two and you will have it with two. Desired count is ignored in the lifecycle of the ecs_service, meaning it can only be used for the bootstrap. The reason for this is that in an environment where scaling changes the desired_count, one does not want the terraform run to set the desired_count back to the initial value.

I just got airship up and running for first time. with the demo-web example. great!

ok now, uhm.. how do I do a rolling deploy? lol

looking at codedeploy now

I’m trying to figure out if it’d be possible to implement changing the deployment controller type to CODE_DEPLOY in the modules.

and so far, so great. But this resource https://www.terraform.io/docs/providers/aws/r/ecs_service.html is missing specification for “Service role for CodeDeploy” which is required when changing deployment type from rolling to blue/green in web gui.

if it works without i’ll submit a pr

https://github.com/blinkist/terraform-aws-airship-ecs-service/pull/44

hi @i5okie I’m traveling a bit, so I can’t check now, looks good so far, but travis failed, maybe you can run a fmt!

Why did you make a map with rolling and blue_green. It is more straight forward to keep using ECS and CODE_DEPLOY as arguments, just as a simple string variable. This way they have a direct relation to ECS documentation.