#airship

Home of Airship ECS Modules ( https://github.com/blinkist/terraform-aws-airship-ecs-service / https://github.com/blinkist/terraform-aws-airship-ecs-cluster )

Archive: https://archive.sweetops.com/airship/

2019-10-03

@Bogdan Airship is not on TF0.12 yet. Maybe try running it under TF0.11

1
2

2019-10-02

Bogdan

Hi guys, I’m encountering the following at terraform init time:

Bogdan

I’m using the following: Terraform v0.12.7

  • provider.aws v2.29.0
  • provider.random v2.2.0
Bogdan

I’ve tried with version 0.9.9, 0.9.8 but not with 0.9.9.1 or 0.9.9.0

Bogdan

any hints appreciated (CC: @ @Mads Hvelplund @Maciek Strömich)

2019-09-23

Morten Hjorth Fæster

Hello - I believe there is a problem with scaling in services in https://github.com/blinkist/terraform-aws-airship-ecs-service ; scale out works as expected but scaling in fails with an empty error message when triggered. There is little more details here https://github.com/blinkist/terraform-aws-airship-ecs-service/issues/79 . I have created the pull request https://github.com/blinkist/terraform-aws-airship-ecs-service/pull/78 which sets a metric_lower_bound for scaling in events and a metric_upper_bound for scaling out. This seem to fix the problem in our case at least.

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Problems scaling in services · Issue #79 · blinkist/terraform-aws-airship-ecs-service

what We are consistently seeing that scale down events are not respected as Cloudwatch reports an error with an empty string as error message. We need scale downs to work to ensure we can gain true…

Fixing autoscaling for ECS services: The previous policy did scale up… by faester · Pull Request #78 · blinkist/terraform-aws-airship-ecs-service

… correctly but caused an exception with an empty error message, when scale downs were attempted.

1

Thanks Morten, I’ve merged it.

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Problems scaling in services · Issue #79 · blinkist/terraform-aws-airship-ecs-service

what We are consistently seeing that scale down events are not respected as Cloudwatch reports an error with an empty string as error message. We need scale downs to work to ensure we can gain true…

Fixing autoscaling for ECS services: The previous policy did scale up… by faester · Pull Request #78 · blinkist/terraform-aws-airship-ecs-service

… correctly but caused an exception with an empty error message, when scale downs were attempted.

1
Release notes from terraform-aws-airship-ecs-service
10:17:40 AM

<https Health port variable (#80)> health check port as variable for alb_handling

health port as variable

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Release notes from terraform-aws-airship-ecs-service
10:17:40 AM

<https Fixing autoscaling for ECS services: The previous policy did scale up…> … correctly but caused an exception with an empty error message, when scale downs were attempted. (<a class=”issue-link js-issue-link” data-error-text=”Failed to load issue title” data-id=”496833286” data-permission-text=”Issue title is private” data-url=”https://github.com/blinkist/terraform-aws-airship-ecs-service/issues/78” data-hovercard-type=”pull_request” data-hovercard-url=”/blinkist/terraform-aws-airship-ecs-service/pull/78/hovercard”…

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Fixing autoscaling for ECS services: The previous policy did scale up… by faester · Pull Request #78 · blinkist/terraform-aws-airship-ecs-service

… correctly but caused an exception with an empty error message, when scale downs were attempted.

2019-09-20

Hi @IvanM, I can take a look. What is your use-case, have both internal and external lb connected ?

IvanM

My use case is to have one Fargate service running and one NLB that will be listening on multiple ports - that means multiple target groups

2019-09-16

Release notes from terraform-aws-airship-ecs-cluster
08:23:50 AM

<https Removed the default use of detailed monitoring. (#17)> Reduces CloudWatch costs for metrics by 80%

blinkist/terraform-aws-airship-ecs-cluster

Terraform module which creates an ECS Cluster with integrated instance scaling and EFS mounting capability - blinkist/terraform-aws-airship-ecs-cluster

Rolf M. Harksen

@ Hi Maarten I created a pull request regarding to Scheduled tasks. I wonder if you would like to use it. I created it because we have a situation, where we have multiple clusters, with the same service in each cluster. Therfore we needed an unique name for the cloudwatch event rule: https://github.com/blinkist/terraform-aws-airship-ecs-service/pull/77

Scheduled task name by Rolf88 · Pull Request #77 · blinkist/terraform-aws-airship-ecs-service

In case you need the same service name in multiple clusters. In case event rule can’t be named the same as the service.

2019-09-08

IvanM

guys I would have a PR for the https://github.com/blinkist/terraform-aws-airship-ecs-service however, I can’t push to new branch Who should I contact in order to push it?

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

1
Maciek Strömich

fork, push to a branch in your repo, create pr against your repo

1

2019-09-05

Release notes from terraform-aws-airship-ecs-service
11:32:42 AM

<https Fixed a bug where scheduled task resources were created by accident (…> …https://github.com/blinkist/terraform-aws-airship-ecs-service/pull/76) Added example of headless service

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2
IvanM

Guys is there pls a way how to in blinkist/airship-ecs-service/aws enable to access ECR from Fargate service in public subnets?

I have a fargate service without ALB/NLB in public subnets. I see that it does not assign public IP - I guess that is the issue

IvanM

even if I assign ALB and have it deployed in public subnet I still can’t pull image from ECR

2019-09-04

Thang Man

hello everyone, I am finding a way to do CI/CD for the ECS services created by the Airship module. Ideally, I would like to make it work with Gitlab CI. Looking forward to hearing your suggestions/experience. Thanks!

IvanM

Hey guys I’m having an issue with ecs service

module "prod-ecs-cluster" {
  source  = "blinkist/airship-ecs-cluster/aws"
  version = "0.5.1"

  name                    = "production"
  create_roles            = false
  create_autoscalinggroup = false
  tags                    = "${var.prod-account-tags}"
}

module "prod_xxx_fargate" {
  source  = "blinkist/airship-ecs-service/aws"
  version = "0.9.8"

  name            = "heartbeat"
  region          = "eu-west-1"
  fargate_enabled = true

  ecs_cluster_id = "${data.aws_ssm_parameter.prod_ecs_cluster_id.value}"

  awsvpc_enabled            = true
  awsvpc_subnets            = "${split(",", data.aws_ssm_parameter.prod_private_subnets.value)}"
  awsvpc_security_group_ids = ["${aws_security_group.prod_heartbeat.id}"]
  load_balancing_type = "none"
  load_balancing_properties_route53_record_type = "NONE"
  container_cpu                = "512"
  container_memory             = "1024"
  container_memory_reservation = "1024"
  container_name               = "heartbeat"
  capacity_properties_desired_capacity     = "1"
  capacity_properties_desired_min_capacity = "1"
  capacity_properties_desired_max_capacity = "1"
  bootstrap_container_image                = "<http<i class="em em-//XXXX.dkr.ecr.eu-west-1.amazonaws.com/XXXX/heartbeat"></i>latest>"
  tags                                     = "${var.prod-tags}"
}

When applying I get this error

module.prod_heartbeat_fargate.aws_cloudwatch_event_target.scheduled_task: Creating...
  arn:                                                             "" => "arn<img src="/assets/images/custom_emojis/aws.png" class="em em-aws">ecs<i class="em em-eu-west-1"></i>XXXX:cluster/production"
  ecs_target.#:                                                    "" => "1"
  ecs_target.0.group:                                              "" => "default"
  ecs_target.0.launch_type:                                        "" => "FARGATE"
  ecs_target.0.network_configuration.#:                            "" => "1"
  ecs_target.0.network_configuration.0.assign_public_ip:           "" => "false"
  ecs_target.0.network_configuration.0.security_groups.#:          "" => "1"
  ecs_target.0.network_configuration.0.security_groups.2188918090: "" => "sg-01fc127021b5c06e2"
  ecs_target.0.network_configuration.0.subnets.#:                  "" => "3"
  ecs_target.0.network_configuration.0.subnets.1208029751:         "" => "subnet-009c1eef5052c165e"
  ecs_target.0.network_configuration.0.subnets.1240492469:         "" => "subnet-016c5d63421f2b356"
  ecs_target.0.network_configuration.0.subnets.3423134812:         "" => "subnet-0b7f0f0a1f22789ec"
  ecs_target.0.task_count:                                         "" => "1"
  ecs_target.0.task_definition_arn:                                "" => "arn<img src="/assets/images/custom_emojis/aws.png" class="em em-aws">ecs<i class="em em-eu-west-1"></i>XXXX<i class="em em-task-definition/production-heartbeat"></i>1"
  rule:                                                            "" => "heartbeat_scheduled_task"
  target_id:                                                       "" => "<computed>"
Releasing state lock. This may take a few moments...

Error: Error applying plan:

1 error occurred:
        * module.prod_heartbeat_fargate.aws_cloudwatch_event_target.scheduled_task: 1 error occurred:
        * aws_cloudwatch_event_target.scheduled_task: Creating CloudWatch Event Target failed: ValidationException: RoleArn is required for target arn<img src="/assets/images/custom_emojis/aws.png" class="em em-aws">ecs<i class="em em-eu-west-1"></i>XXXX:cluster/production.
        status code: 400, request id: ab07b8c1-342d-4b5b-a00d-604cfe95de5c


IvanM

as a solution I reverted back to blinkist/airship-ecs-service/aws v 0.9.4 and set

load_balancing_properties_lb_arn                = "aXXXX"
Mads Hvelplund

i can reproduce your bug report with this snippet. i’ll try to fix it.

Mads Hvelplund

@IvanM & @joshmyers: I think I found the problem. stuff related to scheduled tasks added in PR #73 was being created even for non-scheduled tasks. I can run your example now with my fix. I’ll go over it a final time and create a pull request with the fix.

joshmyers

Awesome :)

Mads Hvelplund

let’s not celebrate until you can see it works I’ve sent you a link to the PR

IvanM

cool!

joshmyers

@IvanM See my comment above

1
Mads Hvelplund

i’ll take a look now

2019-09-03

joshmyers
joshmyers
Scheduled task support by mhvelplund · Pull Request #73 · blinkist/terraform-aws-airship-ecs-service

I added support for deploying ECS scheduled tasks based on CloudWatch event rules, and I updated the website documentation.

joshmyers

Any ideas @Mads Hvelplund?

Mads Hvelplund

I’ll have a look tomorrow after work.

2019-08-30

joshmyers
Only trigger ALB module if ALB != ‘none’ by joshmyers · Pull Request #74 · blinkist/terraform-aws-airship-ecs-service

what If wanting to create an ECS task that does not have an ALB e.g. a scheduled task of some kind that doesn’t need an ALB, because we try and do a lookup on a data source, I am currently getting …

Release notes from terraform-aws-airship-ecs-service
01:47:43 PM

<https Only trigger ALB module if ALB != ‘none’ (#74)> If wanting to create an ECS task that does not have an ALB e.g. a scheduled task of some kind that doesn’t need an ALB, because we try and do a lookup on a data source, I am currently getting [1] If load_balancing_type is set to none, take it that we do not want to create any resources in the ALB module. [1]

  • module.ecs_service.module.alb_handling.data.aws_lb.main: 1 error occurred:
  • module.ecs_service.module.alb_handling.data.aws_lb.main: data.aws_lb.main: Search returned 0 results, please…
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2019-08-07

Release notes from terraform-aws-airship-ecs-service
12:17:38 PM
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2019-08-02

joshmyers
Allow pulling images from private Docker registry by joshmyers · Pull Request #71 · blinkist/terraform-aws-airship-ecs-service

what As per the documentation for Fargate using private docker registries[1] this commit introduces a new variable repository_credentials_secret_arn used to pass in the AWS Secrets Manager path to …

Release notes from terraform-aws-airship-ecs-service
12:37:43 PM
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2019-07-31

Release notes from terraform-aws-airship-ecs-service
09:17:39 AM

<https It seems like AWS maybe doing some extra validation here but without …> …(https://github.com/blinkist/terraform-aws-airship-ecs-service/pull/70) this change, I am getting the below error [1]. As per the…

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

It seems like AWS maybe doing some extra validation here but without by joshmyers · Pull Request #70 · blinkist/terraform-aws-airship-ecs-service

what this change, I am getting the below error [1]. As per the docs [2] when using an ECS condition like ArnEquals, we need to ensure the value is an actual valid ARN of the cluster, and not just t…

1

2019-07-30

joshmyers

Hey folks - long time listener first time caller

joshmyers

Just getting started with Airship, but am having an issue with one of the policies. Think I’ve been staring at a screen too long but any ideas why I’m getting “MalformedPolicyDocument: The policy failed legacy parsing” for:

joshmyers
03:25:11 PM
joshmyers

aws_iam_role_policy.lambda_ecs_task_scheduler_policy BTW

joshmyers

Maybe @ or @

@joshmyers Are you sure you are using eu-west-1 as region ?

joshmyers

@ yes

2019-07-29

Release notes from terraform-aws-airship-ecs-cluster
07:58:44 PM

<https Mixed cluster support (#16)> Switched to launch template instead of launch config.

Added support for mixed instance autoscaling groups.

Added examples

fix travis

blinkist/terraform-aws-airship-ecs-cluster

Terraform module which creates an ECS Cluster with integrated instance scaling and EFS mounting capability - blinkist/terraform-aws-airship-ecs-cluster

2019-07-08

@CumulusOps Would you be so kind and give me a sample of your module definition. I’d like to replicate it.

CumulusOps

i was running 0.12 I did not know that would be an issue until i join the slack channel

yes, porting to 0.12 still needs to happen, but also waiting for this one to allow for a proper rewrite of the deployment mechanics. https://github.com/terraform-providers/terraform-provider-aws/pull/8652

Support EXTERNAL deployment controller for ECS by kumarappan-arumugam · Pull Request #8652 · terraform-providers/terraform-provider-aws

Community Note Please vote on this pull request by adding a reaction to the original pull request comment to help the community and maintainers prioritize this request Please do not leave &quot;…

2019-07-06

CumulusOps

I got an error on airship I believe its the multiline description on the container_secrets in http://variables.tf

CumulusOps

if I remove the multiline description it resolves itself.

2019-06-27

2019-06-26

2019-06-25

Blaise Pabon

I’m traumatized by some early experiences of tfstate gone wild and so I have some naive questions about the modules, particularly @’s terraform-aws-airship-ecs-service … is the strategy to clone the repo and then create a terraform.tfvars file with the settings I want? Or should I apply each of the preparation steps in sequence, finally culminating in a magnificent terraform apply?

Hi Blaise.

So which terraform modules have you used in the past? Or is this the first one? @Blaise Pabon

The module is @ ‘s brainchild.

There’s a guide on how to use the module and all associates resources at http://airship.tf

Airship Modules

Flexible Terraform templates help setting up your Docker Orchestration platform, resources 100% supported by Amazon

Can you tell me what you are using it for? I can hopefully help you with it further!

Blaise Pabon

My hope is to host a number of apps (Jenkins, Jira, Artifactory, DockerEE, <http://webhook.site>) for the folks at the office.

Blaise Pabon

set up cognito for OAUTH

Blaise Pabon

Thank you @, I will go ahead and step through each of the preparation steps and build it up incrementally…. after all, it’s not on my dime!

Blaise Pabon

Hi @, well, I noticed that the example with_nlb was beautifully done and easy to understand, so I ran it. The plan was clean, but when I applied, I got one small error (see https://paste.opennms.eu/vefilidemi.php for the full trace)

Blaise Pabon
Error: Error applying plan:

1 error occurred:
	* module.nlb_service.module.ecs_service.aws_ecs_service.app_with_lb: 1 error occurred:
	* aws_ecs_service.app_with_lb: InvalidParameterException: The target group with targetGroupArn arn<img src="/assets/images/custom_emojis/aws.png" class="em em-aws">elasticloadbalancing<i class="em em-us-east-2"></i>481444911553:targetgroup/showben-cluster-showben-echo-ser/5ffab9ae16044a32 does not have an associated load balancer.
	status code: 400, request id: e6be16b8-39d1-4de6-89cb-1ab9a38ce734 "showben-echo-service"

Potentially the alb listener wasn’t created which causes that error.

Blaise Pabon

I may have screwed up the container name somehow by changing the original project name, but not the container. I will try it with the original settings.

I’m getting a similar error myself. I will see if I can address it.

Blaise Pabon

OK, my uninformed hunch is that I need to have bootstrap image in my ECR, but I imagine that would have appeared in the doc.

Blaise Pabon

heeyyyy, what about this:

output "lb_address" {
  value = "${aws_lb.this.dns_name}"
}
Blaise Pabon

but then:

Blaise Pabon

resource "aws_lb" "this" {
  name               = "${terraform.workspace}-service-nlb"
Blaise Pabon

should that be

dns_name  = "${terraform.worksp......
Blaise Pabon

@: I think I may have been useing the wrong profile. Sorry, it seems to be getting much farther.

Blaise Pabon

Nope, still getting the LB error, despite getting farther…


	* module.nlb_service.module.ecs_service.aws_ecs_service.app_with_lb: 1 error occurred:
	* aws_ecs_service.app_with_lb: InvalidParameterException: The target group with targetGroupArn arn<img src="/assets/images/custom_emojis/aws.png" class="em em-aws">elasticloadbalancing<i class="em em-us-east-2"></i>481444911553:targetgroup/default-cluster-default-echo-ser/adbaa6a3d9394315 does not have an associated load balancer.
	status code: 400, request id: afbc5f21-0f94-4a0e-ba14-6a071ea52539 "default-echo-service"
Blaise Pabon
blaisep/terraform-google-consul

A Terraform Module for how to run Consul on Google Cloud using Terraform and Packer - blaisep/terraform-google-consul

Blaise Pabon
blaisep/terraform-google-managed-instance-group

Modular Google Compute Engine managed instance group for Terraform. - blaisep/terraform-google-managed-instance-group

Blaise Pabon

so I’m sure a lot of this is just me getting used to AWS.

You’re on the right track. I’ll be on from 9am est and can discuss your implementation with you if you like. It’s late for me now so I’m heading to bed. :)

1
Blaise Pabon

Hi @, I think it may be:

 module.alb_shared_services_external.var.https_listeners: variable https_listeners in module alb_shared_services_external should be type list, got string
1
Blaise Pabon

yup! The variable for should be an empty list, not an empty string.

1
Blaise Pabon

https_listeners = []

1

2019-06-13

Release notes from terraform-aws-airship-ecs-service
04:57:36 PM
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

It features secrets handling. As well as provides documentation and new examples for how to use it.

6
2
Mads Hvelplund

2019-06-03

Mads Hvelplund

airship is unlikely to work with 0.12 without a major overhaul. if you use lots of modules, wait a bit before switching to 0.12

1

2019-05-30

Thang Man

it mights be caused by TF 0.12

Thang Man

I have tested fine with TF 0.11.14

2019-05-29

Thang Man

hello everyone

Thang Man

I am still testing airship project

Thang Man

I have deployed successfully a ecs cluster using the ecs-cluster module

Thang Man

but got this error when trying to run terraform apply using the ecs-service cluster

Thang Man

my TF version is 0.12

Thang Man
<http<i class="em em-//module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]\|module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]>"></i> Destroying... [id=stack-demo-web]
<http<i class="em em-//module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]\|module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]>"></i> Destruction complete after 0s
<http<i class="em em-//module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]\|module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]>"></i> Creating...
<http<i class="em em-//module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]\|module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]>"></i> Creation complete after 1s [id=stack-demo-web]
<http<i class="em em-//module.demo_web.module.ecs_service.aws_ecs_service.app[0]\|module.demo_web.module.ecs_service.aws_ecs_service.app[0]>"></i> Creating...

Error: InvalidParameterException: Invalid revision number. Number:
	status code: 400, request id: 7707d912-829d-11e9-b89e-554ff397ffec "demo-web"

  on ecs-service/modules/ecs_service/main.tf line 134, in resource "aws_ecs_service" "app":
 134: resource "aws_ecs_service" "app" {
Thang Man

the task definition has not been changed, but I don’t know why the module still destroy the current one (not changed), and re-create a new task definition with a new revision number

Thang Man

the task definition not changed but every I tried to re-run terraform apply, it show this:

Thang Man
# <http://module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]\|module.demo_web.module.ecs_task_definition.aws_ecs_task_definition.app[0]> must be replaced
-/+ resource "aws_ecs_task_definition" "app" {
      ~ arn                      = "arn<img src="/assets/images/custom_emojis/aws.png" class="em em-aws">ecs<i class="em em-ap-southeast-1"></i>513084766957<i class="em em-task-definition/stack-demo-web"></i>8" -> (known after apply)
      ~ container_definitions    = jsonencode(
          ~ [ # forces replacement
              ~ {
                  ~ command                = [
                      + null,
                    ]
                    cpu                    = 256
                  + entryPoint             = null
                  ~ environment            = [
                      + null,
                    ]
                    essential              = true
                  + healthCheck            = null
                    hostname               = "demo-web"
                    image                  = "nginx:stable"
                    logConfiguration       = {
                        logDriver = "awslogs"
                        options   = {
                            awslogs-group         = "stack/demo-web"
                            awslogs-region        = "ap-southeast-1"
                            awslogs-stream-prefix = "demo-web"
                        }
                    }
                    memory                 = 512
                  + memoryReservation      = null
                  ~ mountPoints            = [
                      + null,
                    ]
                    name                   = "nginx-fe"
                    portMappings           = [
                        {
                            containerPort = 80
                            hostPort      = 80
                            protocol      = "tcp"
                        },
                    ]
                    privileged             = false
                    readonlyRootFilesystem = false
                  - volumesFrom            = [] -> null
                  + workingDirectory       = null
                } # forces replacement,
            ]
        )
        family                   = "stack-demo-web"
      ~ id                       = "stack-demo-web" -> (known after apply)
        network_mode             = "bridge"
        requires_compatibilities = [
            "EC2",
        ]
      ~ revision                 = 8 -> (known after apply)
      - tags                     = {} -> null
        task_role_arn            = "arn<img src="/assets/images/custom_emojis/aws.png" class="em em-aws">iam:role/stack-demo-web-task-role"
    }
Thang Man

let me know if you need further information, thanks~

2019-05-24

Im on vacay at the moment. Will take a look when I’m back folks.

3
2

2019-05-23

Bogdan

I’m trying to use terraform-aws-airship-ecs-service from https://github.com/blinkist but if I’m using the following config:

name = "App"
region = "eu-central-1"
ecs_cluster_id = "dev123"
fargate_enabled = "true"
awsvpc_enabled = "true"
load_balancing_type = "none"
awsvpc_subnets = ["subnet-12,subnet-132,subnet-143"]
bootstrap_container_image = "<http://33.dkr.ecr.eu-central-1.amazonaws.com/app>"
container_cpu = 512
container_memory = 1024
container_port = 4000
awsvpc_security_group_ids = ["sg-4321"]
ssm_enabled = "true"
ssm_paths = ["/accounts/github/test"]
load_balancing_properties_route53_zone_id = "terst321"

I get the following errors:

Error: Error applying plan:

2 errors occurred:
        * module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: 1 error occurred:
        * aws_iam_role_policy.lambda_ecs_task_scheduler_policy: Error putting IAM role policy terraform-20190523154130657200000001: MalformedPolicyDocument: The policy failed legacy parsing
        status code: 400, request id: 3c746ca9-11e9-b313d05c3a7a


        * <http<i class="em em-//module.ecs_task_definition.aws_ecs_task_definition.app\|module.ecs_task_definition.aws_ecs_task_definition.app>"></i> 1 error occurred:
        * <http<i class="em em-//aws_ecs_task_definition.app\|aws_ecs_task_definition.app>"></i> ClientException: hostname is not supported on container when networkMode=awsvpc.
        status code: 400, request id: 47bd85d0-11e9-7b3bd4b40bd4
Blinkist

Blinkist has 30 repositories available. Follow their code on GitHub.

Bogdan

I opened an issue but not sure whether @ has had time to look at it. I’m very tempted to fork it and go my own way..

2019-05-22

jaustinpage

Hi guys, i recently ran into an issue where i really need to deploy 2 containers in 1 ecs task. I know that this is not a supported use case for airship. Is the challenge just in the lambda lookup?

jaustinpage

basically i am trying to determine level of effort to add multi-container support to airship

2019-05-20

2019-05-19

Thang Man

Hello everyone, I am a newbie to AWS ECS

Thang Man

I am looking at the code of Airship, and not sure/don’t know some parts

Thang Man
ECS Service | Airship Modules

Flexible Terraform templates help setting up your Docker Orchestration platform, resources 100% supported by Amazon

Thang Man

This is my TF root dir structure

Thang Man
Thang Man
 ecs_cluster_id = “${local.cluster_id}“

<– I got an error for this

Thang Man
* module.demo_web.local.ecs_cluster_name: local.ecs_cluster_name: 1<i class="em em-12"></i> unknown variable accessed: var.ecs_cluster_id in:
Thang Man

I fixed this

Thang Man

I am also confused with these

Thang Man
awsvpc_subnets
awsvpc_security_group_ids
Thang Man

they are only using for Fargate mode?

Mads Hvelplund

@Thang Man you need to create a cluster before adding services to it. there is a separate module for creating the ecs cluster: https://registry.terraform.io/modules/blinkist/airship-ecs-cluster/aws/0.5.1. if you follow the tutorial, it all gets sewt up: https://airship.tf/getting_started/

Start | Airship Modules

Flexible Terraform templates help setting up your Docker Orchestration platform, resources 100% supported by Amazon

Mads Hvelplund

i haven’t looked at the docs for the two awsvpc vars, but i’m guessing they control which subnet your airship services run in, and which ports are accessible on them.

Thang Man

thanks! let me take it a look again

2019-04-30

Mads Hvelplund

FYI: ecs-deploy doesn’t support secrets unless you run with the changes from https://github.com/silinternational/ecs-deploy/pull/179 .. or rather, it only supports Fargate containers that use secrets, without the fix

add executionRoleArn into NEW_DEF_JQ_FILTER by yu-orz · Pull Request #179 · silinternational/ecs-deploy

If &quot;executionRoleArn&quot; is specified for Task, ecs-deploy will result in an error and a filter will be added because it failed. An error occurred (ClientException) when calling the Register…

2019-04-29

@Mads Hvelplund using ecs-deploy as well, and the drift detection always made sure that ecs-deploy could do its work

2019-04-26

hi @Mads Hvelplund so as we cannot retrieve that from the datasource we need to do something else. You can store a hash of the combined secrets-names in a label, and compare the label for drift detection.

Mads Hvelplund

when your buildserver builds a new docker image, how do you deploy it? by running terraform, or using aws cli/api?

2019-04-25

Mads Hvelplund

Hi @. I got around to look at the drift detection you requested in the PR above. Looking at the code, I’m uncertain about how to proceed. There doesn’t seem to be any special precautions for normal environment variables. What am I missing?

Mads Hvelplund

Looking at the lookup lambda, i can see where you get the env vars, but it doesn’t look like javascript’s describetaskdefintion returns the secrets as part of the container …

Mads Hvelplund

I use ecs-deploy (https://github.com/silinternational/ecs-deploy) on my CI server to update the container image when the code changes. The script fetches the running taskdef and replace the image before uploading a new task def. However, if I run terraform to update something else, it detects a “change” and wants to downgrade to the last task defintion created with Terraform. I’m not sure if this is related to secrets, or ecs-deploy “scrubbing off” something that Airship uses to detect the newest image.

Any ideas?

silinternational/ecs-deploy

Simple shell script for initiating blue-green deployments on Amazon EC2 Container Service (ECS) - silinternational/ecs-deploy

2019-04-10

Mads Hvelplund
Minor bugfixing by mhvelplund · Pull Request #59 · blinkist/terraform-aws-airship-ecs-service

Added stack to custom error. Without it, error origin is lost. Changed Python string building to JS concatenation. String modulo string gives a NaN result every time. Respect the health check grace…

Looks good, very clean, thanks!

Minor bugfixing by mhvelplund · Pull Request #59 · blinkist/terraform-aws-airship-ecs-service

Added stack to custom error. Without it, error origin is lost. Changed Python string building to JS concatenation. String modulo string gives a NaN result every time. Respect the health check grace…

Release notes from terraform-aws-airship-ecs-service
09:12:59 PM

<https Minor bugfixing (#59)> Fixed bug in custom exception and error reporting

Added stack to custom error. Without it, error origin is lost.

Changed Python string building to JS concatenation. string modulo string gives a NaN result every time.

Cleanup.

Respect the health check grace period variable

Statement actions MUST be lists in Terraform.

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Mads Hvelplund

Is there a way to make environment vars for containers “valueFrom” instead of “value”?

Mads Hvelplund

2019-04-03

Mads Hvelplund
11:23:23 AM

Hi guys

I have a problem with service shown in the snippet. I deployed the service successfully earlier, but ran into problems after destroying my environment over night and reapplying it today.

After first terraform apply I get the following error:

Error: Error applying plan:

1 error(s) occurred:

* module.linkmobility.module.live_task_lookup.data.aws_lambda_invocation.lambda_lookup: data.aws_lambda_invocation.lambda_lookup: AccessDeniedException: The role defined for the function cannot be assumed by Lambda.
        status code: 403, request id: 280f95ac-e6a2-4738-935e-5b9c013e9ceb

It seems related to the timing of resource creation, but when I rerun the apply I get a new error every time I rerun:

Error: Error applying plan:

1 error(s) occurred:

* module.linkmobility.module.ecs_service.aws_ecs_service.app_with_lb: 1 error(s) occurred:

* aws_ecs_service.app_with_lb: ClientException: TaskDefinition is inactive
        status code: 400, request id: ab7ef1d7-5601-11e9-8828-d365d379c104 "linkmobility"

I think it’s because the old task defintion never gets deleted, just deactivated, and that trips up the new creation.

Mads Hvelplund

@ & @Maciek Strömich: Any suggestions?

Will try to get back to you this evening, have to work on work stuff now.

“* module.linkmobility.module.live_task_lookup.data.aws_lambda_invocation.lambda_lookup: data.aws_lambda_invocation.lambda_lookup: AccessDeniedException: The role defined for the function cannot be assumed by Lambda.”

interesting, let me get back to you later

Mads Hvelplund

While I’ve been banging on the module for the last few days, I’ve seen a couple of similar situations, where something is a dependency, but isn’t available quickly enough. Re-running fixes most of them, but I’ve hit an impasse here.

Mads Hvelplund

The thing is that deleting a taskdef doesn’t remove it, merely inactivates it. when you deactivate the last taskdef version, the entire taskdef becomes inactive, but it doesn’t disappear.

Mads Hvelplund

looking at the docs i think it never disappears

what you can always do for now

is to add an env var

I’ve ran into issues where I’ve created an ECS service with the module, after that deleted it again, but the old task definitions are more or less kept in AWS. This creates a weird irregularity

Mads Hvelplund

i tried to force a new taskdef with force_bootstrap_container_image but I’ll try the env var now

so for now

container_envvars  {
       FIX = "1"
  }
Mads Hvelplund

well, you could support it now with a lambda, like you handle the live task check

• module.linkmobility.module.live_task_lookup.data.aws_lambda_invocation.lambda_lookup: data.aws_lambda_invocation.lambda_lookup: AccessDeniedException: The role defined for the function cannot be assumed by Lambda.

this however doesn’t explain the task definition issue you have

Mads Hvelplund

Thanks!

container_envvars         = {
    stamp = 1554291327
  }

… worked like a charm.

Mads Hvelplund

like i wrote, the first error disappears when i rerun. i think its timing related.

Mads Hvelplund

like maybe it takes a second for a new policy to propagate, and terraform is already trying to use it

hm shouldn’t be!

Mads Hvelplund

btw, i also submitted a PR to fix a bug in the lambdas. there was some python cut and paste in the error reporting that made all errors have the message “NaN”

Mads Hvelplund

i assume it’s python, since it looked like python string building

Mads Hvelplund

anywho, thanks for the workaround

ah nice, thanks.

np, any time.

2019-03-25

Release notes from terraform-aws-airship-ecs-service
08:37:45 AM
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Release notes from terraform-aws-airship-ecs-service
09:23:05 AM
Fix Terraform Registry issues with examples by maartenvanderhoef · Pull Request #56 · blinkist/terraform-aws-airship-ecs-service

Registry won&#39;t publish currently. Attribute redefined: The argument &quot;filter&quot; was already set at ./examples/with_nlb/main.tf:72,3-9. Each argument may be set only once. (in http://main.tf on …

Release notes from terraform-aws-airship-ecs-service
09:42:47 AM
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

Release notes from terraform-aws-airship-ecs-service
10:53:06 AM

<https Travis cli install + proper config build trigger (#55)> Travis install + proper config

Install travis gem only within the condition branch=master

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2019-03-22

John, can you (privately) paste me your ecs module definition, and disclose a bit more of your plan. Normally the module takes the current taskdefinition unless it’s changed. If it is changed it would create a new task definition with the image of the live definition.

1

2019-03-21

johnbeans

when i run terraform with the airship module it seems to want to try to overwrite the task definition of my service… is there anything i can do to make it skip trying to do that?

  ~ module.fargate_service.module.ecs_service.aws_ecs_service.app_with_lb_awsvpc
      task_definition: "internal-tools-admin-api:15" => "internal-tools-admin-api:12"

2019-03-18

Release notes from terraform-aws-airship-ecs-service
09:27:49 PM

<https Load balancer map removal (#47)> Added an example of create = false Load balancer map removal

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2019-03-14

2019-03-13

Those are the worst.

It can take ages to debug resources that have maps or lists that have one wrong or missing value in them.

Because the terraform parser that handles it says that the error is in a count, or in another component.

2019-03-12

jaustinpage

Hi Guys!

I was running through the airship demo, but, got stuck on some errors on the last step, creation of the fargate service itself. I checked github issues, didnt see anything similar, checked github pr’s, didnt see anything that looked the same.

in trying to debug, i bumped to the 0.8.8 tag, instead of 0.8.6 as listed in the docs

The errors are:

* module.fargate_service.module.iam.aws_iam_role_policy_attachment.ecs_tasks_execution_role: 1 error(s) occurred:
* module.fargate_service.module.iam.aws_iam_role_policy_attachment.ecs_tasks_execution_role: Resource 'aws_iam_role.ecs_task_execution_role' not found for variable 'aws_iam_role.ecs_task_execution_role.id'
* module.fargate_service.module.iam.output.ecs_task_execution_role_arn: Resource 'aws_iam_role.ecs_task_execution_role' does not have attribute 'arn' for variable 'aws_iam_role.ecs_task_execution_role.*.arn'
* module.fargate_service.module.iam.output.lambda_ecs_task_scheduler_role_arn: Resource 'aws_iam_role.lambda_ecs_task_scheduler' does not have attribute 'arn' for variable 'aws_iam_role.lambda_ecs_task_scheduler.*.arn'
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: 1 error(s) occurred:
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: Resource 'aws_iam_role.lambda_ecs_task_scheduler' not found for variable 'aws_iam_role.lambda_ecs_task_scheduler.name'

so it would seem like: <https://github.com/blinkist/terraform-aws-airship-ecs-service/blob/master/modules/iam/main.tf#L22> isnt getting created. This is weird, because I set create=true as well as fargate_enabled=true

jaustinpage

i was curious if there was something simple that i missed, or a pointer of where to look next

Hi @jaustinpage do you have any other errors maybe regarding the creation of IAM resources ? Also, does your user for terraform has AdministratorAccess ?

jaustinpage

@: no, no other errors. lemme make 100% certain i have admin…

jaustinpage

yep, the assumerole i am using has AdministratorAccess

ok, so that’s all good

jaustinpage

im surprised that there isnt a resource creation error, it seems like it never got to the point where it could try to create resources

can you run terraform -v in your root module ?

jaustinpage

one other potential red herring: when i tried to create the ecs cluster, i did get an error, but then manually applied https://github.com/blinkist/terraform-aws-airship-ecs-cluster/pull/10 to fix

jaustinpage
terraform -v
Terraform v0.11.11
+ provider.aws v2.0.0
+ provider.null v2.1.0
+ provider.template v2.1.0

Your version of Terraform is out of date! The latest version
is 0.11.12. You can update by downloading from <http://www.terraform.io/downloads.html>
jaustinpage

ill update tf

ok, let me go through the same loop as you, and see if the new aws provider breaks something

jaustinpage
terraform -v
Terraform v0.11.13
+ provider.aws v2.0.0
+ provider.null v2.1.0
+ provider.template v2.1.0
jaustinpage

updated terraform makes no difference, lemme double check my providers…

jaustinpage

i was 1 version back on the aws provider, lets see if that fixes

jaustinpage

ahh, gotta manually patch for the owners attribute again…

no wait

use 0.5.1

module "ecs" {
    source  = "blinkist/airship-ecs-cluster/aws"
    version = "0.5.1"
  
jaustinpage

ahhh! nice

jaustinpage

much appreciated!

ok, now I’m running ecs service adding

jaustinpage

yep, no luck with provider aws v2.1.0

jaustinpage

@: thank you so much for all the help, by the way, i really appreciate it

ok, i get some other errors regarding policies

jaustinpage

a different set of errors?


* module.fargate_service.module.iam.aws_iam_role_policy.lambda_lookup_policy: 1 error(s) occurred:

* aws_iam_role_policy.lambda_lookup_policy: Error putting IAM role policy terraform-20190312184616302000000001: MalformedPolicyDocument: The policy failed legacy parsing
	status code: 400, request id: 1e46ae1b-44f7-11e9-ac46-ddd75d8a23d7
* module.fargate_service.module.iam.aws_iam_role_policy.lambda_ecs_task_scheduler_policy: 1 error(s) occurred:

* aws_iam_role_policy.lambda_ecs_task_scheduler_policy: Error putting IAM role policy terraform-20190312184616313900000002: MalformedPolicyDocument: The policy failed legacy parsing
	status code: 400, request id: 1e461169-44f7-11e9-8ea3-e7746315f9c9
jaustinpage

hmmm, that would mean that it made it further than it did for me i think…

jaustinpage

i dont know if this helps, but, i ran tf on 0.8.6, then 0.8.8, im not sure if that caused some of the resources to make it further or not…

it should just work, so let me fix this, not sure what’s going on

jaustinpage

ok, let me know what I can do to assist

Release notes from terraform-aws-airship-ecs-cluster
06:59:10 PM

<https Added owners attribute to ecs_ami (#10)> added owners attribute to ecs_ami owner: self -> amazon

blinkist/terraform-aws-airship-ecs-cluster

Terraform module which creates an ECS Cluster with integrated instance scaling and EFS mounting capability - blinkist/terraform-aws-airship-ecs-cluster

@jaustinpage what do you have at ` region = `

jaustinpage

region = "${data.aws_region.current.name}"'

jaustinpage

Line 0: data "aws_region" "current" {}

ok, that’s good

jaustinpage

the reason for that is im creating the same resources in 2 regions. technically i am calling the airship module from a homespun “region” module, so that i can make all the regions the same.

jaustinpage

and, i may have doctored the above logs to hide this fact…

we all have our secrets

so are you using 2 different aws providers atm ?

jaustinpage

3, but only 2 are having airship called at the moment

and you pass the provider block to both modules

jaustinpage

the 3rd one is handling some aws credential stuff that is happening behind the scenes

jaustinpage

i pass the provider block to the homespun region module

jaustinpage

then use default inheritance from there

ok, but for me to understand

you have one root module, in which you create 3 different aws providers

then you create 2 ecs clusters

jaustinpage

let me put together some excerpts

you pass both ecs clusters the provider block like:

module “example” { source = “./example” providers = { aws = “aws.usw2” } }

and pass the same provider block to the relevant ecs service modules

jaustinpage

and then the “modules/region” module looks just like a normal demo.

jaustinpage

inside the modules/region module, because it was handed a single aws provider, we let the default provider inheritance in terraform do the rest of the work

that should cascade down

But i’m not interpolating region names in my resources. Could you interpolate the region name or a short version of it in the name of the ecs service ?

and test

jaustinpage

sure thing, i would have hoped a name conflict to produce a better error message (which would be a tf/provider bug), but ill give it a shot

jaustinpage

name = "${var.name}-${terraform.workspace}-${data.aws_region.current.name}"

how long are the names of your workspaces ?

jaustinpage

‘lower’ and ‘production’

maybe you can substr 0,3 them for now, the ecs cluster name will also be interpolated, and that can result in some issues unfortunately

lets first see if we can get it to work with short names

jaustinpage

hmm, i am getting the 64char limit now

jaustinpage

ill find another way to unique them

jaustinpage

i ran into some other bugs, itl take a sec for me to fix them.

jaustinpage

yeah, got some name collisions with the change in name above, its just gonna take a sec to remove the existing resource and re-add to make sure that all the resources get recreated

jaustinpage

ok, back to the initial state

jaustinpage

of errors that were listed above

Is there a way for you to simplify the setup step-by-step to be able to start ruling out stuff.

jaustinpage

i take it you are having difficulty reproducing?

the errors I had earlier were something else.

jaustinpage

yep, ill simplify.

and was a human error on my side, by bluntly copying the ecs_service block without modifying the region

jaustinpage

messing with the name of both the ecs and fargate_service modules appears to have moved this on to a new error. thanks so much for your help @, i think that was it

jaustinpage

must have been some sort of weird naming collisions just not throwing the errors i would have expected

That happens sometimes, the real errors are hidden, takes some effort to figure those out.

2019-03-03

2019-03-01

hey guys, I’ll try to spend some more time again, started working somewhere new so last weeks were just flying and I needed to be away from my laptop in the evenings.

3
3

2019-02-28

@ i’ll take a look at this

I put in a PR for the ecr-cluster repo a few days ago that gave it a few more features

2019-02-27

Awesome project. I’d like to use NLB Setup. Is my understanding correct that this is a two phase deployment currently? If LB is included from beginning I keep getting LB errors.

Correct, for this to work I need to sit down a bit to remove the maps as input and replace them with normal string vars.

Is there any other workaround to get into a single step deployment or would you find some time soon? I might also crawl through the TF code.

2019-02-18

2019-02-16

2019-02-15

2019-02-14

ansgar

Hey there I just noticed today, that Airship at sometime dropped the support for multiple containers in a single task definition.

I’m quite new to Airship & ECS and am now thinking which way I should go for the service I need to deploy (nginx and php are the two images to run). I don’t want to write everything by hand or just copy most of the code and am looking for a “better way” Could anyone of you give me a hint how to solve this whilst still using the airship-ecs-service module? (Maybe I could update the taskdefinition after the initial creation to contain the two images and find a way for Airship to keep the two images in there on later updates?)

tl;dr How to airship with two containers in a single taskdefinition?

Hi Asgar, this can be possible, but only with a massive rewrite of the module. I removed sidecarring as it made the everything much more complex and because I rather see a nginx as a seperate service than something combined.

I slowly started to understand that especially for PHP applications, Nginx functions as the URI rewriting mechanism which functions as a 1:1 mechanism to the PHP app.

It will require a lot of effort to make these changes, but I’m happy to assist in getting it right if you have suggestions (PR).

ansgar

Thank you for explaining, Maarten. I think for now I’ll mainly copy the http://main.tf of the ecs-service module locally and replace the task-definition and lookup modules. Currently time is quite tight for the project I’m working on, so I’m unsure if I can make a good and clean PR in the next one to two months.

Erik Osterman
05:24:18 AM

2019-02-13

Using Jenkins or gitlab? What’s your setup and how do you test those images?

10:22:09 PM
1
1
1
Erik Osterman
Build software better, together

GitHub is where people build software. More than 31 million people use GitHub to discover, fork, and contribute to over 100 million projects.

1
1
1
Erik Osterman

So many projects called airship

1
1
1

Hi Chris,

Thanks for pointing me to the similar names. Airship is a common word so it was bound to happen. terraform-aws-airship-ecs-service is older and was already on Github ( the open source creativity hub , not marketplace ) under a different name. Another repository related to this project https://github.com/blinkist/airship-tf-albnlb was created on April 26h 2018.

The Terraform AWS Airship Github repositories are directly connected to a Terraform Registry and are used as building blocks for other parties to use as infrastructure. The Terraform AWS Airship ECS service itself has been provisioned https://registry.terraform.io/modules/blinkist/airship-ecs-service/aws/0.8.8 over 10.000 times already. A name change would not be possible without breaking existing infrastructure of other parties.

My projects have Terraform in their names, are written in Terraform and are specifically made around AWS ECS infrastructure and have nothing to do with Openstack nor Kubernetes. I’m sure that developers who use Github can spot the difference.

I hope you understand my point of view, and the difficulties regarding renaming a project like this, I am willing to put a disclaimer of some sorts in the README of my projects. I’m happy to help out in that way.

Best regards,

Maarten van der Hoef

blinkist/airship-tf-albnlb

Airship Terraform module for a generic ALB / NLB. Contribute to blinkist/airship-tf-albnlb development by creating an account on GitHub.

1
1
1
Erik Osterman


My projects have Terraform in their names, are written in Terraform and are specifically made around AWS ECS infrastructure and have nothing to do with Openstack nor Kubernetes. I’m sure that developers who use Github can spot the difference.

1
1
1
Erik Osterman

great point.

1
1
1
Erik Osterman

well written!

1
1
1
Erik Osterman


I am willing to put a disclaimer of some sorts in the README of my projects. I’m happy to help out in that way.

1
1
1
Erik Osterman

this should be all that’s needed

1
1
1

Maybe they can buy the .dev already

1
1
1

2019-02-12

Maciek Strömich

@ depends on the platform. For Elastic Beanstalk we’re treating them as dependency only containers with code being mounted inside the container upon execution. For ECS we’re adding everything inside the container which is version tagged and this version tag is being used upon cloudformation stack deployment.

Do you have a build pipeline for the images?

Maciek Strömich

yes

2019-02-11

Slightly off topic, how you you build and manage your images, ideally in a pipeline ?

2019-02-07

@tamsky not in the current setup. Would take a lot of effort to move to get this multi-container definition working.

1

2019-02-06

@hlarsen The Airship ECS Cluster module is purely on-demand for now. I suggest you take a look at something Jamie started : https://github.com/cloudposse/terraform-aws-ecs-spot-fleet/tree/init

cloudposse/terraform-aws-ecs-spot-fleet

[wip] Terraform module to create a diversified spot fleet for ECS clusters - cloudposse/terraform-aws-ecs-spot-fleet

i5okie

@ I left terraforming for later. Trying to get myself more familiarized with ECS/forgate with cloud formation for now. going to write up terraform config from scratch after I think.

ah ok, why cloud formation ?

aknysh

because TF is for kids and all real men start with CF?

I forgot

reason I don’t do AWS certification, is all the CF

aknysh

associate or professional?

Neither, I never really looked into it tbh, but for sure there will be CF

have you ?

aknysh

yea, I had associate architect, but there was no CF whatsoever (just a bunch of multiple choice q’s)

ah ok, in that case ..

aknysh

you should do it

yeah, will help freelancing a bit maybe

aknysh

professional is much more complicated. Has anyone you know got it?

@Maciek Strömich what do you have again ?

Maciek Strömich

wut?

aws certification

Maciek Strömich

saa, dop and sap

Maciek Strömich

why?

Maciek Strömich

and why are you asking on #airship? (-:

is that associate or professional ?

we can chat everywhere about anything

and was there cloudformation involved ?

Maciek Strömich

sys admin associate, devops professional and solutions architect professional

aknysh

oh wow, you are the man @Maciek Strömich

aknysh

any notes on solutions architect professional?

Maciek Strömich

@aknysh thanks but I don’t care about the certs. company that I work for is in APN and requires some number of certs

Maciek Strömich

@aknysh new or old one?

aknysh

what’s the diff?

Maciek Strömich

huge

aknysh

you got old?

1
Maciek Strömich

the old one was much simpler

Maciek Strömich

yeah

Maciek Strömich

pushed in nov to be able to have the old one

aknysh

ok, i guess i’ll try to get the new one

Maciek Strömich

the new one is built on the newer AWS offering

Maciek Strömich

some code* questions

Maciek Strömich

autoscaling, containers

Maciek Strömich

and as always building solutions in a highly available, fault tolerant and with cost in mind manner

Maciek Strömich

the old one was about ec2, s3, opsworks, storage gateways, some hybrid solutions

Maciek Strömich

and not that much technical

aknysh

you talking about the old and new for the same thing, solution architect professional ?

Maciek Strömich

just theory of cloud concepts

Maciek Strömich

the new one on the other hand (at least based on the practice exams in cloud academy) is much more technical in general

Maciek Strömich

including some “simple” questions about API usage

Maciek Strömich

it gets more in depth on the technical path

Maciek Strömich

if your aws-fu is okish then with a little bit of effort certs are easily passable

aknysh

yea, thanks

Maciek Strömich

@aknysh yeah basically

Maciek Strömich

@aknysh i spent a week of going through the faqs and docs on the topics covered by the cert

Maciek Strömich

and I’m not as fluent in AWS as @

Maciek Strömich

or maybe instead of fluency it’s more about exposure to aws services in a different manner

You have no wet dreams on AWS ?

Maciek Strömich

only if you’re setting up my airship cluster

Maciek Strömich

@aknysh BTW IIRC since beginning of Feb you can only apply for a updated certificate.

anyway, I just got myself a new contract without certification so ..

Maciek Strömich

and since Oct last year you can skip associate exam and pass professional

1
aknysh

that’s good

1
aknysh

mine already expired, so I had to do it again

1
Maciek Strömich

@ kawabunga btw, we should go for beer and kickers after me moving back to Berlin

let’s do !

tamsky

I was hoping to use a sidekick container in ECS and use airship. https://github.com/blinkist/terraform-aws-airship-ecs-service/issues/16#issuecomment-423129398 makes me think this can’t be done:

[ECS TASK [ Container 1 : load data from s3, unzip to /data, sleep or exit ] <--volumes-from mount [ Container 2 : Application server, reads from /data ]]

and the module has eliminated support for multi-container definitions. Is this true?

tamsky

Or is there another way I can use airship (which I like), to implement a sidekick pattern?

2019-02-05

hlarsen

i have some time today to finally look at the code, so i’ll see if that will work for us - thanks!

Thanks @joshmyers for helping out, for everyone else, I’m back from vacay

@i5okie how are you faring now ?

To everyone, I’m thinking of removing the current map input we have. I’ve always used them for aesthetics, but so far this causes also the famous cannot compute count problems. @ You were also a big fan of maps, any last verdict ?

@i5okie and @joshmyers the PR’s that I submitted for the airship ecs cluster module and the ecs service module removed all of the count compute errors from it, and gave it the ability to do spot. It is however a breaking change, as there were like 4 variables that I had to move out of the map blocks and back into stand alone variables.

There’s an examples directory that I have added to the branch that demonstrates adding the variables, and shows that it works without count errors.

2019-02-04

hlarsen

hi all, i’m about to do some experimenting with the end goal being an ECS cluster backed by an ec2 fleet made up of on-demand and spot instances. would the airship modules be a good fit for that, or should i just go with the base terraform resources?

@hlarsen @ I have addedthis pull request that adds those features https://github.com/blinkist/terraform-aws-airship-ecs-cluster/pull/9

This PR includes three features by Jamie-BitFlight · Pull Request #9 · blinkist/terraform-aws-airship-ecs-cluster

launch_template resource is used instead of a launch_configuration, this allows for a more granular tagging. i.e. The instances and the instance ebs volumes get tags. The ASG uses a mixed_instances…

@jonboulle care to take a look ?

joshmyers

airship has shaved a load of edges so you don’t have to. Not sure if it works with on demand and spot instances OOTB (probably not) but a good bet would be to fork it and use as a base

2019-02-01

i5okie

good point. also the minimum and maximum deployment/health values are redundant when its on CODE_DEPOLOY. but it didn’t complain when i left them set. I’ll get rid of the map later today

i5okie

updated PR. travis green

i5okie
07:56:54 PM
1
i5okie

i keep getting this error “module.core.module.container_definition.null_resource.envvars_as_list_of_maps: null_resource.envvars_as_list_of_maps: value of ‘count’ cannot be computed”

i5okie

it doesn’t like more than one envvar in this “container_envvars {..”

i5okie

i’ve tried without comas, with comas, i dont get it.

i5okie

its interpolation.

i5okie

yeah found the issue. completely unrelated to any of these modules.

2019-01-31

i5okie

I just got airship up and running for first time. with the demo-web example. great!

i5okie

ok now, uhm.. how do I do a rolling deploy? lol

i5okie

looking at codedeploy now

i5okie

I’m trying to figure out if it’d be possible to implement changing the deployment controller type to CODE_DEPLOY in the modules.

i5okie

and so far, so great. But this resource https://www.terraform.io/docs/providers/aws/r/ecs_service.html is missing specification for “Service role for CodeDeploy” which is required when changing deployment type from rolling to blue/green in web gui.

i5okie

if it works without i’ll submit a pr

i5okie
Add Deployment Type by i5okie · Pull Request #44 · blinkist/terraform-aws-airship-ecs-service

Added optional variable deployment_type rolling (default) – maps to deployment controller type &quot;ECS&quot; default functionality of the modules blue_green – maps to deployment controller type…

hi @i5okie I’m traveling a bit, so I can’t check now, looks good so far, but travis failed, maybe you can run a fmt!

Why did you make a map with rolling and blue_green. It is more straight forward to keep using ECS and CODE_DEPLOY as arguments, just as a simple string variable. This way they have a direct relation to ECS documentation.

2019-01-28

Hi @johnbeans set the min capacity to two and you will have it with two. Desired count is ignored in the lifecycle of the ecs_service, meaning it can only be used for the bootstrap. The reason for this is that in an environment where scaling changes the desired_count, one does not want the terraform run to set the desired_count back to the initial value.

2019-01-25

johnbeans

so i got the getting started guide working very smooth!

i noticed that if i want to change the desired_capacity after initially following the getting started guide, it does not actually result in expected behaviour. instead, it creates a new empty cluster called “fargate” and with the existing service “demo-web” it just creates a copy of the existing task definition and deploys it without actually changing the number of tasks… am i doing something wrong or using the incorrect property in the module? id ultimately like to increase the desired count from 1 to 2

2019-01-24

Hi @johnbeans! It’s a bit hard without as the ALB does the ‘routing’ based on the hostname

johnbeans

hey @ is this a requirement of AWS or airship’s module? please note im still somewhat new to using AWS with ALB and ECS so pardon my ignorance if its a silly question

The getting started was made to utilize the ALB. But for the ecs module it’s not necessary.

do you want to reach the docker service from the outside ?

johnbeans

yes i do

It’s really much simpler to get a (sub) domain and work from there, but to have something to work without. for use with ALB you do need the ALB https listener as it’s kind of mandatory. What you can do is to supply the ALB with a self signed certificate.

Within load_balancing_properties you can set route53_record_type to none, so you don’t need to provide a route53_zone_id.

You can add the dns name of the alb to the custom_listen_hosts, this make sure a rule is made so that traffic to the dns name of the alb is forwarded to the ecs task.

johnbeans

thanks maarten it sounds like ill be better off just getting a domain that i can sandbox with

2019-01-23

johnbeans

hey all! im fairly new to terraform so i really enjoyed reading about the airship modules for setting up ECS

my only question is… im currently wanting to testdrive how the airship modules creates the ECS cluster and service however it looks like in the getting started guide i am forced to own a domain as a requirement. is there any way i can get through the getting started guide WITHOUT having to setup a route53 host and SSL?

2019-01-21

Release notes from terraform-aws-airship-ecs-service
08:43:11 PM

<https Add more output variables (#42)> More outputs

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2019-01-12

Hi @Lukasz German I’ve created a ticket for that, will have a look what can be done there. I’ll have a look on Monday to see if we can combine it with terraform-aws-ecs-codepipeline

1

2019-01-11

Lukasz German

Thanks @ for this great project and outstanding docs.

Here is a small problem that I encountered with getting started guide. When I tried to apply everything at once (including ECS cluster and service) this errors occur:

* module.fargate_service.module.alb_handling.aws_route53_record.record_alias_a: aws_route53_record.record_alias_a: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_custom_listen_host_redirect_to_https: aws_lb_listener_rule.host_based_routing_custom_listen_host_redirect_to_https: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host: aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host_cognito_auth: aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host_cognito_auth: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_lb_listener_rule.host_based_routing_custom_listen_host: aws_lb_listener_rule.host_based_routing_custom_listen_host: value of 'count' cannot be computed
* module.fargate_service.module.alb_handling.aws_route53_record.record: aws_route53_record.record: value of 'count' cannot be computed

But if you plan and apply in exact moments that you described in the guide it works fine.

Now I’m struggling with adding CI/CD for it. I’m trying to use https://github.com/cloudposse/terraform-aws-ecs-codepipeline but have no idea how to pass variables to buildfile.yml. Have you got any clues?

cloudposse/terraform-aws-ecs-codepipeline

Terraform Module for CI/CD with AWS Code Pipeline and Code Build for ECS https://cloudposse.com/ - cloudposse/terraform-aws-ecs-codepipeline

2019-01-10

Maciek Strömich
11:10:10 AM

@Maciek Strömich has joined the channel

I think if Airship has Traefik with something like this, I think it can be a more complete project, what do you guys think ? https://github.com/containous/traefik/issues/4363

[AWS] ALB Provider · Issue #4363 · containous/traefik

Do you want to request a feature or report a bug? Feature What did you expect to see? I am suggesting an Application Load Balancer (ALB) Provider for AWS to make it easy to expose anything connecte…

1
Release notes from terraform-aws-airship-ecs-service
07:13:12 PM

<https Label support (#38)> Label support Restructure input blocks

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

1

2019-01-08

Thanks Erik! That got time, it is too expensive to do anything with it.

2019-01-07

Erik Osterman
AWS Fargate Price Reduction – Up to 50% | Amazon Web Services

AWS Fargate is a compute engine that uses containers as its fundamental compute primitive. AWS Fargate runs your application containers for you on demand. You no longer need to provision a pool of instances or manage a Docker daemon or orchestration agent. Because the infrastructure that runs your containers is invisible, you don’t have to […]

1

2019-01-06

So checked out Traefik a bit, looks promising and could be cool for a reference architecture. It needs a K/V store for Cluster mode, and DynamoDB support is currently not merged yet in the library it uses https://github.com/abronan/valkeyrie so that’s that.

abronan/valkeyrie

Distributed Key/Value Store Abstraction Library written in Go - abronan/valkeyrie

Without cluster mode every instance of traefik would hit the aws api’s for changes, which doesn’t scale

2019-01-03

jonboulle

hey @, back in the land of the living?

happy NY

1
jonboulle

qq: can you remind me why the airship scheduled tasks use lambda rather than just pure cloudwatch events?

the lambda checks for the current task definition, so after updating the ECS service it will stick with what is currently running.

normal scheduled tasks are configured with a certain task definition, so after updating the service they would refer to a non-active taskdef

have a better idea ?

jonboulle

right! thanks. will ponder it

jonboulle

@ so we are belatedly looking at moving everything to newer airships with the lambda lookup/task definition selector; but this means creating O(N) of all of the resources involved - aws_iam_role.lambda_lookup, aws_iam_role_policy.lambda_ecs_task_scheduler_policy (even if no task scheduling is used), aws_iam_role_policy.lambda_lookup_policy, aws_lambda_function.lambda_lookup, as well as various data sources. ideally we would have O(1), e.g. just one lambda that can be used for multiple defs with the necessary parameters passed in. any thoughts on this?

Cool.. For the purpose of speeding up I assume ? Have you thought of creating seperate env’s per application, this really works well, also limits blast radius.

  1. For the lambda lookup we could create a function_arn param which overrides and disables the current one.
  2. lambda_ecs_task_scheduler_policy can be fixed ..
jonboulle

Partly speeding up but also just general cleanliness/KISS - if we have 100 services, then we don’t really want to have 100 lambdas “just” to help with their deployment, if we can do the same thing with 1

puru
04:15:03 PM

@puru has joined the channel

2019-01-02

Bryan
12:54:26 AM

@Bryan has joined the channel

Welcome @Bryan what brought you here ?

2018-12-29

11:53:10 AM

@ has joined the channel

Welcome aboard @ and @! I’m off for new years partying in Belgrade until the 2nd of January, most likely afk. If there’s anything I can help you with, let me know and I’ll get back to you when I can.

2018-12-28

Next stop, Traefik

1
10:16:59 PM

@ has joined the channel

2018-12-27

Release notes from terraform-aws-airship-ecs-service
09:13:15 AM

<https Service discovery (#28)> Added service discovery.. service_discovery_enabled = true service_discovery_properties { namespace_id = “${aws_service_discovery_private_dns_namespace.test.id}” dns_ttl = “60” dns_type = “A” routing_policy = “MULTIVALUE” healthcheck_custom_failure_threshold = “1” }

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2

2018-12-26

Close to finishing service discovery, @ do you have time to take a look ? branch:service_discovery

  service_discovery_enabled = true

  service_discovery_properties {
    namespace_id                         = "${aws_service_discovery_private_dns_namespace.test.id}"
    dns_ttl                              = "60"
    dns_type                             = "A"
    routing_policy                       = "MULTIVALUE"
    healthcheck_custom_failure_threshold = "1"
  }

Hi merry Xmas - did you manage to fix the weird issue with it?

merry xmas!

yes, i don’t have it anymore, also I found a way to disable container_port now for A type records but setting it to false ..

02:20:30 PM
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

That is such a weird fix.

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

I lolled when it worked out. If this ist the real fix of me problem earlier I don’t know. Lost track at one moment.

Ah good. I can’t do the pr right now as I’m out of my hotel. But I can within 24 hours!

1

I’m at schiphol this video is loading very slowly

nice one

Reviewed the changes. There is one instance of var.tags when it should be local.tags

Apart from that, approved!

2018-12-25

2018-12-23

joshmyers

@ Airship in DevOps weekly, nice!

3

Oh cool, I’m not even subscribed, do you want to fwd that to me ?

let’s see if I can get it if I subscribe now

joshmyers
Devops Weekly List

Devops Weekly List Email Forms

I subscribed but not getting the old one I think, can’t find any active archives.

2018-12-21

10:11:51 PM

@ has joined the channel

Hi everyone. First, thanks for your ECS terraform modules! I was following the instructions https://airship.tf/getting_started/ to set up a fargate cluster but i am getting 6 value of 'count' cannot be computed errors as soon as I add the ecs_service module:

* module.ecs_service_nginx.module.alb_handling.aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host: aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host: value of 'count' cannot be computed
* module.ecs_service_nginx.module.alb_handling.aws_lb_listener_rule.host_based_routing_custom_listen_host: aws_lb_listener_rule.host_based_routing_custom_listen_host: value of 'count' cannot be computed
* module.ecs_service_nginx.module.alb_handling.aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host_cognito_auth: aws_lb_listener_rule.host_based_routing_ssl_custom_listen_host_cognito_auth: value of 'count' cannot be computed
* module.ecs_service_nginx.module.alb_handling.aws_route53_record.record_alias_a: aws_route53_record.record_alias_a: value of 'count' cannot be computed
* module.ecs_service_nginx.module.alb_handling.aws_lb_listener_rule.host_based_routing_custom_listen_host_redirect_to_https: aws_lb_listener_rule.host_based_routing_custom_listen_host_redirect_to_https: value of 'count' cannot be computed
* module.ecs_service_nginx.module.alb_handling.aws_route53_record.record: aws_route53_record.record: value of 'count' cannot be computed

I am using terraform 0.11.11 and I was able to provision all resources, once I add the ecs_service module it fails

Start | Airship Modules

Flexible Terraform templates help setting up your Docker Orchestration platform, 100% supported by Amazon

Hi @ thnks, That’s a lot of errors, and they don’t make much sense to me. Could you post a GIST of your sample code & terraform plan. Tomorrow I have some time, but I’m traveling as well.

Hey, sorry, busy christmas days I created a gist: https://gist.github.com/ulich/3ec5ee0d25df3a5451b7350a932ba8ac

@

  1. You’re not really using the getting started code 1:1 so always best to just start of there and make changes after
  2. vpc_id is not an attribute of the aws_vpc datasource. if you change vpc_id to id it will most likely work.

https://gist.github.com/ulich/3ec5ee0d25df3a5451b7350a932ba8ac#file-ecs-service-nginx-tf-L29

    lb_vpc_id = "${data.aws_vpc.vpc.id}"

ah damn, that was it, thanks a lot. Now it works!

Alright! If you feel documentation is lacking somewhere let me know!

2018-12-18

niek
09:39:55 AM

@niek has joined the channel

@niek is also an author of ECS modules, happy to have you here

@ thx

Release notes from terraform-aws-airship-ecs-service
03:58:09 PM

<https Changed the lambda to be dependent of the policy which belongs to the…> … (https://github.com/blinkist/terraform-aws-airship-ecs-service/pull/30) lambda lookup policy dependency fix

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

lambda lookup policy dependency fix by maartenvanderhoef · Pull Request #30 · blinkist/terraform-aws-airship-ecs-service

What A dependency for the lambda iam role policy adding.. the This to make sure that the datasource lookup is not crashing at invoke because the policy was not there yet.. Never happened to me befo…

Erik Osterman

welcome @niek!

2018-12-17

Do you have an example set up using it currently?

that you can make a change to and test?

  resource "aws_service_discovery_private_dns_namespace" "example" {
    name        = "example.terraform.local"
    description = "example"
    vpc         = "${module.vpc.vpc_id}"
  }

module {
 enable_service_discovery = true

    service_discovery_namespace_id = "${aws_service_discovery_private_dns_namespace.example.id}"
}
10:18:57 AM
blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

can you comment out the port line

Isn’t the port needed for SRV ?

# port = "${var.container_port}"

No. I’ve just read through my code and my collegues code where we ahve used srv records.

and never actually added the port value tehre

I just want to see if that is doing anything funny

even though it is mentioned in the tf documentation

The Service Discovery instance could not be registered.

Task comes up, is healthy

But is it working with A records?

then gets killed

can try again

maybe MULTIVALUE routing with SRV is the problem ?

no, it can do it

But try toggling that… andthen also try commenting out

  # Needed for private namespaces
  health_check_custom_config {
    failure_threshold = "${var.service_discovery_healthcheck_custom_failure_threshold}"
  }

Since it may also be the helth check modifiers that do it

ok, now also have the same with A records, not sure what changed then.

ok good. At least we know its not srv now

You didn’t accedentally create a terraform.local route53 zone did you?

as well as the example.terraform.local zone

no, they make it

‘they’

the borg

no change

Can you do a destroy and apply?

just the service or also the namespace ?

namespace too

if you can easily do it

Also, what network mode are you using?

awsvpc? or host/bridge

awsvpc

same shit, i give up for now

@ @jonboulle If you want to have another avatar or introduction next to your name, let me know

Also, if you can write better than me, let me know

https://airship.tf/introduction/

Introduction | .. ECS made easy

Home of Terraform Airship

2018-12-16

@ https://github.com/blinkist/terraform-aws-airship-ecs-service/blob/service_discovery/modules/ecs_service/main.tf With SRV record as DNS type the Service doesn’t get registered, do you have any idea ?

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

2018-12-15

How to spend a weekend. My start on the ECS service documentation.. https://airship.tf/guide/ecs_service/ Would love to have input on anything guys!

ECS Service | .. ECS made easy

Home of Terraform Airship

Erik Osterman
07:44:19 PM
Erik Osterman

love these diagrams

Erik Osterman
07:45:20 PM

Love them too If you have time please tell me what you don’t like!

Erik Osterman

this is the most documented terraform module on the internet

I’ll have to wear sunglasses outside now.

because your future is so bright?

Erik Osterman

hahah

Steven
07:47:57 PM

@Steven has joined the channel

2018-12-12

Erik Osterman

have you taken a look at the new blue/green deployment feature?

Hi Erik, I don’t think so, do you have more info’s for me ?

Erik Osterman
aws/containers-roadmap

This is the public roadmap for AWS container services (ECS, ECR, Fargate, and EKS). - aws/containers-roadmap

Erik Osterman

Not much

Erik Osterman

see the “Just Shipped” column

Ahh! CodeDeploy I’m less interested in, but the roadmap stuff is great

Erik Osterman

yea, I was really surprised they made this public

2018-12-11

Release notes from terraform-aws-airship-ecs-service
10:53:19 AM

<https Task scheduler fix (#27)> What

Rewrite of the javascript Task scheduler, it wasn’t working properly with AWSVPC networking. Limited the name of the jobs to 32 chars. Fixed a bug which effected ecs services without a load balancer as the condition was still set to match NONE instead of none

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

1
i5okie

I fell off the earth.. looks like my project has been scrapped for the foreseeable future ;(. But now I’ll be playing with Airship the way it was designed haha

i5okie

The question I had was… vpc_security_group_ids = ["${module.ecs_instance_sg.this_security_group_id}","${module.admin_sg.this_security_group_id}"] What is the intent with these SGs? Or what are the suggested rules?

The Autoscaling Group of instances needs to allow incoming traffic from the Load Balancer. I’m describing it here //airship.tf/guide/ecs_cluster/#security-groups>

ECS Cluster | .. ECS made easy

Home of Terraform Airship

2018-12-07

I use airship and cloudposse combined. I like to add the null_label into airship, so i can cascade the conststant labeling and tagging down through all the modules via context.

I have used Traefik, and it is easy to add into the mix

IF I get a moment, i’ll build up an example

2
Ryan Ryke
05:52:58 AM

@Ryan Ryke has joined the channel

2018-12-05

@i5okie I’d love to know how you see your setup working. In my documentation I will have to work-out a few common use-cases. So I’d love to hear yours.

i5okie

Im thinking to just use terraform to document the infrastructure for the ecs. So I’d like to have a small ecs cluster, and some docker images in ecr. like our usual ruby image, then a postgresql / redis service.. I’m still a bit new to this, so im not sure if ruby would be a task or not?

right now we’re primarily on heroku, and have review apps that can get spun up from github PRs. So in my case, I’d want to write some code that devs would run in their command line to spin up a ‘review app’ (ruby + postgresql + redis images) responding to <app>-<hash>.http://mydomain.com as per traefik.. then when QA is done verifying or whatever, we can kill the review app afterwords.

Hi @i5okie I’ve done something similar with Airship, but less dynamic than your idea. For a QA environment I’ve created multiple services of the same application, let’s call it app running on the same ECS cluster. The 5 different services were preconfigured by terraform and had planet names, earth, mars, uranus etc. At PR, or however the deployment was configured, the developer creates a branch named app-mars-[JIRA-TICKET] and the application was then deployed to the app-mars ECS Service. Route53 and a single Application Load Balancer takes care of routing the traffic to the Planet QA ECS Service. A who-is-blocking-what-service-app on Slack takes care of not having developers deploying at the same planet at the time, what never was a problem really.

1
davidvasandani

@ do you have any config you can share regarding this setup? I’m debating between what you’ve described and the Kubernetes namespace per PR that CloudPosse uses.

1

Hi David, I will add more stuff to the documentation this week!

1

The setup I just described are nothing more than multiple ECS Services configured by terraform together with the resources they are configured to. It’s less dynamic than kubernetes, but you can configure a services from top to bottom, redis, db, access to IAM, other AWS resources, kinesis etc.

1

How multiple services then connect to each other is then configured by the environment variables configured within SSM which are turned into ENV variables by the chamber application.

1

Internally applications can talk to each other through an internal application load balancer, and soon, the service registry, which is something I’m building in at the moment.

1

So the moment I have the service registry setup, only then namespaces can become a reality.

1
i5okie

yeah that kind of sounds similar. So i guess in my case, the LB would point to traefik. then traefik would point to one of the deployed apps..

What is the need for traefik here, if the ALB already does the routing ?

i5okie

auto discovery i guess

i5okie

rather uhm reverse proxy

i5okie

i have wildcard *.http://apps.mydomain.com pointing to traefik. so anything i type in http://myapp-v3344.apps.mydomain.com traefik would then say ah okay this goes here..

i5okie
NETBEARS - Blog | Traefik Load Balancer for ECS services

This is a tutorial on how to deploy a Traefik Load Balancer in AWS to create hosts (FQDN) for development applications launched in ECS based on application name and tags.

i5okie

except someone who wrote this has incomplete documentation and i can’t get that to work.

i5okie

so im trying to do basically the same, but starting with terraform, the airship module, and going from there.

I’m seeing where you’re getting at now. It’s a little different idea from what I’m used to and that makes it interesting. The question is how you want to deploy your others services then. Running terraform at PR ? Or are you ok with pre-configuring a set of services to which you can deploy after.

i5okie

so lets forget about PRs for now

i5okie

i want to have this in terraform so its documented, in case we need to tear down the stack and bring it up in the future. or on another aws account.. etc.

i5okie

so for other services.. I was thinking if it’d be possible to do something like how CircleCi config file has the main -image ruby:2.5 for example, and then below you specify -image postgresql:9.6-alpine for example..

so i think that when ‘deploy command’ is triggered, i’l have to spin up the ruby image, and the postgresql/ redis images and then associate those three together. (this probably has nothing to do with terraform at this point)

I’ve used RDS in a shared tenancy model for that as production is also using RDS and I like to have production more or less in sync with everything else

i5okie

what i’ve described above would be insane for production. its for ephemeral apps with like short life-spans of a couple minutes to a couple days tops.

i5okie

it’s quite a challenge lol. i’ve never done anything like this before

Let me know if if you struggle with something on the way! The mindset behind the airship modules is slightly different from your use-case but we’re engineers, so we can make everything work

i5okie

yep, i have to run out for a bit. but i have a question about your config example

i5okie

tty soon

sure, just post!

2018-12-04

Hi Everyone, I’ve started documenting the ecs cluster module here: https://airship.tf/guide/ecs_cluster/. For ‘better’ English I’ll find someone to take a look, but would love to have input from you guys on the structure itself.

ECS Cluster | .. ECS made easy

Home of Terraform Airship

Erik Osterman

wow maarten! looks great

1

Vuerocks

The codeblocks are currently json, very soon hcl https://github.com/PrismJS/prism/pull/1594

Add support for HCL by outsideris · Pull Request #1594 · PrismJS/prism

Supporting HCL(HashiCorp Configuration Language). Close #1252

Erik Osterman

i think you definitely made the right choice for this project

Erik Osterman

going to take a closer look at some of the UI/UX elements and have our guy incorporate that into our docs

nice ::: warning

Erik Osterman

how does this handle multi-line?

you mean, within the ::: warning block

let me see

Erik Osterman

yea

not really

with <br/> it works

this is not br/ just 2 enters by markdown

Erik Osterman

“2 enters”?

2 returns

that’s not really multiline..

ah got it

so you do it by putting two spaces at the end of the line

Erik Osterman

oh, that’s cool

Erik Osterman
07:05:04 PM
Erik Osterman

the writing on this page is exceptional

Erik Osterman

@ you might want to talk with @ about airship for what you guys are doing. It’s a great turnkey solution for ECS used by Blinkist, among others.

07:11:02 PM

@ has joined the channel

Hi @! I’m afk very soon, but happy to talk any time.

Erik Osterman

Basically they’ve built a very easy turnkey solution for using ECS with Terraform. Cloud Posse has our own modules, but @ is going above and beyond with airship.

Erik Osterman

He’s specifically trying to make it easier for smaller startups to get up and running quickly

i5okie
07:31:23 PM

@i5okie has joined the channel

i5okie

hi

i5okie

moved here from general. im trying to implement and automagical experience for our devs to create review apps, after we move from heroku to aws.

i5okie

I’m imagining it being almost like circle-ci. by that i mean have “service images” like postgresdb, and other normal images, like ruby-2.3-8-node-browsers kind. most likely only have a single ecs instance, or cluster of 1, and devs would launch review apps from command line (i’m working on a gem, have a bunch of heroku cli - like features already implemented, this would be next). to either deploy from PR, or from local staged files.

i5okie

came here for some inspiration / suggestions / opinions

i5okie

looks like traefik will be pretty important

joshmyers
09:00:11 PM

@joshmyers has joined the channel

joshmyers

@i5okie Is this an unsolved problems that needs code written or is there something you could take off the shelf?

joshmyers

Traefik is

joshmyers

Airship looks nice!

i5okie

yeah i was looking at airship too.

i5okie

i still don’t really understand how airship can help

Hu @i5okie I’m not really online now, but tomorrow I’ll have plenty of time to show you around what airship is about. I just started documenting ( work in progress ) https://airship.tf . Its guide is only covering the cluster at the moment, the ecs service module which is 99% of the logic isn’t covered yet, just a little bit in the (unfinished) getting started section. You can take a look here: http://github.com/blinkist/terraform-aws-airship-ecs-service . Maybe you can explain your use-case a bit and let’s discuss tomorrow !

.. ECS made easy

Home of Terraform Airship

i5okie

Hmm if i were to use airship, which looks great.. Looking at your guide (great work so far!!!) I think in my use case I’ll need to replace nginx with Traefik

i5okie

i think that’d be a killer combo

joshmyers

Cunning use of the data lambda for bootstrap

joshmyers

@i5okie Have done similar with Traefik and the ECS backend. Worked nicely

i5okie

josh, the only ecs+traefik tutorials I’ve found out there so far. are somewhat useless / not well documented.

i5okie

I’m trying to hack together the airship moduel some of the cloud-posse modules, into a nice ecs stack. then try to figure out how to make a service module to bring-up a traefik service.. one thing at a time i guess

i5okie

So is the plan with Airship to bring it into Cloud-Posse family?

Erik Osterman

For the time being , I think this is an independent project. But honestly, we just want people doing cool stuff to be part of the community.

Erik Osterman

I totally endorse what @ is building with airship + blinkist

Erik Osterman

I know @ is contributing to it among others

i5okie

i like that

2018-12-03

Documenting is close to folding laundry in my happiness index.

2
2
1
Erik Osterman

yea, it’s like pulling teeth

Pulling teeth is short and intense, documenting is more like a week long back pain.

Erik Osterman

lol

Erik Osterman

good point

2018-12-01

2018-11-30

Hi @davidvasandani !

1
davidvasandani

Hello! Looking forward to installing Airship.

1

Cool, I’ve just started with https://airship.tf/getting_started/ let me know if you need some help.

.. ECS made easy | Start

Home of Terraform Airship

1
geertn
08:45:31 AM

@geertn has joined the channel

@ Do you have that initial setup of the service registry again for me, slack history doesn’t show it anymore. Also in your original PR there are some things volume related which I will cover after service discovery.

Erik Osterman

Man… I gotta find someone at slack who can hook us up

Erik Osterman

I have seen they help some communities out

Good Morning Erik

What do you mean ?

Erik Osterman

Oh they can flip a flag and make us a “paid” community

Erik Osterman

Like Kubernetes slack

Erik Osterman

And some other teams I am on

AHHH gotcha, slack history

Erik Osterman

Oh yes - sorry not clear

yeah that would be nice to have some more history, they probably store it anyway

Erik Osterman

They do keep it

We just have to create a history bot to kinesis & s3

Erik Osterman

we’re in the process of setting up a public searchable archive

Erik Osterman
Slack Gives Back to K8s and CNCF Community - Cloud Native Computing Foundation

Slack is giving back to the Kubernetes and CNCF communities with free access as part of their not for profit program. We are also thrilled that they have extended their…

Erik Osterman

Aha, so that’s why…

Release notes from terraform-aws-airship-ecs-service
09:08:10 AM

<https NLB Services (#26)> Network Load Balancing Support added for the ECS Module. The parameter load_balancing_type can be set to either “none”, “network” or “application” and takes care of attaching the related LB type.

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

@ and another question, for a service to connect to other services in the service registry, it does not need to be registered to the registry itself, correct ?

correct

1
Erik Osterman
11:33:09 PM

@Erik Osterman set the channel topic:

2018-11-29

@jonboulle @ replied to your pr comments, if you have time to take a look .

Hi, thanks. Ill look later today.

Introducing AWS App Mesh – service mesh for microservices on AWS | Amazon Web Services

AWS App Mesh is a service mesh that allows you to easily monitor and control communications across microservices applications on AWS. You can use App Mesh with microservices running on Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Container Service for Kubernetes (Amazon EKS), and Kubernetes running on Amazon EC2. Today, App Mesh is available […]

jonboulle

requires awsvpc

think we built that in if there now only would be instances with a bit more eni capability ..

davidvasandani
06:24:50 AM

@davidvasandani has joined the channel

2018-11-27

yeah but CICD is really the least of our problems, so it’s not a decision maker. I was looking for a simple DOC Layout, with a side bar etc. and this looks I don’t need to spend much time on it which is exactly what I want : https://github.com/runatlantis/atlantis/tree/master/runatlantis.io

runatlantis/atlantis

Terraform For Teams. Contribute to runatlantis/atlantis development by creating an account on GitHub.

1

2018-11-26

@jonboulle we now have merge commit set by default, shall we make it squash & merge ?

jonboulle

no strong opinion… I like merge commits for auditability and reverts but that only works with extremely strict commit diligence anyway

still no gif support here

Release notes from terraform-aws-airship-ecs-service
05:28:15 PM

<https Adding lambda_ecs_task_scheduler (#25)> Init commit scheduled tasks

wip

ecs_scheduled_tasks update

small fixes

ecs_task_scheduler

ecs_scheduled_tasks clean up

Readme updates

fmt

Fix after PR Review

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

yeah this can be better

After going through tons of jekyll examples, I think https://vuepress.vuejs.org which is what is powering https://www.runatlantis.io looks to be the least time intensive with the most powerful layout there.

VuePress

Vue-powered Static Site Generator

Terraform For Teams | Atlantis

Atlantis: Terraform For Teams

Erik Osterman

if considering just static site generation, did you consider Hugo?

Erik Osterman
gohugoio/hugo

The world’s fastest framework for building websites. - gohugoio/hugo

Erik Osterman

main reason to use Jekyll is it’s supported by GitHub without any CI/CD systems

Erik Osterman

(30K stars - very popular)

2018-11-25

Probot

GitHub Apps to automate and improve your workflow

Erik Osterman

yea, that one is nice

2018-11-23

so every time someone clicks on the word http://variables.tf on slack

If I drop the redirect I can probably just banner it annoyingly

Erik Osterman

Hahjaha

Erik Osterman

Good one Martin

need to find out what slack gets and what not

Erik Osterman

Are you going to build a Jekyll site?

what’s that ?

“Transform your plain text into static websites and blogs. “

http://variables.tf is probably a good site for a blog

But for airship, a nice project site would be nice with some samples

Erik Osterman
Using Jekyll as a static site generator with GitHub Pages - User Documentation

If you use Jekyll as a static site generator with GitHub Pages, you benefit from more support with setting up, updating, and troubleshooting your site. …

Erik Osterman

It’s the predominant way static sites for projects in GitHub are generated

Erik Osterman

Hundreds of templates out there

Do you have a favourite example of a project ?

Erik Osterman

It’s on my list of things to research. Plan to use our readme spec to generate static sites for all of our modules.

nice, i’ll have a look.. First have to find a way to get the README.md a bit more attractive in general

Erik Osterman
Supported themes

List of Jekyll themes supported by GitHub Pages

funnily enough readme.md does not trick slack

Erik Osterman

Is there a .md TLD?

Erik Osterman

Looks like there is!

2018-11-22

Release notes from terraform-aws-airship-ecs-service
09:48:25 AM

<https Merge pull request #24 from blinkist/lambda_lookup> Lambda lookup

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

1

hehe thanks @Erik Osterman

jonboulle

funky

2018-11-21

Fellas! I’m adding Scheduled jobs to the module, callable with simple list block of multiple jobs. Cloudwatch event triggers an event which hits a lambda, lambda figures out what the current task definition of the service is and will run the command on there. Let me know if you have ideas or concerns.

I think I leave out the container env vars override as it’s too much hassle now, same with placement constraints, as they can be added later.

I’m currently creating a different iam role for this lambda, but I’m opting for using one role defined in the iam module instead.

    ecs_scheduled_tasks = [{
      job_name                           = "vacuum"
      schedule_expression        = "cron(0 12 * * ? *)"
      command                           = "python vacuum_db.py"
      container_envvars_override = "A=B;VAR2=C"
    }]

Work in progress : https://github.com/blinkist/terraform-aws-airship-ecs-service/blob/ecs_scheduled_tasks/modules/ecs_scheduled_tasks/main.tf

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

1

2018-11-19

Need some ideas here:

https://github.com/blinkist/terraform-aws-airship-ecs-service/blob/966757509ca42a0b4d3af541d7b5f864e8ff8ac4/modules/live_task_lookup/output.tf#L33

So I’ve added a lambda for live task lookup to overcome the bootstrapping issue. But it seems that a lambda_invocation datasource works differently from a regular datasource.

We create a Taskdefinition with an image coming from the live_lookup module, when using a normal datasource, it will figure out that when the taskdefinition is the same as the current one and will not create a new one. But when using the lambda invocation as datasource for getting the current image, the plan already describes it will create a new task_definition no matter what. UGH!

blinkist/terraform-aws-airship-ecs-service

Terraform module which creates an ECS Service, IAM roles, Scaling, ALB listener rules.. Fargate & AWSVPC compatible - blinkist/terraform-aws-airship-ecs-service

pfew, found the problem

I had a depends_on in the datasource to the lambda_function..

depends_on in datasources always force recreation..

jonboulle

so it all works now?

Do you think you can get Anatoli to take a look at the nodejs, if not I’ll have to find someone else

also, currently i’m creating the iam resources for the lambda in modules/iam, shall we leave it there or move it to the live_task_lookup

Maybe you can take another look at the logic itself in js

PR is out Jon, sorry that you need to go through Javascript code .. I’ve asked in the node subreddit and one guy with unknown seniority said the code was fine .

2018-11-12

05:27:35 PM

@ has joined the channel

05:27:35 PM

@ set the channel purpose: Home of Airship ECS Modules ( https://github.com/blinkist/terraform-aws-airship-ecs-service )

Erik Osterman
05:27:35 PM

@Erik Osterman has joined the channel

05:27:35 PM

@ has joined the channel

jonboulle
05:27:36 PM

@jonboulle has joined the channel

aknysh
05:29:56 PM

@aknysh has joined the channel

Hi Everyone, welcome to the channel, thanks @Erik Osterman ! Happy to help out Airship module users and to also discuss upcoming changes to the module here.

1

And happy to announce that the latest ECS Service 0.8.0 supports Cognito authentication, http to https redirection. and it can deal with ever changing task definitions by any external pipeline and still allows Terraform to modify the Environment Variables and/or cpu and mem.

3
Nikola Velkovski
08:05:08 PM

@Nikola Velkovski has joined the channel

Erik Osterman

That’s really cool! Like the cognito integration. Should make it easier to expose staging apps

Exactly and drop nginx services with the sole purpose of providing a simple htaccess

2
    keyboard_arrow_up