#airship (2019-02)

airship

Home of Airship ECS Modules ( https://github.com/blinkist/terraform-aws-airship-ecs-service / https://github.com/blinkist/terraform-aws-airship-ecs-cluster )

Archive: https://archive.sweetops.com/airship/

2019-02-28

jamie avatar
jamie

@maarten i’ll take a look at this

jamie avatar
jamie

I put in a PR for the ecr-cluster repo a few days ago that gave it a few more features

2019-02-27

github140 avatar
github140

Awesome project. I’d like to use NLB Setup. Is my understanding correct that this is a two phase deployment currently? If LB is included from beginning I keep getting LB errors.

maarten avatar
maarten

Correct, for this to work I need to sit down a bit to remove the maps as input and replace them with normal string vars.

github140 avatar
github140

Is there any other workaround to get into a single step deployment or would you find some time soon? I might also crawl through the TF code.

2019-02-18

2019-02-16

2019-02-15

2019-02-14

ansgar avatar
ansgar

Hey there I just noticed today, that Airship at sometime dropped the support for multiple containers in a single task definition.

I’m quite new to Airship & ECS and am now thinking which way I should go for the service I need to deploy (nginx and php are the two images to run). I don’t want to write everything by hand or just copy most of the code and am looking for a “better way” Could anyone of you give me a hint how to solve this whilst still using the airship-ecs-service module? (Maybe I could update the taskdefinition after the initial creation to contain the two images and find a way for Airship to keep the two images in there on later updates?)

tl;dr How to airship with two containers in a single taskdefinition?

maarten avatar
maarten

Hi Asgar, this can be possible, but only with a massive rewrite of the module. I removed sidecarring as it made the everything much more complex and because I rather see a nginx as a seperate service than something combined.

I slowly started to understand that especially for PHP applications, Nginx functions as the URI rewriting mechanism which functions as a 1:1 mechanism to the PHP app.

It will require a lot of effort to make these changes, but I’m happy to assist in getting it right if you have suggestions (PR).

ansgar avatar
ansgar

Thank you for explaining, Maarten. I think for now I’ll mainly copy the main.tf of the ecs-service module locally and replace the task-definition and lookup modules. Currently time is quite tight for the project I’m working on, so I’m unsure if I can make a good and clean PR in the next one to two months.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
05:24:18 AM

2019-02-13

github140 avatar
github140

Using Jenkins or gitlab? What’s your setup and how do you test those images?

maarten avatar
maarten
10:22:09 PM
1
1
:-1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Build software better, together

GitHub is where people build software. More than 31 million people use GitHub to discover, fork, and contribute to over 100 million projects.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So many projects called airship

maarten avatar
maarten

Hi Chris,

Thanks for pointing me to the similar names. Airship is a common word so it was bound to happen. terraform-aws-airship-ecs-service is older and was already on Github ( the open source creativity hub , not marketplace ) under a different name. Another repository related to this project https://github.com/blinkist/airship-tf-albnlb was created on April 26h 2018.

The Terraform AWS Airship Github repositories are directly connected to a Terraform Registry and are used as building blocks for other parties to use as infrastructure. The Terraform AWS Airship ECS service itself has been provisioned https://registry.terraform.io/modules/blinkist/airship-ecs-service/aws/0.8.8 over 10.000 times already. A name change would not be possible without breaking existing infrastructure of other parties.

My projects have Terraform in their names, are written in Terraform and are specifically made around AWS ECS infrastructure and have nothing to do with Openstack nor Kubernetes. I’m sure that developers who use Github can spot the difference.

I hope you understand my point of view, and the difficulties regarding renaming a project like this, I am willing to put a disclaimer of some sorts in the README of my projects. I’m happy to help out in that way.

Best regards,

Maarten van der Hoef

blinkist/airship-tf-albnlb

Airship Terraform module for a generic ALB / NLB. Contribute to blinkist/airship-tf-albnlb development by creating an account on GitHub.

:--1:2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


My projects have Terraform in their names, are written in Terraform and are specifically made around AWS ECS infrastructure and have nothing to do with Openstack nor Kubernetes. I’m sure that developers who use Github can spot the difference.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

great point.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

well written!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


I am willing to put a disclaimer of some sorts in the README of my projects. I’m happy to help out in that way.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this should be all that’s needed

maarten avatar
maarten

Maybe they can buy the .dev already

2019-02-12

Maciek Strömich avatar
Maciek Strömich

@github140 depends on the platform. For Elastic Beanstalk we’re treating them as dependency only containers with code being mounted inside the container upon execution. For ECS we’re adding everything inside the container which is version tagged and this version tag is being used upon cloudformation stack deployment.

github140 avatar
github140

Do you have a build pipeline for the images?

Maciek Strömich avatar
Maciek Strömich

yes

2019-02-11

github140 avatar
github140

Slightly off topic, how you you build and manage your images, ideally in a pipeline ?

2019-02-07

maarten avatar
maarten

@tamsky not in the current setup. Would take a lot of effort to move to get this multi-container definition working.

1

2019-02-06

maarten avatar
maarten

@hlarsen The Airship ECS Cluster module is purely on-demand for now. I suggest you take a look at something Jamie started : https://github.com/cloudposse/terraform-aws-ecs-spot-fleet/tree/init

cloudposse/terraform-aws-ecs-spot-fleet

[wip] Terraform module to create a diversified spot fleet for ECS clusters - cloudposse/terraform-aws-ecs-spot-fleet

i5okie avatar
i5okie

@maarten I left terraforming for later. Trying to get myself more familiarized with ECS/forgate with cloud formation for now. going to write up terraform config from scratch after I think.

maarten avatar
maarten

ah ok, why cloud formation ?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

because TF is for kids and all real men start with CF?

maarten avatar
maarten

I forgot

maarten avatar
maarten

reason I don’t do AWS certification, is all the CF

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

associate or professional?

maarten avatar
maarten

Neither, I never really looked into it tbh, but for sure there will be CF

maarten avatar
maarten

have you ?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea, I had associate architect, but there was no CF whatsoever (just a bunch of multiple choice q’s)

maarten avatar
maarten

ah ok, in that case ..

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you should do it

maarten avatar
maarten

yeah, will help freelancing a bit maybe

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

professional is much more complicated. Has anyone you know got it?

maarten avatar
maarten

@Maciek Strömich what do you have again ?

Maciek Strömich avatar
Maciek Strömich

wut?

maarten avatar
maarten

aws certification

Maciek Strömich avatar
Maciek Strömich

saa, dop and sap

Maciek Strömich avatar
Maciek Strömich

why?

Maciek Strömich avatar
Maciek Strömich

and why are you asking on #airship? (-:

maarten avatar
maarten

is that associate or professional ?

maarten avatar
maarten

we can chat everywhere about anything

maarten avatar
maarten

and was there cloudformation involved ?

Maciek Strömich avatar
Maciek Strömich

sys admin associate, devops professional and solutions architect professional

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

oh wow, you are the man @Maciek Strömich

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

any notes on solutions architect professional?

Maciek Strömich avatar
Maciek Strömich

@Andriy Knysh (Cloud Posse) thanks but I don’t care about the certs. company that I work for is in APN and requires some number of certs

Maciek Strömich avatar
Maciek Strömich

@Andriy Knysh (Cloud Posse) new or old one?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

what’s the diff?

Maciek Strömich avatar
Maciek Strömich

huge

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you got old?

1
Maciek Strömich avatar
Maciek Strömich

the old one was much simpler

Maciek Strömich avatar
Maciek Strömich

yeah

Maciek Strömich avatar
Maciek Strömich

pushed in nov to be able to have the old one

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ok, i guess i’ll try to get the new one

Maciek Strömich avatar
Maciek Strömich

the new one is built on the newer AWS offering

Maciek Strömich avatar
Maciek Strömich

some code* questions

Maciek Strömich avatar
Maciek Strömich

autoscaling, containers

Maciek Strömich avatar
Maciek Strömich

and as always building solutions in a highly available, fault tolerant and with cost in mind manner

Maciek Strömich avatar
Maciek Strömich

the old one was about ec2, s3, opsworks, storage gateways, some hybrid solutions

Maciek Strömich avatar
Maciek Strömich

and not that much technical

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you talking about the old and new for the same thing, solution architect professional ?

Maciek Strömich avatar
Maciek Strömich

just theory of cloud concepts

Maciek Strömich avatar
Maciek Strömich

the new one on the other hand (at least based on the practice exams in cloud academy) is much more technical in general

Maciek Strömich avatar
Maciek Strömich

including some “simple” questions about API usage

Maciek Strömich avatar
Maciek Strömich

it gets more in depth on the technical path

Maciek Strömich avatar
Maciek Strömich

if your aws-fu is okish then with a little bit of effort certs are easily passable

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea, thanks

Maciek Strömich avatar
Maciek Strömich

@Andriy Knysh (Cloud Posse) yeah basically

Maciek Strömich avatar
Maciek Strömich

@Andriy Knysh (Cloud Posse) i spent a week of going through the faqs and docs on the topics covered by the cert

Maciek Strömich avatar
Maciek Strömich

and I’m not as fluent in AWS as @maarten

Maciek Strömich avatar
Maciek Strömich

or maybe instead of fluency it’s more about exposure to aws services in a different manner

maarten avatar
maarten

You have no wet dreams on AWS ?

Maciek Strömich avatar
Maciek Strömich

only if you’re setting up my airship cluster

maarten avatar
maarten

Maciek Strömich avatar
Maciek Strömich

@Andriy Knysh (Cloud Posse) BTW IIRC since beginning of Feb you can only apply for a updated certificate.

maarten avatar
maarten

anyway, I just got myself a new contract without certification so ..

Maciek Strömich avatar
Maciek Strömich

and since Oct last year you can skip associate exam and pass professional

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

that’s good

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

mine already expired, so I had to do it again

maarten avatar
maarten

Maciek Strömich avatar
Maciek Strömich

@maarten kawabunga btw, we should go for beer and kickers after me moving back to Berlin

maarten avatar
maarten

let’s do !

tamsky avatar
tamsky

I was hoping to use a sidekick container in ECS and use airship. https://github.com/blinkist/terraform-aws-airship-ecs-service/issues/16#issuecomment-423129398 makes me think this can’t be done:

[ECS TASK [ Container 1 : load data from s3, unzip to /data, sleep or exit ] <--volumes-from mount [ Container 2 : Application server, reads from /data ]]

and the module has eliminated support for multi-container definitions. Is this true?

tamsky avatar
tamsky

Or is there another way I can use airship (which I like), to implement a sidekick pattern?

2019-02-05

hlarsen avatar
hlarsen

i have some time today to finally look at the code, so i’ll see if that will work for us - thanks!

maarten avatar
maarten

Thanks @joshmyers for helping out, for everyone else, I’m back from vacay

maarten avatar
maarten

@i5okie how are you faring now ?

maarten avatar
maarten

To everyone, I’m thinking of removing the current map input we have. I’ve always used them for aesthetics, but so far this causes also the famous cannot compute count problems. @jamie You were also a big fan of maps, any last verdict ?

jamie avatar
jamie

@i5okie and @joshmyers the PR’s that I submitted for the airship ecs cluster module and the ecs service module removed all of the count compute errors from it, and gave it the ability to do spot. It is however a breaking change, as there were like 4 variables that I had to move out of the map blocks and back into stand alone variables.

There’s an examples directory that I have added to the branch that demonstrates adding the variables, and shows that it works without count errors.

2019-02-04

hlarsen avatar
hlarsen

hi all, i’m about to do some experimenting with the end goal being an ECS cluster backed by an ec2 fleet made up of on-demand and spot instances. would the airship modules be a good fit for that, or should i just go with the base terraform resources?

jamie avatar
jamie

@hlarsen @maarten I have addedthis pull request that adds those features https://github.com/blinkist/terraform-aws-airship-ecs-cluster/pull/9

This PR includes three features by Jamie-BitFlight · Pull Request #9 · blinkist/terraform-aws-airship-ecs-cluster

launch_template resource is used instead of a launch_configuration, this allows for a more granular tagging. i.e. The instances and the instance ebs volumes get tags. The ASG uses a mixed_instances…

maarten avatar
maarten

@jonboulle care to take a look ?

joshmyers avatar
joshmyers

airship has shaved a load of edges so you don’t have to. Not sure if it works with on demand and spot instances OOTB (probably not) but a good bet would be to fork it and use as a base

2019-02-01

i5okie avatar
i5okie

good point. also the minimum and maximum deployment/health values are redundant when its on CODE_DEPOLOY. but it didn’t complain when i left them set. I’ll get rid of the map later today

i5okie avatar
i5okie

updated PR. travis green

i5okie avatar
i5okie
07:56:54 PM
1
i5okie avatar
i5okie

i keep getting this error “module.core.module.container_definition.null_resource.envvars_as_list_of_maps: null_resource.envvars_as_list_of_maps: value of ‘count’ cannot be computed”

i5okie avatar
i5okie

it doesn’t like more than one envvar in this “container_envvars {..”

i5okie avatar
i5okie

i’ve tried without comas, with comas, i dont get it.

i5okie avatar
i5okie

its interpolation.

i5okie avatar
i5okie

yeah found the issue. completely unrelated to any of these modules.

    keyboard_arrow_up