#refarch (2024-03)
Cloud Posse Reference Architecture
2024-03-27
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
Hey, getting a weird error when trying to deploy ecs-services
tasks. Some, but not all of our services are failing to deploy with this error:
│ Error: creating ECS Task Definition (taskname): ClientException: When networkMode=awsvpc, the application protocol must be one of [http, http2, grpc]
│
│ with module.ecs_alb_service_task[0].aws_ecs_task_definition.default[0],
│ on .terraform/modules/ecs_alb_service_task/main.tf line 49, in resource "aws_ecs_task_definition" "default":
│ 49: resource "aws_ecs_task_definition" "default" {
│
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
We can’t find any meaningful difference between the services that deploy and the ones that don’t
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
appProtocol
is an optional string and we don’t have it set for any of the other tasks that are working
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
and setting it also doesn’t seem to matter
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
based on the error message, it sounds like awsvpc network mode requires one of those 3 app protocols. When you tried setting appProtocol
, what happened?
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
didn’t make a difference, same error
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
that should go under port_mappings
correct?
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
yes that should be right. Could you share how you configured that variable?
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
port_mappings:
- containerPort: 80
hostPort: 80
protocol: tcp
appProtocol: http
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
I’m assuming Slack reformatted that right? YAML indentation is picky
port_mappings:
- containerPort: 80
hostPort: 80
protocol: tcp
appProtocol: http
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
it doesn’t seem to be picking up the appProtocol
var
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
it’s weird because this works fine for some other ecs-services with nearly the same config
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
yeah that is bizarre. Can you share a config that is working? What’s the difference?
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
port_mappings:
- containerPort: 8080
hostPort: 8080
protocol: tcp
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
oh lol. I’m trying to reproduce locally, one minute
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
we’re on version: 1.417.0
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
I can’t reproduce this. Whenever I add the appProtocol, it is passed all the way through. Could you try describing a component that is working and a component that isnt working? Then check the values that are passed to terraform:
For example, describe the component:
atmos describe component ecs/platform/service/echo-server -s plat-use2-sandbox
Then double check that the port_mappings include appProtocol
under port_mappings.
Then also check that task
network_mode
is awsvpc
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
atmos describe component ecs/platform/service/echo-server -s plat-use2-sandbox
...
vars:
...
containers:
service:
...
port_mappings:
- appProtocol: http
containerPort: 8080
hostPort: 8080
protocol: tcp
...
task:
...
network_mode: awsvpc
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
oh maybe you might have a lifecycle rule configured for the task definition? In that case it could be that terraform is ignoring your changes. When you add or change appProtocol
, if you run terraform plan
does it show changes?
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
standby, we think we might have a larger issue with ECS clusters that is manifesting as this error for some weird reason
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
we deployed a new ECS cluster yesterday and might have broken some stuff by accident
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
if it’s still busted after the ECS cluster is rebuilt, I’ll give this a shot and follow up. thank you!
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
sounds good. let me know!
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
yeah this ended up being a totally separate issue with a weird error message for some reason
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
it looks like it was partly being caused by some cached task definitions in the ecs s3 mirror
![Taimur Gibson avatar](https://avatars.slack-edge.com/2024-02-15/6670596085552_871db72fafcac3be2839_72.png)
![Dan Miller (Cloud Posse) avatar](https://avatars.slack-edge.com/2021-08-12/2389147782305_5729c9d69c393852d209_72.jpg)
yeah I believe this PR was the fix for anyone else looking up this thread in the future: https://github.com/cloudposse/terraform-aws-components/pull/1008
what
ECS Service Upstream for better support of partial task definition.
why
• Fixes issue with bad merges on s3 task definition • Map_secrets not being updated