#refarch (2024-01)

Cloud Posse Reference Architecture

2024-01-03

2024-01-10

johncblandii avatar
johncblandii

Is there any way to turn off the health checks on an ASG?

context: a set of github runners are failing health checks due to the code causing the runner to show unhealthy. it is a temporary time period where the compute is hammered, but we need it to live through that moment and not cause the runner to be decommissioned.

johncblandii avatar
johncblandii

Setting health_check_type to null or "" is ignored so I’m assuming the ASG requires a health check

RB avatar

could you just increase the health_check_grace_period from 300 seconds to something higher?

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group#health_check_grace_period

I don’t think health checks can be disabled

RB avatar

you could add HealthCheck to suspended_processes

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group#suspended_processes

See scenario 6 on https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-suspend-resume-processes.html

Scenario 6: HealthCheck is suspended • Amazon EC2 Auto Scaling stops marking instances unhealthy as a result of EC2 and Elastic Load Balancing health checks. Your custom health checks continue to function properly. After you suspend HealthCheck, if you need to, you can manually set the health state of instances in your group and have ReplaceUnhealthy replace them.

Suspend and resume a process for an Auto Scaling group - Amazon EC2 Auto Scaling

Suspend and then resume one or more of the standard processes that are built into Amazon EC2 Auto Scaling.

johncblandii avatar
johncblandii

oh that could work. we turned off autoscaling for now, but we can look into that for sure.

thx @RB

1

2024-01-11

2024-01-24

johncblandii avatar
johncblandii

Heads up: https://github.com/cloudposse/terraform-aws-components/blob/main/modules/s3-bucket/README.md.

That doc says prefix: logs/, but the code will create a double slash since the format is %s/%s/ resulting in: logs//some-name.

A simple fix is to change this line to include a replace(//, /, ...) sort of thing, but there could be a better fix of clearing a suffix / or something else.

We’re tackling this internally soon, but want to raise it for the larger community

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Gabriela Campana (Cloud Posse) looks like a ticket to fix is needed

1
bananadance1
Gabriela Campana (Cloud Posse) avatar
Gabriela Campana (Cloud Posse)
Log in with Atlassian account

Log in to Jira, Confluence, and all other Atlassian Cloud products here. Not an Atlassian user? Sign up for free.

2024-01-29

johncblandii avatar
johncblandii
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Gabriela Campana (Cloud Posse) we can add this as an internal task, no commitment on when it will get fixed.

1
johncblandii avatar
johncblandii

I might put someone on pushing up a fix as well. Just wanted to create a ticket to not forget it

1
Gabriela Campana (Cloud Posse) avatar
Gabriela Campana (Cloud Posse)

@Erik Osterman (Cloud Posse) is this related to https://cloudposse.atlassian.net/browse/DEV-1591?

Log in with Atlassian account

Log in to Jira, Confluence, and all other Atlassian Cloud products here. Not an Atlassian user? Sign up for free.

    keyboard_arrow_up