#aws (2023-07)

aws Discussion related to Amazon Web Services (AWS)

aws Discussion related to Amazon Web Services (AWS)

Archive: https://archive.sweetops.com/aws/

2023-07-02

Adnan avatar

Anybody using GuardDuty? How do your bills look like?

Stoor avatar

It all depends on usage honestly, it ranges from few dollars to thousands depending on accounts. But as an example, a account with like 7000$ spend, around 150$ of it is guardduty, the account hosts a web application.

2023-07-03

Thomas Poetke avatar
Thomas Poetke

Hello, I have a question related to the different EC2 Terraform modules. Is it really needed to have 3 seperated modules for single instance, instance group and autoscaling? For me, it looks like ec2 group is just the ec2 module with a count. at the moment it looks hard to maintain all 3 modules

Thomas Poetke avatar
Thomas Poetke

Just asking, because of all three modules have similiar merge requests, I change already something in the single ec2 module, which is still missing at ec2group module. looks like its hard to solve the pipeline for ec2 group module, alot of stuck pr’s there

Alex Jurkiewicz avatar
Alex Jurkiewicz

Are you asking why CloudPosse has three? It’s probably a mix of two reasons:

  1. Modules which have fewer knobs are easier to reason about
  2. As new requirements came up, CloudPosse created new modules to meet new needs
Thomas Poetke avatar
Thomas Poetke

@Alex Jurkiewicz the EC2 module with a count would be the same like https://github.com/cloudposse/terraform-aws-ec2-instance-group but less to maintain. the instance group module is totally outdated at the moment

cloudposse/terraform-aws-ec2-instance-group

Terraform Module for provisioning multiple general purpose EC2 hosts for stateful applications.

Alex Jurkiewicz avatar
Alex Jurkiewicz

If that’s true, there are probably many users of one or the other who don’t want to spend time migrating

2023-07-04

Balazs Varga avatar
Balazs Varga

In an ALB… DO I need to enable preserve mode ? I see it is disabled default

BATeller avatar
BATeller

The preserve mode for AWS Application Load Balancer (ALB) is useful when it comes to managing client connections and maintaining “stickiness”. By enabling preserve mode, you can maintain the connection between a client and your application’s instances, even if the targets of the load balancer change.

This is especially beneficial in scenarios where you have applications that maintain stateful connections or utilize real-time protocols. For example, when you have WebSockets or long-polling connections, the connections between the client and the instances need to be maintained.

Normally, ALB will close connections to targets as they are deregistered, but in preserve mode, existing connections remain open and new requests from existing connections are sent to other targets. This can be useful in situations where you are updating or scaling down your backend services and want to minimize disruptions.

1

2023-07-05

Daniel Ade avatar
Daniel Ade

hey my ec2 instance has been stuck on this, anyone encountered this before

Hao Wang avatar
Hao Wang

It is not stuck, need to enter username/password

Daniel Ade avatar
Daniel Ade

I made this on terraform with the same project we worked on lol, how do you put the username and password in?

Hao Wang avatar
Hao Wang

ssh key should be used, instead of username/password

Alex Jurkiewicz avatar
Alex Jurkiewicz

It looks like you’ve connected to the serial console. That’s not really a recommended access path. Use SSH or SSM Session Manager

2023-07-06

Balazs Varga avatar
Balazs Varga

do you use spot instances ? do you know anything about pricing? will it increase and reach on demand in near future ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
AWS Docs GPT

AI-powered Search and Chat for AWS Documentation

8
1
Alex Jurkiewicz avatar
Alex Jurkiewicz

This is much better than I assumed it would be. The answers are good for easy questions but quickly fail once you start asking anything which is harder

AWS Docs GPT

AI-powered Search and Chat for AWS Documentation

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I tried only one test which previously yielded me a false answer (halucination) on ChatGPT. My question was how to enable end to end TLS using an ALB to pods on Kubernetes. The wrong answer was terminating TLS on the ALB and using security groups to protect the pod. Using this new service I got the right answer, which involved a tls sidecar.

Adnan avatar

I’m really a bad prompt engineer

Domagoj avatar
Domagoj

having same issues here guess we destroyed it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Heh, they must have run out of GPT credits

2023-07-07

2023-07-10

Balazs Varga avatar
Balazs Varga

is there any issue in aws ohio ?

Balazs Varga avatar
Balazs Varga

problem is not there .

Sergei avatar

Hi , we have multiple AWS accounts across multiple regions with Transit Gateways used for connectivity. Most of the times all works good. I am trying to troubleshoot a connectivity issue between two different ec2s placed in vpcs in the same account in different regions. Does not seem like connectivity works but from what I see all the route tables, TGW route tables TGW-to-TGW peering and security groups rules are place. Does anyone know of any methods on how to troubleshoot this kind of problem?

Darren Cunningham avatar
Darren Cunningham
What is Reachability Analyzer? - Amazon Virtual Private Cloud

Identify network connectivity issues in your virtual private cloud (VPC) using Reachability Analyzer.

Sergei avatar

Thank you for a suggestion! Reachability Analyzer is a great tool, but isn’t it limited to traffic inside VPC only? I looked at it some time ago and it seemed quite limited. Good idea though, I will at least jog my memory and see what it can do for me in this scenario

Sergei avatar

from the Reachability Analyzer docs: *Source and destination resources* The source and destination resources must be in the same Region. The source and destination resources must be in the same VPC or in VPCs that are connected through a VPC peering connection or a transit gateway. The source and destination resources can belong to different AWS accounts in the same organization from AWS Organizations.

Sounds promising

Sergei avatar

Does not seem this can help in my case with cross region traffic. I can enable organisational wide cross account trust but I don’t see an option for Analyzer to do cross-region tests.

Alex Jurkiewicz avatar
Alex Jurkiewicz

Try asking AWS support. They can often do this sorry if troubleshooting for you

2
Sergei avatar

Thank you everyone, I managed to get it resolved with few rounds of Reachability Analyzer!

2023-07-11

Balazs Varga avatar
Balazs Varga

is there a way to autosync snapshots over regions? based on tag maybe. or all new is fine as well

Alex Jurkiewicz avatar
Alex Jurkiewicz

For what service? Check AWS Backup

Balazs Varga avatar
Balazs Varga

for Velero. Velero creates the snapshots for me and I would like to copy/sync them to different region to able to restore backup in different region on different cluster … with aws backup I saw I can do the snapshot and copy them, but I don’t want to create snapshots by aws backup.

Mike Shade avatar
Mike Shade
Replicating objects - Amazon Simple Storage Service

Set up and configure replication to allow automatic, asynchronous copying of objects across Amazon S3 buckets.

Balazs Varga avatar
Balazs Varga

yeah, I know this. it is good for objects… but I have snapshots in the “aws” s3 bucket, so nnot managed by me.

Alex Jurkiewicz avatar
Alex Jurkiewicz

What do you mean? If you’re using Velero, you can write to any S3 bucket of your choice?

I assume you’re talking about EKS btw, you still haven’t said

Balazs Varga avatar
Balazs Varga

not eks, self managed cluster with kops. OK I can write to any S3 bucket, but can I set 2 ? so until my primary cluster is running it should write to 2 locations. … when pirmary goes down then I can restore everything on secondary in different region.

Balazs Varga avatar
Balazs Varga

in case of aws region where my primary cluster is running goes down and I don’t want to run multi region cluster

Alex Jurkiewicz avatar
Alex Jurkiewicz

You don’t want to write to two buckets. Write to one and replicate the bucket to one in another region

Alex Jurkiewicz avatar
Alex Jurkiewicz

See the link Mike posted before

Balazs Varga avatar
Balazs Varga

yeah, I saw that but will that work for snapshots as well? Guess it is working only if I use the https://velero.io/docs/v1.11/file-system-backup/ this one has a higher inconsistency issue than using CSI https://velero.io/docs/v1.11/csi/ or original cloud api call. Or using CSI will be the option for me and that can go to the bucket I choose. with cloud api, I still can do the snapshot copy to region but for that I think I need a job or something else.

tommy avatar

S3 can be replicated to other region/account

Chris Wash avatar
Chris Wash

Wondering if anyone has started w an approach for how to use and/or incorporate the Aurora Blue/Green deployments feature offered by AWS with existing Terraform projects that manage Aurora clusters?

New – Fully Managed Blue/Green Deployments in Amazon Aurora and Amazon RDS | Amazon Web Servicesattachment image

When updating databases, using a blue/green deployment technique is an appealing option for users to minimize risk and downtime. This method of making database updates requires two database environments—your current production environment, or blue environment, and a staging environment, or green environment. You must then keep these two environments in sync with each other so […]

Alex Jurkiewicz avatar
Alex Jurkiewicz

We are using blue-green heavily for upgrades, especially for 5.7 to 8 migrations. We don’t manage the process with Terraform, we use clickops. We lock our Terraform state, migrate by hand over a week or two, then perform state surgery to reconcile Terraform with reality.

I think terraform is quite weak at managing complex project state transitions, so this is a pattern we use often for things like migrations, restore from snapshot, etcetera etcetera

New – Fully Managed Blue/Green Deployments in Amazon Aurora and Amazon RDS | Amazon Web Servicesattachment image

When updating databases, using a blue/green deployment technique is an appealing option for users to minimize risk and downtime. This method of making database updates requires two database environments—your current production environment, or blue environment, and a staging environment, or green environment. You must then keep these two environments in sync with each other so […]

1
1
Joe Perez avatar
Joe Perez

@Chris Wash any additional insight here from your experience?

2023-07-12

Balazs Varga avatar
Balazs Varga

do you use change manager for managing organization? E.g put approval process to creating/ deleting accounts, OU-s, assigning scp-s … etc etc. Maybe for assigning the assume role to able to login to another account ?

Gabriela Campana (Cloud Posse) avatar
Gabriela Campana (Cloud Posse)

@Jeremy G (Cloud Posse)

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Does GitOps + CODEOWNERS + Branch protections + IAC + PRs count for change management? Then that’s what we have in our refarch

Ibansal avatar
Ibansal

Hello All, I am using “aws ecs wait services-stable” in my CI/CD pipeline to wait for ecs service to be healthy and then make the Jenkins job successful. By default this command waits for 10 minutes to make the decision based on 2 parameter ( “maxAttempts”: 40, & “delay”: 15 sec ) . Is there any way to increase this timeout as there is no direct option provided by aws cli command. if anyone has any suggestion then Please let me know. Refrences:- https://github.com/boto/botocore/blob/b54ceeaca5d4c1316fae0a0496422572d1f6aba5/botocore/data/ecs/2014-11-13/waiters-2.json#L42-L72

    "ServicesStable": {
      "delay": 15,
      "operation": "DescribeServices",
      "maxAttempts": 40,
      "acceptors": [
        {
          "expected": "MISSING",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "failures[].reason"
        },
        {
          "expected": "DRAINING",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "services[].status"
        },
        {
          "expected": "INACTIVE",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "services[].status"
        },
        {
          "expected": true,
          "matcher": "path",
          "state": "success",
          "argument": "services | [@[?length(deployments)!=`1`], @[?desiredCount!=runningCount]][] | length(@) == `0`"
        }
      ]
    },
Hao Wang avatar
Hao Wang
    "ServicesStable": {
      "delay": 15,
      "operation": "DescribeServices",
      "maxAttempts": 40,
      "acceptors": [
        {
          "expected": "MISSING",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "failures[].reason"
        },
        {
          "expected": "DRAINING",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "services[].status"
        },
        {
          "expected": "INACTIVE",
          "matcher": "pathAny",
          "state": "failure",
          "argument": "services[].status"
        },
        {
          "expected": true,
          "matcher": "path",
          "state": "success",
          "argument": "services | [@[?length(deployments)!=`1`], @[?desiredCount!=runningCount]][] | length(@) == `0`"
        }
      ]
    },
Hao Wang avatar
Hao Wang

oh this is not for ECS service

Hao Wang avatar
Hao Wang

as a workaround, may run services-stable multiple times in for loop

Hao Wang avatar
Hao Wang

from this PR, https://github.com/boto/botocore/pull/1267, awscli supports custom waiter configuration

#1267 Expose waiter configuration

Adding so we can properly address aws/aws-cli#2761

This will allow WaiterConfig to be passed to waiter.wait() calls allowing customers to adjust waiter’s maxAttempts and delay settings. This is based on how PaginationConfig works. The current keys are formatted to match the waiter models.

waiter.wait(WaiterConfig={
    'delay': 5,
    'maxAttempts': 5,
})
Ibansal avatar
Ibansal

Thanks @Hao Wang, I will try this solution

1

2023-07-13

Balazs Varga avatar
Balazs Varga

another question with

create-db-cluster

can I specify a certificate to be used ? maybe created by amazon ? if yes, then I could set a custom domain for rds and then without changing dsn I could change the endpoint

Hao Wang avatar
Hao Wang

seems not, the cert is assigned by AWS which can be downloaded in public

Hao Wang avatar
Hao Wang

yeah, confirmed from a few sources

Hao Wang avatar
Hao Wang

Only AWS cert in different regions can be used

Balazs Varga avatar
Balazs Varga

ok thanks

1
Nishant Thorat avatar
Nishant Thorat
CIS AWS Foundations Benchmark v2.0 - Securing AWS cloud resources

Bring your AWS environment into compliance with CIS AWS Foundations Benchmark and prove it at any time with automated reports and dashboards.

2023-07-14

Balazs Varga avatar
Balazs Varga

about aws backup. how can I set an aurora serverless backup to be backed up every hour? can I somehow set the backup windows to infinite ?

BATeller avatar
BATeller

From my understanding with Aurora Serverless snapshots are taken automatically every 5 minutes, but AWS does not allow customizing the snapshot frequency

Also the limit on infinite backups is because the backup window cannot overlap with the weekly maintenance window for the DB cluster

Not sure your specific use-case but could you not automate (through a cron or lambda) to trigger a manual snapshot on some set interval?

1
Balazs Varga avatar
Balazs Varga

As I know yes, but from RDS I cannot copy the snapshots to a different region to able to restore from that snapshot. I would like to use aws backup apps to have a hourly task that backups the serverless DB and copy it to another region…. Really cannot do this with aws backup apps ?

Balazs Varga avatar
Balazs Varga

I only want to have a copy of the snapshot in a second region… and have this hourly, not daily.

Alex Atkinson avatar
Alex Atkinson

I can’t spot the details on CloudFront object invalidation latency (cache flush) in the docs. Does anyone know what the performance profile on this action looks like? Back in the day I remember finding POPs with latency in the 10s-100s of seconds of latency.

Alex Atkinson avatar
Alex Atkinson

AH. Google can tell when I give up on it, because the next result is what I’m looking for. https://repost.aws/knowledge-center/cloudfront-serving-outdated-content-s3

Push updated Amazon S3 content from CloudFront

I’m using Amazon CloudFront to serve objects stored in Amazon Simple Storage Service (Amazon S3). I updated my objects in Amazon S3, but my CloudFront distribution is still serving the previous ver…

Alex Atkinson avatar
Alex Atkinson

I’m sure that detail used to be in the CloudFront invalidation docs…

Alex Jurkiewicz avatar
Alex Jurkiewicz

10-100secs sounds like a reasonable p95 interval to me. I’ve seen invalidations take 10+mins in unusual circumstances.

Alex Atkinson avatar
Alex Atkinson

Depends. Anywhere you’re caching api responses that’s a long time. Fastly’s ~150ms global purge on objects/surrogate key is what I’ve come to expect. Their VCL language offers a lot of capability over cf also. Cf is really basic. But sometimes that’s all you need.

Alex Jurkiewicz avatar
Alex Jurkiewicz

Sorry, what I meant by sounds reasonable is that it reflects reality for CF

1
Alex Atkinson avatar
Alex Atkinson

It’s better than it used to be if I remember right. I think I saw it taking +/- 10m to flush. Eesh

2023-07-18

2023-07-19

forswearbeetle avatar
forswearbeetle

Hello, I created a youtube video on how to create AWS real-world architecture here is a short description of the video https://youtube.com/shorts/7USwGdSFsfc for the full video here the link: https://youtu.be/VGwO7IYYPXE

4

2023-07-20

2023-07-22

2023-07-23

2023-07-25

Adnan avatar

Hi all. Is there a way to find out which feature gates are enabled on EKS clusters?

AFAI was able to see, feature gates are not configurable with EKS. But I was wondering how to find out which of them are enabled. I found this in the docs

“The feature gates that control new features for both new and existing API operations are enabled by default.”

Does that mean that we can assume all features gates that control new features are enabled? I know I just repeated what it says but maybe someone can confirm or correct it.

Gabriela Campana (Cloud Posse) avatar
Gabriela Campana (Cloud Posse)

@Andriy Knysh (Cloud Posse) @Dan Miller (Cloud Posse)

2023-07-26

Nishant Thorat avatar
Nishant Thorat
How to Monitor AWS IAM Root Users at Scale: Best Practices

Protect your AWS environment by monitoring IAM root users at scale. Discover the top best practices for IAM monitoring and avoid common security pitfalls.

andrey.a.devyatkin avatar
andrey.a.devyatkin
fivexl/terraform-aws-cloudtrail-to-slack

Parse AWS CloudTrail events and send alerts to Slack for events that match pre-configured rules

2023-07-27

Abra avatar

Hi everyone, I have a CloudFormation infrastructure repo, What’s the best way to delivery (deploy/update/delete) the stack from the git repo to AWS CloudFormation service.

    keyboard_arrow_up