#aws (2022-06)

aws Discussion related to Amazon Web Services (AWS)

aws Discussion related to Amazon Web Services (AWS)

Archive: https://archive.sweetops.com/aws/

2022-06-01

Hamed Pourshafiee avatar
Hamed Pourshafiee

wave Hello, team!

RO avatar

Hi Everyone, Learning a lot of AWS and just joining today here. When I have questions can I simply paste them on this channel?

2
managedkaos avatar
managedkaos

welcome, @RO. yes, just post. scroll back in the channel and you will see some examples. wave

venkata.mutyala avatar
venkata.mutyala

Check out #office-hours as well

RO avatar

Thanks for the reply. For questions about cloudformation, tools like sceptre, which channel should I use for it?

venkata.mutyala avatar
venkata.mutyala

#aws or even #office-hours (if you wanted to have it discussed during the weekly office hours)

2022-06-02

shivanshu avatar
shivanshu

Not a question but I learned the hard way that the eks package on terraform has quite a few subtle bugs that make using terraform to deploy eks clusters with managed nodegroups quite annoying. My focus is exclusively on compute, so I can only talk about multi gpu instances (mainly p4ds and p3ds) which have the efa networking. Turns out that the variable that decides the number of EFA NiCs to attach to the instances has differing names in cloudformation API and the eks module in terraform.

Carlos Reyna (Infrascension) avatar
Carlos Reyna (Infrascension)

@shivanshu - Deplyoing EKS clusters with Cloud Posse modules works great. Highly recommend it.

Alex Jurkiewicz avatar
Alex Jurkiewicz

your terminology is a bit confusing. in Terraform, modules and resources are different things, and packages is not really a used term

shivanshu avatar
shivanshu

Sorry. Coming from a python background I interchangeably use module and package. The module eks has some issues is what I meant

Alex Jurkiewicz avatar
Alex Jurkiewicz

gotcha. There are many eks modules, are you talking about the cloudposse one? It might be worth a bug report, but I’m not sure ‘module parameter names differ from API names’ would be fixed

shivanshu avatar
shivanshu

And the eksctl package is completely busted if you work with on-demand capacity reservations

shivanshu avatar
shivanshu

anyone else had similar issues on AWS?

2022-06-03

2022-06-06

Zeeshan S avatar
Zeeshan S

Hello Everyone,

What is the most simple solution to rotate IAM user access keys and store them in parameter store.

pjaudiomv avatar
pjaudiomv

For ssm param store probably just a lambada

1
Amaury Ortega avatar
Amaury Ortega

a lambda function with a scheduled event using eventbridge

Creating an Amazon EventBridge rule that runs on a schedule - Amazon EventBridge

Learn how to create an EventBridge rule that runs on a regular schedule.

2

2022-06-07

Gabriel avatar
Gabriel

Hi Everyone, For those who used IAM only and migrated to SSO ….

I have a certain IAM groups and roles configuration in a management account and roles like developer and admin in sub accounts. Now after migrating to SSO, what exactly is SSO replacing in the current IAM configuration? For example, does SSO replace the developer/admin roles in the sub accounts or can I keep them and continue using them with SSO? In my case it is especially relevant in the CLI context where these roles are used for different tasks on developer computers (macs).

Wédney Yuri avatar
Wédney Yuri

For each account I have multiple roles, each role is connected to AWS CLI through this command:

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-sso.html

Configuring the AWS CLI to use AWS Single Sign-On - AWS Command Line Interface

If your organization uses AWS Single Sign-On (AWS SSO), your users can sign in to Active Directory, a built-in AWS SSO directory, or another iDP connected to AWS SSO and get mapped to an AWS Identity and Access Management (IAM) role that enables you to run AWS CLI commands. Regardless of which iDP you use, AWS SSO abstracts those distinctions away, and they all work with the AWS CLI as described below. For example, you can connect Microsoft Azure AD as described in the blog article

1

2022-06-08

pianoriko2 avatar
pianoriko2

Hello @channel Can anyone help with a script to Identify cloudwatch log-groups without logstreams, and groups with streams older than a year..

2022-06-09

Ananya Chowdhury avatar
Ananya Chowdhury

Hi Folks

I have aws SES email identities configured in one aws account and want to move those identities to another aws account. Is it possible to do? and do we have any documentation for that which can be referred. Will the verfication email will again be triggered if it is moved to new aws account? Can anyone please help me with the clarification.

Катерина Кучернюк avatar
Катерина Кучернюк

Hey folks) AWS User Group Ukraine is running a virtual AWS Tech Conference #StandWithUkraine! Join us to discuss Digital Transformation with AWS and meet peers from the global AWS community.

When? June 30 Where? Online

How to join?

  1. Register for free and get full access to the event.
  2. You can support freedom in Ukraine by buying a ticket of any type. All profit will go to Ukrainian charity funds.

Sign up here: https://bit.ly/3zsQkq5 It’s going to be AWSome!

1

2022-06-10

yegorski avatar
yegorski

Anyone been to AWS Summit at Javits Center? Is it worth the time going?

2022-06-11

Gabriel avatar
Gabriel

In the aws_rds_cluster resource definition/docs it says …

To manage cluster instances that inherit configuration from the cluster (when not running the cluster in serverless engine mode), 
see the aws_rds_cluster_instance resource.

I am confused about the when not running the cluster in serverless engine mode

Does anybody understands what this means? Does this mean aws_rds_cluster_instance is not supported in engine mode is serverless? Does this mean it automatically scales replicas in engine mode is serverless?

Alex Jurkiewicz avatar
Alex Jurkiewicz

It means that if you use serverless mode, you shouldn’t attempt to manage the cluster’s instance(s) with Terraform. It will sort of work, but not really.

1
1

2022-06-13

2022-06-15

mikesew avatar
mikesew

Hi folks. I’m trying to write standards/guidelines for AWS RDS instance type standards. For AMD-based instances (ie. m5a), I believe they’re supposed to give comparable performance to intel (m5) with about .. 10% less cost. Are there any cons when using say, RDS (where it’s a PaaS and you don’t really care about the individual OS compatibility)?

keen avatar

@mikesew I haven’t thought about trying arm RDS instances (hadn’t noticed availability). do they allow you to slide them into existing clusters/replace existing instances with them? that said - general rule of thumb here is always going to be testing for your self. if a 10% difference is worth the effort, model your workload, and simulate it. see how it compares. roll it into early environments (dev..) and evaluate for true compatibility with your application stack. if you’re doing green field, probably worth just pulling the trigger to try it in your dev environments. at least then you can always write around incompatibility/performance concerns without any actual cost. (unless it breaks your frameworks.) but I’d venture that 10% might not be worth the investment for an existing platform.

1
mikesew avatar
mikesew

I was going to make the same question for ARM graviton instance family, but I understand that is a much bigger issue switching architectures (but for a tantalizing 20+% savings). With AMD , it seems like I’m not losing anything here - it’s still x86, and the OS / potential compatibility issues are masked from me as a PaaS customer.

keen avatar

ohhh amd. i read arm :). yeah zero concerns going amd64!!

keen avatar

i mean yeah your workloads might perform worse in one area ot better in another but that’s alreadt the case with any instance size or type!

keen avatar

(blame the phone keyboard…)

mikesew avatar
mikesew

I appreciate the reservations, I have the same . I’m trying to create some guidelines for my org. Our original standard only allowed Intel, but now I think we’re going to relax it to allow alternative instances like r5b in addition to M5, r5 and t3.

2022-06-16

Balazs Varga avatar
Balazs Varga

Hello all, What is the best way to automate subaccount creation. Let’s say I have a main account and I would like to run k8s clusters under different accounts so they won’t bother each other. Terraform ? Ansible?

zetta avatar

you can run your terraform from your main account and create the sub accounts directly from there. and even create the resources, obviously the more you add there will exponentially increase on time for the run.

zetta avatar
Create new AWS accounts with Organizations and Terraform - The Cloudly Engineerattachment image

Let’s use AWS organizations with Terraform to create new AWS accounts or invite existing AWS accounts in AWS organizations!

Tomasz Krzyżanowski avatar
Tomasz Krzyżanowski

IMO you should stick to Terraform for such tasks - TF will take care of your future changes and will report if something was Clickoopsed - as in TF everything is all about taking care of state and moving towards desired state

Balazs Varga avatar
Balazs Varga

thanks guys. ALready started to migrate my code from ansible to terraform, but this is another great point to move forward with that ticket

2022-06-17

2022-06-20

Seth S avatar

Hi there. I’m looking to provision RDS aurora using cloudposse terraform-aws-rds-cluster v0.47.2

** I hope this hasn’t been asked many times…

We have a cluster setup in one region, but would like to re-use the same module(cloudposse) to provision a replicant of the DB instance in another region.

I noticed the support for mult-region (ie using global db) exists. • requires global_database_identifiercluster_type should be set to global • and I believe the 'secondary' should have the source_region provided would be the region string for the primary What I do not see is the binding for provider So I most likely do not understand how it’s handled, or it’s not required. If it’s not required - why?

The simple example I’m using as a reference is here:

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_global_cluster#new-postgresql-global-cluster

The relevant part in the cloudposse TF is here: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_global_cluster#new-postgresql-global-cluster

• I also looked at variables.tf in this module and did not find anything obvious re: region. Do I need to worry about setting the provider for input to cloudposse module?

Alex Jurkiewicz avatar
Alex Jurkiewicz

overriding module providers is a base part of the Terraform language

Alex Jurkiewicz avatar
Alex Jurkiewicz
Providers Within Modules - Configuration Language | Terraform by HashiCorpattachment image

Terraform is an open-source infrastructure as code software tool that enables you to safely and predictably create, change, and improve infrastructure.

Alex Jurkiewicz avatar
Alex Jurkiewicz

you only need this if you want to deploy both modules in a single Terraform configuration. It might be simpler to have one configuration per region

Seth S avatar

Thank you @Alex Jurkiewicz.

I did encounter provider meta argument to use the aliased provider. However, as I understand it, it would not be possible to use this when directly using the https://github.com/cloudposse/terraform-aws-rds-cluster module.

Even with one config per region:

  1. use terraform-aws-rds-clusterto setup primary
  2. use terraform-aws-rds-cluster again. to setup secondary a. this of course requires setting the correct cluster type: global + and providing global_database_identifier
  3. setting up the secondary would require setup aws provider for a different region a. this avoids using the provider meta-argument suggested In this way (2) is a different config than (1) where each instance is provisioned within the context of a multiple provider; one for each region. Does this seem like a conventional approach in general (or using the terraform-aws-rds-cluster module?)
cloudposse/terraform-aws-rds-cluster

Terraform module to provision an RDS Aurora cluster for MySQL or Postgres

Alex Jurkiewicz avatar
Alex Jurkiewicz

i think you might be confused about how this works. You can do the following in a single Terraform configuration:

provider "aws" { region = "us-west-2" }
provider "aws" { region = "eu-west-1", alias = "ew1" }
module "db_primary" {
  # module inputs...
}
module "db_secondary" {
  providers = { aws = aws.ew1 }
  # module inputs...
}

If you use two Terraform configurations, it looks like this: Configuration 1:

provider "aws" { region = "us-west-2" }
module "db_primary" {
  # module inputs...
}

Configuration 2:

provider "aws" { region = "eu-west-1" }
module "db_secondary" {
  # module inputs...
}

Does this make sense?

Seth S avatar

Yes it makes complete sense… and I discovered prior to reading your reply that provider can be also added to module blocks and not just resource blocks - as all the examples show. Thank you!

1

2022-06-21

2022-06-22

Grummfy avatar
Grummfy

anybody have some experience with aws codedeploy for ECS? is there anyway to put some paramter to the hook of the lifecycle deployment? because it’s only lambda call I don’t want to create a lambda for each of my evenironment and hook for each app

2022-06-23

Aamir Ahmad avatar
Aamir Ahmad

Did anyone here have any experience with AWS SSM automation of the quicksetup to roll it out to entire infrastructure?

2022-06-26

Катерина Кучернюк avatar
Катерина Кучернюк

Hi everyone) AWS User Group Ukraine is running a virtual AWS Tech Conference! Don’t miss it!

Dr. Werner Vogels, CTO at Amazon will be the keynote! He’ll share his ideas on Next-Gen Cloud Computing.

Also, you’ll meet 12 top speakers from AWS, AWS User Groups, AWS heroes and, sure, Ukrainian AWS professionals, who will talk about #DevOps, #data and #backend.

When? June 30 Where? Online

How to join? You can register for free or buy charity ticket. *All profit will go to Ukrainian charity funds.

Check agenda and sign up for free here: https://bit.ly/3zsQkq5 It’s going to be AWSome!

jonathan.herman avatar
jonathan.herman

may I share this in some of my other slacks?

Катерина Кучернюк avatar
Катерина Кучернюк

@jonathan.herman Yes, of course. I will be grateful if you share this event

brenden avatar
brenden

Hey All, Anyone have any experience in AWS Application Insights? Or how are you monitoring serverless microservices?

I’m using AWS Application Insights with a SAM template, mainly to take advantage of the auto instrumentation of some basic monitoring, metrics and dashboards for api gateway, lambda, state machine, sqs etc?

I’m struggling to find an option to set a notification alarm state trigger to SNS (which just sends to pagerduty)?

I don’t want to use Opscenter - there’s no integration with pager duty, as PD just supports cloudwatch. I’ve got a similar issue with using event bridge rules sinxe I think its just going to be a Application Insights Problem Detected which gives me the resource group arn which is just my serverless stack so my pager duty alert is just going to be dev_abcname has a problem instead of getting details like dev_abcname_lamba123 has been throttled which application insights has already created an alarm for I just can’t see any method to add an sns alarm action.

Snip from SAM template:

Resources:
  resResourceGroup:
    Type: "AWS::ResourceGroups::Group"
    Properties:
      Name: !Sub "${paramEnvironment}_${paramServiceName}"

  resApplicationInsights:
    Type: AWS::ApplicationInsights::Application
    Properties: 
      AutoConfigurationEnabled: true
      OpsCenterEnabled: false
      ResourceGroupName: !Sub "${paramEnvironment}_${paramServiceName}"
    DependsOn: resResourceGroup

2022-06-28

jonjitsu avatar
jonjitsu

I want to automate the creation of new AMIs when ubuntu releases new AMIs. Is there some kind of sns type subscription I can do similar to amazon linux? Or do I have to write a cron job that polls for changes to the right public ssm parameter for the particular ami I’m interested in? I know I could use event bridge for ssm parameters in my own account but I don’t think I can use it to track events happening on the ssm parameters on a third parties account like ubuntus.

Andy avatar

Are there any up to date alternatives to aws cli tools like awless and saws? both projects haven’t had much github activity recently

Darren Cunningham avatar
Darren Cunningham

cw haven’t used it much, but it’s at least semi-active

1

2022-06-29

shivanshu avatar
shivanshu

I migrated my FSx into my eks cluster by creating a PV and a PVC. But when I try to attach the pvc to my pods I get this:

Events:
  Type     Reason       Age                     From               Message
  ----     ------       ----                    ----               -------
  Normal   Scheduled    6s                      default-scheduler  Successfully assigned default/neox-0 to ip-10-0-98-61.ec2.internal
  Warning  FailedMount  <invalid> (x5 over 2s)  kubelet            MountVolume.MountDevice failed for volume "pv-new" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name fsx.csi.aws.com not found in the list of registered CSI drivers

Saw an issue around it that’s as of yet unsolved. Anyone got any ideas?

2022-06-30

Катерина Кучернюк avatar
Катерина Кучернюк

Hi everyone! There are 2 more speakers ahead.

You can still register in our amazing AWS Tech Conference and receive the recordings after event.

Register here and support freedom in Ukraine https://www.aws-user-group.com.ua/

AWS Tech Conferenceattachment image

Join AWS User Group Ukraine in a virtual AWS Tech Conference #StandWithUkraine on June, 30th! Let’s discuss with global AWS community Digital Transformation on AWS with speakers from AWS, AWS heroes and Ukrainian companies. It`s going to be AWSome!

    keyboard_arrow_up