#aws (2021-10)

anyone deployed AWS Inspector for EC2 instances in a private subnet?

RDS Event Subscriptions: I’m trying to send RDS event subscriptions to an SNS topic, then have a lambda send those as a webhook to MS Teams. but following the AWS blog post about SNS to teams webhooks, the message I see in my teams channel is:

 {"Event Source":"db-snapshot","Event Time":"2021-10-01 02:09:04.371","Identifier Link":"<https://console.aws.amazon.com/rds/home?region=us-west-2#snapshot:id=rds:dev-mpa-spa-db-01-2021-10-01-02-09>","Source ID":"rds:dev-mpa-spa-db-01-2021-10-01-02-09","Source ARN":"arn:aws:rds:us-west-2:730458288754:snapshot:rds:dev-mpa-spa-db-01-2021-10-01-02-09","Event ID":"<http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_Events.html#RDS-EVENT-0090>","Event Message":"Creating automated snapshot"}

.. I want to customize this to something more readable. The python code in the lambda looks like below, and it’s basically blindly sending out the contents of the Message object. Anybody customized their events notifications before?

def lambda_handler(event, context): 
    url = "<https://outlook.office.com/webhook/xxxxxxx>"    
    msg = {
        "text": event['Records'][0]['Sns']['Message']
    }
    encoded_msg = json.dumps(msg).encode('utf-8')
    resp = http.request('POST',url, body=encoded_msg)
    print({
        "message": event['Records'][0]['Sns']['Message'], 
        "status_code": resp.status, 
        "response": resp.data
    })

How can I test connectivity for created kafka service? It seems apps cannot use it Here is the config I used to create AWS MSK

module "kafka" {
  source = "../../external_modules/cloudposse/terraform-aws-msk-apache-kafka-cluster"
  # version = "0.6.3"
  namespace = "testnamesapce"
  stage     = "dev"
  name      = "msk"
  vpc_id    = module.vpc.vpc_id
  security_groups        = ["sg-XXXXXXXXXXXX", "sg-XXXXXXXXXXXX"]         
  subnet_ids             = ["subnet-XXXXXXXXXXXX", "subnet-XXXXXXXXXXXX"]
  kafka_version          = "2.8.0"
  number_of_broker_nodes = 2 # this has to be a multiple of the # of subnet_ids
  broker_instance_type   = "kafka.t3.small"
  broker_volume_size     = "100"
}

When I use netcat from EKS pods, I can reach Zookeeper nodes Error log from apps

2021-10-04 10:10:13 WARN  o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker z-2.dev.mjd92j.c17.kafka.us-east-1.amazonaws.com:2182 (id: -1 rack: null) disconnected
2021-10-04 10:10:13 WARN  o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker z-3.dev.mjd92j.c17.kafka.us-east-1.amazonaws.com:2182 (id: -2 rack: null) disconnected
2021-10-04 10:10:14 WARN  o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Bootstrap broker z-1.dev.mjd92j.c17.kafka.us-east-1.amazonaws.com:2182 (id: -3 rack: null) disconnected

looks like either your security groups or route tables aren’t allowing traffic

network looks fine

yeah it does

well…silly question…are you running the netcat from outside in or from an instance within the VPC?

I’m running netcat from the same pod where app is running

any chance the IAM policy associated doesn’t have the appropriate perms?

honestly though I’m just throwing out random guesses in hopes that I help you stumble upon something

yep, checking everything, just wondering what might be the root cause as in overall it looks fine

typically for me when I’ve seen unexpected timeouts it’s either networking (usually security groups) or IAM…usually with the latter though it’s not perms but rather an expired session

Hi all, hope you are doing well.. currently I am using sticky session for caching the user session..for a wordpress site.. but now I am facing some performance issues with that and I am planning to move to elasticcache redis how to achieve this ?? Any reference would be really helpful.. or any basic idea

Does anyone know if there is a cloudposse terraform package for cicd on aws that uses code commit? we have a requirement for code commit, but all the packages seem to rely on github. Thanks for any help.

Is it possible to set these prarameters for AWS MSK(Kafka)

KAFKA_ADVERTISED_LISTENERS=<PLAINTEXT://kafka-server:9092>,PLAINTEXT_<HOST://localhost:29092>

Faced with this error but for AWS MSK https://stackoverflow.com/questions/35788697/leader-not-available-kafka-in-console-producer

solved by adding auto.create.topics.enable=true

Good morning. I am not able to get AWS codepipeline to work with the “cloudposse/ecs-codepipeline/aws” module. I get an error:

Error: POST <https://api.github.com/repos/><name><app>/hooks: 404 Not Found []
│
│   with module.ecs_push_pipeline.module.github_webhooks.github_repository_webhook.default[0],
│   on .terraform/modules/ecs_push_pipeline.github_webhooks/main.tf line 7, in resource "github_repository_webhook" "default":
│    7: resource "github_repository_webhook" "default" {

Here is the main.tf for the pipeline:

module "ecs_push_pipeline" {
  source = "cloudposse/ecs-codepipeline/aws"
  version = "0.28.1"
  name                        = var.name
  namespace                   = var.namespace
  stage                       = "stage"
  image_repo_name             = var.imgRepoName
  region                      = var.aws_region
  github_oauth_token    = "<secure_token_from_github_org_oauth_creation>"
  github_webhooks_token = "<github_repo_webhook_secret>"
  repo_owner                  = var.owner
  repo_name                   = var.repo
  branch                      = "master"
  service_name                = "test-app-service"
  ecs_cluster_name            = "${var.name}-ecs-cluster"
  privileged_mode             = true
  cache_bucket_suffix_enabled = false # important: see <https://github.com/cloudposse/terraform-aws-codebuild/issues/91>
}

we have to use the deprecated but still supported aws oauth access instead of codestar connect is not an option as we are multi-region and it is not supported in ap or eu. Any help greatly appreciated.

Hai Everyone. I have a problem with cloudformation. The current status is update_rollback_failed for ECS Service. I want to update parameter which takes docker image to latest. how can I do that as I’m unable to create change set and update the template as well. Any help would be great for me

Hey everyone, so I have a simple question that I’m probably searching poorly for an answer for, but can’t find. So even just posting a link to an article about it is enough for me .

How do you request an internal ELB from say a node application?

For reference what I’m trying to do: My company has always worked in a monolith architecture, which is becoming a huge pain for us, so we want to split off some stuff into micro services. I’m trying to start with the most basic setup possible that allows us to easily add on to/move to better architecture in the future. We run a grails (Groovy/Java based back end) on elastic beanstalk instances. I want to launch the first service in a few EC2 instances (1 per environment) to start.

What I’m trying to figure out right now is the routing to keep the request from having to go out to the internet and back in to the ELB we are currently using. I know I can use a new ELB that’s internal to route requests dynamically based on URI so that we don’t have to hard code IPs/change per environment. Is there a specific internal IP/url the load balancer is always launched to? Or how can I consistently request it from a grails/node application. Am I overthinking it?

Anyone noticing issues logging into AWS using the built-in SSO? Just started?

AWS Dashboard seems to be offline for the last ~30 minutes.

Seems to be limited to us-east, I am able to login specifically to canada region

This is why you deploy SSO in any other region

is it us-east-1 or all of us-east!?

*New customers can access two Availability Zones in US West (Northern California).

is it a support ticket to get this unlocked

Does anyone have experience implementing centrailized security logging in AWS? I’ve created an Organization level cloud trail, but I need to clean up the resources in the sub accounts.

has anyone seen this before … https://twitter.com/swade1987/status/1448584648771133441?s=20

Hi everyone, I seen big swings in the creation times of RDS MYSQL being created from a snapshot. Sometimes it’s 30min, other times more than 2h in which case terraform times out. Any of you had similar experience?

Hello everyone! I’m looking at this module to create a website and host it in AWS S3. I see that it stores all the traffic (and what not) logs also in another S3 bucket. I wonder how do people usually go about analyzing (mostly reviewing) them :thinking_face:. Is it possible to store/send them to CloudWatch also/instead? So far, I can only download them to see what’s going on, but I’m sure that’s far from the trend

Hi Everyone- question about RDS/postgres and schema permissions. We have an RDS instance that was set up with a bastion host that forwards traffic from port 8887 to 5432 on the RDS instance. I’ve been accessing the instance with SSH tunneling and have successfully created and populated tables. I wanted to POC a data viz program and changed the instance to publicly accessible but only open to traffic on port 5432 for a couple of IP addresses. I can successfully connect to the instance now, but all the schemas I created while using SSH tunneling disappear. I’ve checked out permissions for the user but nothing stands out.

Is there a setting somewhere in RDS/postgres that connects port to schema privileges?

Hi all, we got EKS that holds our web app When we try to upload a evan a small, 250 KB file it is throwing 413 error response that is for file size limit exceed. The php.ini file seems to be ok. • post_max_size = 12M • upload_max_filesize = 10M • memory_limit = 128M Any ideas what else we should check?

Folks,

Issue #42 of my low-volume (once a week) newsletter “AWS Security Digest” is out.

What you will find:

• Highlight of the week
• Change since last week on AWS Managed IAM Policies
• Curated Cloud Security Newsletters
• AWS API changes
• IAM Permissions changes
• Most upvoted posts on r/AWS
• Top shared links on Twitter (by cloudsec folks)
• Most engaged Tweets from the community

Adopt a slow-tech approach by reading only essential, digest summary of what is going on in the AWS Security landscape.

With already 300+ subscribers with famous folks from @netflix and @amazon, you can’t go wrong :)

https://app.mailbrew.com/zoph/aws-security-digest-HrkhwqNrwBBk

:wave: Is there a way to get the number of bytes written to S3 per day for a given AWS acccount (bonus if you can wildcard the bucket name, e.g. foo-dev-*)?

FYI, new feature for containers on AWS launches in less than 3 hours on Twitch: https://twitter.com/iamvlaaaaaaad/status/1453754371880230918

AWS pre-announcing a new feature is… interesting.

I made an aws profile switcher in GO based on awsp if anyone wants to check it out https://github.com/pjaudiomv/awsd