#aws (2019-07)

aws Discussion related to Amazon Web Services (AWS)

aws Discussion related to Amazon Web Services (AWS) Archive: https://archive.sweetops.com/aws/

2019-07-31

ruan.arcega avatar
ruan.arcega

hi guys, i have one question about authentication and authorization with EKS i’ll try explain my pain…

We have all users and groups centralized in Google G suite, i am finding some way to connect G Suite users/groups to EKS. Manage auth is painful, i want to still using Google G suite, and i am thinking and watching this solution, i don’t know whether works, so, try using AWS Cognito (as identity management) + aws-iam-authenticator into EKS.

Why AWS Cognito? G suite use SAML pattern, and with AWS Cognito has possibility to connect with SAML providers.

I am accepting solutions to get users/groups from Google G suite and use into EKS! Anyone have some experience with EKS authentication ?

mumoshu avatar
mumoshu

AWS has an API called AssumeRoleWithSAML that should work with any IdP that supports SAML. That said, I think you can connect G suite directly to AWS so that the G suite user name [email protected] is accessible via {{SessionName}} in the iam authenticator config(https://github.com/kubernetes-sigs/aws-iam-authenticator#full-configuration-format) which frees you from creating a 1-to-1 mapping of your G suite user to EKS user

kubernetes-sigs/aws-iam-authenticator

A tool to use AWS IAM credentials to authenticate to a Kubernetes cluster - kubernetes-sigs/aws-iam-authenticator

mumoshu avatar
mumoshu

Actually I’m using OneLogin as a SAML IdP instead of G Suite. All I need in the iam authenticator config is several mapRoles entries, one per IAM role

- roleARN: arn:aws:iam::000000000000:role/Developer
    username: developer:{{SessionName}}
    groups:
    - developers
- roleARN: arn:aws:iam::000000000000:role/Admin
    username: admin:{{SessionName}}
    groups:
    - admins

The username is translated to something like admin:<my email provided by IdP via SAML, e.g. [email protected]`

ruan.arcega avatar
ruan.arcega

thanks @mumoshu for your reply i was lost looking many solutions, i guess your suggestion good, but i have one issue now… using aws-iam-authenticator i have to configure identity provider into IAM Console until this step ok.

Into SAML settings i have to setup some parameters like: Single sign on URL, Audience URI (SP Entity ID). I not found Single sign on URL for example after configure into IAM Console.

For these reasons i am looking at AWS Cognito… After this step above is resolved, I can configure aws-authenticator as you show me …

Blaise Pabon avatar
Blaise Pabon

It depends a bit on where you want to consolidate…. For example @mumoshu uses Onelogin to handle all the identities, other people might use Okta. I use gSuite because most of my apps are based there, so when I run Gitlab in EC2, I use google omniauth. I think Cognito is good if you have lots of apps in your EKS that require auth because you can create a Cognito pool and it will create the session tokens for them. However, those apps are already using Google as IdP, then there is little sense in duplicating.

2019-07-30

Partha avatar
Partha

Hi All, Is there any way to setup alert for RDS Slow QRY log

Jonathan Le avatar
Jonathan Le

You can have the Slow Query Logs go to Cloudwatch Logs and then setup something to trigger off what lands in Slow Query Log

Jonathan Le avatar
Jonathan Le

If you forward the Cloudwatch Logs to something like ELK or Splunk or Sumo, you could then setup an alert in one of those things

Jonathan Le avatar
Jonathan Le

At my last startup, we ended up sending the logs to ELK and sent a slack notification to have someone review if something was being weird. this helped us turn the bad reports and eventually quiet the alert

Partha avatar
Partha

How to forward the logs to Splunk or loggly

Andy avatar
Lambda function for RDS Slow Query

Lambda functions are just another great tool provided by AWS to solve issues in a modern way! Using Lambda functions, you can run a micro service without a need to have a server and think of how to configure and maintain it! There are lots of use cases for Lambda functions; here I used it to implement a service which sends alerts in case there is a slow query running in RDS. Of course slow queries are important for developers as it helps them to debug better and improve performance of the application. You can find the code here but there are some other things to be considered: As you may know, there are some ways to trigger a Lambda function. In this case, using CloudWatch Events to schedule it periodically makes sense. The lamda function should have some permissions to get RDS Logs and send alerts using SNS. To find out how to define required rules, please see this AWS documentation. You are also asked to do this when creating Lambda function. There is a parameter named ‘distinguisher’ which is actually the keyword specifying the occurrence of slow query. For ‘Postgresql’ RDS it can be ‘ Parameters Group in RDS should be configured to log slow queries. To know how to do this please see AWS documentation or this guide:Enabling slow query log on Amazon RDS

Partha avatar
Partha

Thank you so much Let me work on it

2019-07-29

Sharanya avatar
Sharanya

Hey Folks, Trying to find some Terraform Modules related to AWS - app stream service ( for creating fleets and stacks) any help appreciated

2019-07-28

2019-07-26

PePe avatar

Anyone have an idea to how to change the the ecs agent docker image directory to use something else instead of /var/lib/docker ?

github140 avatar
github140

How about building your own AMI using packer? Base AMI is the AWS ECS one.

PePe avatar

we will do that for sure later

PePe avatar

I wanted to just an ebs volume for now

github140 avatar
github140

The configuration has to be changed, so an AMI build is necessary.

PePe avatar

cool, thanks

PePe avatar

in an ECS optimized instance the agent is already pulled and started

2019-07-24

chrism avatar
chrism

one thing azure does better than aws is separate the machine from the data; I can resize a vm in azure with a slider / if the hardware under its gone to pot they’ll notify you and migrate the machine. AWS is just one big rotating middle finger to your vms when it comes to stuff like that. Change it yourself; or we’ve murdered it, hope there wasn’t anything important on there

joshmyers avatar
joshmyers

Any folks using fargate here? Good tooling/frameworks?

joshmyers avatar
joshmyers

Am tempted to go with airship as feels robust

joshmyers avatar
joshmyers

But also kinda like the idea of fargate cli for unlimited staging environments

joshmyers avatar
joshmyers

Which means the ECS is out of Terraform control

ciastek avatar
ciastek

Don’t forget about limit of 50 Fargate tasks per region, per account.

maarten avatar
maarten

Limits can be changed, fargate is just very expensive if you’re doing 50

Steven avatar
Steven

Fargate is a lot cheaper than it used to be. Fargate limits can be increased to 1000’s if needed. I think I have it at 3000 in 2 accounts.

:--1:3

2019-07-23

mmarseglia avatar
mmarseglia

is there a decent tool to remove all resources within an AWS account?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

there are a few of them out there

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
gruntwork-io/cloud-nuke

A tool for cleaning up your cloud accounts by nuking (deleting) all resources within it - gruntwork-io/cloud-nuke

mmarseglia avatar
mmarseglia

ooh nuke

mmarseglia avatar
mmarseglia

thank you

mmarseglia avatar
mmarseglia

there are quite a few out there..

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

three that are maintained

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
AWS launches a new tool to help you optimize your EC2 resources – TechCrunch attachment image

Here is a small but potentially handy update if you’re an AWS EC2 user. The company today launched a new feature called “EC2 Resource Optimization Recommendations,” which does exactly what the name promises. It’s not flashy, it’s not especially exciting, but it may jus…

2019-07-22

Sharanya avatar
Sharanya

looking for a way - when I have a s3 new file upload, I need Jenkins to trigger a new job — CAN anyone help me out ?

Suresh avatar
Suresh

create a SNS topic for s3 events and create HTTPS subscription on sns

1

2019-07-19

chrism avatar
chrism

The aws eks providers (<https://github.com/terraform-aws-modules/terraform-aws-eks>) pretty nasty; luckily the cloudposse splits it out as I like. Hate nested objects as params in modules (looking at you aws-eks), terraforms dog rough when it comes to isolating things. its nice to have a flat hierarchy of mods

:--1:2

2019-07-18

Maciek Strömich avatar
Maciek Strömich

hey, is anyone using localstack’s cloudformation mocks in their local pipelines?

chrism avatar
chrism

EKS is … hmm Decided to pop an EKS cluster up via rancher… also hmm but with extra (no ones tested this)

Are there any good TF, secure eks with calico network isolation and NLB (my god does EKS feel like its optimized for AWS’s wallet)

chrism avatar
chrism

https://eksctl.io/ seems quite nice

chrism avatar
chrism

in the world of “lets all make a new tool for every job”

chrism avatar
chrism

Interesting tool; no userdata for the amis though like tf

chrism avatar
chrism

The spot configs nice though

chrism avatar
chrism

It’s impressive how they made cloudformation sooooooooo god damn slow

Maciek Strömich avatar
Maciek Strömich

don’t tell me about it

chrism avatar
chrism

Not being from the parallel universe the created EKS; assuming node groups are dumpable; without manually intervening via the UI like its 1999, is there a way to make sure nodes from the ASG are registered with the ILB All a chap wants is (private net) ILB > (K8 Workers / nginx ingress)

chrism avatar
chrism

I tried the k8 ALB module; interesting but who wants to pay per ingress

chrism avatar
chrism

if i lean back just hard enough I can see that terraform does what I want

chrism avatar
chrism

in that it unglues the extra cloud formation so I can do stuff that wont evaporate

2019-07-17

Maxim Tishchenko avatar
Maxim Tishchenko

hey everyone, I have AutomatedSnapshotFailure alarm for my elasticsearch but i don have any log, any error in cluster (it it green) does anyone know whot could be a reason of AutomatedSnapshotFailure ? AutomatedSnapshotFailure = Insufficient data

2019-07-16

sarkis avatar
sarkis
Using AWS Application Load Balancer and Network Load Balancer with EC2 Container Service attachment image

Amazon Web Services recently released new second generation load balancers: Application Load Balancer (ALB), and Network Load Balancer…

daveyu avatar
daveyu

is using an Aurora Serverless DB cluster with a Heroku app not possible because of this limitation?
You can’t give an Aurora Serverless DB cluster a public IP address. You can access an Aurora Serverless DB cluster only from within a virtual private cloud (VPC) based on the Amazon VPC service.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that would appear to be the case. that said, you can deploy a connection proxy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
CrunchyData/crunchy-proxy

PostgreSQL Connection Proxy by Crunchy Data (beta) - CrunchyData/crunchy-proxy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
mysql/mysql-proxy

MySQL Proxy is a simple program that sits between your client and MySQL server(s) and that can monitor, analyze or transform their communication. Its flexibility allows for a wide variety of uses, …

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@daveyu actually you should also be able to create an NLB with a target group to an IP

daveyu avatar
daveyu

I’ll try it out. Thanks!

daveyu avatar
daveyu

an internet-facing NLB in a public subnet with a target group of Aurora Serverless endpoint IP addresses worked. thanks @Erik Osterman (Cloud Posse)!

1
daveyu avatar
daveyu

a downside to this approach.. it seems the NLB’s health checks keep “waking” the cluster, causing it to scale up with no other traffic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Can you disable all healthchecks?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


Passive health checks enable the load balancer to detect an unhealthy target before it is reported as unhealthy by the active health checks. You cannot disable, configure, or monitor passive health checks.

daveyu avatar
daveyu

It doesn’t look like I can.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s too bad. but look at it this way… it’s a great way to keep your serverless database awake

daveyu avatar
daveyu

Yep. It works out fine for prod env.

PePe avatar

ALB you mean

PePe avatar

?

daveyu avatar
daveyu

nope, network load balancer

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@daveyu that’s awesome! glad there’s an easy/reliable/scalable workaround

2019-07-15

me1249 avatar
me1249

Check out your target group configuration

2019-07-14

David avatar
David

is there a way to map port 443 on the LB to port 80 on the web server if so where do I make that modification?

Alex Siegman avatar
Alex Siegman

It’s on the load balancer if it’s a classic, application load balancer will be a forward to a target group, target group will specify the destination port

2019-07-13

dalekurt avatar
dalekurt

@hlesta You can look at using VPC peering.

hlesta avatar
hlesta

Is It works with app mesh? Aws support told me that is not possible share app mesh service or be used with cross account role

hlesta avatar
hlesta

Now I’m thinking that I should use cloud map to route internally in different accounts using vpc link

hlesta avatar
hlesta

The current architecture accounts is

The inbound traffic enters in an account called shared services and this traffic is routed to other accounts but for example, when account A with eks app send traffic to eks in account B , it should be sent using aws backbone instead internet

2019-07-12

Alex Siegman avatar
Alex Siegman

This just went GA today: https://aws.amazon.com/cdk/

AWS Cloud Development Kit - Amazon Web Services

AWS Cloud Development Kit (CDK) is a software development framework to model and provision your cloud application resources using familiar programming languages.

Alex Siegman avatar
Alex Siegman

Aiming at Terraform it seems.

hlesta avatar
hlesta

Hello there someone has some experience using app mesh with k8s making routing im different aws accounts? I have the idea to connect a eks pod app with another eks pod app in another account

hlesta avatar
hlesta

The request should be do in aws backbone without go out to internet

2019-07-11

Maciek Strömich avatar
Maciek Strömich

How do you deal with multiple services running on ecs where everyone of them is configured with awsvpc networking?

Maciek Strömich avatar
Maciek Strömich

deal in terms of placing services on instances which e.g. have enough available NICs and doing autoscaling

Maciek Strömich avatar
Maciek Strömich

do you spread applications across multiple clusters and generally don’t care about this issue or do you have a fancy way of having a single ecs cluster with multiple services up and running?

maarten avatar
maarten
ECS ENI Density Increases · Issue #7 · aws/containers-roadmap

Instances running in awsvpc networking mode will have greater allotments of ENIs allowing for greater Task densities.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Great link

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Elastic Network Interface Trunking - Amazon Elastic Container Service

Each Amazon ECS task that uses the awsvpc network mode receives its own elastic network interface (ENI), which is attached to the container instance that hosts it. There is a default limit to the number of network interfaces that can be attached to an Amazon EC2 instance, and the primary network interface counts as one. For example, by default a

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(Via previous link)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
[EKS]: Next Generation AWS VPC CNI Plugin · Issue #398 · aws/containers-roadmap

We are working on the next version of the Kubernetes networking plugin for AWS. We&#39;ve gotten a lot of feedback around the need for adding Kubenet and support for other CNI plugins in EKS. This …

fast_parrot2
dalekurt avatar
dalekurt

Did anyone catch today’s keynote at AWS Summit in NYC?

dalekurt avatar
dalekurt

Any thoughts on AWS CDK for IaC?

2019-07-08

laxman71290 avatar
laxman71290

Hi, I am setting up my qa env using terraform and after the ec2 is provisioned the instance is failing health check. Any suggestions on why this is happening. I am using the right port number for health check.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@laxman71290 health check from what? A load balancer or from outside?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

check security groups in any case

Maciek Strömich avatar
Maciek Strömich
Template format error: Unresolved resource dependencies [] in the Resources block of the template

anyone has any ideas what to look for in cloudformation template?

Maciek Strömich avatar
Maciek Strömich

doesn’t matter. fat fingers putting !Ref without pointing it to a resource

2019-07-07

2019-07-03

2019-07-02

pericdaniel avatar
pericdaniel

Anyone create an ALB to point to a KOPS k8 cluster?

PePe avatar
EKSworkshop.com attachment image

Amazon EKS Workshop

:--1:1
PePe avatar

dunno if that could help

Issif avatar
Issif

are you sure than ALB are available as ingress with kops? I think remember only ELB are possible right now

Issif avatar
Issif

because ALB needs at least a targetgroup

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’ve deployed the alb-ingresss-controller to a kops cluster

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re currently using it with one customer, but would probably not recommend it unless absolutely necessary.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

using #helmfile

2019-07-01

vishnu.shukla avatar
vishnu.shukla

arnawsiam:user/ad.dt is not authorized to perform: kms:DescribeKey on resource: arnawskms:us-east-1:634880740321:key/ac43ea17-741a-4347-ac04-88d42ffec899

aaratn avatar
aaratn

Looks like issue with IAM policy

vishnu.shukla avatar
vishnu.shukla

han it got fixed, thanks

vishnu.shukla avatar
vishnu.shukla

anyone can help me to fix this error

vishnu.shukla avatar
vishnu.shukla

user is trying to access the SQS and SNS and user has full access for SQS and SNS

Nikola Velkovski avatar
Nikola Velkovski

@vishnu.shukla I think you should remove your account id

vishnu.shukla avatar
vishnu.shukla

@Nikola Velkovski thanks but got fixed i just created a custom policy for KMS and attached to the user and it worked.

Nikola Velkovski avatar
Nikola Velkovski

What I was saying that it’s not the best practice from a security point of view to expose your AWS account id

vishnu.shukla avatar
vishnu.shukla

thats not my AWS id, I edited the AWS ID and user before sending

Nikola Velkovski avatar
Nikola Velkovski

ah nice!

Maciek Strömich avatar
Maciek Strömich

@Nikola Velkovski (-:

Nikola Velkovski avatar
Nikola Velkovski

¯_(ツ)_/¯

ruan.arcega avatar
ruan.arcega
EKSworkshop.com attachment image

Amazon EKS Workshop

:--1:3
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh, I didn’t realize that eksctl was a joint effort.

omerfsen avatar
omerfsen

And now it is offical documentation of aws Eks user guide (added end of May) so it is suggested setup

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I thought it was only by Weaveworks + community

vitaly.markov avatar
vitaly.markov

yeah, awesome tool for creating EKS cluster compared to AWS EKS Console (web)

2
    keyboard_arrow_up