#aws (2019-07)
Discussion related to Amazon Web Services (AWS)
Discussion related to Amazon Web Services (AWS)
Archive: https://archive.sweetops.com/aws/
2019-07-01
arniam:user/ad.dt is not authorized to perform: kms:DescribeKey on resource: arnkms634880740321:key/ac43ea17-741a-4347-ac04-88d42ffec899
Looks like issue with IAM policy
han it got fixed, thanks
anyone can help me to fix this error
user is trying to access the SQS and SNS and user has full access for SQS and SNS
@vishnu.shukla I think you should remove your account id
@Nikola Velkovski thanks but got fixed i just created a custom policy for KMS and attached to the user and it worked.
What I was saying that it’s not the best practice from a security point of view to expose your AWS account id
thats not my AWS id, I edited the AWS ID and user before sending
ah nice!
@Nikola Velkovski (-:
¯_(ツ)_/¯
Oh, I didn’t realize that eksctl was a joint effort.
And now it is offical documentation of aws Eks user guide (added end of May) so it is suggested setup
I thought it was only by Weaveworks + community
yeah, awesome tool for creating EKS cluster compared to AWS EKS Console (web)
2019-07-02
Anyone create an ALB to point to a KOPS k8 cluster?
dunno if that could help
are you sure than ALB are available as ingress with kops? I think remember only ELB are possible right now
because ALB needs at least a targetgroup
We’ve deployed the alb-ingresss-controller to a kops cluster
we’re currently using it with one customer, but would probably not recommend it unless absolutely necessary.
using #helmfile
2019-07-03
2019-07-05
heh, interesting read
2019-07-07
2019-07-08
Hi, I am setting up my qa env using terraform and after the ec2 is provisioned the instance is failing health check. Any suggestions on why this is happening. I am using the right port number for health check.
@laxman71290 health check from what? A load balancer or from outside?
check security groups in any case
Template format error: Unresolved resource dependencies [] in the Resources block of the template
anyone has any ideas what to look for in cloudformation template?
doesn’t matter. fat fingers putting !Ref without pointing it to a resource
2019-07-10
2019-07-11
How do you deal with multiple services running on ecs where everyone of them is configured with awsvpc networking?
deal in terms of placing services on instances which e.g. have enough available NICs and doing autoscaling
do you spread applications across multiple clusters and generally don’t care about this issue or do you have a fancy way of having a single ecs cluster with multiple services up and running?
Great link
(Via previous link)
Did anyone catch today’s keynote at AWS Summit in NYC?
Any thoughts on AWS CDK for IaC?
2019-07-12
This just went GA today: https://aws.amazon.com/cdk/
Aiming at Terraform it seems.
Hello there someone has some experience using app mesh with k8s making routing im different aws accounts? I have the idea to connect a eks pod app with another eks pod app in another account
The request should be do in aws backbone without go out to internet
2019-07-13
@hlesta You can look at using VPC peering.
Is It works with app mesh? Aws support told me that is not possible share app mesh service or be used with cross account role
Now I’m thinking that I should use cloud map to route internally in different accounts using vpc link
The current architecture accounts is
The inbound traffic enters in an account called shared services and this traffic is routed to other accounts but for example, when account A with eks app send traffic to eks in account B , it should be sent using aws backbone instead internet
2019-07-14
is there a way to map port 443 on the LB to port 80 on the web server if so where do I make that modification?
It’s on the load balancer if it’s a classic, application load balancer will be a forward to a target group, target group will specify the destination port
2019-07-15
Check out your target group configuration
2019-07-16
is using an Aurora Serverless DB cluster with a Heroku app not possible because of this limitation?
You can’t give an Aurora Serverless DB cluster a public IP address. You can access an Aurora Serverless DB cluster only from within a virtual private cloud (VPC) based on the Amazon VPC service.
that would appear to be the case. that said, you can deploy a connection proxy
e.g. https://github.com/CrunchyData/crunchy-proxy for postgres
and https://github.com/mysql/mysql-proxy for mysql
@daveyu actually you should also be able to create an NLB with a target group to an IP
I’ll try it out. Thanks!
an internet-facing NLB in a public subnet with a target group of Aurora Serverless endpoint IP addresses worked. thanks @Erik Osterman (Cloud Posse)!
a downside to this approach.. it seems the NLB’s health checks keep “waking” the cluster, causing it to scale up with no other traffic
Can you disable all healthchecks?
Passive health checks enable the load balancer to detect an unhealthy target before it is reported as unhealthy by the active health checks. You cannot disable, configure, or monitor passive health checks.
It doesn’t look like I can.
that’s too bad. but look at it this way… it’s a great way to keep your serverless database awake
Yep. It works out fine for prod env.
ALB you mean
?
nope, network load balancer
@daveyu that’s awesome! glad there’s an easy/reliable/scalable workaround
2019-07-17
hey everyone, I have AutomatedSnapshotFailure alarm for my elasticsearch
but i don have any log, any error in cluster (it it green)
does anyone know whot could be a reason of AutomatedSnapshotFailure ?
AutomatedSnapshotFailure = Insufficient data
2019-07-18
hey, is anyone using localstack’s cloudformation mocks in their local pipelines?
EKS is … hmm Decided to pop an EKS cluster up via rancher… also hmm but with extra (no ones tested this)
Are there any good TF, secure eks with calico network isolation and NLB (my god does EKS feel like its optimized for AWS’s wallet)
https://eksctl.io/ seems quite nice
in the world of “lets all make a new tool for every job”
Interesting tool; no userdata for the amis though like tf
The spot configs nice though
It’s impressive how they made cloudformation sooooooooo god damn slow
don’t tell me about it
Not being from the parallel universe the created EKS; assuming node groups are dumpable; without manually intervening via the UI like its 1999, is there a way to make sure nodes from the ASG are registered with the ILB All a chap wants is (private net) ILB > (K8 Workers / nginx ingress)
I tried the k8 ALB module; interesting but who wants to pay per ingress
if i lean back just hard enough I can see that terraform does what I want
in that it unglues the extra cloud formation so I can do stuff that wont evaporate
2019-07-19
The aws eks providers (<https://github.com/terraform-aws-modules/terraform-aws-eks>) pretty nasty; luckily the cloudposse splits it out as I like. Hate nested objects as params in modules (looking at you aws-eks), terraforms dog rough when it comes to isolating things. its nice to have a flat hierarchy of mods
2019-07-22
looking for a way - when I have a s3 new file upload, I need Jenkins to trigger a new job — CAN anyone help me out ?
2019-07-23
is there a decent tool to remove all resources within an AWS account?
there are a few of them out there
ooh nuke
thank you
there are quite a few out there..
three that are maintained
2019-07-24
one thing azure does better than aws is separate the machine from the data; I can resize a vm in azure with a slider / if the hardware under its gone to pot they’ll notify you and migrate the machine. AWS is just one big rotating middle finger to your vms when it comes to stuff like that. Change it yourself; or we’ve murdered it, hope there wasn’t anything important on there
Any folks using fargate here? Good tooling/frameworks?
Am tempted to go with airship as feels robust
But also kinda like the idea of fargate cli for unlimited staging environments
Which means the ECS is out of Terraform control
Don’t forget about limit of 50 Fargate tasks per region, per account.
Limits can be changed, fargate is just very expensive if you’re doing 50
Fargate is a lot cheaper than it used to be. Fargate limits can be increased to 1000’s if needed. I think I have it at 3000 in 2 accounts.
2019-07-26
Anyone have an idea to how to change the the ecs agent docker image directory to use something else instead of /var/lib/docker ?
How about building your own AMI using packer? Base AMI is the AWS ECS one.
we will do that for sure later
I wanted to just an ebs volume for now
The configuration has to be changed, so an AMI build is necessary.
cool, thanks
in an ECS optimized instance the agent is already pulled and started
2019-07-28
2019-07-29
Hey Folks, Trying to find some Terraform Modules related to AWS - app stream service ( for creating fleets and stacks) any help appreciated
2019-07-30
Hi All, Is there any way to setup alert for RDS Slow QRY log
You can have the Slow Query Logs go to Cloudwatch Logs and then setup something to trigger off what lands in Slow Query Log
If you forward the Cloudwatch Logs to something like ELK or Splunk or Sumo, you could then setup an alert in one of those things
At my last startup, we ended up sending the logs to ELK and sent a slack notification to have someone review if something was being weird. this helped us turn the bad reports and eventually quiet the alert
How to forward the logs to Splunk or loggly
Thank you so much Let me work on it
2019-07-31
hi guys, i have one question about authentication and authorization with EKS i’ll try explain my pain…
We have all users and groups centralized in Google G suite, i am finding some way to connect G Suite users/groups to EKS. Manage auth is painful, i want to still using Google G suite, and i am thinking and watching this solution, i don’t know whether works, so, try using AWS Cognito (as identity management) + aws-iam-authenticator into EKS.
Why AWS Cognito? G suite use SAML pattern, and with AWS Cognito has possibility to connect with SAML providers.
I am accepting solutions to get users/groups from Google G suite and use into EKS! Anyone have some experience with EKS authentication ?
AWS has an API called AssumeRoleWithSAML that should work with any IdP that supports SAML.
That said, I think you can connect G suite directly to AWS so that the G suite user name user@yourdomain is accessible via {{SessionName}} in the iam authenticator config(https://github.com/kubernetes-sigs/aws-iam-authenticator#full-configuration-format) which frees you from creating a 1-to-1 mapping of your G suite user to EKS user
Actually I’m using OneLogin as a SAML IdP instead of G Suite. All I need in the iam authenticator config is several mapRoles entries, one per IAM role
- roleARN: arn:aws:iam::000000000000:role/Developer
username: developer:{{SessionName}}
groups:
- developers
- roleARN: arn:aws:iam::000000000000:role/Admin
username: admin:{{SessionName}}
groups:
- admins
The username is translated to something like admin:<my email provided by IdP via SAML, e.g. [email protected]`
thanks @mumoshu for your reply i was lost looking many solutions, i guess your suggestion good, but i have one issue now… using aws-iam-authenticator i have to configure identity provider into IAM Console until this step ok.
Into SAML settings i have to setup some parameters like: Single sign on URL, Audience URI (SP Entity ID). I not found Single sign on URL for example after configure into IAM Console.
For these reasons i am looking at AWS Cognito… After this step above is resolved, I can configure aws-authenticator as you show me …
It depends a bit on where you want to consolidate…. For example @mumoshu uses Onelogin to handle all the identities, other people might use Okta. I use gSuite because most of my apps are based there, so when I run Gitlab in EC2, I use google omniauth.
I think Cognito is good if you have lots of apps in your EKS that require auth because you can create a Cognito pool and it will create the session tokens for them. However, those apps are already using Google as IdP, then there is little sense in duplicating.