#aws (2021-11)

aws Discussion related to Amazon Web Services (AWS)

aws Discussion related to Amazon Web Services (AWS)

Archive: https://archive.sweetops.com/aws/

2021-11-01

2021-11-02

Yusuf avatar

Hi everyone, I was wondering if I can categorize my logs in cloud watch based on a filter. I.e having somewhere to look at all logs that has a tag “failed-error”. If there isn’t a way to do this with cloud watch kindly suggest any third-party solution you know. Thanks

Max avatar

Hey, you can write queries in cloudwatch insights to categorize your logs https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html

Analyzing log data with CloudWatch Logs Insights - Amazon CloudWatch Logs

Use CloudWatch Logs Insights and CloudWatch Logs to search and analyze your log data.

2021-11-03

Or Azarzar avatar
Or Azarzar

Great technical blog by Jonathan Rau, our CISO, about securing your AWS EC2 Instances with Microsoft Defender

https://blog.lightspin.io/microsoft-defender-for-endpoint-on-aws

Microsoft Defender for Endpoint on AWS: Part 1attachment image

This blog serves as the first part of our series that deep dives into Microsoft Defender for Endpoint on AWS.

Sarah Donehower avatar
Sarah Donehower

Hi Everyone! I’m using a bastion host on AWS and am trying to add a new user. The user generates a new RSA key pair. I add the public key to the authorized_keys file on the host. But they still get Permission denied (publickey,gssapi-keyex,gssapi-with-mic). I’ve tried myself and it still fails. Its not a file/directory permissions issue because I can currently connect with the key/pair I generated a couple of months ago. How can I troubleshoot this further?

managedkaos avatar
managedkaos

on the local system, have they set the permissions on the key to 400 or 600 so the private key is only readable by the user?

also, (if you haven’t already) try conencting with ssh -vvv to get lots of debug out put. the answer might be in there as well.

Sarah Donehower avatar
Sarah Donehower

the permissions are correct.

my new findings: I can ssh into the bastion instance with the new credentials. But when I use the port forwarding command to create the ssh tunnel… that generates the permission denied error

1
Sarah Donehower avatar
Sarah Donehower

Any thoughts on why ssh works, but ssh with port forwarding fails depending on they key used?

1
managedkaos avatar
managedkaos

can you share your port forwarding command?

Typically, i use it like this:

ssh -o ExitOnForwardFailure=yes -f -N -L 5432:database-hostname.us-west-2.rds.amazonaws.com:5432 jump-host.example.com
Sarah Donehower avatar
Sarah Donehower
ssh -N -i id_rsa_aws_bastion  -L 8887:test-floodlight-db.ccoajwmnofma.us-east-1.rds.amazonaws.com:5432 [email protected] -v
managedkaos avatar
managedkaos

yep, just eyeballing it that looks good to me

Sarah Donehower avatar
Sarah Donehower

Ok. Well, when I originally added the new public keys to the host, I was connecting as root through the console’s EC2 Instant connect (the in-browser terminal for an ec2 instance). Then I ssh’d into the terminal as ec2-user and updated the authorized_keys file… and it worked.

Does that make sense? Can the perms of the user updating the authorized_keys file change the results? Or is the console connection a bad choice?

Sarah Donehower avatar
Sarah Donehower

either way, soooooo annoying.

Sarah Donehower avatar
Sarah Donehower

Thanks for your help @managedkaos!

1

2021-11-04

Justin D avatar
Justin D

Hello,

Sadly I’m working in an environment where most of our resources are not controlled with IaC; or else this would be relatively simple. I’m needing to keep a subset of tags (~5) from an EC2 instance in sync with its associated volumes and snapshots. An easy example of what I would be looking for is if a billing tag changed in 6 months, I would need to replicate out to its volumes and snapshots.

We already have solution with some custom coded automation, but wanted to hear if there are any projects or AWS tools that would provide this type of functionality.

loren avatar

might do more than you need, but there’s this one, https://github.com/GorillaStack/auto-tag

GitHub - GorillaStack/auto-tag: Automatically tag AWS resources on creation, for cost assignmentattachment image

Automatically tag AWS resources on creation, for cost assignment - GitHub - GorillaStack/auto-tag: Automatically tag AWS resources on creation, for cost assignment

Justin D avatar
Justin D

Oh, nice!

RB avatar

cloud custodian has a policy for this too

RB avatar

it’s open source and free so you’d have to come up with your own custodian policy

beaur97 avatar
beaur97

Does anyone have experience adding splunk universal forwarder to elastic beanstalk? Running into an “issue” that I’m stumped on and not sure where to ask anymore

2021-11-05

AugustasV avatar
AugustasV

Any ideas how to turn off aws cloudwatch alarms at specific hours?

Darren Cunningham avatar
Darren Cunningham

Have a CloudWatch Rule that runs a Lambda that toggles alarms off/on as desired

AugustasV avatar
AugustasV

something like math expresion? Or cloudwatch Event Rules?

ikar avatar

this is what we use:

ubuntu@Tony04:~$ crontab -l
#Ansible: mongo db backup
0 3 * * * aws cloudwatch disable-alarm-actions --alarm-names Tony04-CPU; ~/backup.daily.sh

#Ansible: re-enable CPU alarm few minutes after backup
0 4 * * * aws cloudwatch enable-alarm-actions --alarm-names Tony04-CPU
Darren Cunningham avatar
Darren Cunningham

I was referring to CloudWatch Rules as you can use them like cron jobs that trigger jobs (Lambda, etc)

AugustasV avatar
AugustasV

@ikar thank you will go this way, but will trigger those using lambda functions. Filtering through tags

2021-11-07

2021-11-08

AugustasV avatar
AugustasV

If I will use math expression on aws cloudwatch alarms, does it mean that I will increase number of alarm querying per minute by doing that?

Kian Sajjadi avatar
Kian Sajjadi

Has anyone ever had an issue where you can access an object in a public bucket, but when an ecs container tries to access that same object its access denied?

Todd Harpersberger avatar
Todd Harpersberger

You may want to make sure there is a VPC endpoint so that the ecs container doesnt try to traverse the internet for it.

Kian Sajjadi avatar
Kian Sajjadi

Is there a reason that it shouldn’t have access to a public object in a bucket on the same account? When I ssh into the fargate container I can curl objects on buckets that aren’t on the same account

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

From my Twitter, a thing that may be of interest to y’all//twitter.com/iamvlaaaaaaad/status/1457676356033171465)

Zach avatar

Can we see some evidence on this claim that running kubernetes costs $1M annual please

3
Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

My experience That’s it!

Zach avatar

based on what though? the resources? training? the labor? surely you didn’t just pluck this number out of the air

1
Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

You have to train people, you have to migrate to k8s. After that you have to maintain and operate k8s. All that comes with missed opportunity costs.

I added all those costs

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

I don’t have an Excel spreadsheet for it, it’s not that precise

bradym avatar
The 17 Ways to Run Containers on AWSattachment image

There may be 50 ways to leave your lover, but there are 17 ways to run containers on AWS. I will now enumerate them.

17 More Ways to Run Containers on AWSattachment image

It started as a meme, but it turned into a real post on “The 17 Ways to Run Containers on AWS.” Apparently my list continues to be a source of amusement

1
Balazs Varga avatar
Balazs Varga

is it possible that If I have cert arn with a.com and *.a.com cert and I would like to use it with nlb then nlb will use just the 1st cert (a.com) ?

Mohammed Yahya avatar
Mohammed Yahya

awesome GraphQL for AWS https://github.com/cloudgraphdev/cli

GitHub - cloudgraphdev/cli: The GraphQL API for AWSattachment image

The GraphQL API for AWS. Contribute to cloudgraphdev/cli development by creating an account on GitHub.

1
loren avatar

very cool

GitHub - cloudgraphdev/cli: The GraphQL API for AWSattachment image

The GraphQL API for AWS. Contribute to cloudgraphdev/cli development by creating an account on GitHub.

Eugene avatar

Sorry if this is a naive question but does this try to address the pitfalls of using AWS’s advertised GraphQL solution of AppSync? And if so what are they?

loren avatar

no, it’s more of an inventory query solution for cloud resources, based on GraphQL syntax

loren avatar


CloudGraph lets any cloud professional answer questions like, “What KMS keys do I have in us-west-2?”, “How much am I paying for my environment?”, and, “What resources in my production environment aren’t tagged correctly?” in the time it takes to put on the pants you should already be wearing for your next zoom meeting. Ask any question about your cloud environments, and get back answers instantly in a single place with a single standardized API, for all of your cloud providers.

loren avatar

i’m not affiliated, it just looks really interesting

Eugene avatar

Oh I see it’s just using GraphQL to access thanks

Mohammed Yahya avatar
Mohammed Yahya

Example: you can create frontend - backend webapp to query your AWS inventories using GraphQL ( new technology) VS RESt API Imagine how this web app could be used:

• cost estimations

• managment for Landing zone

• Security Checks

davidvasandani avatar
davidvasandani

General PSA when working with Fargate (via AWS Support):
Below is the summary of our conversation:

You had contacted us since you wanted to know whether STOPSIGNAL or SIGTERM sent to the running container is sent when you stop a task. While on chat, I informed you that unfortunately this feature is not available through ECS yet. The STOPSIGNAL is not supported by ECS and there is no way to specify any other custom signal for StopTask.

We are aware of this limitation and there is currently a feature request to have this available with ECS. However, there is no ETA provided as this requires extensive testing and several approvals before making any changes into production environment. You can keep an eye on the GitHub issue on the containers-roadmap repository [1] for future updates regarding this [2] - https://github.com/aws/containers-roadmap/issues/359

Later, you asked on how common is it for AWS to not have spare capacity and not replace a terminated spot task. (SERVICE_TASK_PLACEMENT_FAILURE event). I informed you that this is a known issue and there is a Feature Request open mentioning the use of “on demand” instances when spot is not available. https://github.com/aws/containers-roadmap/issues/773

Add support for `--stop-signal` `docker run` flag · Issue #359 · aws/containers-roadmapattachment image

Hi, When running Centos 7 based container with systemd there's a graceful systemd service shutdown issue. If I run container (not in ECS) with option "docker run –stop-signal=$(kill -l RT…

[ECS] : Capacity Strategy to Fall back to OD only When No More Spot Capacity Available · Issue #773 · aws/containers-roadmapattachment image

Community Note Please vote on this issue by adding a reaction to the original issue to help the community and maintainers prioritize this request Please do not leave "+1" or "me to…

1

2021-11-09

Andrea Cavagna avatar
Andrea Cavagna

Hi everyone! is there someone here who partecipate to the AWS reInvent?

It could be a great idea to have the opportunity to meet each other!

Personally I’m happy to partecipate, and also, I will present with @Eric Villa Leapp at the Open-Source lounge on Monday at 5 PM

Hope to see you all

1
Andrea Cavagna avatar
Andrea Cavagna

@Erik Osterman (Cloud Posse) will you partecipate?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yep! I will be there

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’ll DM you my mobile

Andrea Cavagna avatar
Andrea Cavagna

Awesome!

robwdux_sweetops avatar
robwdux_sweetops

I’ll be there too

1
Rhys Davies avatar
Rhys Davies

Hi all, I’m using ECS’s Docker Compose integration and everything is working well EXCEPT the output from the docker compose command is so verbose and noisy that even CircleCI wants me to write it to a file and inspect it after because it goes over the 100MB limit that there log output supports.

Rhys Davies avatar
Rhys Davies

I’ve also noted that docker compose --loglevel doesn’t work? Has anyone else experienced this?

Rhys Davies avatar
Rhys Davies

I guess my general question isn’t entirely AWS related but I do wonder how other infra/ops people quiet logs in this sort of situation? Are y’all just piping to grep or awk/sed?

Rhys Davies avatar
Rhys Davies

It specifically the part where it’s creating resources in ECS, so the LogGroup, ECS Service, CloudMap etc. takes some time and each update writes another 20-30 lines to the screen - I don’t want to lose the ability to track progress of my CI pipe, but I also can’t find any flags to shut up ECS/Docker Compose

2021-11-10

Almondovar avatar
Almondovar

Hi all, this is an EKS related question, can someone tell me how to trigger an instance refresh? is it EKS, AMI version or something on the ASG has to change to trigger it? we are looking for the minimum impact so i guess eks update is out of the question, thanks!

z0rc3r avatar

kubectl drain $node, then delete instance in question and let autoscaling group spin up a fresh worker node

Almondovar avatar
Almondovar

Thank you very much Igor!

2021-11-11

Almondovar avatar
Almondovar

Hi all, which one would you use between Cloudcraft vs. Lucidchart to have automatically updated charts as we edit the infra via console or terraform? thanks!

2021-11-12

2021-11-15

Eugene avatar

A bit of a networking question, but would be glad if someone could give their take on it. It seems that AWS recently has a way to use NLB with ALB so that you can take advantage of things like SSL termination on the ALB while still using the NLB to do non HTTP traffic (https://aws.amazon.com/blogs/networking-and-content-delivery/application-load-balancer-type-target-group-for-network-load-balancer/). If I didn’t have any non HTTP needs, is there a purpose to use an NLB? The only reason we used an NLB is that our company wanted to use API Gateway and wanted to call an ECS Service behind an ALB, but API Gateway only allows NLBs to use AWS Private Link to create a connection. Otherwise, the ALB has to be public if we want to use API Gateway to reroute it but it ruins the point, since the ALB is still exposed to the public.

Application Load Balancer-type Target Group for Network Load Balancer | Amazon Web Servicesattachment image

Application Load Balancer (ALB) is a fully managed layer 7 load balancing service that load balances incoming traffic across multiple targets, such as Amazon EC2 instances. ALB supports advanced request routing features based on parameters like HTTP headers and methods, query string, host and path based routing. ALB also offloads important capabilities including TLS termination, […]

Eugene avatar

I guess I’m confused why AWS doesn’t provide way to use a private ALB through API Gateway directly.

Application Load Balancer-type Target Group for Network Load Balancer | Amazon Web Servicesattachment image

Application Load Balancer (ALB) is a fully managed layer 7 load balancing service that load balances incoming traffic across multiple targets, such as Amazon EC2 instances. ALB supports advanced request routing features based on parameters like HTTP headers and methods, query string, host and path based routing. ALB also offloads important capabilities including TLS termination, […]

Michael Warkentin avatar
Michael Warkentin

Anyone know if there’s a way to set up bucket replication on the s3 bucket created by the AWS-cloudfront-s3-cdn module? A bit confused over how I can ensure the failover bucket stays up to date.

Yage Hu avatar
Yage Hu

I’m evaluating migrating my CI pipelines away from CodePipeline and CodeBuild. Any recommendations? Are there hosted CI that is cheaper or comparable to CodePipeline?

managedkaos avatar
managedkaos

depending on where your code is hosted, I would suggest GitHub Actions, GitLab CI, or Bitbucket Pipelines, These services keep your CI close to your code and the configuration for this can be versions right along with the application code. Not sure on exact pricing, but the convenience alone is worth quite a bit in my book.

1

2021-11-16

Yusuf avatar

Hi there, Does anyone know how to set AWS Managed Apache Kafka data retention to forever?

Bhavik Patel avatar
Bhavik Patel

Hi everyone,

Hoping to get some advice from some of you. I’m currently migrating our infrastructure off of Heroku into AWS. Our dev shop has one monolith and a handful of micro-services that are build with Django. The idea here is to move from a monolith to micro-services. Each application also has a celery worker and beat. I’m planning on migrating over to AWS using Fargate and initially I will be supporting the celery worker and beat with a spot instance. Has anyone had experience doing a similar migration? The two pieces that i’m trying to iron out is how we will be implementing continuous deployment and how service to service communication will work for applications that will only interact within the VPC. From what I’m reading. Codedeploy is used to help with the blue/green deployments via traffic shifting from one image to the other and AppMesh is a service mesh that will enable me to do do service to service communication. Does Codedeploy work with AppMesh?

Yusuf avatar

Quick answer to your question. Yes AppMesh works with AWS.

Bhavik Patel avatar
Bhavik Patel

Sorry, I don’t think my question was clear. Does Codedeploy work with AppMesh? Both are AWS services.

Bhavik Patel avatar
Bhavik Patel

https://github.com/aws/aws-app-mesh-roadmap/issues/72#issuecomment-702894797

This issue made me wonder if it was possible to use both together

[AppMesh][request]: Canary deployment · Issue #72 · aws/aws-app-mesh-roadmapattachment image

Tell us about your request Schema for automatic virtual router shifting a percentage of production traffic from one virtual node to a new one and monitor rollback alarms. Which integration(s) is th…

Yusuf avatar

It’s not supported out of the box but there are ways around it.

Bhavik Patel avatar
Bhavik Patel

@Yusuf Thank you! Seems like using CodeDeploy along with AppMesh is a bit redundant if I was planning on using it for Blue/Green deployments

Balazs Varga avatar
Balazs Varga

hello all, We are facing a really strange issue. We have k8s cluster and have spot fleet for nodes. we have a java app that can start on intel all time and cannot start few times on amd epyc… Any idea ?

Balazs Varga avatar
Balazs Varga

instance types are m5 vs m5a

Max avatar

HI, any logs you have to share ?

Balazs Varga avatar
Balazs Varga

will try to collect.

Balazs Varga avatar
Balazs Varga

need a little time to reproduce.

2021-11-17

Balazs Varga avatar
Balazs Varga

aurora serverless? I read it somewhere the volume cluster behind this service is way slower than gp2… is that right ? If yes, then can we somehow move to gp2 or just w/o serverless?

Ray Myers avatar
Ray Myers

Anyone using Cloud Custodian? As we’re moving to more accounts, we’re finding it desirable to put all the policy lambdas in one account. However, we then run into an EventBridge rule limit which we’re requesting be raised. It also seems that we could change the way the policies are packaged to not require so many lambdas. Seeing if anyone has run into this, or suggests an alternative to Cloud Custodian such as AWS Config.

julie avatar

Do you have AWS Control Tower enabled? are you using AWS Organizations? These tools allow you to manage many accounts under an “organizational unit” or under the entire “org”. I recommend it as you can create SCPs

Ray Myers avatar
Ray Myers

@julie Yes we are using Control Tower and Orgs. Service Control Policies look like they could handle some of our rules - especially tagging. Probably not everything but worth looking into thanks!

julie avatar

dont quote me, but I think CT and Orgs use AWS config rules under the hood. I love using Config rules so that might be something to look into as well.

2021-11-18

Zach avatar

Marc uses it heavily [e] oh lol I was in wrong slack. Marc isn’t in this one

RB avatar

cloud custodian uses aws config for inventory information. I’ve never run into the event bridge limitation tho. have you checked out the cloud custodian gitter?

kapilt and his posse are super helpful and if it’s a limitation they will know immediately

Ray Myers avatar
Ray Myers

I will try the gitter, good idea. All the policies run in scheduled mode, hence the eventbridge rules, but haven’t tried Config as a data source or the Config Rule execution mode.

Balazs Varga avatar
Balazs Varga

does anybody has good config for aurora mysql serverless? Or the default innodb and mysql conf is fine ?

jose.amengual avatar
jose.amengual

Aurora does nor behave the same as another mysql

jose.amengual avatar
jose.amengual

I never had to change any innodb settions or anything

Balazs Varga avatar
Balazs Varga

and do you use serverless in prod? We are facing with slow query issue.

jose.amengual avatar
jose.amengual

when we tested serverless it was slow but it all came down to the cpu size etc

jose.amengual avatar
jose.amengual

and that is when they get you with the chargers

jose.amengual avatar
jose.amengual

it gets pretty expensive quick

Balazs Varga avatar
Balazs Varga

Ok thanks. Have you ever tested v2 serverless?

jose.amengual avatar
jose.amengual

no

Balazs Varga avatar
Balazs Varga

thanks

2021-11-19

Michael Warkentin avatar
Michael Warkentin
Lambda function URLs - AWS Lambda

With Lambda function URLs, you can invoke your Lambda function through a dedicated HTTP(S) endpoint without having to integrate with other AWS services.

1
Zach avatar

oh wow built in ALB basically

Lambda function URLs - AWS Lambda

With Lambda function URLs, you can invoke your Lambda function through a dedicated HTTP(S) endpoint without having to integrate with other AWS services.

Zach avatar

can we alias a r53 record to it I wonder?

Michael Warkentin avatar
Michael Warkentin

Looks like the Lambda Function URL functionality has been disabled for now.

Zach avatar

must have been released too early

2021-11-20

2021-11-21

Patrick Jahns avatar
Patrick Jahns
Hunters Research: Is AWS Recycling your Access Keys?attachment image

Hunters’ research team discovered that temporary AWS API access key IDs, issued by AWS, are not unique and could repeat, which can impair AWS security tools detection capabilities.

2021-11-22

2021-11-23

Almondovar avatar
Almondovar

Hi colleagues, we are using aws iot service and we have been wondering whitch is the best place to store securely online some certificates without the need to maintain servers (like hashicorp vault etc.) Can we do it securely with some s3 buckets for example? thanks!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Almondovar are you joining us for office hours today? #office-hours

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we can discuss

mike.sh avatar
mike.sh

hey people, I was asked if I could create regular AWS EKS control plane and make local, on premises bare-metals join that EKS – any thoughts, experience, ideas ?

(last time I did EKS it was ~1.15, I’ve just used terraform with eks module, run terraform apply - it created control plane, worker group within my vpc and that was it)

Zach avatar


Amazon EKS Anywhere is a new deployment option that helps you create and operate Kubernetes clusters on your own on-premises infrastructure using VMware vSphere starting today, with support for bare metal deployments coming in 2022.
https://aws.amazon.com/eks/eks-anywhere/

Amazon EKS Anywhere – Amazon Web Services

Create and operate Kubernetes clusters on your own infrastructure

mike.sh avatar
mike.sh

yeah, but can I join EKS Anywhere workers to EKS cluster ? I do not want to manage control plane

Zach avatar

well I thought thats what it did but perhaps not

mike.sh avatar
mike.sh

yeah, I was googling and docsing week long, but havent found anything useful. in the end it makes sense, b/c how would EKS support bare-metals (and probably all other options across all the world) options and features? I guess AWS says ‘we do only our workers, get lost’

mike.sh avatar
mike.sh

maybe they will add support joining workers populated from EKS Distro to the regular EKS, but thats not on their roadmap atm

2021-11-24

Almondovar avatar
Almondovar

Hi all, is it possible to have gui access to an ec2 linux server, but use only aws authentication? something like using the web terminal of the SSM, but doing it with gui instead? Thanks!

AugustasV avatar
AugustasV

linux and gui? why you need something like that, GUI on linux is just commands wrapper. Better write some bash scripts to make things easier.

bradym avatar

As someone who uses linux as my main operating system every day, I beg to differ.

Alex Jurkiewicz avatar
Alex Jurkiewicz

sure, you can install a desktop environment onto the ec2 instance and connect with SSH and xterm forwarding

Alex Jurkiewicz avatar
Alex Jurkiewicz

there should be plenty of guides if you search something like “linux gui over ssh”

Almondovar avatar
Almondovar

Apologies if i derailed the conversation - the point of this post was not to ask how to install graphical interface on an amazon linux ec2 instance, rather than, solving how people can access it the easiest but secure way

Almondovar avatar
Almondovar

@Alex Jurkiewicz - i found this Run Your Favorite Graphical X Applications Over SSH article, you think that if we use the web terminal of systems manager, it will make it possible to achieve browser access? thank you!

Grummfy avatar
Grummfy

perhaps you can play with pam module, because gui or cli it’s the same

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

In my view, it’s still early as heck for Proton, but they added Terraform support as a preview now: https://aws.amazon.com/about-aws/whats-new/2021/11/aws-proton-terraform-infrastructure/

TL;DR: Proton wants to be a pretty UI over Terraform (modules). It will help answer questions like “what’s the latest stable version of module X?” and allow non-tech users to use Terraform and Terraform modules by filling variables in a nice UI. Kind of like Service Broker, or a service platform in which you fill in details about what you want and next, next, next, finish (and then Proton commits it to your IaC repo maybe?)

^^^ I could be wrong, this is just my current understanding

Alex Jurkiewicz avatar
Alex Jurkiewicz

So there is:

• CloudFormation (AWS)

• Terraform (Hashicorp)

• CloudFormation CDK (AWS)

• Terraform CDK (Hashicorp)

• Cloud Control (AWS) And now Proton is thrown in the mix.

It seems like AWS thinks neither CF or TF as-is are ideal, and is throwing ideas at the wall to see what sticks. So we devs are going to be stuck in a churning market for the next few years, either continuing to use CF/TF and miss out on innovation, or risk betting on a losing technology

bradym avatar

I don’t know about anyone else, but I never adopt anything new from AWS for at least a year if I can avoid it. Their initial GA releases seem too rough around the edges to be worth touching until they’ve been proven and had some time to mature.

1
Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

Eh, not really. I should do a flowchart for this too

IaC has options:

• Declarative ◦ CloudFormation ◦ Terraform ◦ Bonus declarative using k8s’ continuous reconciliation loop: ACK, Crossplane

• Imperative ◦ CDK which “compiles” to CloudFormation ◦ CDKTF which “compiles” to Terraform ◦ Pulumi which does direct changes (I think/kinda?)

• Tooling ◦ Managed applies and things around that: Atlantis, Terraform Cloud, Spacelift, Env0, Scalr ◦ Nice UIs with “Installation wizard” instead of editing HCL/YAML/Code: Proton They all solve different things. We don’t have one programing language or one IDE, so having one way to do infra as code won’t be a thing. And options are good!

aimbotd avatar
aimbotd

I mean, cloudformation was a hackathon project that was built in 24 hours to help provision internal aws services. It wasnt originally designed for general availability

loren avatar

Reading through the way Proton “compiles” things, I’d liken it to something like Terragrunt + TFC

loren avatar

Hmmm, it relies on CodeStar for repository connections, which somehow doesn’t support CodeCommimt? Nor GitLab? But it does support BitBucket?

bradym avatar

Can’t wait to hear what Cory Quin has to say about that

loren avatar
New – AWS Proton Supports Terraform and Git Repositories to Manage Templates | Amazon Web Servicesattachment image

Today we are announcing the launch of two features for AWS Proton. First, the most requested one in the AWS Proton open roadmap, to define and provision infrastructure using Terraform. Second, the capability to manage AWS Proton templates directly from Git repositories. AWS Proton is a fully managed application delivery service for containers and serverless […]

loren avatar

Ahh, not like TFC then…
AWS Proton is not the one managing the provision of infrastructure. Therefore it is important that in the process of provisioning the infrastructure, there is a step that notifies AWS Proton of the status of the deployment.

1

2021-11-25

DaniC (he/him) avatar
DaniC (he/him)

in case folks missed https://aws.amazon.com/blogs/aws/aws-free-tier-data-transfer-expansion-100-gb-from-regions-and-1-tb-from-amazon-cloudfront-per-month/ ( after some behind the doors info been mentioned https://blog.cloudflare.com/aws-egregious-egress/. Is good to se this sort of actions from folks like AWS and co.

Balazs Varga avatar
Balazs Varga

hell all. we are using m5 and m5a instances (intel and amd cpu). We see significant performance differences between them. is that possible ? Our java code times out on AMD cpu and we see strange issues in our app if we run on m5a types…

DaniC (he/him) avatar
DaniC (he/him)

hi folks, i’m trying to find a sweet spot to improve the experience of developing / test lambda or ecs containers + rds (in private subnets, of course ) using least privileged access.

*Context*

Have a bunch of various solutions using lambda deployed inside VPC to interact with RDS deployed on private subnet. Equally same apply when having a container running as part of ECS talking with RDS.

*Challenges*

• developing/ debugging the lambda locally while setting breakpoints connected to RDS is very painful as you need to first overcome the network access: using a Bastion and a ssh tunnel over it is okay-ish although rough …

• because the initial RDS and the additional services were deployed using TF and the least privileges, running the code locally requires extensive effort to go over the IAM *Tried out and … failed*

• thought of moving the local dev env to Cloud9 where i could associated an existing/ working IAM locked policy as extended instance profile but … Issue #1 : by default Cloud9 gets deployed in public subnet. Issue #2: created an env using private subnet and SSM but sadly i couldn’t modify the associated instance profile from Console

Has anyone faced similar situations and if so able to share their stories ?

loren avatar

Have you tried localstack?

DaniC (he/him) avatar
DaniC (he/him)

i have tried in the past life but it was a hit and miss in terms of supported services. Since then i’ve stayed away

loren avatar

It works pretty well for lambda at least. Can condition the endpoint for the AWS session on the env LOCALSTACK_HOSTNAME. But yeah if you start using services/APIs that aren’t yet implemented, then you end up contributing to moto a lot

1
loren avatar

Something else you might try, to simplify the networking between local dev and remote rds in private subnets, is tailscale…

DaniC (he/him) avatar
DaniC (he/him)

that could do it in a way as i just came across https://tailscale.com/blog/github-codespaces/ time to explore. THX

1
loren avatar

Tailscale is basically magic as far as I’m concerned. One of those things that makes computers good again

1
Alex Jurkiewicz avatar
Alex Jurkiewicz

I suggest using a system to upload code changes directly to an already-existing function in your development environment.

For example, we deploy dev environments using Terraform (same IaC as production). But when developers are using a dev environment, generally they only change function code. So we have a little script for them which compiles their code locally and updates an existing function. It takes ~5secs to run, which means the dev iteration loop is short.

DaniC (he/him) avatar
DaniC (he/him)

that is an idea indeed, thanks for sharing @Alex Jurkiewicz

Laurynas avatar
Laurynas

Hey, any good info on multi region active-active Architectures? I’d like to route users to their assigned aws region based on authentication cookie. e.g if user has eu cookie they are redirected to us-west-1 alb

Alex Jurkiewicz avatar
Alex Jurkiewicz

you can’t do that without application logic. That is, you would need application code to read the cookie, decide the destination, and redirect to an internal hostname (or something along those lines)

The specifics of how you should implement multi-region active-active are very dependent on how your application works, and what metrics you care / don’t care about.

Take a look at AWS Global Accelerator, and RDS/DynamoDB global databases. They are often two key technologies to build a solution out of

1

2021-11-26

2021-11-28

2021-11-29

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)
Announcing Pull Through Cache Repositories for Amazon Elastic Container Registry | Amazon Web Servicesattachment image

Organizations, development teams, and individual developers who have chosen to use containers to host their applications may prefer, or perhaps are required, to source all images from Amazon Elastic Container Registry to take advantage of its high availability and security. To satisfy those requirements, customers have needed to take on the burden of manually pulling […]

2
Andy Miguel (Cloud Posse) avatar
Andy Miguel (Cloud Posse)

@Erik Osterman (Cloud Posse) ^^

Announcing Pull Through Cache Repositories for Amazon Elastic Container Registry | Amazon Web Servicesattachment image

Organizations, development teams, and individual developers who have chosen to use containers to host their applications may prefer, or perhaps are required, to source all images from Amazon Elastic Container Registry to take advantage of its high availability and security. To satisfy those requirements, customers have needed to take on the burden of manually pulling […]

1
Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

PM said on Twitter that Dockerhub support is coming: https://twitter.com/Sravan_R_/status/1465378657216593923

@pgarbe Thanks for sharing the launch! Pull through cache supports ECR Public and http://Quay.io images right now, but we have another announcement coming out later today for Docker Hub images :)

CloudFormation is also coming very soon

Vlad Ionescu (he/him) avatar
Vlad Ionescu (he/him)

From this GitHub issue: all Docker official images are now mirrored to ECR Public: https://gallery.ecr.aws/docker/

ECR Public Gallery

Amazon ECR Public Gallery is a website that allows anyone to browse and search for public container images, view developer-provided details, and see pull commands

DaniC (he/him) avatar
DaniC (he/him)

Hopefully soon folks will move out of DockerHub, since the quota limit is a pain to live with (especially those with small pockets )

Antarr Byrd avatar
Antarr Byrd

Anyone else at reinvent?

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I am by the expo

Antarr Byrd avatar
Antarr Byrd

Me too. What area?

2021-11-30

Yusuf avatar

Hi guys, is there a better way to track request count with aws application load balancer? I have connected cloudwatch metrics to aws managed grafana to display daily requests on the dashboard but the data provided isn’t useful. We needed something to give us a figure of daily traffic. Any other recommendation to achieve this is appreciated too.

Thanks

Andrea Cavagna avatar
Andrea Cavagna

anyone @ aws re:Invent? would love to meet community people

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Just pulling in…

    keyboard_arrow_up