#aws (2019-06)
Discussion related to Amazon Web Services (AWS)
Discussion related to Amazon Web Services (AWS)
Archive: https://archive.sweetops.com/aws/
2019-06-02
2019-06-03
2019-06-04
new tool I released :
@Issif I mean if I use a classic AWS key/secret key, they workflow in your tool will use AWS CLI profile? I would rather use aws-vault
I understand you using federation/STS. Merci pour le retour
If I get it, you don’t add you keys in .credentials/.config but you use aws-vault to store them and get them on the fly?
but after getting them, when you create a session, AWS provides you a token and this token is used by CLI or anything else to auth your requests
I guess you can keep this pattern, as you could do with AWS cli or else
yep I don’t store my keys in aws cli config as they are stored in plain text
have you had any problems using this in assume_role setting with MFA with ruby gems ?
I was trying terraforming to do a backup and with aws-vault it does not seems to work
although any other cli command works
does your script use the credentials from ENV or try to grab a profile in AWS CLI
is not my script, I’m using terraforming project
it uses the ruby aws sdk
so it should use the session env , region etc
aws-vault exec home -- env | grep AWS
perfect, my tool ca use env vars
it should work as well
I’m curious about a test if you can
I will do it later once I get my backlog cleared . Will keep you updated
thank, I let you comment and close your issue if you can
it populates them as env variables in temp context
by the way there is no licence in your released code.
oh shit, thanks, I thought it was
fixed yesterday
2019-06-05
Interesting articles ^
2019-06-10
2019-06-11
….presumably anyone paying aws bills (who isn’t at the PO level) is using a a credit card with rewards… for non-SP businesses, though, as mentioned that card is a personal card, so probably not the best idea to use for business expenses.
I keep on getting cache missed from CloudFront in front of Api Gateway for Serverless Image Handler
Do I need to enable API cache on the API Gateway Stage for CloudFront to cache the images?
2019-06-12
@Igor - yes
Although API Gateway does use Cloudfront - it’s not fully featured
2019-06-13
- join #aws-reinvent
@me1249 The Serverless Image Handler page states “The solution generates a Amazon CloudFront domain name that provides cached access to the image handler API.” Is that misleading then?
Another overview page says “modified images are cached in CloudFront”
CC @Andriy Knysh (Cloud Posse)
Unless the intent is to generate the images that are needed and then save them in the S3 bucket
Actually, I may be wrong
Are you using and edge or regional endpoint?
2019-06-14
The solution generates cloudfront distribution and caches images there. At least it’s supposed to do so, but before I also noticed that in some cases the images were not cached with Miss from CloudFront response header
I didn’t go into the details, but guess the cache request cache headers are generated incorrectly on the origin
I would check the cloudfront distribution and see what and how it uses the cache headers
And maybe try to override the origin headers
I can’t even see the CloudFront distribution in front of the API Gateway
I see the one in front of the S3 bucket that stores the utility that generates the images and original images
The API Gateway one presumably is under an AWS-owned account
Question: Can somebody share which AWS Namespace option I should choose?
The Cold Start Prerequisite sections states that you need to have a domain and a namespace setup (I assume the meaning “within AWS”). However, when I choose to create a namespace within AWS I am presented with three options, of which I must choose one… but the Cold Start instructions do not indicate which one I should choose:
1) API calls - the application discovers service instances by specifying the namespace name and service name in a DiscoverInstances request.
2) API calls and DNS queries in VPCs - AWS Cloud Map automatically creates an Amazon Route 53 private hosted zone that has this name.
3) API calls and public DNS queries - AWS Cloud Map automatically creates an Amazon Route 53 public hosted zone that has the same name.
2019-06-19
Funny issue we had where I work the other day which I’d like to share. One instance could not retrieve from ECR, but another one in another VPC with the same credentials could . The VPC had a S3 endpoint configured with a restrictive policy..
aha! tricky
2019-06-24
A question regarding certificate management: I need to set up a bunch of very lightweight HTTP webservers (only a single endpoint) through ECS in different regions (and within each region within 2-3 AZs for HA). The web server Docker images will be built from scratch with a single self-contained binary (with Go). I don’t want to use an ALB, but just leverage Route53’s latency based routing. I know that in case of an outage I will loose some payloads before DNS re-routing kicks in due to DNS caching (3min), but that’s not a problem. This means I need to do TLS termination on the web servers using Let’s Encrypt certificates. Certificate creation can be easily automated with certbot which can cache the certificates. However, those caches are on the specific machines and CSRs are limited to 20 a week, which doesn’t scale. Does anybody have a good solution for centrally managing Let’s Encrypt certificates and keeping them up to date on specific servers?
There’s tools that can do this - traefik for example has the ability to store/maintain LE certs; cert-manager in kuberenetes, etc. I just went through needing to automate 1000s of these. Every thing I researched was basically “build your own service to manage it.” We ended up settling on using traefik enterprise because it fit our use case (we needed the reverse proxy anyways), but it doesn’t fit your design parameters.
I didn’t find anything that is basically an off the shelf certificate management service with like an api you could use while building your containers or what not.
@squidfunk for the simplicity I would store generated files in S3 protected with KMS or in SSM encrypted parameter store. then download/get objects from either in entrypoint.sh. This setup works for us pretty well but we don’t have anything critical. for more critical setups we use elb with acm.
@Maciek Strömich that’s a great idea, thanks! How do you make sure that certificates are re-generated and web servers are restarted? Cloud Watch Events + S3 Lambda triggers?
as I said it’s nothing critical and if a cert expires nothing bad happens. someone (one of 3 people) on internal network will get a cert warning. we regenerate the certs in a semi automatic fashion based on tasks from our GRC calendar (someone has to run a make command) and last step in that target is to kill the containers that are currently running using aws command line tools.
Just as an FYI: I solved it by using the Terraform ACME provider which will automatically generate certificates via route53 DNS challenge and save them in SSM. Terraform will also perform automatic renewal on the next apply when necessary (i.e. expiration date is x days away).
Question regarding integrating AWS with Azure AD SSO - more specifically using ’aws-azure-login (<https://github.com/dtjohnson/aws-azure-login>) on top of that. I did everything according to this tutorial - <https://docs.microsoft.com/en-us/azure/active-directory/saas-apps/amazon-web-service-tutorial> - and I’m able to login via web browser, but when I configure aws-azure-login` and try to login, I receive following error message. Any clues how to solve it?
Has someone seen something like this? Or have an idea what could be wrong?
I’m banging my head against the wall for a week already
I managed to solve the issue myself. Turns out there I needed to change SAML link in settings from AWS one to MS one.
2019-06-25
Does this mean we don’t need to open support tickets any more to raise our limits for EC2 instances?
looks like it’s just wrapping the existing way of requesting increases
it adds a few nice features though, like service quota templating
@Erik Osterman (Cloud Posse) And here I thought it was just us that do the weekly “ask AWS for more of X” dance.
maybe I’m being mean….
looks to see if it has an API to programmatically create requests 
Quite nice to see everything in one place though, even incl. non-adjustable limits
Seems to be missing quite a few things (e.g. CloudFront + Lambda@Edge), but it’s a good start
It’d also be nice to see what the current usage value a limit is, which they must know to actually apply a limit (you’d assume)
ya, I would love to see that as well
@keen You joke, and yet I think this is the main reason we pay for the Business support (at a painful percentage of our monthly AWS cost), to get quick limit increase turnarounds. I’d drop it in a heartbeat if this turns out to be quick, easy and painless.
Last thing I’d love to see is an articulation of the hard limits (since these are soft limits only, for those that are increasable) - I get why AWS doesn’t want to publish them, but it’s nice to know exactly where the headroom extends to for some of the limits. I couldn’t get a sufficient answer from them for the absolute maximum number of CloudFront distributions.
@Lee Skillen i think they aren’t able to provide you with a hardlimit because it changes over time depending on new hardware installation/hardware upgrades/technology evolving/any other reason
Yup
aws is constantly evolving and tries to squeeze as much as possible from their setup
I agree! That’s what I meant by “I get why AWS doesn’t want to publish them”. They might not even always know for certain.
business support is nice to have. especially when your cf stack goes rogue and hangs in e.g. UPDATE_FAILED_ROLLBACK_IN_PROGRESS
also save us from few other headaches in the past when we put too much trust in developer hands
Has anyone ever run into CodeDeploy failing at AllowTraffic step?
My instance is showing up in the targets as Healthy.. but CodeDeploy just times out after 5min on AllowTraffic
2019-06-26
that’s one of the best aws news I got today
the other one is the ability to use secrets property in aws::taskdefinition pointing directly to ssm in cloudformation
seems like aws is more and more showing it’s “be nice to law enforcement” face
We’re still waiting on cross-region support for Aurora Postgres - That would really make my day.
we are waiting for it too.
2019-06-27
Anyone here using aws-azure-login and has found an alternative ?
What’s the objective/end goal?
aws-azure-login is currently used company wide to login employees to AWS who are in the AD of Azure. This itself works great for engineers.
Installation is an npm install of a package and it needs a bit of configuration for ~/.aws/config
However, currently, other non-engineers, for example content people, update images on S3 have their own created iam users with specific s3 bucket access and login with a tool like Cyberduck. Currently IAM Users+keys are created for them and it would be much better and safer to provide them access through AD as well.
But supporting the installation of aws-azure-login for the regular users would be too much work for IT and it would be great to have a simpler tool which would be easier to install / maintain.
ok, that’s great context. agree that you might not want to have them get all that other stuff setup.
perhaps a better solution would be a self-hosted S3 browser sitting behind a web/oidc proxy?
can Sumone help me out with this “The role “arniam:role/Admin” cannot be assumed. There are a number of possible causes of this - the most common are: * The credentials used in order to assume the role are invalid * The credentials do not have appropriate permission to assume the role * The role ARN is not valid”
@Sharanya this is unfortunately quite open ended. There are too many possibilities of what could be wrong.
My Terraform is on 0.12.3 , Is there any changelog with Providers ?
@Erik Osterman (Cloud Posse)
do you mean that everything was working and it stopped after upgrading?
yea
it was working perfect locally without providers
i don’t understand what “locally” means in this context and what “without providers” means since terraform cannot function without providers.
you mean you are able to assume the role not using terraform? e.g. with the aws cli?
by Locally I mean terraform was able to do “init” and plan ,
without provider
nice stuff.
very