#aws (2020-2)

aws Discussion related to Amazon Web Services (AWS)

aws Discussion related to Amazon Web Services (AWS) Archive: https://archive.sweetops.com/aws/

2020-02-14

Hemanth avatar
Hemanth

Looking for some free tool/options/advice to generate Reports based on CPU utilization of EC2 instances, primarily to check if they cross above 80% and 10 % between certain period ? Thoughts ?

imiltchman avatar
imiltchman

Does CloudWatch+Excel not meet your use case?

Hemanth avatar
Hemanth

looking to make it automated by sending reports in an email

imiltchman avatar
imiltchman

Does it have to be a report; the alerts generated from CloudWatch are not sufficient?

2020-02-13

2020-02-12

Hemanth avatar
Hemanth

Hello, I am trying to attach an CSV file from local to email as attachment AWS CLI (SES) First i tried

cat <<EOF > ./message.json
{
  "Data": "From: [[email protected]>\nTo: <mailto:[email protected]|[email protected]](mailto:[email protected])\nSubject: Report\nMIME-Version: 1.0\nContent-type: Multipart/Mixed; boundary=\"NextPart\"\n\n--NextPart\nContent-Type: text/plain\n\nReports: report\n\n--NextPart\nContent-Type: text/csv;\nContent-Disposition: attachment; filename=\"report.csv\";\npath=\"report.csv\"\n;Content-Transfer-Encoding: base64;\n--NextPart--"
  }
EOF
cat message.json
aws ses send-raw-email --raw-message <file://message.json>

i also tried modifying

  {
"Data": "From: [[email protected]>\nTo: <mailto:[email protected]|[email protected]](mailto:[email protected])\nSubject: [Subject]\nMIME-Version: 1.0\nContent-type: Multipart/Mixed; boundary=\"NextPart\"\n\n--NextPart\nContent-Type: text/plain\n\n[Body]\n\n--NextPart\nContent-Type: text/comma-separated-values;\nContent-Disposition: attachment;\nContent-Transfer-Encoding: base64; filename=\"report.csv\";\npath=\"report.csv\";--NextPart--"
}

Both the methods didn’t work for me, not sure how to modify next to achieve what i am trying to do ?

rms1000watt avatar
rms1000watt

@Erik Osterman does the maintainer of ssm-diff live in this slack? rofl https://github.com/runtheops/ssm-diff/pull/27

Configurable overwrite argument by rms1000watt · Pull Request #27 · runtheops/ssm-diff

This PR lets you run the command like: ssm-diff –overwrite false apply To prevent overwrites in special cases.

Erik Osterman avatar
Erik Osterman

Doesn’t look like it!

Configurable overwrite argument by rms1000watt · Pull Request #27 · runtheops/ssm-diff

This PR lets you run the command like: ssm-diff –overwrite false apply To prevent overwrites in special cases.

Erik Osterman avatar
Erik Osterman

I’ll be happy to reach out to him and invite to slack, if you can DM me his email

rms1000watt avatar
rms1000watt

I was trying to find his email, but then google actually pointed me back to the setup.py

rms1000watt avatar
rms1000watt

lol

Erik Osterman avatar
Erik Osterman

(ok, gonna delete that to avoid spam)

Erik Osterman avatar
Erik Osterman

invite sent!

2020-02-11

David avatar
David

I am trying to replace a bastion host that was used for port forwarding an RDS db to localhost, and replace it with an EC2 with SSM permissions. The command we used before was ssh -i ${sshPrivateKeyPath} -L ${localPort}:${remoteDbUri} -Nf ${publicBastionUri}

remoteDbUri would be something like db.private:5432, with both a private DNS name and a port.

In SSM, I found the AWS-StartPortForwardingSession document, but that won’t let me specify the db.private part I need.

Anyone know how I can do this?

maarten avatar
maarten

I battled with the same problem and created a terraform project + scripts for it. Take a look here: https://github.com/Flaconi/terraform-aws-bastion-ssm-iam

Flaconi/terraform-aws-bastion-ssm-iam

AWS Bastion server which can reside in the private subnet utilizing Systems Manager Sessions - Flaconi/terraform-aws-bastion-ssm-iam

David avatar
David

Incredible, thank you! Your module (plus looking at https://www.reddit.com/r/aws/comments/df6uip/ssm_tunnelling_ec2_what_about_rds/fhcm3e1/?context=3) got me to where I needed.

Thank you so much @maarten!

Erik Osterman avatar
Erik Osterman
AWS CLI v2 is now generally available | Amazon Web Services attachment image

We’re excited to announce the v2.0.0 GA release of the AWS CLI version 2 (v2). AWS CLI v2 builds on AWS CLI v1 and includes a number of features and enhancements based on community feedback. New Features The AWS CLI v2 offers several new features including improved installers, new configuration options such as AWS Single […]

Erik Osterman avatar
Erik Osterman

Finally an updated cli with binary distribution and built in SSO support

loren avatar
loren

Well, AWS SSO support… At least there is credential_process for others I guess

Chris Fowles avatar
Chris Fowles

This topic describes the changes in behavior between AWS CLI version 1 and AWS CLI version 2. It covers some backward-compatibility concerns and other items that might require script changes.

Chris Fowles avatar
Chris Fowles

ecr get-login has been removed and replaced with ecr get-login-password

Chris Fowles avatar
Chris Fowles

or actually

Chris Fowles avatar
Chris Fowles

RTFM The older aws ecr get-login command is still available in the AWS CLI version 1 for backward compatibility.

Chris Fowles avatar
Chris Fowles

ignore me

2020-02-10

David avatar
David

For those who use CircleCi, how do you manage rotating the IAM User credentials you supply to your CI workflows?

JJ Ferman avatar
JJ Ferman

Right now I just use the CircleCI Environment Variables for the repo

David avatar
David

Gotcha, thanks! I’m not sure that would work for my use case as we don’t want devs to be able to get the key values ever, but it’s good to know all solutions that exist

Erik Osterman avatar
Erik Osterman

I have thoughts on this - but haven’t implemented it

Erik Osterman avatar
Erik Osterman

What I’ve wanted to do is setup a cron job (in codefresh parlance) that calls the STS API to get the short lived credentials. Then update the shared secrets on codefresh. That way if credentials leak, their validity is limited.

Erik Osterman avatar
Erik Osterman

I assume something similar could be done on circle

Erik Osterman avatar
Erik Osterman

Of course, the ideal way is to have something like a runner that runs on-prem

Erik Osterman avatar
Erik Osterman

in codefresh, this is venona

Erik Osterman avatar
Erik Osterman

If have the runners on prem, those can run as a pod and assume a temporary role with credentials

JJ Ferman avatar
JJ Ferman

@David When you store an env var in circle CI it’s a write only operation. So you can’t see the variables once their set.

David avatar
David

My understanding was that devs could ssh onto CircleCi servers and run env to see env values. Or they could make a workflow that prints env values to the circle output

JJ Ferman avatar
JJ Ferman

ah yes, that’s true. But’s there’s nothing stopping them from doing that in the code either right?

David avatar
David

well if you use CircleCi Contexts, it ensures that devs can’t see the values.

But the sad part is that Circle has an API for updating standard env vars, but not env vars managed using contexts

Erik Osterman avatar
Erik Osterman

That sucks

Erik Osterman avatar
Erik Osterman

GitHub actions also doesn’t support setting secrets via API. We were trusted by that since we want to programmatically update all of our repos.

Erik Osterman avatar
Erik Osterman

#codefresh supports it though

:--1:1
curious deviant avatar
curious deviant

Hello .. Would you be open to using the AWS Secrets Manager or the Parameter Store ? There’s an orb that CircleCI provides. I haven’t spiked it out yet but is on my list :)

Erik Osterman avatar
Erik Osterman

I think the problem though we’re trying to solve here is how to have “short lived credentials” for AWS users

Erik Osterman avatar
Erik Osterman

I guess one could write short lived credentials to ASM and SSM

Erik Osterman avatar
Erik Osterman

but then you still need long lived credentials in Circle to access the short lived ones

Erik Osterman avatar
Erik Osterman

I don’t think it’s optimal

2020-02-03

Maciek Strömich avatar
Maciek Strömich

Hey, does anyone seen a boto3 behaviour where 1st request fails with unable to find credentials exception? I’ve recently started to observe this behaviour on an multi-docker elastic beanstalk environment (which until ~thursday was running flawlessly). For me it happens only when a new instance is brough up by autoscaling and worker starts to send records to firehose. What’s even more strange is that when debug was enabled to have a littlebit more verbose output in the logs for boto problem perished.

Maciek Strömich avatar
Maciek Strömich

BTW firehose credentials are supposed to be obtained from task IAM role

    keyboard_arrow_up