#announcements (2018-11)

cloudposse Cloud Posse Open Source Community <#CB84E9V54 geodesic> <#CB6GHNLG0 terraform> <#CBW0HJDS8 release-engineering> <#CB2PXUHLL random> <#CB9N1MMFV releases> <#CB7CA7X0D docs>

This channel is for workspace-wide communication and announcements. All members are in this channel. Archive: https://archive.sweetops.com

2018-11-30

GreetBot avatar
GreetBot
08:29:02 AM

hey everyone give a warm welcome to @chhed13! Good to have you here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

welcome @chhed13

2
1
chhed13 avatar
chhed13

Hi everybody!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hey Andrey, where are you from?

chhed13 avatar
chhed13

From Russia. But currently I am on re:Invent

chhed13 avatar
chhed13

We’ve met once there with @antonbabenko

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

antonbabenko avatar
antonbabenko

Hi @chhed13 ! Great to have you here, enjoy your trip further :)

1
OScar avatar
OScar

Sorry we didn’t meet last night, it was difficult to get around!

antonbabenko avatar
antonbabenko

Yeah, it was. Which city you are based at? See you at other events in the world

GreetBot avatar
GreetBot
06:33:04 PM

hey everyone give a warm welcome to @Valdemir! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @chhed13! what’d you think of #aws-reinvent this year?

chhed13 avatar
chhed13

It’s too crowded for me, worse then 2017. I am overwhelmed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, i went last year and had the same experience. decided to skip it this time around. then saw they were adding a lot more repeat talks and had some FOMO.

Zapier avatar
Zapier
08:08:15 PM

@Erik Osterman (Cloud Posse) created a new channel #packer. Join if this sounds interesting!

OScar avatar
OScar

Anyone know of a module or something that can help me get all Cloudwatch logs from several accounts to the audit account?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you mean the new capability that was just announced?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(forget what they called it)

joshmyers avatar
joshmyers

Organizational cloudtrail

joshmyers avatar
joshmyers

Think the PR to add functuoanlity is merged but probably waiting for a release

joshmyers avatar
joshmyers

Needs to be run in the master account org

loren avatar
loren

Can’t you do that already? Just create the bucket in the audit account, and point the trail in each account at that bucket? At least, that’s how we’re doing it, and no limit on the org boundary, so still works if you need multiple payer accounts for compliance reasons…

joshmyers avatar
joshmyers

That is you managing a individual trail per account and telling them all to log into a single bucket

loren avatar
loren

Is this implementation different? Or is it just creating the trail for you, from the master account?

joshmyers avatar
joshmyers

Organizational trails you create a new trail in master, with an added Boolean of isOrganizationalTrail and it will create and manage in all underling accounts of the org. Master config is applied to all

:--1:2
1
loren avatar
loren

Meh. If it’s just creating the trail, I like our approach better. Rather have terraform manage the config for each trail

joshmyers avatar
joshmyers

Hmm, yeah, think I’d rather less code and a single account to manage. There are other benefits like an underling account can’t turn it off or update the trail, which is an attack vector if an account is compromised

:100:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

exactly, this is what I like about it. check & balances.

loren avatar
loren

Yeah, we already manage these compliance related resources in child accounts, including the IAM role assignment, so no chance of that

loren avatar
loren

Thank goodness for the permission boundary feature… Makes it possible to delegate IAM permissions to their roles without escalation risk

joshmyers avatar
joshmyers

What if your production account IAM creds were compromised? Or someone managed to impersonate a service that has cloudtrail access?

joshmyers avatar
joshmyers
Disrupting AWS logging – Cyber Free attachment image

So you’ve pwned an AWS account — congratulations — now what? You’re eager to get to the data theft, amirite? What about that whole cyber…

1
loren avatar
loren

I don’t see how this feature changes that risk, you still have something, somewhere with permissions to these things, and it’s still centrally controlled

joshmyers avatar
joshmyers

I generally want to setup alerts if anyone messes with CT

joshmyers avatar
joshmyers

Because it is in one place that can be more locked down than each underling account

joshmyers avatar
joshmyers
08:52:52 PM
joshmyers avatar
joshmyers

Hook that up to SNS

loren avatar
loren

Eh, I don’t really see any difference. Once you have the module written, it’s just a matter of being able to implement it, and if you have multiple payer accounts then you need to be able to implement multiple times anyway

loren avatar
loren

The org-centric features are certainly nice for those who do things manually, through the console though

joshmyers avatar
joshmyers

I agree that where Terraform is concerned, sometimes it is just easier to manage a thing in each account rather than turn on in one and magic behind the scenes, treat others differently.

OScar avatar
OScar

@joshmyers I was chatting with @Erik Osterman (Cloud Posse) the AWS Control Tower is going to be interesting to see in action, I feel it does a lot of what cloudposse framework tackles

joshmyers avatar
joshmyers

Looks pretty nice. Hopefully not all magic behind the scenes and APIs are exposed (can’t see api docs for it)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea, hopefully it can make some things easier.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I just got off the phone with @Jan and he gave me some AWESOME ideas for how to simplify the geodesic coldstart process that I can’t wait to implement

2
4
bowtie2
loren avatar
loren

I’m not a fan of magic, either, but it might be nice for control tower to be a bit more magic than implied by this pic…

loren avatar
loren

That’s actually the landing zone, but the two teams are working closely

joshmyers avatar
joshmyers

OScar avatar
OScar

Oooohh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i added a bunch more emoji. let me know what i’m missing!

Jan avatar

Table flip is missing for sure

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cool-doge

joshmyers avatar
joshmyers

picard_fail

:--1:1
picard_fail1
1
fidget_spinner1
fiesta_parrot2
grumpycat1

2018-11-29

GreetBot avatar
GreetBot
01:26:36 PM

hey everyone give a warm welcome to @pigglesticks! Good to have you here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

welcome @pigglesticks

pigglesticks avatar
pigglesticks

Thanks @Andriy Knysh (Cloud Posse)

pigglesticks avatar
pigglesticks

I think I might be in the wrong timezone over here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

what timezone are you in?

pigglesticks avatar
pigglesticks

I am based in Cape Town

pigglesticks avatar
pigglesticks

UTC+2

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

oh nice, I have a friend here in Florida, he is from Cape Town originally

Jan avatar

I’m from Cape Town

Jan avatar

Live in Germany now

pigglesticks avatar
pigglesticks

I am originally from the UK, but I’ve lived in SA for 8 years now

pigglesticks avatar
pigglesticks

I’ve fallen for the lifestyle

pigglesticks avatar
pigglesticks

How long in Germany Jan?

Jan avatar

2 years

Jan avatar

Almost

joshmyers avatar
joshmyers

@pigglesticks Best out of the UK

pigglesticks avatar
pigglesticks

Theresa May’s dancing is enough to scare anyone away for good

joshmyers avatar
joshmyers

hahaha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Kenny Inggs is also in South Africa - Cape Town if I am not mistaken

Zapier avatar
Zapier
10:31:54 PM

@Erik Osterman (Cloud Posse) created a new channel #ansible. Join if this sounds interesting!

2018-11-28

davidvasandani avatar
davidvasandani

I really have to say this is the most active DevOps Slack team I’m apart of. Way to go!

1
1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @davidvasandani!

i5okie avatar
i5okie

hey i have a general question.. i’ve just configured my bastion host to automatically look at aws iam users, sync them, and copy their ssh public keys. so now i can ssh into bastion. great.

my previous method was sshing in using a shared key, then using a key on bastion to ssh into the private EB instances.

question: now that my users are sshing into bastion, and there are now like 10 users, how do I seamlessly ssh to the private instances without copying the key to all their .ssh folders?

extra note: i wrote a gem that replicates Heroku cli functionality. so its all automated $ mygem ssh –app myappname –env myenvstage #and it automatically sshes through bastion to one of the app instances. so internally now i have to change it to use ssh <iamuser>@privatehost instead of [email protected] and im just trying to figure this out. any suggestions?

tamsky avatar
tamsky


how do I seamlessly ssh to the private instances without copying the key to all their .ssh folders

  1. Heroku app can instruct ssh (via SSH(1) flag: -F configfile) to use a custom config file that enables ForwardAgent yes and User <iamuser> – users use ssh-agent and forwarding to access private hosts.
  2. Also add <iamuser> accounts on all private hosts, so you can continue to track/attribute/remove access/usage.
:--1:1
i5okie avatar
i5okie

thanks

i5okie avatar
i5okie

key being, how can i automate this, maybe via a single line command.. instead of telling everyone to change their ssh_config

GreetBot avatar
GreetBot
05:55:45 PM

hey everyone give a warm welcome to @adamhkaplan! Good to have you here

maarten avatar
maarten

@i5okie Try adding this into your .ssh/config and then directly ssh into the ec2 instance.

Host bastionhost
    ForwardAgent yes

Host 10.* <- fix your range here
     ProxyCommand ssh [email protected] -W %h:%p
i5okie avatar
i5okie

im trying to do it without that

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

welcome @adamhkaplan

adamhkaplan avatar
adamhkaplan

joshmyers avatar
joshmyers

@i5okie could your gem template some SSH config? A bit yuk.

joshmyers avatar
joshmyers

I think it is generally acceptable for folks to have to add some SSH config in order to access things, and preferably not as a shared user.

i5okie avatar
i5okie

yeah i just got this going on my bastion: https://github.com/widdix/aws-ec2-ssh

widdix/aws-ec2-ssh

Manage AWS EC2 SSH access with IAM. Contribute to widdix/aws-ec2-ssh development by creating an account on GitHub.

i5okie avatar
i5okie

pretty neat

maarten avatar
maarten
kislyuk/keymaker

Lightweight SSH key management on AWS EC2. Contribute to kislyuk/keymaker development by creating an account on GitHub.

i5okie avatar
i5okie

now the last leg… im trying to avoid modifying ssh_config.. in theory i could use sshkit and dynamically change ssh_config on the fly. but “yuk” is perfect word here

joshmyers avatar
joshmyers

What is wrong with having users add some SSH config? I think this is pretty standard practice in onboarding docs

i5okie avatar
i5okie

we have multiple aws accounts, multiple vpcs…

i5okie avatar
i5okie

that sshconfig is going to look like a book

i5okie avatar
i5okie

bigger hurdle is, there might be some vpc cidr overlapping

i5okie avatar
i5okie

might be easier to script the code that syncs users, to also copy the ssh key into their home folders..

joshmyers avatar
joshmyers

SSH config does wildcard matching which works nicely with CNAMEs

i5okie avatar
i5okie

josh, one of the accounts has vpcs that overlap the other account’s ip address. different bastion hosts for similar address space

i5okie avatar
i5okie

10.1.0.0 10.1.0.0 for example

joshmyers avatar
joshmyers

Attach an EIP to each bastion and CNAME to it, bastion.production.bla, bastion.staging.bla ?

i5okie avatar
i5okie

but ssh config starts with remote host.. if that wildcard matches, then it picks which bastion host.. right

i5okie avatar
i5okie

so bastion host ip doesn’t matter. its the private instance ip address

i5okie avatar
i5okie

i guess another option is to sftp the key to bastion, right before they ssh through it

i5okie avatar
i5okie

or scp

i5okie avatar
i5okie

stupid elastic beanstalk.

Zapier avatar
Zapier
07:14:30 PM

@Erik Osterman (Cloud Posse) created a new channel #lax. Join if this sounds interesting!

tamsky avatar
tamsky

keymaker has been useful

i5okie avatar
i5okie

my private hosts are all elastic beanstalk hosts.

i5okie avatar
i5okie

with instance refresh, etc etc. so i need to get that automated somehow.

i5okie avatar
i5okie

I’ve just added this copy_ssh_key function to the script i’ve been using: https://github.com/i5okie/aws-ec2-ssh/blob/master/import_users.sh#L161-L180

i5okie/aws-ec2-ssh

Manage AWS EC2 SSH access with IAM. Contribute to i5okie/aws-ec2-ssh development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@i5okie have you seen our github-authorized-keys service?

i5okie avatar
i5okie

nope

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the idea here is that you use AuthorizedKeysCommand (or something like that) to avoid needing to provision keys

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it can also provision users too

i5okie avatar
i5okie

yeah the aws-ec2-ssh script does that by getting the ssh public key associated with your iam user

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i think the bastion container implements support for that.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ah right - forgot you’re using aws iam (which is nice)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if we did it all over again, i would implement it slightly differently

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i would use pam.d to automatically query IAM on user connection

tamsky avatar
tamsky

this is what keymaker does, uses PAM and a script

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

which would then provision the user (if permitted) and then add their key

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is possible with pam_exec

i5okie avatar
i5okie

hmm sounds like keymaker and aws-ec2-ssh are very similar. next question is, does it work with Amazon Linux 2 ami? because they’ve taken over the AuthorizedKeysCommand for use with ssm

tamsky avatar
tamsky

Yeah, it’s pretty odd text in the release notes:
“to support an upcoming feature to read SSH public keys”

https://aws.amazon.com/amazon-linux-2/release-notes/

Amazon Linux 2 - 10/31/2018 Update

OpenSSH daemon configuration file /etc/ssh/sshd_config updates

The OpenSSH daemon configuration file /etc/ssh/sshd_config has been updated. The AuthorizedKeysCommand value is configured to point to a customized script, /opt/aws/bin/curl_authorized_keys to support an upcoming feature to read SSH public keys; from the EC2 instance metadata during the SSH connection process.
i5okie avatar
i5okie

and broken functionality of all these scripts that use that command lol

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i didn’t know that

i5okie avatar
i5okie

i don’t like how they are forcing people to use ssm to connect to instances.

i5okie avatar
i5okie

if it was a one-line command, that’d be nice. otherwise its kinda like meh

i5okie avatar
i5okie

ooh

i5okie avatar
i5okie

“and home directory sharing (via optional EFS integration)” this is very interesting re: keymaker

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
07:40:05 PM
GreetBot avatar
GreetBot
07:45:34 PM

hey everyone give a warm welcome to @Stephen! Good to have you here

3
:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I setup https://www.linkedin.com/groups/13649396/ if LinkedIn is your thang and you want a badge on your profile

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
12:12:12 AM
GreetBot avatar
GreetBot
01:16:36 AM

hey everyone give a warm welcome to @rohit! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @rohit!

rohit avatar
rohit

Hi

rohit avatar
rohit

I have a question related to cloudposse/terraform-null-label

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sure, let’s move to #terraform

rohit avatar
rohit

Just curious, how did you generate the image used here https://docs.cloudposse.com/helm-charts/

rohit avatar
rohit

i see lot of people generating something like this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh haha, that was actually just a screenshot from one of our slide decks

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

which I had a fancier answer

rohit avatar
rohit

rohit avatar
rohit

but lot of people are generating something like this

rohit avatar
rohit

so i was wondering how people do it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i literally googled everyone of those projects to find a nice transparent logo and placed them

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so i’ve seen various implementations

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

some use favicons

rohit avatar
rohit

gotcha

rohit avatar
rohit

i thought there was some tool to generate something like this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Clearbit

Easily embed any company’s logo in your project with this simple and free API from Clearbit

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, something to generate a montage

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you find one, lmk!!

rohit avatar
rohit

will do

rohit avatar
rohit

@Erik Osterman (Cloud Posse) what did you use clearbit for ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh just googling logo api

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

never used it

rohit avatar
rohit

when i searched for Cloud Posse your pic showed up

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh, haha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, i think their algo is weak

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i tried another one of these services and it returned one of our customer’s logo for [cloudposse.com](http://cloudposse\.com)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i guess they pulled it from our logo slider on the homepage

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

alas, google FTW

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
04:46:14 AM
rohit avatar
rohit

yeah their algo looks weak

2018-11-27

GreetBot avatar
GreetBot
12:02:43 PM

hey everyone give a warm welcome to @jerry! Good to have you here

2
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @jerry! glad you stopped by

jerry avatar
jerry

Thanks Erik. Couldnt resist Thank you and the team for all the great content you have up there.

gyoza avatar
gyoza

i wish i could open source the stuff i work on

gyoza avatar
gyoza

i would get mad internet points

joshmyers avatar
joshmyers

@gyoza Interested now…

gyoza avatar
gyoza

i built an autohealing federated prometheus cluster thing using consul-template, consul, and some python..

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sounds awesome!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we haven’t had the opportunity to explore this yet - would love to see this for kubernetes

1
gyoza avatar
gyoza

the consul template looks ilke hieroglyphics

1
gyoza avatar
gyoza

so if the master goes down and a new one comes up it will resync, and the child nodes, if one of them dies its fine

gyoza avatar
gyoza

a new one will come up and resync

gyoza avatar
gyoza

as long as they all dont die at once

gyoza avatar
gyoza

should be ok:P

gyoza avatar
gyoza

not using LTS

gyoza avatar
gyoza

any of you guys using thanos?

Jan avatar

Exploring it

gyoza avatar
gyoza

yea, i like it for the LTS stuff

gyoza avatar
gyoza

RONCO style

GreetBot avatar
GreetBot
07:41:39 PM

hey everyone give a warm welcome to @samhagan! Good to have you here

3
GreetBot avatar
GreetBot
11:41:38 PM

hey everyone give a warm welcome to @john294! Good to have you here

1
john294 avatar
john294

Hello!

Jan avatar

hey hey

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hey john! Glad you signed up

GreetBot avatar
GreetBot
07:56:35 AM

hey everyone give a warm welcome to @chialun! Good to have you here

4

2018-11-26

 avatar
05:00:01 PM

There are no events this week

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

will be nice if AWS adds MongoDB and Kafka as managed services https://seekingalpha.com/news/3410999-aws-develops-new-services-amid-open-source-pushback

AWS develops new services amid open-source pushback

Amazon (AMZN -4.3%) continues to use open-source software to build out AWS despite tensions with the open-source community, according to The Information.AWS is reportedly developing two new cloud serv

rms1000watt avatar
rms1000watt

Lol, it’s nice when my peers organically stumble on cloudposse projects: https://github.com/cloudposse/prometheus-to-cloudwatch in particular

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

whoot! yea, that’s been a popular one

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) worked on that

GreetBot avatar
GreetBot
10:30:42 PM

hey everyone give a warm welcome to @Ben Hecht! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

welcome @Ben Hecht !

:--1:2

2018-11-25

GreetBot avatar
GreetBot
09:58:42 AM

hey everyone give a warm welcome to @boz! Good to have you here

2018-11-23

GreetBot avatar
GreetBot
12:49:33 PM

hey everyone give a warm welcome to @gauravg! Good to have you here

Jan avatar

welcome

gauravg avatar
gauravg

Thanks

gauravg avatar
gauravg
cloudposse/terraform-datadog-monitor

Terraform module to provision Standard System Monitors (cpu, memory, swap, io, etc) in Datadog - cloudposse/terraform-datadog-monitor

Jan avatar

I think most of the folks are not around, given that its thanks giving holidays

Jan avatar

I could try though

gauravg avatar
gauravg

yeah

Jan avatar

whats up?

gauravg avatar
gauravg

I am getting this error Error: resource 'datadog_monitor.cpu_usage' config: unknown module referenced: label

Jan avatar

im also on a high speed train crossing Germany right now so my internet is not super reliable

gauravg avatar
gauravg

good luck

gauravg avatar
gauravg
module "datadog_cpu_usage_us_east_1" {
  source     = "./module/cpu"
  namespace  = "test"
  stage      = "dev"
  name       = "els1"
  attributes = "us-east-1"
  delimiter  = "-"
  tags       = "ci"
}

provider "datadog" {
  api_key = "${var.datadog_api_key}"
  app_key = "${var.datadog_app_key}"
}

variable datadog_api_key {}
variable datadog_app_key {}
gauravg avatar
gauravg

This is my main.tf in root ^

Jan avatar

let me see

Jan avatar
module "label" {
  source     = "git::<https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.3.1>"
  namespace  = "${var.namespace}"
  stage      = "${var.stage}"
  name       = "${var.name}"
  attributes = "${var.attributes}"
  delimiter  = "${var.delimiter}"
  tags       = "${var.tags}"
}
Jan avatar
cloudposse/terraform-datadog-monitor

Terraform module to provision Standard System Monitors (cpu, memory, swap, io, etc) in Datadog - cloudposse/terraform-datadog-monitor

Jan avatar

did you do a terraform init

Jan avatar

or terraform get -update

gauravg avatar
gauravg

yes it’s giving error on same

Jan avatar

mm

gauravg avatar
gauravg

the difference is in source. Instead of using githib url, I am using my local which has same module of cpu

Jan avatar

for sure

Jan avatar

do you have a local copy of the "git::<https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.3.1>" module?

gauravg avatar
gauravg

oh yeah, got it now. Somehow that copy on different location

gauravg avatar
gauravg

Let me try to move it and then try

gauravg avatar
gauravg

it works

gauravg avatar
gauravg

Thanks Jan

Jan avatar

brilliant :–1:

gauravg avatar
gauravg

Now let me see why monitor not created

Jan avatar

its one of those things with terraform, if you are using local, relative path, modules you gotta be careful to validate the paths

Jan avatar

@gauravg did you go through the cold-start?

Jan avatar

for geodesic

gauravg avatar
gauravg

nope

Jan avatar

dang

Jan avatar

oki

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan I can give you some tips on provisioning AWS accounts, if you ready

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

The docs were written when we provisioned accounts manually in the AWS console

Jan avatar

Yea im ready

Jan avatar

I suspected that might be the case

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules

:--1:1
Jan avatar

just because of the env calls

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so, in short, are you able to login to the root account from geodesic?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

root means the master Org account

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

let’s switch to #geodesic

Jan avatar

kk

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Boss as a Service | Hire a boss, get stuff done attachment image

BaaS helps you achieve your goals, making sure that you follow through on what you plan.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

funny ^

GreetBot avatar
GreetBot
02:53:00 PM

hey everyone give a warm welcome to @ManojH! Good to have you here

GreetBot avatar
GreetBot
11:39:05 PM

hey everyone give a warm welcome to @Bryan! Good to have you here

2018-11-22

GreetBot avatar
GreetBot
08:19:43 AM

hey everyone give a warm welcome to @Duong! Good to have you here

GreetBot avatar
GreetBot
01:24:08 PM

hey everyone give a warm welcome to @Bogdan! Good to have you here

Jan avatar

wellcome

Bogdan avatar
Bogdan

Hey! I’m the HUG Zurich leader and long-time user/fan of CloudPosse’s TF modules joining at @antonbabenko’s invitation

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

welcome to the group!

Bogdan avatar
Bogdan

We just had a Meetup focused TF with @antonbabenko as a guest speaker yesterday: https://www.meetup.com/Zurich-HashiCorp-User-Group/events/255859299/

Zurich HUG #1: Inaugural event attachment image

Wed, Nov 21, 2018, 6:30 PM: Hi everyone!It is our pleasure to announce the launch of the Zurich HashiCorp User Group with a great selection of speakers.Agenda:18:30 - 18:35: Arrival and networking18:3

bowtie1
1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hey @Bogdan welcome! Thanks for sharing the link

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

welcome @Duong

3
antonbabenko avatar
antonbabenko

@Andriy Knysh (Cloud Posse) thanks.

antonbabenko avatar
antonbabenko

If anyone wants to see my talk at Hashicorp User Group meetup in New York city 3rd of December - Join me at Terraform Modules and Best Practices http://meetu.ps/e/G3zHX/1MhZb/a

Terraform Modules and Best Practices attachment image

Mon, Dec 3, 2018, 6:00 PM: Join us for the next NYC HashiCorp User Group meetup with Anton Babenko and Microsoft!Schedule:6:00 - 6:40PM Food, beverages, socializing6:40 - 7:10PM Anton Babenko, Terrafo

:--1:2

2018-11-21

maarten avatar
maarten

@Nikola Velkovski Fargate is just extremely expensive, there is a reason AWS only has monthly pricing example for running Fargate tasks a few hours a day. A 1CPU fargate with 2GB of memory costs 54 USD/month. Leaving out EBS that’s about the same price as a c5.large..

:100:3
Nikola Velkovski avatar
Nikola Velkovski

Thanks @maarten

Nikola Velkovski avatar
Nikola Velkovski

That sealed the deal more or less lol

maarten avatar
maarten

so yeah for just a few services it’s perfect, also it’s good to isolate certain services for security purposes, and you can run Fargate next to regular ECS here.

integratorz avatar
integratorz

Hey @Erik Osterman (Cloud Posse), just taking a look at some of the modules that Cloud Posse has for Terraform and wanted to hop into the slack channel.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

awesome! feel free to ask questions in #terraform if you get stuck on anything

GreetBot avatar
GreetBot
09:34:11 PM

hey everyone give a warm welcome to @Martin! Good to have you here

OScar avatar
OScar

@here I’m guessing folks are still working (you workaholics!). I should talk right? Just wanted to wish everyone in this amazing community a safe and happy Thanksgiving!

2
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Thanks @OScar

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Happy thanksgiving everybody

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks @OScar !

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Have a good one…

OScar avatar
OScar

@here anyone have this issue trying to assume the role?

Enter passphrase to unlock /conf/.awsvault/keys/:

An error occurred (InvalidClientTokenId) when calling the GetCallerIdentity operation: The security token included in the request is invalid.

OScar avatar
OScar

I am able to assume the cnc-root-admin via the web console however

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Probably something wrong in the AWS config

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Sec

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you want you can PM me your config assuming no secrets are in it!

OScar avatar
OScar

I am also able to assume the role via a plain aws-vault login cnc-root-admin command and the web console opens with the assumed role

OScar avatar
OScar

ok

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Let’s move to #geodesic

2018-11-20

joshmyers avatar
joshmyers

Maybe?

Nikola Velkovski avatar
Nikola Velkovski

kinda

Nikola Velkovski avatar
Nikola Velkovski

but thanks

Nikola Velkovski avatar
Nikola Velkovski

I am slowly compiling the list I just need to dig through a lot of blogs, docs etc

GreetBot avatar
GreetBot
08:12:12 AM

hey everyone give a warm welcome to @zerocool.jothi! Good to have you here

maarten avatar
maarten

@ramesh.mimit afaik, Terraform is not forcing any certain patterns on ARN’s. I’m running TF with the new format without problems. Resources affected are mostly resources Terraform doesn’t care about, container instances, task-id’s, and they mentioned services, although services don’t have an ARN afaik.

:--1:2
Nikola Velkovski avatar
Nikola Velkovski

I guess it will be like when they switched the instances ids to longer ones, terraform will be agnostic in this case.

:--1:1
GreetBot avatar
GreetBot
06:46:49 PM

hey everyone give a warm welcome to @mishinev! Good to have you here

1
rms1000watt avatar
rms1000watt
07:02:34 PM

The right thing to do, when @Erik Osterman (Cloud Posse) sends you stickers.. is put them on your boss’s office window @stobiewankenobi

:100:4
1
stobiewankenobi avatar
stobiewankenobi

Oh that is one handsome dude

rms1000watt avatar
rms1000watt

in the reflection? I know right

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

HAHAH

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that was fast

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i just mailed them last friday.

rms1000watt avatar
rms1000watt

OScar avatar
OScar

@here pardon my ignorance, but i want to ask something that may make sense for others, just not clear to me:

The IAM User that is created for the [root.cloudposse.co](http://root\.cloudposse\.co) repository. Which is used to manage other accounts. Can that be a federated user since it assumes IAM Roles? or?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so there is no specific account designated as “the account to manage other accounts”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

let’s move to #geodesic

Zapier avatar
Zapier
01:11:13 AM

@Erik Osterman (Cloud Posse) created a new channel #packages. Join if this sounds interesting!

GreetBot avatar
GreetBot
02:41:13 AM

hey everyone give a warm welcome to @integratorz! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @integratorz! what bring you around?

ramesh.mimit avatar
ramesh.mimit

@maarten thanks for the confirmation

2018-11-19

GreetBot avatar
GreetBot
11:03:42 AM

hey everyone give a warm welcome to @j.baldzer! Good to have you here

 avatar
05:00:01 PM

There are no events this week

OScar avatar
OScar

@here I am excited this morning because I am deploying @AWS accounts using Geodesic!

:--1:4
joshmyers avatar
joshmyers

Nice! How’d it go?

OScar avatar
OScar

I am on it right now Guiding them, super stoked!

joshmyers avatar
joshmyers

Anything missing from docs? Rough edges / ordering issues? What have you got running?

OScar avatar
OScar

Yep i am going through the “Cold Start” for this exercise! https://docs.cloudposse.com/reference-architectures/cold-start/

1
OScar avatar
OScar

I followed the Cold Start on my lab, and setup root and stage architectures

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sorry I’ve been AWOL for the last few days. Drove up to Oregon for a mini work “vacay” for Thanksgiving

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

catching up on all the messages now!

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jan is doing the same right now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

he found a couple of issues with the #docs

Jan avatar

I found a few other things I wanna think through

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cool - remember we’re so familiar with this stuff that getting your outside feedback is what we really need

Jan avatar

I ma used to building vpc’s with terraform and rendering a k8s cluster.yaml template and launcing into the TF created vpc

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@mcrowe banged his head against the while for a while until we got these docs online

Jan avatar

I did see from the docs that you are launching from a rendered cluster spec

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jan you mean using terraform to provision the VPC and launching kops in that VPC?

Jan avatar

exactly

Jan avatar

I only looked at your example

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

gotcha, yes, there are some pros with that and i’ve seen others do it that way

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

our approach was slightly different, but i’d be open to supporting both methods

Jan avatar

and I think even if it its a supported case it should be easy enough to add

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

our approach is to let kops manage it’s VPC and all resources

Jan avatar

Yea

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and use terraform to manage it’s own VPC

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then peer them

Jan avatar

So I have in the past had vpc’s that have services running in them where I want to add k8s

Jan avatar

I lied having the programmatic blast radius separation between k8s / infra / rds data

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. https://slack.cloudposse.com/ - cloudposse/geodesic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is the kops manifest template

Jan avatar

I mean what happens for example if you create the vpc with kops and then after that launch other EC2 resources into said vpc

Jan avatar

if you do a kops destroy cluster will it kill the vpc?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i believe so… but i just saw an issue raised by a customer where it was not, but haven’t looked into it

Jan avatar

the nice aspect of the other wey is that you cant destroy a TF controlled vpc without first destroying the k8s cluster

Jan avatar

blast radius control

Jan avatar

any how I need to walk to my colleges house and its like -3 so no more typing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, we can discuss more later

Jan avatar

:–1:

Jan avatar

I am also keen to explore gitlab in place of github

Jan avatar

as we use gitlab

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have a customer using GitLab. That’s not a blocker. Personally, I’m still not a fan or sold on GitLab.

Jan avatar

Ah brilliant, would love to know more about any changes required to use to gitlab

ramesh.mimit avatar
ramesh.mimit

anyone aware then terraform is going to support new ECS ARN formats?

pecigonzalo avatar
pecigonzalo

I believe this will be auto supported

Nikola Velkovski avatar
Nikola Velkovski

@ramesh.mimit just a quick update, I have enabled the new arns in our testing acccount for ECS and it seems that everything is working fine as before.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@maarten might know more about that

Nikola Velkovski avatar
Nikola Velkovski

Hey People I am not sure if this is the right place to ask but does anyone have a comparison chart for fargate and ECS. I know that fargate is way more limited than ecs but I cannot seem to find anything that compares those 2 on the interwebz.

Nikola Velkovski avatar
Nikola Velkovski

any help is appreciated

2018-11-18

Jan avatar

not this year

GreetBot avatar
GreetBot
01:15:58 PM

hey everyone give a warm welcome to @monsoon.anmol.nagpal! Good to have you here

davidvasandani avatar
davidvasandani

Hi @monsoon.anmol.nagpal

monsoon.anmol.nagpal avatar
monsoon.anmol.nagpal

Hello

Jan avatar

Where would you say is the best place to get started

Jan avatar

docs wise?

Jan avatar

I have a 7 hour train journey tomorrow so I figure I can do a deep dive

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jan do you want to get started with the reference architecture

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have our QuickStart guide that links to everything, but it needs a lot of work

Jan avatar

Oh I did notice one thing just before, let me dig it up

Jan avatar

was a dead link to local dev

Jan avatar

There are a few changes I will explore, I need to build with CIS compliance in mind

Jan avatar

will see how it works out

Jan avatar

thanks for the quick-start, will take a look there

Jan avatar

terraform is one of my strong areas so should be able to follow through most of it

Jan avatar

I mean almost all the tool chain you are using is my strong area

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Cool! I am afk today, but will be around all day tomorrow

Jan avatar

epic

Jan avatar

Will be here asking stuffs depending on my internet connection

2018-11-17

GreetBot avatar
GreetBot
10:49:14 AM

hey everyone give a warm welcome to @demeringo! Good to have you here

antonbabenko avatar
antonbabenko

Who is going to AWS re:invent and want to chat about stuff we do?

OScar avatar
OScar

I’ll be there all week, let me know when we can meet up @antonbabenko

antonbabenko avatar
antonbabenko

Great! I will ping you when I am there.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also we created #aws-reinvent

OScar avatar
OScar
07:40:28 PM

I am!! Let’s meet up, grab a drink and talk shop!

:--1:1

2018-11-16

GreetBot avatar
GreetBot
09:00:42 AM

hey everyone give a warm welcome to @Jan! Good to have you here

Jan avatar

o/

maarten avatar
maarten

hi @Jan !

Jan avatar

hey hey

Jan avatar

Dutch or Afrikaans?

maarten avatar
maarten

My Afrikaans is Braai

maarten avatar
maarten

Dutch is better

Jan avatar

Dutch I guess based on time

Jan avatar

cool cool

maarten avatar
maarten

I can also do a bit of Xhosa

maarten avatar
maarten

when drunk

Jan avatar

some nice ideas you have going on in your terraform

Jan avatar

haha awesome, I can speak Dutch when drunk

Jan avatar

and Xhosa a little

maarten avatar
maarten

What are you working on atm ?

Jan avatar

I was rewriting an example multi aws account org, with IAM roles an d sts assume patterns. Opinionated modules for vpc’s peering, logging etc

Jan avatar

doing a workshop with some engineers for a company I joined recently

Jan avatar

stumbled across your sweetops

Jan avatar

would have been amazing to have this several years back

Jan avatar

So today im exploring doing the same setup with your modules

Jan avatar

and then see if I can work CIS benchmark compliance into it

maarten avatar
maarten

Myself I’m not cloudposse, more of a pro-active lurker, but happy to help out, most of the cloudposse guys show up US working times.

Jan avatar

makes sense

Jan avatar

tx

joshmyers avatar
joshmyers

Have folks implemented https://github.com/cloudposse/root.cloudposse.co out of the box switching out CP variables for their own? I’m hitting orderings issues which is expected during any bootstrap but they aren’t documented from what I can see..

cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Jan avatar

what sort of ordering issues?

joshmyers avatar
joshmyers

Ordering of resources. i.e. account-settings > users > iam

Jan avatar

I will be doing a similar test setup shortly

Jan avatar

will let you know if I see the same issues

joshmyers avatar
joshmyers

Only used in anger for an hour or so

OScar avatar
OScar
03:33:22 PM

@joshmyers I’m actually using the root, audit and stage modules on Monday for a client! Ping me here if you have questions, I’ll reciprocate

joshmyers avatar
joshmyers

@OScar I got up and running pretty quick after poking around but wasn’t sure if I was missing some docs

OScar avatar
OScar
03:57:27 PM

Cool, one thing you might be missing, is when using the CloudTrail module and say if you wish to use KMS key to encrypt the content. There is a parameter for the KeyID to pass. I am going to use it, as we need to encrypt. But I’ll know more on Monday!

OScar avatar
OScar
03:58:27 PM

My point being, not sure how the cross account kms works since the audit trail bucket is on its own account and we are using a kms key from another account

joshmyers avatar
joshmyers

Cool, good luck for your engagement on Monday!

joshmyers avatar
joshmyers

You can grant KMS perms for cross account, but ONLY if you don’t use the standard KMS key and instead create a custom one. IIRC you can’t change policies easily on default generated KMS keys

OScar avatar
OScar
05:46:44 PM

Thanks @joshmyers I did create a key via Terraform, I think the permissions or policy attached to that key may be the problem it sounds like.

OScar avatar
OScar
05:47:33 PM

So on the audit account bucket, we would provide said permissions?

joshmyers avatar
joshmyers

What is the problem you are seeing?

joshmyers avatar
joshmyers

You may need a trust policy on the key, something like:

joshmyers avatar
joshmyers

{ “Version”: “2012-10-17”, “Id”: “key-default-1”, “Statement”: [ { “Sid”: “Enable IAM User Permissions”, “Effect”: “Allow”, “Principal”: { “AWS”: “arnawsiam:root” }, “Action”: “kms:“, “Resource”: “” } ] }

joshmyers avatar
joshmyers

Note the resource is * because the policy is on the key.

OScar avatar
OScar
06:06:14 PM

Cool gonna give that a try later today, appreciate it @joshmyers

joshmyers avatar
joshmyers

No worries, hope it helps :–1:

joshmyers avatar
joshmyers

If first confirm if/what your problem is

joshmyers avatar
joshmyers

Note that I’ve seen CloudTrail lie to me about why access to a key was denied.

OScar avatar
OScar
06:11:07 PM

To answer your question @joshmyers when I clicked and opened a log entry from an account in the audit central bucket, I would get access denied.

joshmyers avatar
joshmyers

OK, so that would be calling a decrypt function, which needs access to the KMS key that was used to encrypt it. No need to tell it which key this is, it already knows, but probably can’t access it.

GreetBot avatar
GreetBot
12:08:56 PM

hey everyone give a warm welcome to @mumoshu! Good to have you here

GreetBot avatar
GreetBot
03:26:11 PM

hey everyone give a warm welcome to @Shane! Good to have you here

Shane avatar
Shane

@Erik Osterman (Cloud Posse) thanks for the invite

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

welcome @Shane & @mumoshu! awesome to have you guys onboard. We have an #atlantis channel where we can coordinate efforts. @Andriy Knysh (Cloud Posse) is the one on my team who worked on it.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hi @mumoshu and @Shane

:--1:2
Zapier avatar
Zapier
04:57:48 PM

@Erik Osterman (Cloud Posse) created a new channel #helmfile. Join if this sounds interesting!

rajcheval avatar
rajcheval

I am considering using your terraform code build module. I have a few questions: 1. Does it support git repos in CodeCommit? 2. currently it creates a IAM role and policy. In our scenario we will be running code build in 1 AWS account. We will actually run terraform inside code build. We will be using terraform to provision infrastructure in other AWS accounts. so the role inside code build has to be able to assume roles in other accounts. This is why I was trying to determine if it is possible to update the policy statements for the role created by the module. Perhaps we can pass in a role Arn. I am open to any other suggestion you have for me. Thanks

loren avatar
loren

i think we do exactly what you’re describing, far as using codebuild to run terraform. but we wrote our own module, rather than use cloudposse’s. we create iam policies in one module, and use the resulting policies in our ci module. here’s the ci module, https://github.com/plus3it/terraform-aws-codecommit-flow-ci

plus3it/terraform-aws-codecommit-flow-ci

Implement an event-based CI workflow on a CodeCommit repository - plus3it/terraform-aws-codecommit-flow-ci

:--1:1
rajcheval avatar
rajcheval

@Andriy Knysh (Cloud Posse) I can certainly open a PR. I reviewed the module code. My plan is to add a new parameter that will pass in the name of an existing Role ARN. In addition to this I was thinking of adding a new boolean that can be called createrole. If it is false role/policy/policy attachment will not be created. The supplied value of existing role will be used. Let me know if this is acceptable.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@rajcheval we can definitely provide an external IAM policy to the module. Please open a PR if you know how to do it. If not, open an issue and we’ll get to it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@loren thanks for sharing

:--1:2
GreetBot avatar
GreetBot
08:39:34 PM

hey everyone give a warm welcome to @gyoza! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hey John! welcome

gyoza avatar
gyoza

yo

gyoza avatar
gyoza

I work for blackberry now lol

gyoza avatar
gyoza

(shrug)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cool! whatcha working on over there?

gyoza avatar
gyoza

Well, Cylance got bought out, so we’re doing the same thing as of yesterday

gyoza avatar
gyoza

just operating under blackberry

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
08:43:58 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

heh, i missed the news

gyoza avatar
gyoza

yea its everywhere

gyoza avatar
gyoza

Did you see that hashicorp absorbed atlantis?

gyoza avatar
gyoza

im sure you have

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yep! that was a surprise announcement. hope to see #atlantis will get more engineering resources

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve forked, but hoping to ultimately converge on functionality

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are you guys using some flavor of atlantis?

gyoza avatar
gyoza

i was planning on doing it

gyoza avatar
gyoza

i had a demo working with bitbucket

gyoza avatar
gyoza

but i stopped as soon as i saw it was going to be in terraform-0.12

gyoza avatar
gyoza

also the pay-wallish features being released

gyoza avatar
gyoza

im looking forward to that

GreetBot avatar
GreetBot
02:01:44 AM

hey everyone give a warm welcome to @dio! Good to have you here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hi @dio

dio avatar

Hello!

2018-11-15

OScar avatar
OScar

@here i might have asked this, but maybe it wasn’t clear. Using the Geodesic Framework, setting up CloudTrails that send logs to central Audit account S3. All good there, I have that in place. My question is; what module do i use to send other AWS events to the bucket from all of the accounts?

OScar avatar
OScar

I am super stoked to be using this framework with a company that I hope will be a Case Study for it!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Other than CloudTrail events, it’s on a case by case basis

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Let’s move to #geodesic

GreetBot avatar
GreetBot
07:20:04 PM

hey everyone give a warm welcome to @bob! Good to have you here

1
1
bob avatar

@Igor Rodionov tnx for invite

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @bob! welcome

Igor Rodionov avatar
Igor Rodionov

@bob you are welcome

2018-11-14

GreetBot avatar
GreetBot
09:35:14 AM

hey everyone give a warm welcome to @Kasun! Good to have you here

1
GreetBot avatar
GreetBot
01:14:49 PM

hey everyone give a warm welcome to @Jorge Rodrigues! Good to have you here

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jorge Rodrigues @Kasun @JustPerfect @ay-ay-ron welcome!

GreetBot avatar
GreetBot
02:54:53 PM

hey everyone give a warm welcome to @nutellinoit! Good to have you here

1
:--1:1
nutellinoit avatar
nutellinoit

Hi everyone!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hey @nutellinoit

GreetBot avatar
GreetBot
11:19:29 PM

hey everyone give a warm welcome to @bchain! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @bchain! let me know if we can help with anything…

bchain avatar
bchain

Thanks, just ran across Cloudposse. Awesome stack of tools!

1
GreetBot avatar
GreetBot
03:49:10 AM

hey everyone give a warm welcome to @rbadillo! Good to have you here

1
rbadillo avatar
rbadillo

Hi Team, quick question

rbadillo avatar
rbadillo
cloudposse/prometheus-to-cloudwatch

Utility for scraping Prometheus metrics from a Prometheus client endpoint and publishing them to CloudWatch - cloudposse/prometheus-to-cloudwatch

rbadillo avatar
rbadillo

My understanding is that you tell that tool which exporter to scrape and it will send the metrics to AWS, is that correct?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/prometheus-to-cloudwatch

Utility for scraping Prometheus metrics from a Prometheus client endpoint and publishing them to CloudWatch - cloudposse/prometheus-to-cloudwatch

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

^ to scrape kube-state-metrics

rbadillo avatar
rbadillo

This is amazing, I was looking for a tool like this

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it has some limitations though (see the open issue), which we did not address yet

rbadillo avatar
rbadillo

let me check

rbadillo avatar
rbadillo

I see, so let me explain my situation

rbadillo avatar
rbadillo

let’s say that I have 1 EC2 server and I have the node_exporter running locally (At the Ec2 box) Port 9100. If I run your tool locally in the same EC2 server and tell it to scrape http://localhost:9100/metrics . Those metrics will ended up in CloudWatch. correct?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

If that’s a Prometheus endpoint, yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

It uses Prometheus client to read the metrics

rbadillo avatar
rbadillo

by prometheus endpoint you mean any /metrics endpoint that can be scrape by a Prometheus server ?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Yep

rbadillo avatar
rbadillo

perfect

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

To be clear, the tool was just an experiment. The correct official way would be to write a Prometheus operator using the operator framework, which would be managed by Kubernetes and would scrape metrics and send them to CloudWatch

rbadillo avatar
rbadillo

I understand but this is a great start for I was planning on writing

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you manage to resolve the issue, we accept PRs

rbadillo avatar
rbadillo

I think I know how to fix it

rbadillo avatar
rbadillo

I will take a look next week and let you know guys

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that would be awesome! thanks

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks @rbadillo

rbadillo avatar
rbadillo

you’re welcome, talk to you later

GreetBot avatar
GreetBot
06:01:27 AM

hey everyone give a warm welcome to @Rok Carl! Good to have you here

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hey @Rok Carl welcome!

Rok Carl avatar
Rok Carl

glad to be here

2018-11-13

Nikola Velkovski avatar
Nikola Velkovski

Is AWS IAM down for everyone ?

maarten avatar
maarten

not very fast here..

maarten avatar
maarten

aws iam get-user 0.52s user 0.22s system 59% cpu 1.252 total

Nikola Velkovski avatar
Nikola Velkovski

I get a 505

Nikola Velkovski avatar
Nikola Velkovski

ok now it’s ok

GreetBot avatar
GreetBot
08:48:17 PM

hey everyone give a warm welcome to @masterwill! Good to have you here

masterwill avatar
masterwill

Glad to be here

4
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @masterwill! let me know if we can help

GreetBot avatar
GreetBot
11:41:01 PM

hey everyone give a warm welcome to @ay-ay-ron! Good to have you here

ay-ay-ron avatar
ay-ay-ron

Hello everyone!

3
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey there!

GreetBot avatar
GreetBot
06:08:14 AM

hey everyone give a warm welcome to @JustPerfect! Good to have you here

1

2018-11-12

Zapier avatar
Zapier
04:11:28 PM

@sarkis created a new channel #prometheus. Join if this sounds interesting!

 avatar
05:00:01 PM

There are no events this week

Zapier avatar
Zapier
05:27:36 PM

@maarten created a new channel #airship. Join if this sounds interesting!

GreetBot avatar
GreetBot
11:42:50 PM

hey everyone give a warm welcome to @Amos! Good to have you here

Amos avatar

Hello there, I’ve got a quick question for someone about this project: https://github.com/cloudposse/terraform-aws-kops-vpc-peering

cloudposse/terraform-aws-kops-vpc-peering

Terraform module to create a peering connection between a backing services VPC and a VPC created by Kops - cloudposse/terraform-aws-kops-vpc-peering

Amos avatar

I’m attempting to peer a VPC in another region but get this error message

* module.cluster_to_vpn.module.vpc_peering.data.aws_vpc.requestor: data.aws_vpc.requestor: InvalidVpcID.NotFound: The vpc ID 'vpc-xxxxxx' does not exist
	status code: 400
Amos avatar

I have had success peering within my same region, but when I enter the vpc id for the remote region it breaks

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

welcome @Amos!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

let’s move to #terraform

2018-11-11

yurchenko avatar
yurchenko

yo guys i keep on getting “unable to connect to helm server” with build harness

yurchenko avatar
yurchenko

what build harness version should i be using

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrmmm we are using the latest version with helm on Codefresh

yurchenko avatar
yurchenko

hey, erik, weird it stopped working yesterday

yurchenko avatar
yurchenko

i am using 0.5.5

yurchenko avatar
yurchenko

might be a different problem

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We don’t do anything funky in the container

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We just install helm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

On alpine

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Let’s see… so can you try using kubectl?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Just to test the connection.

yurchenko avatar
yurchenko

it breaks on make/helm/upsert

yurchenko avatar
yurchenko

sure

yurchenko avatar
yurchenko

what should i run?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Let’s move to #release-engineering

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Actually let’s move to #codefresh

yurchenko avatar
yurchenko

ok

Zapier avatar
Zapier
06:26:03 PM

@Erik Osterman (Cloud Posse) created a new channel #codefresh. Join if this sounds interesting!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Or #codefresh :-)

2018-11-10

SweetOps avatar
SweetOps
06:00:11 PM

• Are you hiring? Post a link to your job ad in our #jobs channel.

• Looking for work? Let everyone know by promoting what you do in the #jobs channel by sharing your LinkedIn profile and GitHub links.

• Are you a freelancer/consultant? Feel free to engage in self-promotion in the #jobs channel by sharing a link to your website and a tidbit about what you do.

2018-11-09

GreetBot avatar
GreetBot
11:20:14 AM

hey everyone give a warm welcome to @g0nz0! Good to have you here

pecigonzalo avatar
pecigonzalo

Welcome!

g0nz0 avatar
g0nz0

hi there

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hey there!

mmarseglia avatar
mmarseglia

i’ve looked into Chef inspec to write tests for terraform, to ensure I’m creating resources correctly. anyone else doing testing beyond terraform validate and if so, what?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Taking a little different approach

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Goal is to make it easy for anyone to write tests without installing ruby and the kitchen sink

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Using bats-core we can write simple tests that will catch the essential problems

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Plan, apply, reapply test for idempotency, destroy, etc

mmarseglia avatar
mmarseglia

interesting..

mmarseglia avatar
mmarseglia

i’ll have to look into this too..

mmarseglia avatar
mmarseglia

thanks!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Sorry at DMV

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Will share example

mmarseglia avatar
mmarseglia

dept. of motor vehicles?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ya

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Takes for ever in Los Angeles

mmarseglia avatar
mmarseglia

if the experience is anything like my state (RI), i wouldn’t wish it on my worst enemy.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ya same!

OScar avatar
OScar

OMG #SweetOps is the shizzle! @here, just wanted to thank you for all the stuff you’ve put out on github. I’ve got a todo on my part to enter an issue and then actually implement it very soon on the KMS module! Thank you, just thank you for all this goodness!

:--1:6
1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @OScar! on behalf of all of us here, it really means a lot to us to know we’re helping others out…

1
OScar avatar
OScar

I am committed to contributing! Thanks so much @Erik Osterman (Cloud Posse)

Zapier avatar
Zapier
08:15:55 PM

@Erik Osterman (Cloud Posse) created a new channel #aws-reinvent. Join if this sounds interesting!

OScar avatar
OScar
09:13:13 PM

So I don’t look forward to going to DMV myself. Just had this beast shipped from FL and now registering in CA!

2
1
Matthew avatar
Matthew

Pick me up on Wilshire downtown LA

2
GreetBot avatar
GreetBot
09:51:01 PM

hey everyone give a warm welcome to @ellisera! Good to have you here

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh nice!! R8?

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

welcome @ellisera! have a look around…

ellisera avatar
ellisera

thank you!

2018-11-07

pecigonzalo avatar
pecigonzalo

Welcome!

pecigonzalo avatar
pecigonzalo

@OScar we use a similar workflow to what @Andriy Knysh (Cloud Posse) explained, I believe its the “main” way for multi account

GreetBot avatar
GreetBot
04:55:31 PM

hey everyone give a warm welcome to @Richard Pearce! Good to have you here

OScar avatar
OScar

@pecigonzalo thank you for confirming the approach, it is nice to know other folks agree!

:--1:1
OScar avatar
OScar

@Richard Pearce welcome to our group!

Richard Pearce avatar
Richard Pearce

thanks and greetings

Richard Pearce avatar
Richard Pearce

I was looking at https://github.com/cloudposse/terraform-null-smtp-mail do you have anything similar that can send slack messages ?

cloudposse/terraform-null-smtp-mail

Terraform module to send transactional emails via an SMTP server (e.g. mailgun) - cloudposse/terraform-null-smtp-mail

GreetBot avatar
GreetBot
05:06:57 PM

hey everyone give a warm welcome to @nian! Good to have you here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hi @Richard Pearce

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for slack, I believe we don’t have anything for terraform

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/slack-notifier

Command line utility to send messages with attachments to Slack channels via Incoming Webhooks - cloudposse/slack-notifier

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

which we use in Docker containers to send Slack messages about build/deploy status from CI/CD pipelines

Richard Pearce avatar
Richard Pearce

thank you @Andriy Knysh (Cloud Posse) I will take a look.

GreetBot avatar
GreetBot
05:09:39 PM

hey everyone give a warm welcome to @RobH! Good to have you here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

here is an example of a Codefresh pipeline step that sends a Slack message when the app gets deployed to staging

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  send_slack_notification:
    title: Send notification to Slack channel
    stage: Deploy
    image: cloudposse/slack-notifier
    environment:
    - "SLACK_WEBHOOK_URL=${{SLACK_WEBHOOK_URL}}"
    - SLACK_USER_NAME=CodeFresh
    - "SLACK_ICON_EMOJI=:rocket:"
    - SLACK_FALLBACK=Deployed to Staging environment
    - SLACK_COLOR=good
    - SLACK_PRETEXT=${{CF_COMMIT_MESSAGE}}
    - SLACK_AUTHOR_NAME=Auto Deploy Robot
    - SLACK_AUTHOR_LINK=<https://cloudposse.com/>
    - SLACK_AUTHOR_ICON=<https://cloudposse.com/wp-content/uploads/sites/29/2018/02/small-cute-robot-square.png>
    - SLACK_TITLE=App Updated
    - SLACK_TITLE_LINK=${{CF_BUILD_URL}}
    - "SLACK_TEXT=The latest changes have been deployed to\n :point_right: https://${{APP_HOST}}"
    - SLACK_THUMB_URL=<https://cloudposse.com/wp-content/uploads/sites/29/2018/02/SquareLogo2.png>
    - SLACK_FOOTER=Helm Deployment
    - SLACK_FOOTER_ICON=<https://cloudposse.com/wp-content/uploads/sites/29/2018/02/kubernetes.png>
    - SLACK_FIELD1_TITLE=Environment
    - SLACK_FIELD1_VALUE=Staging
    - SLACK_FIELD1_SHORT=true
    - SLACK_FIELD2_TITLE=Repository
    - SLACK_FIELD2_VALUE=${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}
    - SLACK_FIELD2_SHORT=true
    - SLACK_FIELD3_TITLE=Namespace
    - SLACK_FIELD3_VALUE=${{NAMESPACE}}
    - SLACK_FIELD3_SHORT=true
    - SLACK_FIELD4_TITLE=Branch/Tag
    - SLACK_FIELD4_VALUE=${{CF_BRANCH_TAG_NORMALIZED}}
    - SLACK_FIELD4_SHORT=true
    - SLACK_FIELD5_TITLE=Release
    - SLACK_FIELD5_VALUE=App
    - SLACK_FIELD5_SHORT=true
    - SLACK_FIELD6_TITLE=Commit
    - SLACK_FIELD6_VALUE=[${{CF_SHORT_REVISION}}](\$\{\{CF_COMMIT_URL\}\})
    - SLACK_FIELD6_SHORT=true
    - SLACK_FIELD7_TITLE=Build Time
    - SLACK_FIELD7_VALUE=<!date^${{CF_BUILD_UNIX_TIMESTAMP}}^{date_num} {time_secs}|Time format failed!!!>
    - SLACK_FIELD7_SHORT=true
    - SLACK_FIELD8_TITLE=Commit Time
    - SLACK_FIELD8_VALUE=<!date^${{GIT_TIMESTAMP}}^{date_num} {time_secs}|Time format failed!!!>
    - SLACK_FIELD8_SHORT=true
    - SLACK_FIELD9_TITLE=Trigger
    - [email protected]${{CF_BUILD_TRIGGER}}
    - SLACK_FIELD9_SHORT=true
    - SLACK_FIELD10_TITLE=Commit Author
    - [email protected]${{CF_COMMIT_AUTHOR}}
    - SLACK_FIELD10_SHORT=true
    when:
      condition:
        all:
          executeForPullRequest: "'${{CF_PULL_REQUEST_NUMBER}}' != ''"
          executeForOpenPR: "'${{CF_PULL_REQUEST_ACTION}}' != 'closed'"
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

welcome @RobH

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The same pattern we use for the smtp module can be extended to the slack notifier

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Since it consumes envs for everything …

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We do have a slack/sns module, but don’t think that’s what you are looking for

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-aws-sns-lambda-notify-slack

Terraform module to provision a lambda function that subscribes to SNS and notifies to Slack. - cloudposse/terraform-aws-sns-lambda-notify-slack

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Feature request: add a Slack provider · Issue #15187 · hashicorp/terraform

Hi, Since we already have a PagerDuty provider I think it would be cool to also have a Slack provider, this way we could automate some integrations like PagerDuty to Slack or other monitoring tools…

GreetBot avatar
GreetBot
05:35:35 PM

hey everyone give a warm welcome to @davidvasandani! Good to have you here

davidvasandani avatar
davidvasandani

Hi all!

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hey @davidvasandani

1
joshmyers avatar
joshmyers

Terraform is a CRUD tool, that doesn’t really align with slack notifications. The slack-notifer docker solutions work for me

:--1:1
sarkis avatar
sarkis

I agree with slack notifications bit, but I do see the use in having a provider to manage the notification webhooks and slack extensions…

sarkis avatar
sarkis

or rather codifying all that

SweetOps avatar
SweetOps
08:02:42 PM

Have we helped you in some way? We’d love to know! If you could leave us a testimonial it would make our day.

GreetBot avatar
GreetBot
01:54:28 AM

hey everyone give a warm welcome to @mpmsimo! Good to have you here

mpmsimo avatar
mpmsimo

Hey everyone!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hi @mpmsimo

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hey! Glad you joined Michael

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@mpmsimo is from the #atlantis community

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) is responsible for all of our extensions to Atlantis

mpmsimo avatar
mpmsimo

Awesome, good to know! You guys have some awesome information out on the net about Terraform + Atlantis. Definitely looking forward to learning from you all.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
How to use Terraform with Teams using Atlantis (#GitOps) attachment image

GitOps is where everything, including infrastructure, is maintained in Git and controlled via a combination of Pull Requests and CI/CD pipelines. Reduce the learning curve for new devs by providing a familiar, repeatable process. Use Code Reviews to catch bugs and increase operational competency. Pr

:100:2
:--1:3
2
mpmsimo avatar
mpmsimo

Yes! Funnily enough this is the article that pointed me over here. I was just giving that post and the slide deck a review earlier today.

How to use Terraform with Teams using Atlantis (#GitOps) attachment image

GitOps is where everything, including infrastructure, is maintained in Git and controlled via a combination of Pull Requests and CI/CD pipelines. Reduce the learning curve for new devs by providing a familiar, repeatable process. Use Code Reviews to catch bugs and increase operational competency. Pr

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

Zapier avatar
Zapier
06:11:57 AM

@Erik Osterman (Cloud Posse) created a new channel #terraform-aws-modules. Join if this sounds interesting!

2018-11-06

GreetBot avatar
GreetBot
08:19:17 AM

hey everyone give a warm welcome to @Nikola Velkovski! Good to have you here

3
1
Nikola Velkovski avatar
Nikola Velkovski

Hi everyone!

GreetBot avatar
GreetBot
09:20:21 PM

hey everyone give a warm welcome to @Tee! Good to have you here

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hey @Tee welcome

Tee avatar

Thanks

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@ramesh.mimit we did not use bitbucket with CodeBuild, but here’s how we access private GitHub repos using CodePipeline/CodeBuild (by providing the OAuth token with permissions to access the private repos)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-cicd

Terraform Module for CI/CD with AWS Code Pipeline and Code Build - cloudposse/terraform-aws-cicd

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-cicd

Terraform Module for CI/CD with AWS Code Pipeline and Code Build - cloudposse/terraform-aws-cicd

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-codebuild

Terraform Module to easily leverage AWS CodeBuild for Continuous Integration - cloudposse/terraform-aws-codebuild

ramesh.mimit avatar
ramesh.mimit

@Andriy Knysh (Cloud Posse) thank you for replying. I have noticed, its not terraform issue. Its CodeBuild UI issue.

OScar avatar
OScar

@here this is more of a question around some best practices regarding deploying central logging for multi-account AWS environment. My question is: do I deploy a single bucket for all accounts? Or a bucket per environment?

So we have say: Security_Test Logging_Test

Security_Prod Logging_Prod

What in your experience works best, if then all S3 bucket content needs to go into say Splunk anyway?

OScar avatar
OScar
12:09:34 AM

this may be wrong, so I am looking to see how you’ve done it.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

that’s not a simple question @OScar, depends on many factors. How do you provision the resources for each account. How do you control security and access to the accounts. I would provision a bucker per account since it’s easier to deploy and destroy stuff in one account w/o affecting other accounts

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

on the other hand, CloudTrail logs feel like special (security and audit reasons)

OScar avatar
OScar

Agreed, this is the scenario we call multi-account/Organization logging. Which I understand many companies want to implement

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so what we do with just CloudTrail logs, we use a separate audit account for the bucket (which only special people could access)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we provision the bucket for CloudTrail logs in that audit account

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and then provision CloudTrail(s) in the other accounts and point them to the bucket in audit

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) good stuff, so the bucket would be created literally in its own account for that sole purpose it sounds like vs. say, in an existing “master” account?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes completely separate account

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and you control what people can assume roles to access that account (admin or read-only)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and here CloudTrail(s) for all other accounts pointing to the bucket in audit https://github.com/cloudposse/terraform-root-modules/blob/master/aws/cloudtrail/main.tf

cloudposse/terraform-root-modules

Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and here how we control access to the account (add users to the groups) https://github.com/cloudposse/terraform-root-modules/blob/master/aws/iam/audit.tf

cloudposse/terraform-root-modules

Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) awesome, I am going to follow these steps to specifically provision the audit account and its cloudtrail and s3 bucket

:--1:1
GreetBot avatar
GreetBot
03:19:55 AM

hey everyone give a warm welcome to @javier.moya! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

welcome @javier.moya!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We finally got some cloudposse stickers! PM me your mailing address if you want some

2018-11-05

GreetBot avatar
GreetBot
10:32:18 AM

hey everyone give a warm welcome to @rolf! Good to have you here

 avatar
05:00:01 PM

There are no events this week

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

welcome @rolf :–1:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@OScar here are some examples for CloudTrail. We setup the bucket in a separate AWS audit account (so just a few people if any will have access to it)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and then setup CloudTrail(s) for all other accounts (root, prod, staging, etc) https://github.com/cloudposse/terraform-root-modules/tree/master/aws/cloudtrail

cloudposse/terraform-root-modules

Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) thank you, it looks like with the sample [main.tf](http://main\.tf) file on the root modules repo, we simply tell it to assume a role in order to deploy CloudTrail to that account as per your comment. Ok I will try this, thanks for your response!

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) @Erik Osterman (Cloud Posse) One other question; I managed to see logs for multiple accounts in that same bucket! But the data logged is basic. It appears i need to further configure in order to see VPC logs, or other services and sending logs to the S3 bucket, is this done via CloudWatch SNS Topics? or am I way off?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

use event_selector variable

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-cloudtrail

Terraform module to provision an AWS CloudTrail and an encrypted S3 bucket with versioning to store CloudTrail logs - cloudposse/terraform-aws-cloudtrail

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) how would i use the event_selector parameter for cloudtrail?

cloudposse/terraform-aws-cloudtrail

Terraform module to provision an AWS CloudTrail and an encrypted S3 bucket with versioning to store CloudTrail logs - cloudposse/terraform-aws-cloudtrail

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it’s a variable in the module

OScar avatar
OScar

yep understand ` event_selector = “”` what would a sample value look like?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

event_selector = [ { read_write_type = “All” include_management_events = true

data_resource {
  type   = “AWS::S3::Object”
  values = [“arn<img src="/assets/images/custom_emojis/aws.png" alt="aws" class="em em--custom-icon em-aws">s3:::“]
}   }  ]
OScar avatar
OScar

ah duh!

OScar avatar
OScar

ok my goal is to be able to log events from a lot of services

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it’s a list of maps

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

each map is for one service

:--1:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)


it looks like with the sample [main.tf> file on the root modules repo, we simply tell it to assume a role in order to deploy CloudTrail to that account</span](http://main.tf)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

assuming roles is not a feature of the module nor CloudTrail. It’s how we login to AWS accounts using geodesic. You need to consider the root modules as examples of invocation (not using them verbatim)

OScar avatar
OScar

Thank you @Andriy Knysh (Cloud Posse) understood on not using root modules verbatim - i do appreciate the ability to assume a role since executing terraform under said role would help deploy the CloudTrail to that account…but totally understand your point. I am looking at the AWS:cloudtrail bucket events object now..

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

OScar avatar
OScar

@here whenever i use the variable kms_key_id from this location https://github.com/cloudposse/terraform-aws-cloudtrail/blob/master/variables.tf i get a terraform error.

Error: Error running plan: 3 error(s) occurred:

  • module.cloudtrail.output.cloudtrail_id: Resource ‘aws_cloudtrail.default’ not found for variable ‘aws_cloudtrail.default.id’
  • module.cloudtrail.output.cloudtrail_arn: Resource ‘aws_cloudtrail.default’ not found for variable ‘aws_cloudtrail.default.arn’
  • module.cloudtrail.output.cloudtrail_home_region: Resource ‘aws_cloudtrail.default’ not found for variable ‘aws_cloudtrail.default.home_region’
cloudposse/terraform-aws-cloudtrail

Terraform module to provision an AWS CloudTrail and an encrypted S3 bucket with versioning to store CloudTrail logs - cloudposse/terraform-aws-cloudtrail

OScar avatar
OScar

any thoughts?

catdevman avatar
catdevman

Are you using this output immediately?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

sounds like a race condition

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you create the key in other module

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you can test it by providing the key_id directly

:--1:1
OScar avatar
OScar

thank you @Andriy Knysh (Cloud Posse) let me try that now

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) i think this is where the depends_on can be applied in terraform for cloudtrail module

OScar avatar
OScar

so that the key gets created first

OScar avatar
OScar

let me try that

catdevman avatar
catdevman

If you have both the key creation in a module and this module you will need to use a depends on but you’ll have to use a null_resource to do the connection (I believe until 0.12 is out and stable)

OScar avatar
OScar

hmm , @catdevman lost me a bit

catdevman avatar
catdevman

Sorry. Is your KMS creation in a module?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

to test the theory, use terraform apply -target to create the key first, and then -target to create the CloudTrail module. It could be something else so I’d test it first

maarten avatar
maarten

normally dependencies also work cross-module ie. if kms_id comes as an attribute from the aws_kms_key resource, a condition like this would not happen.

catdevman avatar
catdevman

Yeah just wanted to make sure that if the KMS key was being created by a module you didn’t run into: https://github.com/hashicorp/terraform/issues/18239

I know this has bit me numerous times in the past.

Module Cannot be Used in depends_on · Issue #18239 · hashicorp/terraform

Terraform Version v0.11.7 Terraform Configuration Files Note that the below Terraform configuration file leaves out the actual modules some-module and another-module for sake of readability. module…

maarten avatar
maarten

this is not really an issue as long as one can use the output of something as input for something else. Things get difficult when that’s not possible, and then your null_resource comes into play.

OScar avatar
OScar

Thanks everyone - I think I first need to make sure I am creating all the resources for AWS KMS in order to use the key. As of now I only have this snippet:

resource “aws_kms_key” “cloudtrail_kms_key” { description = “KMS key used for encrypting logs.” deletion_window_in_days = 10 }

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-kms-key

Terraform module to provision a KMS key with alias - cloudposse/terraform-aws-kms-key

:--1:1
OScar avatar
OScar

thank you again @Andriy Knysh (Cloud Posse) let me add this module real quick!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you do not provide the key, do you see the same errors? (just checking that it’s some kind of race condition issue and not something else)

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) if i don’t use the key parameter, cloudtrail module works

GreetBot avatar
GreetBot
09:19:38 PM

hey everyone give a warm welcome to @egonzales! Good to have you here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hi @egonzales

OScar avatar
OScar
09:27:35 PM
OScar avatar
OScar

@Andriy Knysh (Cloud Posse) checkout my entire code base here..

OScar avatar
OScar

I am still getting the error sadly

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

try terraform plan/apply -target=module.kms_key

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

to create the key first

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if works, we’ll take a look at the race condition

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and don’t use master for the module sources, pin to a release

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) this worked! terraform apply -target=module.kms_key

OScar avatar
OScar

key is created

OScar avatar
OScar

i’ll do the other command now

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(you need to update the description and alias for kms_key module)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

the other command could be just terraform apply (no need to use -target)

OScar avatar
OScar

Thanks @Andriy Knysh (Cloud Posse) to confirm, I created the CloudTrail and S3 Bucket successfully. The CloudTrail module does include the kms_key_id which I pasted prior to running terraform apply

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you don’t need to paste anything though

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

terraform apply -target=module.kms_key will save the created resource into the state file

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

next, terraform apply will read the state file and see the created key and use the ID

OScar avatar
OScar

looks like my s3 bucket policy is not working…

OScar avatar
OScar

` module.cloudtrail_s3_bucket.module.s3_bucket.aws_s3_bucket.default: 1 error(s) occurred:

  • aws_s3_bucket.default: Error putting S3 policy: MalformedPolicy: Policy has invalid resource status code: 400, request id: B2B6570282B3F8D2`
OScar avatar
OScar

that is the message after i executed my last command

OScar avatar
OScar

MalformedPolicy is a new error that happened only after I used the kms_key_id in the cloudtrails module

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so if you use kms_key_id = "", it works?

OScar avatar
OScar

actually if i omit the kms_key_id param for the cloudtrail module, the CloudTrail gets created

OScar avatar
OScar

if i use kms_key_id="" it does not throw and error

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

strange

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you could not find why it does that, we’ll have to take a look and apply the code with the key

OScar avatar
OScar

so i also tried copying the key id from the web console and pasting it in the kms_key_id value, and that is when i got the policy error for the s3 bucket

OScar avatar
OScar

So for now, i’ll have to forgo using encryption

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) just to be clear, I am not using Chamber, just wanted to create the key in AWS to use for the cloudtrail module

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea I understand. We did not, at least recently, test the CloudTrail module with an external KMS key . I’ll have to apply your code and see what happens. Thanks for finding the issues @OScar

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-cloudtrail

Terraform module to provision an AWS CloudTrail and an encrypted S3 bucket with versioning to store CloudTrail logs - cloudposse/terraform-aws-cloudtrail

OScar avatar
OScar

Hi @Andriy Knysh (Cloud Posse) just wanted to get back to you with this one. I did make it work, and mainly I was missing a KMS Policy that allowed CloudTrail to encrypt/decrypt. So I don’t believe the issue was with your KMS Module

cloudposse/terraform-aws-cloudtrail

Terraform module to provision an AWS CloudTrail and an encrypted S3 bucket with versioning to store CloudTrail logs - cloudposse/terraform-aws-cloudtrail

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so it’s an addition to the CloudTrail module?

OScar avatar
OScar

Yeah so because i was specifying a key in CloudTrail module, the key policy needed to be modified..

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

can you open a PR for that?

OScar avatar
OScar

@Andriy Knysh (Cloud Posse) sure, for the CloudTrail Module you mean? The PR would be to ensure that the KMS Policy is added or correct when a user specifies a kms_key_id…correct?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes please

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it’s a nice finding which we did not pay attention to

1
OScar avatar
OScar

I’ll clone and make a change, then do a PR for sure!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and let’s move to #terraform

1
ramesh.mimit avatar
ramesh.mimit

I am trying to create a codebuild project using bitbucket private repo using terraform, but when the project is created, on codebuild console, in source section, it shows the repo as public repo. However, I have used the private repo using Oauth

ramesh.mimit avatar
ramesh.mimit

#terraform anyone faced this issue before? or how to create the codebuild project for private repo using terraform..

ramesh.mimit avatar
ramesh.mimit

resource “aws_iam_role” “terraform” { name = “terraform”

assume_role_policy = <<EOF { “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Principal”: { “Service”: “codebuild.amazonaws.com” }, “Action”: “sts:AssumeRole” } ] } EOF }

resource “aws_iam_role_policy” “terraform” { role = “${aws_iam_role.terraform.name}”

policy = <<POLICY { “Version”: “2012-10-17”, “Statement”: [ { “Effect”: “Allow”, “Resource”: [ “” ], “Action”: [ “logs:CreateLogGroup”, “logs:CreateLogStream”, “logs:PutLogEvents” ] }, { “Effect”: “Allow”, “Action”: [ “ec2:CreateNetworkInterface”, “ec2:DescribeDhcpOptions”, “ec2:DescribeNetworkInterfaces”, “ec2:DeleteNetworkInterface”, “ec2:DescribeSubnets”, “ec2:DescribeSecurityGroups”, “ec2:DescribeVpcs” ], “Resource”: “” } ] } POLICY }

resource “aws_codebuild_project” “terraform” { name = “test-project” description = “test_codebuild_project” build_timeout = “5” service_role = “${aws_iam_role.terraform.arn}”

artifacts { type = “NO_ARTIFACTS” }

environment { compute_type = “BUILD_GENERAL1_SMALL” image = “aws/codebuild/nodejs:6.3.1” type = “LINUX_CONTAINER”

environment_variable {
  "name"  = "SOME_KEY1"
  "value" = "SOME_VALUE1"
}

environment_variable {
  "name"  = "SOME_KEY2"
  "value" = "SOME_VALUE2"
  "type"  = "PARAMETER_STORE"
}   }

source { type = “BITBUCKET” location = “https://[email protected]/rameshmimit/puppet-module-puppet.git” git_clone_depth = 1 auth { type = “OAUTH” } } }

2018-11-04

GreetBot avatar
GreetBot
06:17:13 AM

hey everyone give a warm welcome to @ramesh.mimit! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Welcome Ramesh!

2018-11-03

GreetBot avatar
GreetBot
07:47:00 AM

hey everyone give a warm welcome to @nnamani.kenechukwu! Good to have you here

nnamani.kenechukwu avatar
nnamani.kenechukwu

I am so excited to be here

1
nnamani.kenechukwu avatar
nnamani.kenechukwu

Just a quick question. What could cause the Cloudfront loogging to S3 to abruptly stop? I have checked that the S3 has write permission enabled. Also, there were old logs from the cloud front in the S3 but not new ones.

maarten avatar
maarten

Hi @nnamani.kenechukwu

maarten avatar
maarten

So the cloudfront logs are not flushed in real time, I forgot how what the timeframe is in which the logs will be flushed to s3. So if a particular endpoint is not receiving enough traffic it can take a while. That could be one reason.

maarten avatar
maarten

“Note, however, that some or all log file entries for a time period can sometimes be delayed by up to 24 hours. When log entries are delayed, CloudFront saves them in a log file for which the file name includes the date and time of the period in which the requests occurred, not the date and time when the file was delivered. “

nnamani.kenechukwu avatar
nnamani.kenechukwu

@maarten thank you for your prompt response. The last logs from the cloudfront to the S3 buckets dates back to 2017 which makes me think that something is wrong.

maarten avatar
maarten

haha, yes, that’s not a normal delay

maarten avatar
maarten

I’d use AWS Support for something like this, they can figure that out.

maarten avatar
maarten

A few things you can do tho

maarten avatar
maarten

Did logging stop or did traffic to the CF distribution stop

maarten avatar
maarten

If you run apache bench against the cloudfront endpoint for a while you should see logs quite quickly normally

nnamani.kenechukwu avatar
nnamani.kenechukwu

Alright. I will go with your recommendations. But I do know that the traffic to the CF didn’t stop. However l, will run the Apache bench and also reach out to the AWS support.

nnamani.kenechukwu avatar
nnamani.kenechukwu

Thanks a lot

maarten avatar
maarten

Sure, good luck!

GreetBot avatar
GreetBot
08:09:17 PM

hey everyone give a warm welcome to @github140! Good to have you here

1
GreetBot avatar
GreetBot
08:26:49 PM

hey everyone give a warm welcome to @OScar! Good to have you here

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Nice coffee cup! :-)

OScar avatar
OScar

@here I am trying to setup CloudTrail logs for several AWS accounts. I deployed the trail and bucket with my config using this terraform repo https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket . Do I have to deploy via Terraform with each of the accounts? sorry, first time using CloudTrail…

cloudposse/terraform-aws-cloudtrail-s3-bucket

S3 bucket with built in IAM policy to allow CloudTrail logs - cloudposse/terraform-aws-cloudtrail-s3-bucket

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes that’s accurate

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you check our root modules project on GitHub you’ll see some examples

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Sorry - on phone so hard to link out

2018-11-02

joshmyers avatar
joshmyers

1.9billion valuation!? Welp

mmarseglia avatar
mmarseglia

has anyone worked on multi-region S3 website architecture in Terraform, similar to what’s documented here https://read.iopipe.com/multi-region-s3-failover-w-route53-64ff2357aa30

Multi-region S3 failover /w Route53 – IOpipe Blog attachment image

A couple weeks before writing this post, AWS had a single-region failure of S3. It was the worst failure of S3 ever, and it took down many…

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@mmarseglia sounds interesting, but no, we did not work on that

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

sounds like you need to create a) buckets with cross–region replication; b) health checks for buckets and CloudFront; c) Failover policy in Route53

mmarseglia avatar
mmarseglia

i tried doing some of it but got stuck on the s3_bucket_policy

mmarseglia avatar
mmarseglia

i have a module that creates the buckets. to do that, i had to pass two providers each configured for a different region.

:--1:1
mmarseglia avatar
mmarseglia

but I got stuck on the policy because it doesn’t accept provider as a parameter. so it tries to create a policy using the default provider, which is in a different region than the replica bucket

mmarseglia avatar
mmarseglia

i’d have to get some code available to fully illustrate, i guess.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Setting Up Permissions for CRR - Amazon Simple Storage Service

When setting up cross-region replication, you must acquire necessary permissions as follows:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

looks like you need just one policy

mmarseglia avatar
mmarseglia

oh good!

mmarseglia avatar
mmarseglia

i think i see a way forward

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Yea the policy should be for the same provider as the source bucket

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Both source and destination buckets must have versioning enabled.

The source and destination buckets must be in different AWS Regions.

Amazon S3 must have permissions to replicate objects from the source bucket to the destination bucket on your behalf.

2018-11-01

GreetBot avatar
GreetBot
07:16:17 AM

hey everyone give a warm welcome to @solairerove! Good to have you here

1
maarten avatar
maarten

hi @solairerove what brings you here?

solairerove avatar
solairerove

hi, erik send me link after interview

maarten avatar
maarten

That’s great, welcome

solairerove avatar
solairerove

thx)

GreetBot avatar
GreetBot
09:46:21 AM

hey everyone give a warm welcome to @inactive! Good to have you here

1
maarten avatar
maarten

Hi @inactive, what are you working on ?

inactive avatar
inactive

Hi Maarten

inactive avatar
inactive

I am currently working with Terraform. I found one of your modules and one thing led to another and eventually got me to join your Slack channel

GreetBot avatar
GreetBot
01:04:46 PM

hey everyone give a warm welcome to @onzyone! Good to have you here

onzyone avatar
onzyone

hello everyone

1
maarten avatar
maarten

Ah there are a few who can help you out in case you get stuck, especially in PST working hours

inactive avatar
inactive

Yes, thanks. I submitted an issue on Github a few days ago. Just waiting for a response.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks @inactive, remind me what repo you submitted the issue to

inactive avatar
inactive
Provide a way to specify visibility for Elastic Load Balancers to be internal · Issue #58 · cloudposse/terraform-aws-elastic-beanstalk-environment

Hello, I recently started using your module and was able to create a new Elastic Beanstalk environment successfully. However, I need my load balancers to be internal facing only. I could not find a…

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

need to look into that. If you know how to implement and test, submit a PR, we’ll review promptly

inactive avatar
inactive

thank you, @Andriy Knysh (Cloud Posse). I am not entirely sure how to implement this myself. I know that it is possible to do, since it all boils down to a cloud formation template

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

welcome @solairerove @inactive @onzyone

inactive avatar
inactive

hello!

onzyone avatar
onzyone

hi there

GreetBot avatar
GreetBot
01:14:53 PM

hey everyone give a warm welcome to @andreveelken! Good to have you here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you have questions or need help, just ask

onzyone avatar
onzyone

Thank you!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

hey @andreveelken

andreveelken avatar
andreveelken

hey

onzyone avatar
onzyone

I am looking for a simple module that I can use to create CNAMES for AWS resource endpoints

onzyone avatar
onzyone

I did notice that there is one, but it also creates a cert …

onzyone avatar
onzyone
cloudposse/terraform-aws-acm-request-certificate

Terraform module to request an ACM certificate for a domain name and create a CNAME record in the DNS zone to complete certificate validation - cloudposse/terraform-aws-acm-request-certificate

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

that one is to request an SSL cert and validate using DNS

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

take a look at these:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-route53-alias

Terraform Module to Define Vanity Host/Domain (e.g. [brand.com](http://brand\.com)) as an ALIAS record - cloudposse/terraform-aws-route53-alias

onzyone avatar
onzyone

I looked at this one, but the type is hardcoded

cloudposse/terraform-aws-route53-alias

Terraform Module to Define Vanity Host/Domain (e.g. [brand.com](http://brand\.com)) as an ALIAS record - cloudposse/terraform-aws-route53-alias

onzyone avatar
onzyone

but it makes sense based on the name of the module

onzyone avatar
onzyone

line 11 ` type = “A”`

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-route53-cluster-zone

Terraform module to easily define consistent cluster domains on Route53 (e.g. [prod.ourcompany.com](http://prod\.ourcompany\.com)) - cloudposse/terraform-aws-route53-cluster-zone

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-route53-cluster-hostname

Terraform module to define a consistent AWS Route53 hostname - cloudposse/terraform-aws-route53-cluster-hostname

onzyone avatar
onzyone

This one looks like what I am looking for

cloudposse/terraform-aws-route53-cluster-hostname

Terraform module to define a consistent AWS Route53 hostname - cloudposse/terraform-aws-route53-cluster-hostname

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

:–1:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if just for plain RDS (not Aurora), we have a similar module https://github.com/cloudposse/terraform-aws-rds/blob/master/main.tf#L91

cloudposse/terraform-aws-rds

Terraform module to provision AWS RDS instances. Contribute to cloudposse/terraform-aws-rds development by creating an account on GitHub.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

terraform-aws-route53-alias is to create aliases to existing AWS resources

maarten avatar
maarten

Aliases are the way to go saves the requester another request.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes

onzyone avatar
onzyone

interesting …

onzyone avatar
onzyone

I have always used cnames

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-rds-cluster

Terraform module to provision an RDS Aurora cluster for MySQL or Postgres - cloudposse/terraform-aws-rds-cluster

onzyone avatar
onzyone

but if there is a better way of doing it … I am open to try

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

better way is not to use CNAMEs as @maarten pointed out

onzyone avatar
onzyone

ya it is for a Postgres rds instance right now

onzyone avatar
onzyone

so the rds-cluster module will work well too

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you have already AWS resources, you can always create an ALIAS or a new hostname in a diff DNZ zone to point to a resource

onzyone avatar
onzyone

this is perfect

onzyone avatar
onzyone

what is the recommended way for sending in the zone_id?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you only need CNAME in prob just one case: you have a domain name (e.g. [example.com>) in let’s say GoDaddy, and you want to point it to a diff domain let’s say <http://prod.example.net|prod.example.net](http://example\.com) in Route53

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

zone_id you can find in Route53

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

or, if you create it in TF, you already have that ID as the output

onzyone avatar
onzyone

ya … it is already in Route53, i can get it via cli too

:100:1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and you are right you want to use a new hostname for an RDS instance. Because if you use just aws_db_instance.default.address and give it to the application, next time you recreate the instance, the endpoint will change (and the app crash)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

with a dedicated hostname (e.g. [postgres.example.net](http://postgres\.example\.net)), even if you recreate the instance, the hostname will be updated as well to point to the new instance

onzyone avatar
onzyone

ya that is what I was driving to

onzyone avatar
onzyone

this is the first time that I have used TF … but it is a great tool so far

onzyone avatar
onzyone

and cloudposse github is great too!!!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea, everybody here loves it

2
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

let’s move to #terraform

GreetBot avatar
GreetBot
03:40:58 PM

hey everyone give a warm welcome to @dustinvb! Good to have you here

SweetOps avatar
SweetOps
04:06:23 PM

Have you signed up for our Newsletter? It covers everything on our technology radar. Receive updates on what we’re up to on GitHub as well as awesome new projects we discover.

GreetBot avatar
GreetBot
04:41:19 PM

hey everyone give a warm welcome to @btai! Good to have you here

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @dustinvb! check out the #release-engineering channel - that’s probably the most relevant

    keyboard_arrow_up