#geodesic (2019-01)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2019-01-30

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’ve added okta support

2019-01-26

chrism avatar
chrism

Packer runs fine on windows though , you can run it in docker as well. Very few reasons for vbox thesedays

loren avatar
loren

i understand that, it’s really a vagrant box that packer is creating, with a virtualbox provider

loren avatar
loren

of course packer runs fine itself on windows, but packer launches virtualbox locally to create the image/ovf, and then converts it to a vagrant box

loren avatar
loren

so, somehow or other, i need packer to be able to run virtualbox

2019-01-25

chrism avatar
chrism

and alpine is awesome for docker images

cool-doge2
chrism avatar
chrism

I’m trying to glue a couple of things together but I’ve a feeling the only way it’ll work with the audit/root org setup is if I change the cloudwatch module

by default cloudtrail in root/data etc creates a log-group to match the org namespace (it doesnt set the cloud_watch_logs_role_arn/cloud_watch_logs_group_arn so the resulting log group is null as their’s no cloudwatch)

chrism avatar
chrism

adding

module "cloudwatch_log" {
  source            = "git::<https://github.com/cloudposse/terraform-aws-cloudwatch-logs.git?ref=tags/0.2.2>"
  namespace         = "${var.namespace}"
  stage             = "${var.stage}"
  retention_in_days = 180
}

and the cloudwatch role+group arn to the cloudtrail tries to create module.cloudwatch_log.aws_iam_role.default which fails

chrism avatar
chrism

aws_iam_role.default: Error creating IAM Role testns-root-log-group: MalformedPolicyDocument: Invalid principal in policy: com.amazon.balsa.error.InvalidPolicyException: The passed in policy has a statement with no principals!

chrism avatar
chrism

Remind me to mount the local volume on a folder so I can sync in changes rather than typing make all every few minutes

joshmyers avatar
joshmyers

You homedir gets mounted to /localhost in geodesic

chrism avatar
chrism
04:39:44 PM
picard_fail1
chrism avatar
chrism

you know I’ve seen that everytime I open the shell and still didnt click

joshmyers avatar
joshmyers

For exactly that reason

chrism avatar
chrism

its been a long week

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sorry for our contribution to that!!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so our (my) dev workflow is to develop in my regular IDE

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

keep a geodesic shell open to the dev/sandbox account

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cd /localhost/...../project

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and iterate that way

:100:1
chrism avatar
chrism

that makes perfect sense

chrism avatar
chrism

not randomly typing vi and duplicating stuff backwards like a pleb

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, and forgetting that the one thing that you actually made in the shell manually

chrism avatar
chrism

oh that never happens whistles innocently

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s something else that’s cool:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Add Better Support for Minikube · Issue #204 · cloudposse/geodesic

what Add support for Docker for Mac (DFM) Kubernetes or Minikube why Faster LDE, protyping Testing Helm Charts, Helmfiles howto I got it working very easily. Here&#39;s what I did (manually): Enabl…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I haven’t formalized a workflow around it, but I did a POC

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we can actually use geodesic with minikube

chrism avatar
chrism

neat; we all run rancher locally / docker for desktop minikubes handy for quick pocs

chrism avatar
chrism

ahh localhost mounts to the user folder. Still need to map all my work lives on a separate ssd.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

look at /usr/local/bin/<script>

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think you can override it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if not, we would accept PR to support overriding it

chrism avatar
chrism

cool I’ll have a dig as I assume the home paths used for aws-vault

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that too

chrism avatar
chrism

its easy enough to get docker to dance

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and a local “Cache”

chrism avatar
chrism

windows subsystem; where home is home but not quite home

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are you using WSL?

chrism avatar
chrism

yep

chrism avatar
chrism

ubuntu 18

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, that was A MAJOR pain

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it took us a few PRs to get that right

chrism avatar
chrism

works REALLY well so you did a good job there

chrism avatar
chrism

Think I got the cloudwatch issue down. I already had it scripted up before splitting all the orgs so I’ve made it sorta hybrid for the time being. https://gist.github.com/ChrisMcKee/0ca78c207fa7c3aca3b973c824aab069

chrism avatar
chrism

Now to wait for the SIEM monitoring company to start screaming… probably shouldnt have changed it all on a friday

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what company do you use? (curious)

chrism avatar
chrism

dm’d

joshmyers avatar
joshmyers

Hopefully not AlienVault

joshmyers avatar
joshmyers

I remember doing some due diligence a while back and found private keys in their AMIs picard_fail

1
chrism avatar
chrism

lol cough

chrism avatar
chrism

we use all the aws stuff as well as the managed service but it all feeds into the one place for monitoring as well as the aws notifications. Much as I’d love to trust a single service

chrism avatar
chrism

There’s a distinct lack of silver bullets

loren avatar
loren

any good guides on using docker in WSL? i was looking into it a while ago, but at the time it seemed either/or not both

chrism avatar
chrism
Setting Up Docker for Windows and WSL to Work Flawlessly attachment image

With a couple of tweaks the WSL (Windows Subsystem for Linux, also known as Bash for Windows) can be used with Docker for Windows.

loren avatar
loren

Thanks! I swear it was probably the week before this article posted that I was last looking for an updated guide! Can’t keep up!

Setting Up Docker for Windows and WSL to Work Flawlessly attachment image

With a couple of tweaks the WSL (Windows Subsystem for Linux, also known as Bash for Windows) can be used with Docker for Windows.

:--1:1
chrism avatar
chrism

Works fine

loren avatar
loren

ahh, now i remember the problem, i need to be able to use virtualbox locally (for packer, nothing persistent), and installing docker for windows enables hyper-v, which breaks virtualbox

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhhh yea, I’ve had similar issues when trying to use other VMs

loren avatar
loren

Maybe I can get terraform to launch i3 metal instances to run virtualbox and packer for me

2019-01-24

raehik avatar
raehik

Appears that geodesic doesn’t have a terminfo file for Termite

raehik avatar
raehik

upon running the geodesic image script:

raehik avatar
raehik
 ✗   (none) ~ ⨠  echo $TERM
xterm-termite
tput: unknown terminal "xterm-termite"
tput: unknown terminal "xterm-termite"
-> Run 'assume-role' to login to AWS
tput: unknown terminal "xterm-termite"
tput: unknown terminal "xterm-termite"
(~4 more times)
raehik avatar
raehik

for anyone else with this problem, run it in tmux or simply export TERM=screen-256color

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks for posting your fix!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
unknown terminal "xterm-termite" · Issue #365 · cloudposse/geodesic

what On Arch linux someone reported in slack that they get this error: ✗ (none) ~ ⨠ echo $TERM xterm-termite tput: unknown terminal &quot;xterm-termite&quot; tput: unknown terminal &quot;xterm-term…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

geodesic is just alpine under the hood

2019-01-23

raehik avatar
raehik

font recs for a stubborn Arch user

raehik avatar
raehik

symbola is awful and emoji fonts don’t include the math symbols in geodesic prompt T_T

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’ve had a few issues in the past related to the prompt

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(fwiw, on OSX and Amazon Linux it’s working for me)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Can’t lookup the font right now.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you set PROMPT_STYLE=plain it should use no funky characters

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

raehik avatar
raehik

Cheers. stick with Symbola for now. Surprised that DejaVu doesn’t include those chars

chrism avatar
chrism

I keep seeing DEPRECATED: /usr/local/bin/init-terraform is no longer needed. Use tfenv instead. init-terraform still works (and is still prompted as you change folders), but how is the other supposed to work?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sec (docs haven’t been updated)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

In each project folder in /conf

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Add a file like this one:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

\# Import the remote module
export TF_CLI_INIT_FROM_MODULE="git::<https://github.com/cloudposse/terraform-root-modules.git//aws/ecs?ref=tags/0.33.0>"
export TF_CLI_PLAN_PARALLELISM=2

use terraform
use tfenv
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Read up on this here:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/tfenv

Transform environment variables for use with Terraform (e.g. HOSTNAMETF_VAR_hostname) - cloudposse/tfenv

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

basically, we discovered we don’t need wrappers to use terraform with remote state and remote modules

chrism avatar
chrism

Ah cool; similar to .env files in node projects

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we can populate TF_CLI_ARGS* envs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes

chrism avatar
chrism

thats nice; though I’ve nothing against a good shell script

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so this common direnv interface works well with both terraform, kops etc.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea, though I’m not crazy about string manipulation in bash

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and if we’re going to use a formal language, default to go so we don’t need to install a lot of deps and can easily distribute with cloudposse/packages

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so you’ll dig this:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

direnv has support for a stdlib

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that can be extended

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so we’ve added our own extensions…… (partially for backwards compat)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but users can add their own helpers

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(or override ours)

chrism avatar
chrism

Cool; its a damn sight easier to script stuff up and know it’ll run in linux. And as a general distribution mechanism rather than having to keep repos filled with scripts

chrism avatar
chrism

All makes sense though; the make file systems a nice addition; I tend to write them with terraform envs, docker dealt with making sure people had the tools but linking it all up to the WSL auth (aws vault) has dealt with the headache of keeping things secure

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea, really excited with how it’s all come together.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Optionally, add a Makefile like this one:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


\## Fetch the remote terraform module
deps:
	terraform init


\## Reset this project
reset:
	rm -rf Makefile *.tf .terraform
me1249 avatar
me1249

Hey guys - do you generally recommend that people create a new docker image based on the cloudposse geodesic image, or fork the geodesic repo and customize as needed?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think it depends

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Here’s how I recommend you get started.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Get everything up and running using cloudposse/geodesic; reduce the moving pieces

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then consider forking cloudposse/geodesic and adding your extensions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
nikiai/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform - nikiai/geodesic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s an example of someone else - they forked and switched to debian =P

me1249 avatar
me1249

Thanks

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Use geodesic image as base for your own image

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Look at cloudposse/testing.cloudposse.co as an example

me1249 avatar
me1249

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(though [testing.cloudposse.co](http://testing\.cloudposse\.co) is quite out of date)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The reference-architectures are most current

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@me1249 got your PRs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’m going to get that taken carre of

me1249 avatar
me1249

2019-01-22

chrism avatar
chrism

@Erik Osterman (Cloud Posse) https://github.com/cloudposse/terraform-root-modules/commit/547302316db329f492411bff44aae045bd70e430#diff-b4b339745fca6a54353bf7f392d8dc60 was the naming change intentional? Terraform hates refactoring

Terraform will perform the following actions:
  - aws_organizations_account.audit
  + module.audit.aws_organizations_account.default
chrism avatar
chrism

breaks out terraform state mv

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes, the terraform-root-modules are best forked by an organization

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

they are your starting off point

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we refactored all the root modules to support SSM and made them more DRY by using local submodules

chrism avatar
chrism

Yeah I quite like the submodule approach (as it saves extra shit to pull /versions to track)

Forking Root is something I shall definitely be doing.

Whats with the SSM though? its just storing ARNs (in this bit anyway) which are all named by convention

chrism avatar
chrism

I should probably do the fork first and make use of that good old S3 versioning to uncorrupt my state file

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so the SSM allows us to programmatically reference variables by name

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. we cannot progammatically reference an output by name from the terraform remote state provider

chrism avatar
chrism

:–1: cool

joshmyers avatar
joshmyers

Refactoring Terraform is painful.

chrism avatar
chrism

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!! Not half

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Nico You might run into a few bugs with the referrence-architectures

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

there are a few people who have recently run through it and can maybe help if you get stuck

Nico avatar

ok cool, I’ll definitely reach out

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(check the open issues)

Nico avatar

cool, thank you

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I can also give you (or anyone else) an overview via Zoom

tolstikov avatar
tolstikov

please ping me when you’ll schedule it with someone, I’ll try to join!

Nico avatar

yeah I will definitely need some guidance once I actually start working on the architecture

2019-01-21

Jan avatar

ha that gnu make on osx thing is a pita

tolstikov avatar
tolstikov

brew install remake

Jan avatar
rocky/remake

Enhanced GNU Make - tracing, error reporting, debugging, profiling and more - rocky/remake

chrism avatar
chrism

Is there a reason the org names are filtered to a predefined list rather than just allowing anything?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but I would be open to hearing your use-case

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have for an early customer chosen one unique TLD per account so they don’t even share that.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also, we make the distinction between a company’s “service discovery domain” and it’s “branded” or “vanity” domains.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think it makes pretty good sense to keep strong consistency on the service discovery domain, while letting the branded domains stray as far as necessary .

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

branded domains are what the public uses.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhh, the org “account” names

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Soooooooooo yes, that’s terraform being terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s very tricky to reference dynamic properties

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use SSM parameter store to reduce that burden

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but the the deal is this, we define some accounts and those accounts have some purpose that we cannot easily generalize without going much deeper into code generation.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

On top of that, AWS doesn’t allow easy programatic deletion of accounts, so it’s incredibly painful to test. Requires, logging into each suborg manually and setting up the master account creds, before going to terraform to try and delete it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s to reduce the number of settings

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the more predictable they are, they more assumptions we can make

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

… if we’re talking about terraform-root-modules, the idea here is that these are hyper opinionated.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

All of our other terraform-aws-* modules are designed to be building blocks

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but at some point, you need to stop generalizing and implement something. That’s terraform-root-modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@chrism would be happy to give you a high level over view of everything and some of the design decisions.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Of course, we want to capture all that in documentation, but that is not caught up.

chrism avatar
chrism

Everyone loves documenting things

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

lol

chrism avatar
chrism

I was thinking more along the lines of root / audit / maybe data would be opinionated, if its not in the list assume its more of a blank slate like dev or prod sans the pre-included docker

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

soooooo what I’m leaning towards right now is adding more types of accounts

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

rather than renaming the current accounts

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it makes it impossible to document a moving target

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what are the account names you’d like to see?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

chrism avatar
chrism

To be honest its probably a bit domain specific to us but data-prod rather than just data. and something akin to dev-{team}. I can see myself being buried in orgs which isn’t great but at the same time it reduces risk around shared areas between teams / improves the ability to audit / split billing.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

another idea that @Jan had is to supply another “set” of accounts

chrism avatar
chrism

_I also realise i can do similar with policies in aws like we do in our original 1 org structure _

chrism avatar
chrism

I’m simply thinking of dumb accounts basically that you can setup as needed / make use of the centralised auth with

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So the part that I found tricky was NS delegation.

Jan avatar

The dns delegation 8s easy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

didn’t you end up hardcoding it?

Jan avatar

Nope I created a private an additional module for making a private zone

Jan avatar

Depedancy is having a vpc before

Jan avatar

But that fits into my plan of creating a vpc that fits into my subnet strategy and have a k8s cluster launch into that vpc

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s totally possible to do what you want to do, just with trade offs.

Jan avatar

I found the getting the networking / vpc stuff more interesting

1
chrism avatar
chrism

Admittedly not using any of the K8 / domain stuff as we use rancher for K8 management and multiple vpcs per env (we just run stateless apps in k8)

chrism avatar
chrism

Tapping my foot waiting for AWS to match Azure + Google and make EKS free

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, seriously

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s really far behind in as it relates to k8s

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Would love to see the rancher stuff. I bet we could expand the geodesic story to support the rancher cli.

chrism avatar
chrism

Ranchers pretty neat for what it does (Api proxy / rbac auth); we run the rancher control plane (HA) from our datacenter rather than AWS so its a bit of manual wiring to get the K8 plane setup. But it does talk to EKS which is good (from a compliance front) as it takes away 6 vms we’d otherwise have to manage

chrism avatar
chrism

In an ideal world it would be more terraform friendly

chrism avatar
chrism

as would everything

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, getting everything in under terraform is nice

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in the end though, it’s hard to expect one tool to do the job

chrism avatar
chrism

live the dream

chrism avatar
chrism

We’ve used it to manage dualDNS providers for a couple of years now; I just hope the 0.12 conversion tool works well

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, srsly

chrism avatar
chrism

I just see the words breaking change; look at all the repos of terraform and think FOOOOK

chrism avatar
chrism

its either that or tag everything to run with <.12

2019-01-19

2019-01-18

daveyu avatar
daveyu

trying to hack my way through using the reference architecture with an existing root account. got prod and audit accounts created with make root, but now running into this:

➜ make children
make[1]: *** No rule to make target `prod/validate'.  Stop.
make: *** [prod/validate] Error 2
daveyu avatar
daveyu

artifacts/Makefile.env looks right:

ACCOUNTS_ENABLED = prod audit

and the account IDs are in artifacts/acounts.tfvars

daveyu avatar
daveyu

where else should I look for problems?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhhh yes, there’s a bug.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

edit tasks/Makefile.child

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

find the macro called child

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

there is an = sign after it. that works in some makes and not in others

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

remove the = and it will work.

daveyu avatar
daveyu

beautiful. thanks

daveyu avatar
daveyu

if it helps..

GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jan ran into this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we spent some time debugging it. Yea, I’ve been developing it on an AWS WorkSpace instance which is Amazon Linux

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

on OSX this happens

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Fix Make Error · Issue #10 · cloudposse/reference-architectures

what When trying to hack my way through using the reference architecture with an existing root account, I got prod and audit accounts created. When I run make root, it errors ➜ make children make[1…

loren avatar
loren

sometimes you need to make make!

make/install: MAKE_MAKE_VERSION ?= 4.1
make/install: MAKE_SOURCE ?= <http://ftp.gnu.org/gnu/make/make-$(MAKE_MAKE_VERSION).tar.gz>
make/install: | $(BIN_DIR)
	@echo "[make]: MAKE_SOURCE=$(MAKE_SOURCE)"
	$(CURL) -o make.tar.gz "$(MAKE_SOURCE)"
	tar xvf make.tar.gz
	cd make-* && ./configure --bindir=$(BIN_DIR) && sudo make install
	rm -rf make*
	which make
	make --version
Fix Make Error · Issue #10 · cloudposse/reference-architectures

what When trying to hack my way through using the reference architecture with an existing root account, I got prod and audit accounts created. When I run make root, it errors ➜ make children make[1…

party_parrot1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Lol

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Make inception

loren avatar
loren

Yeah, wasn’t sure it would work, but it does! Got tired of running into problems on different platforms due old make versions, and the newer features I like to use

tamsky avatar
tamsky

I commented on the GH issue and shared my method for detecting and bypassing as best I could, the old 3.81 version of make.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrmm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so I think @Jan had success on OSX Make 3.81

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

without updating make

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(using the earlier fix in the issue linked, by removing the extraneous =)

Jan avatar

Yep, all worked after removing the trailing =

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
12:31:54 AM
:--1:1
pecigonzalo avatar
pecigonzalo

Im a simple man, I see colors and design, I upvote

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Geodesic inside of tmate over the web

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Add tmate-session script by osterman · Pull Request #359 · cloudposse/geodesic

what Support remote debugging of geodesic containers why This is useful for pairing with other developers or debuging remote k8s pods, ECS tasks, atlantis, etc.. references #356 (related use-c…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

mumoshu avatar
mumoshu
01:34:09 AM

@mumoshu has joined the channel

2019-01-17

nutellinoit avatar
nutellinoit

★ repo

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
09:23:46 AM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

WOOHOO! thanks @nutellinoit

2
Jan avatar

What version of kops / k8s is supported currently?

joshmyers avatar
joshmyers
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

joshmyers avatar
joshmyers
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan we recently deployed 1.10.12 with kops

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
kubernetes/kops

Kubernetes Operations (kops) - Production Grade K8s Installation, Upgrades, and Management - kubernetes/kops

kubernetes/kubernetes

Production-Grade Container Scheduling and Management - kubernetes/kubernetes

Jan avatar

Awesome

Jan avatar

Will start at that Version

2019-01-16

pecigonzalo avatar
pecigonzalo

I was going over geodesic, im trying to understand what goofys is used for, I understand you mount some s3 buckets, but what for?

pecigonzalo avatar
pecigonzalo

as afaik you use terraform backends, so state should work/happen through that

joshmyers avatar
joshmyers

@pecigonzalo it was used to mount secrets from s3 to a local filesystem

joshmyers avatar
joshmyers

I think @Erik Osterman (Cloud Posse) is wanting to move away from it and instead use SSM

pecigonzalo avatar
pecigonzalo

OK, that makes sense, like sensitive tfvars and other variable/config/secrets

pecigonzalo avatar
pecigonzalo

thanks

joshmyers avatar
joshmyers

mounts private keys by the looks of it

joshmyers avatar
joshmyers
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
12:08:39 AM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

who will be lucky #200?

2019-01-15

Jan avatar

is it safe to assuem this is the current state to follow?

Jan avatar
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

Jan avatar

for getting k8s into geodesic

Jan avatar

Cool so I have kops creating a cluster in a vpc with a private hosted zone

Jan avatar

all working

Jan avatar

need to polish a bit

Jan avatar

and stuff

Jan avatar

but I have my previous idea, I define my vpc and launch k8s into it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Nice @Jan

Jan avatar

also using the kops-backing-services

Jan avatar

mostly

Jan avatar

so with the cluster running as per geodewssic aws/kops method

Jan avatar

mostly

Jan avatar
helmfiles ⨠  helmfile apply
Adding repo coreos-stable <https://s3-eu-west-1.amazonaws.com/coreos-charts/stable>
"coreos-stable" has been added to your repositories

Updating repo
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "incubator" chart repository
...Successfully got an update from the "coreos-stable" chart repository
...Successfully got an update from the "cloudposse-incubator" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈

Comparing prometheus-operator coreos-stable/prometheus-operator
Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"
configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"
Error: plugin "diff" exited with error

err: exit status 1
failed processing /conf/helmfiles/releases/prometheus-operator.yaml: exit status 1
Jan avatar

Is rbac being taken into account?

Jan avatar
  • rbacEnable: {{ env “RBAC_ENABLED” default “false” }}
Jan avatar
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles. Works with helmfile.d - cloudposse/helmfiles

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan by default our helmfiles don’t use RBAC

Jan avatar

I am asking if in the geodesic reference arch rbac is being used

Jan avatar

ah nm

Jan avatar

read your response wrong

Jan avatar

so….. I can understand its easier to not

Jan avatar

but its much much worse

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

agree

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

do you have all the rbac resources (service account, role binding, etc.) provisioned in the cluster?

Jan avatar

just added

Jan avatar

more exploring how much is or is not dealt with in the current ref arch

Jan avatar

and extending it to do what I need

Jan avatar

hAHAHA

Jan avatar
Error: identified at least one change, exiting with non-zero exit code (detailed-exitcode parameter enabled)
identified at least one change, exiting with non-zero exit code (detailed-exitcode parameter enabled)
Error: plugin "diff" exited with error
Jan avatar
helm/helm

The Kubernetes Package Manager. Contribute to helm/helm development by creating an account on GitHub.

Jan avatar

this by itself is a better place to start

chrism avatar
chrism

Hey; I love the centralised management of roles/users using this setup; but how would you restrict a users access at a sub-org level with the current setup? Say for example you have a user jim in root who only has access to the data org, and you want to restrict that to just elasticache in the data org.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

there are a few options

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

1) create more groups with fine grained access control like you describe; then add users to those groups.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

2) add the policies directly to the IAM user.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I like (1) the best.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what you describe is a highly specific requirement (not wrong!), so we haven’t tried to come up with any patterns to generalize that.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

…perhaps after seeing more patterns like this, we can whip up a module for it

chrism avatar
chrism

I’ll have a prod around, the less unicorns the better

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cool

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

happy to give you a high level overview / demo of what it all looks like

chrism avatar
chrism

I’m guessing the simplest way would be to create a role within the sub-organisation to (as per the example) allow only access to one area and a group. tbh I probably need to RTFM on iam + suborgs. Previously we’d manage the users in each org (which is a headache in itself)

chrism avatar
chrism

Love the wrapper around aws-vault and the sts usage; its a really nice way to standardise access

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

right - avoid managing users in suborgs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, the geodesic shell has made it A LOT easier to distribute an advanced toolchain

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we used to try to do it all “natively” (e..g on osx) and always ran into problems

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

old version of XYZ package

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

“works on my machine”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

communicating upgrades etc

chrism avatar
chrism

We’re a mix of windows / nix / mac so its good to see something that wont turn me from a developer into desktop support

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

do you guys run WSL?

chrism avatar
chrism

Docker is a god send for tooling

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we have early support for that.

chrism avatar
chrism

yeah all in WSL

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

a few community members were using geodesic in that setting.

chrism avatar
chrism

docker for windows; wsl talking over tcp. Works relatively seamlessly

Jan avatar

Make vs make :(

chrism avatar
chrism

We’re mostly using the ref setup (setup individually) for the org management. Our actual infrastructure in the orgs will be a tad more hybrid as we have our own terraform modules etc / splits across regions

Jan avatar

Tool chains in docker are the future

chrism avatar
chrism

Yeah docker’s pretty sweet for simplifying tooling; we already use it as an ops tool for running scans and long custom chains of tools to analyse deployments / nmap / etc. Kubernetes is a mixed bag though; its like creating a magic-bag for developers to throw half baked code into

chrism avatar
chrism

Good work chaps; its nice to see an opinionated, secure by default, design

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @chrism!

2019-01-14

Jan avatar

What’s the rational in having the reference architecture route 53 zones public?

Jan avatar

I get that you need to have a vpc existing in order create a private zone

Jan avatar

Mmmm actually

Jan avatar

Re reading the tf docs I need test it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan you need them public to be able to access the resources in the clusters, e.g. [us-west-1.prod.cloudposse.co](http://us\-west\-1\.prod\.cloudposse\.co) is to access the k8s cluster in prod

Jan avatar

Absolutely not the case

Jan avatar

What I want is public people to access a elb via a set record that has nothing to do with the zones I would use for env cloudposse. Co

Jan avatar

Why leak all the info about all the environments you have and the record in them

Jan avatar

I will change our version of the ref arch to build internally

Jan avatar

So for example.. Have cloudposse.co as a public zone with an associated private zone

Jan avatar

All internal zones should have internal zones

Jan avatar

It is an anti pattern to expose anything (including dns hosts information) for anything not intended to be publicly consumed

Jan avatar

I normally do what I am describing with split horizon dns

Jan avatar

Especially so with any direct connect or vpg/vpn peering

Jan avatar

Ah Yea aws call it split-view

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan you can do it with private zones as you described

Jan avatar

Yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

another reason to have public zones is when you use kops

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so I guess there are many diff use-cases here

Jan avatar

What I was asking is, given the focus on security in geodesic, the is seems wrong in the reference architecture

Jan avatar

Kops works perfectly fine for private

Jan avatar

Kops has no hard reliance on public r53 zones

Jan avatar

I will rework our fork of the reference architecture tonight and see if I can resolve this

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

btw, kops works without any DNS at all

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it supports gossip mode

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we used this with our caltech vanvalen project

2019-01-11

Jan avatar

wow the removal and deletion of aws org accounts is a pita

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Jan avatar

Oh yea not even trying via tf

Jan avatar

Doing it manually and it’s a grind

2019-01-10

Jan avatar

is anyone using the new reference architecture with geodesic?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Jan avatar

I mean in here that isnt you I feel bad bombarding just you with questions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yeah probably not…. this is so new that I don’t think anyone has had the chance to run through it yet.

Jan avatar

Hehe then I will be the crash test dummy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Jan avatar

got make root working

Jan avatar

though get make children

make[1]: *** No rule to make target `staging/validate'.  Stop.
make: *** [staging/validate] Error 2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrmmm that’s odd.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Those targets are generated from the ACCOUNTS_ENABLED macro

Jan avatar

so I did make root - all passed, users created etc then directly to make child

Jan avatar
ACCOUNTS_ENABLED = staging prod testing data corp audit
Jan avatar

so in the artifacts the Makefile.env is populated

Jan avatar

so I think in the root makefile where we previously removed the --export-accounts argument that has had a knock on effect later

Jan avatar

busy testing

Jan avatar

mm no

2019-01-09

Jan avatar

in the new reference arch…

the org cidr range, whats the expectation there?

Jan avatar

\# Network CIDR of Organization
org_network_cidr    = "10.0.0.0/8"
org_network_offset  = 100
org_network_newbits = 8    # /8 + /8 = /16
Jan avatar

that all the sub accounts fit within that cidr?

Jan avatar

So at this point I would love to have the new reference arch make root have a --dry-run option so I can still go verify and edit all the configuration before it starts bootstrapping

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

make root/init

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that does not apply

Jan avatar

cheers

Jan avatar

and then despite having set the region to eu-central-1 there is a hard expressed parameter or a variable thats only doing its default of using us-west-2 for the tfstate-backend s3 bucket

Jan avatar
* module.tfstate_backend.aws_s3_bucket.default: 1 error(s) occurred:


• aws_s3_bucket.default: Error creating S3 bucket: IllegalLocationConstraintException: The us-west-2 location constraint is incompatible for the region specific endpoint this request was sent to.
Jan avatar

simply adding the following to the Dockerfile.root solves it for this case


\# Jan hacking
ENV TF_VAR_region="${aws_region}"

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)


that all the sub accounts fit within that cidr?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, the idea was that we could peer the VPCs from any of the accounts, so the VPCs should have not-overlapping ranges

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks for finding the issue, we’ll fix that

Jan avatar

totally

Jan avatar

and I do like the setting of the cidr’s per account/vpc

Jan avatar

but setting into a pre calculated range doesnt fit, especially for me

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

You can override them per account

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we only calculate it if account_cidr (i think) is not set

Jan avatar

perfect

Jan avatar

and im trying to work out how to use it

Jan avatar

same as I dont want to provision the aws_orgs part

Jan avatar

as I have the accounts already

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks @Jan

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

any suggestions and improvements are welcome

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it was the first version of the ref architectures, definitely has room for improvements

Jan avatar

I have forked the 3 projects and will work on fixing the tf state bucket var

Jan avatar

Already validated it works

Jan avatar

And will try run a modded version of the reference architecture that fits my needs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re using tvenv instead of using TF_VAR_ in Dockerfile

Jan avatar

I think I found the issue with the tfstate-backend vars

Jan avatar
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Jan avatar
Jan
06:27:52 PM
Jan avatar

should those not rather be ENV TF_VAR_

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no more TF_VAR_ ever

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Jan avatar

mmmm really

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

standardize around normal envs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that can be used by kops, terraform, scripts etc

Jan avatar

so im trying to track down why its reverting to the default value of us-west-2

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

use tfenv to export them to terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrmm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what do you have in root.tfvars config?

Jan avatar

\# The default region for this account
aws_region = "eu-central-1"

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrmm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, let’s address on our call

Jan avatar

I ended up with everything getting added in eu-central-1 and a error from s3 when it tried to create the bucket with region set to us-west-2

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s TF_STATE_BUCKET_REGION

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we don’t se tthat in the bootstrap (oversight)

Jan avatar

I got around it by adding ENV TF_VAR_region="${aws_region}" to the Dockerfile.root

Jan avatar

ah!

Jan avatar

good catch

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(let me confirm)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

slightly wrong

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

TF_BUCKET_REGION

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the problem is when going multi-region with terraform is where the statefiles belong

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrm

Jan avatar

so im not trying to do multi region

Jan avatar

lol

Jan avatar

lets clarify it later

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s the other problem with the coldstart

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you’re right

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i’ll tell you the fix

Jan avatar

yea I saw the default value

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

add a config template here:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for tfstate-backend

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then add something like that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then it will generate a terraform.tfvars for that project

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

since each project might be in a different region

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(so there’s nothing wrong with setting TF_VAR_region in the Dockerfile - just I’m not advocating it anymore)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I like the explicitness of terraform.tfvars

Jan avatar

will happily stick to the new convention

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

… anyways this is a

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you track a list of these issues I’ll get them fixed

Jan avatar

:–1:

Jan avatar

that worked great btw, thanks

2019-01-08

Jan avatar

Almost all the additions I will need

Jan avatar

Will do an upgrade today

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jan we still need to work on injecting kops to existing subnets/vpcs

Jan avatar

yes though that will fall to my backlog as I will do the vpc creation via kops for now

Jan avatar

its a very easy though

Jan avatar

simply populate the cluster manifest with the vpc id, the subnet ID’s and make sure the cidr is the same

Jan avatar

then just kubectl create -f

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes, just the devil in the details. :-)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jan regarding the kops manifest, we’ve been using gomplate (go templates) because terraform templates not not enough

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(no conditionals)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

like you said, easy enough to parameterize

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Though I think maybe we would create a separate manifest

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

since the changes would be significant.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

have a look at the latest changes for how we provision kops settings from terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I am quite fond of it. We write them to SSM from terraform. Consume them with chamber and call build-kops-manifest or kops

Jan avatar

Cool

Jan avatar

Will be Re setting everything up with the cleaner reference architecture tomorrow morning

Jan avatar

And doing kops vpcs

2019-01-07

daveyu avatar
daveyu

Not sure if I configured something wrong, but is there a way to keep an assume-role session active indefinitely?

[2]+  Stopped                 aws-vault exec ${AWS_VAULT_ARGS[@]} $role -- bash -l
-> Run 'assume-role' to login to AWS
 ⧉  testing
 ✗   (none) ~ ⨠  assume-role
Enter passphrase to unlock /conf/.awsvault/keys/:
aws-vault: error: Failed to start credential server: listen tcp 127.0.0.1:9099: bind: address already in use
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrm…..

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so yes, it should be possible. We do run the aws-vault server in the background

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I am using a VM (AWS WorkSpace), so it doesn’t work for me

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


aws-vault: error: Failed to start credential server: listen tcp 127.0.0.1:9099: bind: address already in use

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This doesn’t look right

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Are you using a recent version of geodesic?

daveyu avatar
daveyu

It’s only annoying because I find I have to exit out of geodesic completely before I can successfully assume-role again.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrmmm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If that’s the case, that’s a bug

daveyu avatar
daveyu

yeah i just rebuilt FROM cloudposse/geodesic:0.57.0

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, that’s recent

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sec

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

to help me triage this…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are you running multiple geodesic containers at the same time?

:-1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are you using aws-vault server mode natively on yourmac?

:-1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you run your geodesic shell and then run ps uxaww do you see aws-vault server running?

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

when you start the shell, do you see the message:

* Started EC2 metadata service at <http://169.254.169.254/latest>
:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ugh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(was hoping for the opposite order of thumbs)

daveyu avatar
daveyu

aws-vault exec --assume-role-ttl=1h --server lootcrate-testing-admin -- bash -l

daveyu avatar
daveyu

guessing im running into problems at the 1h mark

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, try this:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

AWS_VAULT_ASSUME_ROLE_TTL=24h

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

set that in the Dockerfile and rebuild

daveyu avatar
daveyu

ok

daveyu avatar
daveyu

fyi aws-vault: error: Maximum duration for assumed roles is 12h0m0s

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrm…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, well start with that I guess

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

now, for indefinite, that “used to” work.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I wonder why I don’t see this [2]+ Stopped aws-vault exec ${AWS_VAULT_ARGS[@]} $role -- bash -l

daveyu avatar
daveyu

ok this is getting somewhere:

aws-vault: error: Failed to get credentials for lootcrate (source profile for lootcrate-testing-admin): ValidationError: The requested DurationSeconds exceeds the MaxSessionDuration set for this role.
	status code: 400, request id: ef5c9f5d-12c6-11e9-bd1b-b5609fd02acf
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh interesting.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(a) I didn’t know it was possible to scope it to a role (b) I feel like it’s a red herring since this used to work

daveyu avatar
daveyu
09:57:30 PM
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@daveyu what version of geodesic are you suing?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

see above

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

0.57

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so the way it’s supposed to work is actually unreltaed to that ttl

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

basically, it could be 60 seconds

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the way it’s supposed to work is when something needs AWS access, it uses the metadata api

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that is proxied by aws-vault server which mocks the API

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then it fetches some temporary credentials.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Can you open an issue against cloudposse/geodesic with this info

daveyu avatar
daveyu

sure thing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’ll have someone look into it.

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

PSA: there have been a lot of awesome updates to geodesic in the past month

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you haven’t upgraded, do give it a shot.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you’re using k8s, then the kube-ps1 prompt is really slick

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you want to run multiple k8s clusters (kops) per account, that’ now supported using the direnv pattern

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re using geodesic regularly with #atlantis, so if that’s interesting hit me up

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve added a script called aws-config-setup which makes it easier for users to setup aws-vault (no copy pasta)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re adding iam-authenticator support to kops

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve enhanced kubens and kubectx autocomplete with fzf

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve added tfenv to automatically export envs for terraform consumption

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for terraform-root-modules, we’ve added support for multple new account types: identity, corp, security as well as generalized the pattern so we can add more account types easily

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve started embracing SSM with terraform so that we automatically populate settings for consumption with chamber (no more copy pasta)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

anyways, if any of this sounds interesting and you want more info/demo, hit me up.

sarkis avatar
sarkis

I think CloudPosse needs a newsletter

sarkis avatar
sarkis

I’d love to get updates like this weekly or monthly

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Newsletter Signup attachment image

Newsletter Signup

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Newsletters attachment image

Expert Cloud Architects

sarkis avatar
sarkis

thanks @Andriy Knysh (Cloud Posse)! signed up

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@sarkis yes - we’re well overdue for the next edition

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I not had a chance to it together

sarkis avatar
sarkis

catching up with all the new geodesic changes

sarkis avatar
sarkis
kubernetes-sigs/aws-iam-authenticator

A tool to use AWS IAM credentials to authenticate to a Kubernetes cluster - kubernetes-sigs/aws-iam-authenticator

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

right now, we’re working on getting the slack archives published on a static site so we can have the history available

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are you guys looking at k8s now?

sarkis avatar
sarkis

not yet

sarkis avatar
sarkis

at least a quarter out

sarkis avatar
sarkis

im just staying in touch with everything - getting ready

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s cool - at least it’s on the horizon

2019-01-05

2019-01-04

Jan avatar

in the tf root modules is there a list someplace as to what all the backing services relate to?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

can you expand on “relate to”?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’ve jotted down some notes on #atlantis with #geodesic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Document Atlantis with Geodesic · Issue #355 · cloudposse/docs

what Explain this mind warping concept Introduction When you run your infrastructure with geodesic as your base image, you have the benfit of being able to run it anywhere you have docker support. …

sarkis avatar
sarkis

Nice write up! I thought of an edge case I think - not sure unless I can tias though.

Document Atlantis with Geodesic · Issue #355 · cloudposse/docs

what Explain this mind warping concept Introduction When you run your infrastructure with geodesic as your base image, you have the benfit of being able to run it anywhere you have docker support. …

sarkis avatar
sarkis

Left a comment

2019-01-03

Jan avatar

wow man, trying to get back into this after a 2 week brake is hurting my mind

:100:2
joshmyers avatar
joshmyers

Too much and !

Jan avatar

haha

    keyboard_arrow_up