#geodesic (2019-07)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2019-07-31

Lee Skillen avatar
Lee Skillen

As a service that runs on .io (we have the .com as well, and we’re migrating to it), I’d heartily recommend avoiding .io if you can. It’s mostly OK, but occasionally has worldwide resolution issues.

1
Lee Skillen avatar
Lee Skillen

It’d be fine unless you’re running something that’s mission-critical though. A blog? Fine. Something that your pipeline needs for deployments? Negative.

Tim Jones avatar
Tim Jones

Another question - are you guys still supporting the terraform iam user modules? I noticed terraform-aws-iam-system-user was updated to terraform v0.12 but not terraform-aws-iam-user (I submitted a PR https://github.com/cloudposse/terraform-aws-iam-user/pull/3)? I’m curious to know how it, terraform-aws-organization-access-group, terraform-aws-iam-assumed-roles, and terraform-aws-organization-access-role are all meant to work together…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, it’s just a slog. We have over 100 modules.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

upgrading them a little bit at a time.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve upgrading 40+ modules to 0.12 and terratest

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(but this is all out of pocket for cloudposse)

2
joshmyers avatar
joshmyers

Ya’ll rock! @Andriy Knysh (Cloud Posse) is a machine

Tim Jones avatar
Tim Jones

I can only imagine the chore!! Hopefully my contrib helps, since I’m looking at adopting your modules. They follow really great practices from what I can see.

Tim Jones avatar
Tim Jones

My company is happy to let me dedicated paid time to Open Source projects that we use in turn.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @Tim Jones!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

most of the slow down for us accepting 0.12 PRs is that we want to have testing in place so all future PRs can be merged faster.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re using the 0.12 upgrade as the excuse to add them.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) on our side is doing most of the work.

Tim Jones avatar
Tim Jones

Cool! I’d like to help out here I can, and can easily use my fork until PRs get merged…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks! sorry for the hold ups

Tim Jones avatar
Tim Jones

No need to apologise!! We’re already grateful for the awesome contributions you’ve all made!

Tim Jones avatar
Tim Jones

To my other question though: can you give a quick run-down of how terraform-aws-iam-user, terraform-aws-organization-access-group, terraform-aws-iam-assumed-roles, and terraform-aws-organization-access-role are made to be used together? For instance, I have the master account, along with develop, staging, & production accounts. I create the real-life users in the master account with terraform-aws-iam-user, but I’m not sure how I then give the, access to the other accounts & control things like MFA requirements…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

fwiw, have you had a chance to look through our terraform-root-modules repo? this is a collection of blueprints we use that leverage all the modules together.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terraform-aws-organization-access-group is used when you have a role in another account and you want to allow users in your root account (or identity account) easily assume roles in the other account.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so you deploy terraform-aws-organization-access-group for each role that exists in another account that is assumable

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terraform-aws-iam-assumed-roles was one of our earlier modules. We use it for provisioning a few default roles in the root account. it also enforces some best practices around MFA and password resets.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

terraform-aws-iam-user is used when you don’t have SSO. it provisions an IAM account suitable for humans

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(we also have terraform-aws-iam-system-user for bot accounts. e.g. external CI/CD systems)

Tim Jones avatar
Tim Jones

So really I should be using terraform-aws-iam-assumed-roles for human users, and then simply delegate their access with an terraform-aws-organization-access-group with the role ARNs of the org accounts?

Tim Jones avatar
Tim Jones

Ah, I think I see it now, create the individual users with terraform-aws-iam-user, manage the settings of all the users with terraform-aws-iam-assumed-roles and delegate their account access with terraform-aws-organization-access-group`

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the terraform-aws-organization-access-group is somewhat redundant with terraform-aws-iam-assumed-roles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the terraform-aws-organization-access-group module was designed to be a more general way to assume any role in other accounts

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

while the terraform-aws-iam-assumed-roles is designed specifically to assume roles in the “root” (apex) account.

2019-07-29

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:09 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:01:34 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Aug 07, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

Tim Jones avatar
Tim Jones

Hi all! I’m trying to use the https://github.com/cloudposse/reference-architectures repo to put some order into an ad-hoc AWS I inherited. I have two questions: 1) in configs/root.tfvars I have to change the domain for the entire infra. Now currently the company domain [example.com> is pointed to a heroku app and I can’t make AWS the SOA for the entire domain. If I use something like [infra.example.com> will that then make all the accounts register as <account](http://infra.example.com).<http://infra.example.com|infra.example.com](http://example.com)?? 2) In running make rootI get the following error. I can’t create the accounts/root dir by hand as the first step in the Makefile removes it!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We suggest distinguishing between your branded/vanity domains and you infrastructure service discovery domains

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. a product might have dozens of branded domains, but you’ll only have one infra

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so for our customers, we usually register a new domain on route53 or repurpose an existing one that was unused

Tim Jones avatar
Tim Jones

aahh, so you’d suggest the ‘root’ account to ‘own’ [companyexmaple.net](http://companyexmaple.net) or something completely separate from the company itself

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, precisely

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the .net, .io, .org, .dev, .sh, .co domains are what we typically use

2019-07-24

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Public #office-hours with cloud posse starting now! https://zoom.us/s/508587304 join if you have any questions or want to listen in.

2019-07-23

joshmyers avatar
joshmyers

@chrism Re Geodesic on windows, are you running native Windows? Bash? WSL? I literally have no idea with windows these days but a dev in the team is a Windows user

chrism avatar
chrism

Windows 10 + Docker for windows. Ubuntu 18 / WSL (from the store)

joshmyers avatar
joshmyers

Do you need to install docker in WSL? Colleague has docker for windows installed but in a “bash” environment, cannot see DOCKER

chrism avatar
chrism

yeah

chrism avatar
chrism

one mo

chrism avatar
chrism
Setting Up Docker for Windows and WSL to Work Flawlessly attachment image

With a couple of tweaks the WSL (Windows Subsystem for Linux, also known as Bash for Windows) can be used with Docker for Windows.

1
joshmyers avatar
joshmyers

Thanks @chrism

joshmyers avatar
joshmyers

So yeah, it looks like assume-role interactive doesn’t work, and also /localhost/.aws/config doesn’t exist, are you setting AWS_DATA_PATH ?

joshmyers avatar
joshmyers

@chrism Are you always running assume-role $ROLENAME in that case?

chrism avatar
chrism
ENV ASSUME_ROLE_INTERACTIVE=false
ENV TF_BUCKET_PREFIX_FORMAT="basename-pwd"
# Needed because multi provider started failing with wsl and dynamic linking
ENV TF_PLUGIN_CACHE_DIR=/tmp

in my docker file

1
joshmyers avatar
joshmyers

how does your aws config end up in the right place? It looks like WSL has a mapping of e.g. /mnt/c/Users/work/.aws

joshmyers avatar
joshmyers

inside WSL

joshmyers avatar
joshmyers

or is your aws config inside WSL? >_<

chrism avatar
chrism

you configure vault + aws in wsl

chrism avatar
chrism

otherwise you’re in the weird world of virtualised chmod

chrism avatar
chrism

wsl’s my default commandline; i rarely need win-cmd.

1
joshmyers avatar
joshmyers

OK ta, this dev is a windows windows user, grrrr

joshmyers avatar
joshmyers

Are you able to write to /root/.aws/config inside WSL land?

joshmyers avatar
joshmyers

Feels like I need to do some reading around WSL

chrism avatar
chrism

thats just normal linux shit

chrism avatar
chrism

the aws config should be in ~~/.aws/config the vault ~~.aws-vault/ etc

chrism avatar
chrism

basically dont sudo su

chrism avatar
chrism

main issues I had with the folders was sudo altering the owner of files

chrism avatar
chrism

nothing as heart warming as fighting your way through config just to get a worker node to reply banana on request

joshmyers avatar
joshmyers

WSL seems broken…. there isn’t a /root/.aws dir inside WSL, but we can’t seem to create one

joshmyers avatar
joshmyers

We’re getting some weird input/output error trying to create /root/.aws

chrism avatar
chrism

it should be /home/username/.aws I don’t get why its in root

joshmyers avatar
joshmyers

hmm

chrism avatar
chrism
joshmyers avatar
joshmyers

For some reason he is root as soon as enters WSL…. there is nothing in /home/

joshmyers avatar
joshmyers

whoami = root

chrism avatar
chrism

thats… screwy

joshmyers avatar
joshmyers

Said he just installed WSL from the store and doesn’t know anything else

joshmyers avatar
joshmyers

¯_(ツ)_/¯

chrism avatar
chrism

burn it start again

joshmyers avatar
joshmyers

Yeah, he’s already not happy

chrism avatar
chrism

Never had that one and I’ve been using wsl since insider release came out

chrism avatar
chrism

Legacy wsl you could force it to open as root lxrun /setdefaultuser root newer ones ` ubuntu config –default-user ` but I prefer to keep it as a real user; seems crazy to do everything as root

joshmyers avatar
joshmyers

Indeed

joshmyers avatar
joshmyers

So how do you keep it as real usernames? Default that should be the case?

chrism avatar
chrism

just the default tbh

joshmyers avatar
joshmyers

OK, thanks for all the info dude! Out my depth in windows world

chrism avatar
chrism

It’s a fun system of virtualised virtualisation

2019-07-22

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:05 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:01:09 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Jul 31, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2019-07-19

joshmyers avatar
joshmyers

hmmm, how are folks doing Terraform 0.12 and geodesic?

joshmyers avatar
joshmyers
02:51:21 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Direnv with Terraform 0.12 by osterman · Pull Request #500 · cloudposse/geodesic

what Use an empty cache folder to initialize module why Terraform 0.12 no longer allows initialization of folders with even dot files =( Example of how to use direnv with terraform 0.12 usage e…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but this is what @Josh Larsen and @daveyu are doing https://sweetops.slack.com/archives/CB84E9V54/p1563307247187300

much more elegant than my workaround

deps:
	mkdir -p modules
	terraform init modules
	cp modules/* ./
	rm -rf modules

2019-07-17

joshmyers avatar
joshmyers

@chrism Another quick thing, any issues running the Geodesic wrapper script on windows? i.e. make install so you get a [root.foo.co](http://root.foo.co) in your /usr/local/bin to call ?

chrism avatar
chrism

works fine

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

2019-07-16

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Actually, @chrism what @Erik Osterman (Cloud Posse) was referring to was a command I added to Geodesic called role-server. I have only tried it when using credentials that have/require virtual MFA, and with that setup, here is how it works:

You start your first Geodesic shell and run role-server. It uses aws-vault to assume an IAM role, creating temporary credentials with a life of AWS_VAULT_ASSUME_ROLE_TTL (default 1h meaning 1 hour) and a role session with life of AWS_VAULT_SESSION_TTL (default 12h). aws-vault then runs as a credential server, using the same interface that the AWS credential server uses on EC2 instances. It runs in debugging mode so you will see all the activity printed out on the screen as it runs. For this reason, I usually hide the window.

Then you start a second Geodesic shell using the same Docker image. This second shell, as it starts up, finds the credential server is running and immediately assumes the role it is serving. This shell will work uninterrupted until the role session runs out (12 hours). When the temporary credentials expire, aws-vault renews them as long as the role session is valid. When the session runs out, the shell will stop having credentials, but back in the window that I hid, running the role server, aws-vault will ask for the MFA token. Type it in and you get a new session for another 12 hours.

And of course, you are not just limited to 1 useful Geodesic shell. You can run several at the same time and they will all share the same credential server, as long as they are all running the same Docker image. The Geodesic wrapper script that gets installed to launch the Docker image does not run multiple copies of the image, but rather starts multiple bash shells running inside the same image. This is how they are able to share the credential server (and the kubecfg).

1
Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

@Erik Osterman (Cloud Posse) @joshmyers I believe the reason the organizations module is not enabled in root.tfvars in the ref arch is that it is not needed because it is not practical. From a cold start, you have to use ClickOps to:

  • create the root account,
  • add billing info,
  • buy a root domain,
  • connect the root domain to the root account and Route 53,
  • create the organization,
  • verify the root account email address with AWS,
  • request an increase in the limit of the number of accounts allowed under your organization,
  • wait for the organization to finish initializing (can take an hour or more), and
  • wait for the requested limit increase to be approved and implemented.

Because of the long waits and the interaction with AWS support and email, none of the above is done with Terraform. After it is all done, Terraform does not need to manage the organization at all. New accounts are automatically added to the organization when they are created because they are created as “child” accounts of the “root” account.

joshmyers avatar
joshmyers
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

I have never seen it used. It was written over a year ago, when the whole terraform-root-modules project was new. My guess is that it was an attempt at automating the cold start that was abandoned when it was realized that the above-mentioned tasks really needed to be done via ClickOps.

joshmyers avatar
joshmyers

OK, should prob remove it then, I’ll open a PR

joshmyers avatar
joshmyers

Heads up that I applied that module to root, seems fine…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Josh Larsen any opinions on this? https://github.com/cloudposse/geodesic/pull/500

Direnv with Terraform 0.12 by osterman · Pull Request #500 · cloudposse/geodesic

what Use an empty cache folder to initialize module why Terraform 0.12 no longer allows initialization of folders with even dot files =( Example of how to use direnv with terraform 0.12

1
Josh Larsen avatar
Josh Larsen

yes this looks great! might be worth mentioning that people may need to change the make deps target on modules that use overrides

Direnv with Terraform 0.12 by osterman · Pull Request #500 · cloudposse/geodesic

what Use an empty cache folder to initialize module why Terraform 0.12 no longer allows initialization of folders with even dot files =( Example of how to use direnv with terraform 0.12

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, I need to do that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@oscarsullivan_old

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

basically, this is to get around the fact that 0.12 does not allow the initialization of modules to a non-empty directory (previously terraform permitted dot files)

oscarsullivan_old avatar
oscarsullivan_old

What’s an example of initialising an empty directory?

daveyu avatar
daveyu

much more elegant than my workaround

deps:
	mkdir -p modules
	terraform init modules
	cp modules/* ./
	rm -rf modules
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha, more or less the same

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just using ENVs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

though in this case, we keep using .modules/ by passing it always as an argument to terraform (~ TF_CLI_ARGS_plan=.module, TF_CLI_ARGS_apply=.module)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or TF_CLI_PLAN=.module and TF_CLI_APPLY=.module when using cloudposse/tfenv

loren avatar
loren

Does every terraform 0.12 command support a directory as an argument? I know in 0.11 it was not consistent

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Not sure if every command supports it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The ones we care about do: init, plan, apply, destroy

loren avatar
loren

I think it was import or state that did not

2019-07-15

chrism avatar
chrism

Is there a quick way when in geodesic to refresh your vault session

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Do you use vault server?

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:06 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:00:35 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Jul 24, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

chrism avatar
chrism

sorry i mean aws-vault

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

right, but aws-vault server mode

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we have support for that in geodesic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and it will automatically refresh your tokens

chrism avatar
chrism

oh; have to prod that. I was digging through the shell

chrism avatar
chrism

ta

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy (Cloud Posse) spent a lot of time to get this right.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

there were all kinds of edge cases (like what to do when running on an EC2 instance like an AWS Workspace)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

…what to do when the clock is behind

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

…what to do if already in a shell with a role

chrism avatar
chrism

i get the clock issue all the time since updating from ~40odd to current. There’s a whole pile of related issues with docker4win /hyperv and that though. I generally just run hwclock -s

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy (Cloud Posse) do you know why the organizations module is not enabled in root.tfvars?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

In the ref arch…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@joshmyers pointed this out and I don’t understand either

2019-07-08

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:07 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:03:24 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Jul 17, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2019-07-04

joshmyers avatar
joshmyers

Does GEODESIC_PORT serve a purpose?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes, it’s used for things like kubectl proxy and teleport logins. Basically where you want to access something in geodesic via http

joshmyers avatar
joshmyers

Figured so, can see some bits for teleport

joshmyers avatar
joshmyers

Don’t remember needing to do anything with kubectl proxy though…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Nice for accessing kube dashboard when you don’t have all the other infra in place like keycloak

2019-07-03

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Public Office Hours starting now! Join me here: https://zoom.us/meeting/register/dd2072a53834b30a7c24e00bf0acd2b8

Have any questions? This is your chance to ask us anything.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

anybody think this would be cool? https://github.com/genuinetools/img

genuinetools/img

Standalone, daemon-less, unprivileged Dockerfile and OCI compatible container image builder. - genuinetools/img

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

compile geodesic to an exe

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so it wouldn’t require even docker

loren avatar
loren

Slick

2019-07-01

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:01 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:02:21 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Jul 10, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

    keyboard_arrow_up