#geodesic (2018-11)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic

Archive: https://archive.sweetops.com/geodesic/

2018-11-02

Dombo avatar

What do you guys use to host the packages available at apk.cloudposse.com?

Dombo avatar

Managed service or self hosted?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Good question

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/alpinist

Automatic Alpine Linux Package (apk) Repository Generation using AWS Lambda, S3 & SSM Parameter Store - cloudposse/alpinist

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s a fork

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Very cool little lambda deployed via cloud formation

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

PR is still open because we’ve been using it for demos

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

With Atlantis

Dombo avatar

haha dude

Dombo avatar

I was like “I do not want to host an APK repo”

Dombo avatar
forward3d/alpinist

Automatic Alpine Linux Package (apk) Repository Generation using AWS Lambda, S3 & SSM Parameter Store - forward3d/alpinist

Dombo avatar

Checked back here to see if you’d hear of it

Dombo avatar

TUrns out you’ve done the hard part for me already

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you’re going to host one I can finish up the PR next week

Dombo avatar

I’m happy to jump on with you?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think it’s ready to merge and release

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Then our CloudPosse packages repo is how we generate our packages

Dombo avatar

Yeah noted

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We are going to consolidate the vendor and install folders in packages. Our justification for doing it this way is so we don’t have lock in on apk. We use it to install native binaries (e.g. for local Dev) and maybe one day will support other packages like Debian.

Dombo avatar

I might be missing something but I’m unsure how consolidating those folders will change anything? That Makefile in install is a manifest and quite useful as a new-comer to the repo.

Dombo avatar

You can achieve multi-package support by simply adding more steps to the individual Makefiles in the vendor dir

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Exactly! This is the plan

Dombo avatar

Disclaimer: not exactly a Makefile pro but it’s easy enough to follow

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yep, you got it :-)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh the install folder was just how we used to do it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The vendor folder is the new way

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We are going to add a make target to preserve the original behavior

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But keep all the logic in the vendor folder.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh btw, if there are any packages missing you would like feel free to open an issue or PR (preferred) and we can add them

Dombo avatar

Yeah was just thinking that - most of my daily tools are there

2018-11-07

tamsky avatar

Does anyone have an example repo that uses the “multi-stage docker build pattern” against geodesic? I’m hoping not to reinvent the wheel for customizations.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/prod.cloudposse.co

Example Terraform/Kubernetes Reference Infrastructure for Cloud Posse Production Organization in AWS - cloudposse/prod.cloudposse.co

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/staging.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Staging Organization in AWS. - cloudposse/staging.cloudposse.co

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

tamsky avatar

oh, I didn’t know those draw from geodesic – but I should have thought of that

tamsky avatar

tnx!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

2018-11-11

tamsky avatar

what do folks do when they need a later version of a package than what is listed in the cloudposse/packages repo? I’m looking for a later version of terraform than 0.11.8 listed at https://github.com/cloudposse/packages/blob/master/vendor/terraform/VERSION

cloudposse/packages

Cloud Posse installer and distribution of native apps, binaries and alpine packages - cloudposse/packages

tamsky avatar

or a later version than what’s available in the apk aports: https://git.alpinelinux.org/cgit/aports/tree/community/terraform/APKBUILD

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Submit a PR against our packages repo

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The only reason it’s not the latest is we haven’t had a chance to update it yet.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@tamsky

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also, anything you want a package for, feel free to open an issue or submit a PR

tamsky avatar

There seem to be at least two different version directives for terraform within cloudposse/packages:

diff -r 5c58433395f3 install/Makefile
--- a/install/Makefile  Tue Oct 23 18:32:59 2018 -0700
+++ b/install/Makefile  Sun Nov 11 15:53:45 2018 -0800
@@ -297,7 +297,7 @@
 teleport:
        $(CURL) <https://get.gravitational.com/teleport/$(TELEPORT_VERSION)/teleport-v$(TELEPORT_VERSION)-${OS}-$(ARCH)-bin.tar.gz> -o - | tar -C $(INSTALL_PATH) -zx --wildcards --strip-components=1 --overwrite --mode='+x'  */tsh */tctl */teleport
 
-export TERRAFORM_VERSION ?= 0.11.7
+export TERRAFORM_VERSION ?= 0.11.10
 # Releases: <https://github.com/hashicorp/terraform/releases>
 ## Install Terraform
 terraform:
diff -r 5c58433395f3 vendor/terraform/VERSION
--- a/vendor/terraform/VERSION  Tue Oct 23 18:32:59 2018 -0700
+++ b/vendor/terraform/VERSION  Sun Nov 11 15:53:45 2018 -0800
@@ -1,1 +1,1 @@
-0.11.8
+0.11.10
tamsky avatar

and there seems to be some conflict between what version of terraform gets installed in geodesic. geodesic appears to use the apk manifest in //packages.txt to install /usr/bin/terraform. in the cloudposse/packages image, it installs /usr/local/bin/terraform binary directly from vendor (no apk), but this binary does not get copied into geodesic during the multi-stage build.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

to use multi-stage pattern to install packages, use this:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. https://slack.cloudposse.com/ - cloudposse/geodesic

tamsky avatar

So that may mean there are 3 different version directives of terraform in play?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

There are 3 methods to install

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Alpine packages

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Make directives

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Docker multi stage

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The install folder is preserved for legacy reasons

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It will be migrated to use the new make system for backwards compatibility

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But for now we just have both

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@tamsky i see what you’re saying

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

//packages/vendor/terraform/VERSION is used by the new system

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

//packages/install/Makefile we’re going to create a wildcard target to map the old targets to the new targets for backwards compatibility

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


geodesic appears to use the apk manifest in //packages.txt to install /usr/bin/terraform.
in the cloudposse/packages image,

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

correct - alpine packages tend to get installed in /usr/bin, so we preserved that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


in the cloudposse/packages image, it installs /usr/local/bin/terraform binary directly from vendor (no apk)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is determined by the INSTALL_PATH environment variable that defaults to /usr/local/bin

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we didn’t want non-package packages going into /usr/bin

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

/usr/local/bin should have priority over /usr/bin (but i haven’t verified)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i’m open to discussion/suggestions on better ways of handling this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@yurchenko did you get to the bottom of it?

yurchenko avatar
yurchenko
06:26:00 AM

@yurchenko has joined the channel

2018-11-12

tamsky avatar


/usr/local/bin should have priority over /usr/bin (but i haven’t verified)

geodesic’s Dockerfile only lists two COPY directives related to the packages image

COPY --from=packages /packages/install/ /packages/install/
COPY --from=packages /dist/ /usr/local/bin/

so everything in --from=packages /packages/bin/ is lost.

tamsky avatar

for now, I’ve fixed my problem of getting v0.11.10 in geodesic by adding +COPY --from=packages /packages/bin/terraform /usr/bin/terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@tamsky see make dist

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the idea is that you copy everything you want into /dist

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then COPY --from=packages /dist /usr/local/bin

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

by staging things into the /dist folder, allows you to specify the packages in an ENV, so that you only specify a single COPY statement

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if it would help, i can do a quick zoom

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@tamsky are you unblocked?

2018-11-13

tamsky avatar

I made good progress yesterday, so I feel pretty good about it all.

Thanks for the reply and offer to help.

tamsky avatar


allows you to specify the packages in an ENV,

this sounds like a key concept. I’ll take a look at make dist today.

tamsky avatar

Do you have any existing techniques/strategies for caching the .terraform/plugins/linux_amd64/ directory? – maybe as a runtime volume-mount?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i want to do that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i tried it in vane at some point, but the problem is that it’s per terraform project

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i wish .terraform/plugins could be mapped to /var/lib/terraform or something

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(and without symlinks)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but i haven’t found any envs to do that

tamsky avatar


i tried it in vane at some point, but the problem is that it’s per terraform project

it could be a greedy/opportunistic algorithm… all local images that match a pattern get mounted somewhere and an init job/script links/copies them into $HOME/.terraform/plugins

(and without symlinks)

sounds like you know something here. does terraform require the inodes in .terraform/plugins/linux_amd64/ to be regular files and not symlinks?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no, i’m just “anti-symlink” hacks

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

especially where generating symlinks programmatically

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if it’s a one shot deal, like ln -s /lib64 /lib, thats cool

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i guess it’s more related to tfstate. we used to link tfstate to a persistent volume

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but if the symlinks were stale or acidentally pointing to the wrong place, it can have catastrophic outcomes. .terraform/plugins should be pretty safe to symlink. Just wish there was an ENV for it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the init-terraform script would be the place to do it.

tamsky avatar

well, tbh, I’d expect any mistakes in the tree or data below .terraform/plugins to be ignored by the terraform binary…. Seems like it’s an opportunistic cache, and the cache itself does not need versioning… since each provider binary within the cache is versioned.

joshmyers avatar
joshmyers
cli: Opt-in local cache directory for plugins by apparentlymart · Pull Request #16000 · hashicorp/terraform

To avoid surprising users by creating a global plugin directory they don&#39;t know about, we elected to auto-install plugins into the local .terraform directory to keep things contained. However, …

joshmyers avatar
joshmyers

Does TF_PLUGIN_CACHE_DIR do what you want?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

wha?? i missed that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s cool

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, that’s what i want.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

now can we have the same for modules?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Feature Request: Module cache dir à la plugins · Issue #16268 · hashicorp/terraform

References #15613 #16000 Similar to the update in 0.10.7 that cached plugins in a shared dir it would also be nice to be able to keep modules in a shared location to stop duplication on things like…

tamsky avatar

only if you trust .terragrunt-cache

tamsky avatar

oh, maybe we can setup geodesic to bind mount the local user’s .terraform/plugins, and that way the local user’s cache can be shared?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just set TF_PLUGIN_CACHE_DIR=/localhost/.terraform/plugins

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we always mount $HOME to /localhost

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(so we can use .ssh among other things)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i caution though against using /localhost/.terraform, unless your host os is linux

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

since you’ll have conflicts (possibly) unless plugins are versioned by arch

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

maybe use /localhost/.geodesic/terraform

tamsky avatar


plugins are versioned by arch
they are…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in the filenames?

tamsky avatar

in the tree structure

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

nice

tamsky avatar

this works… my mac’s homedir .terraform.d/plugins used to only have a darwin_amd64/ dir, now it also has a new linux_amd64/ dir

1
tamsky avatar

now if we can just get rid of the ssh-add passphrase prompt

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

do you never pull private terraform modules?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. https://slack.cloudposse.com/ - cloudposse/geodesic

tamsky avatar

I’d love it if my local OSX ssh-agent could be shared/forwarded…. like as described here: https://github.com/avsm/docker-ssh-agent-forward/pull/10#issuecomment-410510815

Socat-based approach by djmaze · Pull Request #10 · avsm/docker-ssh-agent-forward

@avsm Thanks for the great idea you had here! I changed this so it doesn&#39;t need local volumes anymore. This makes it work with boot2docker on Linux as well. Even more, it works with any remote …

tamsky avatar

I can report success using geodesic. :smiley: I use the [root.cloudposse.co](http://root.cloudposse.co) style custom Dockerfile for my environment. I’m able to run terraform on source from my private repo.

tamsky avatar


do you never pull private terraform modules?

I do, but it’s the main tree I use, and I COPY it into my geodesic build.

I have two different make targets in my tree. One directs terragrunt to use a module { src= that points to a private git repo, the other uses the local directory as the module source (I use terragrunt ENV var and include() directives do this).

Given that my docker image has all the private tf source files I need, I just skip the ssh-add prompt.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, and use https:// for all the public modules

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i’ve wanted to forward the SSH agent as well

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the socat based approach felt difficult to generalize for the average user

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

plus introduces more native dependencies

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

currently, i like that bash+docker is all that’s needed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

on linux, it’s possible to bind mount the socket

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but I assume you’re on OSX

tamsky avatar

yes, and therein lies a feature discrepancy between docker-machine and native linux

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the other idea i’ve had is to run sshd inside the container

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then instead of exec’ing you treat it like a host

tamsky avatar
Support for sharing unix sockets · Issue #483 · docker/for-mac

Expected behavior When mounting a directory containing unix sockets the sockets should function the same as they do on a Linux host. Actual behavior The socket is &#39;there&#39;, but non-functiona…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what do you think about thta?

tamsky avatar

the terrible thing about ssh keyfile passphrases, is their encryption is terrible

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hahah, but locally?? on your mac?? does it matter

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we could -v ${HOME}/.ssh/authorized_keys:/root/.ssh/authorized_keys

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so it would be easy to ssh into the container. plus, if you have your keys in your OS keychain, then no passwords + agent forwarding.

tamsky avatar
The default OpenSSH key encryption is worse than plaintext

The eslint-scope npm package got compromised recently, stealing npm credentials from your home directory. We started running tabletop exercises: what else wo…

tamsky avatar

yeah, it kinda does matter

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hah do you know @lvh?

tamsky avatar

no, don’t know em

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(i think he wrote that post)

tamsky avatar

I prefer to allow the OS keychain manager to manage secrets…. which we’re kinda looking the other way when it comes to aws-vault as well – we’re kind-of assuming that PBKDF2 is “good enough”

tamsky avatar

… and typing it all the time isn’t a burden

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but taking a step back, i don’t understand why exposing an SSH daemon inside the geodesic container would be a bad thing (for local workstations) and how this “openssh key encryption” issue is related

tamsky avatar

they’re not related… you’ve proposed a different way in, that allows us to bring our agent with us….

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

right

tamsky avatar

the encryption issue stems from the current setup – and I don’t want to type the passphrase (my keychain manager does that for me)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

aha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, thanks - now I get you.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so the way it works today, you don’t like for the aforementioned reasons.

tamsky avatar


a different way in, that allows us to bring our agent with us

this would be really smooth

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if we moved to the sshd approach, that would be cool?

tamsky avatar

yeah – I could imagine a key in ~/.ssh/geodesic_authorized_keys or similar, since I’d guess not everyone wants an ~/.ssh/authorized_keys file at all.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea - we could generate one for geodesic

tamsky avatar

detect the existence of the file in the wrapper, and set an ENV var in docker run which triggers the daemon to launch?

tamsky avatar

oh, “generate one for geodesic” => wrapper generates a new key and auth_keys file for geodesic access kinda thing?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, something like that

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think we should add ENV TF_PLUGIN_CACHE_DIR=/localhost/.terraform.d/plugins to the main geodesic

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thoughts?

2018-11-15

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@here i might have asked this, but maybe it wasn’t clear. Using the Geodesic Framework, setting up CloudTrails that send logs to central Audit account S3. All good there, I have that in place. My question is; what module do i use to send other AWS events to the bucket from all of the accounts?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

what other events that you want to log are you asking for?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

as always, curious if you have any more suggestions

tamsky avatar

I think the reason I haven’t had many suggestions is because I haven’t been able to use it daily. That’ll change now.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha, ok, preparing to duck for cover!

1
lvh avatar
lvh
08:04:39 PM

@lvh has joined the channel

2018-11-19

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/packages

Cloud Posse installer and distribution of native apps, binaries and alpine packages - cloudposse/packages

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so that now install/ targets are just proxied to the vendor/<package>/install

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(all backward compatible)

2018-11-20

joshmyers avatar
joshmyers

What are folks thoughts on Terragrunt usage in Geodesic? I note some usage in places but not everywhere…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We haven’t committed to it wholeheartedly. Generally averse to wrappers as they often break interoperability.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But I do like how we can extend root modules (e.g. for adding users)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It could rather trivially be used everywhere in geodesic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

To date, we have been Docker multi stage to copy and keep things dry

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I know one company moved to git submodules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So I think we have 3 good approaches with various trade offs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But it’s a topic that I like to debate. I don’t feel like it’s 100% solved.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Part of what we want is a solution that is tool agnostic, which is why terragrunt is not ideal

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We need to version kops, Helmfiles, and other things. Terraform isn’t the end-all-be-all

joshmyers avatar
joshmyers

Indeed. adding more layers to geodesic such as Terragrunt make things harder to rationalise

joshmyers avatar
joshmyers

Although I totally see why it is helpful >_<

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Part of my motivation for adding it was to show how flexible the strategy is that we have

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That it works with terragrunt like it works with everything else

joshmyers avatar
joshmyers

There are aspects of Terragrunt I don’d mind leaning on too much. Others like the overrides, I’m not so sold on.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I like the easy state auto init, env mapping, and ability to create some thing like a poor mans overlay file system (are those the overrides?)

joshmyers avatar
joshmyers

In terms of the directory structure and recursively looking up a directory until it reaches a terraform.tfvars, allows only 2 level deep child/parent overrides IIRC. Agree on above likes, and before/after hooks

joshmyers avatar
joshmyers

Feels weird using it when there is some overlap/similar things going on with geodesic itself

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea so the terragrunt directory structure doesn’t make sense with geodesic reference architectures

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We recommend sticking with our poly repo approach

tamsky avatar

@joshmyers when you say “overrides” are you talking about terraform’s *[override.tf](http://override.tf) feature?

tamsky avatar


recursively looking up a directory until it reaches a terraform.tfvars, allows only 2 level deep child/parent overrides

I do this, but in a more generic way in the Makefile I use. It just looks for a placeholder filename that indicates the root of the environment: export UPSTREAM_ROOT_PREFIX := $(shell FILE=.environment_root ; for i in . .. ../.. ../../.. ; do test -e $$i/$$FILE && echo "$$i/" && exit 0 ; done ; echo "unable-to-find-$$FILE" ; exit 1 )

tamsky/terrabase

Contribute to tamsky/terrabase development by creating an account on GitHub.

1
joshmyers avatar
joshmyers
gruntwork-io/terragrunt

Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules. - gruntwork-io/terragrunt

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We do something similar with the build-harness “autodiscovery”

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
export BUILD_HARNESS_PATH ?= $(shell until [ -d "$(BUILD_HARNESS_PROJECT)" ] || [ "`pwd`" == '/' ]; do cd ..; done; pwd)/$(BUILD_HARNESS_PROJECT)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

nice to see someone love make as much as we do

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
[email protected]=123456789012/

account-specific resources are placed within an AWS account directory.
directory names the root account email and the 12-digit AWS account number.

that’s kind’a nice

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@here pardon my ignorance, but i want to ask something that may make sense for others, just not clear to me:

The IAM User that is created for the [root.cloudposse.co](http://root.cloudposse.co) repository. Which is used to manage other accounts. Can that be a federated user since it assumes IAM Roles? or?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so there’s often a two-phased approach here since a “virgin” account has no IAM users

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so step 1, you use the master AWS account credentials to setup the scaffolding

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

part of that scaffolding is to provision an IAM user for yourself (and anyone else on the team)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

step 2, is you use your personal IAM account to provision everything else

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is the ugly “coldstart” problem…

OScar avatar

Ok so here is my scenario, I am thinking of doing this in the master account (without using the actual master account):

Login to the root account with the root credentials and do the following: Create new IAM group cloud_admin Assign AdministratorAccess policy to the group Create an IAM user with the name aws_admin Add the user to the group Enable MFA for the user (we recommend using Google Authenticator as Virtual MFA device) Generate Access Key ID and Secret Access Key for the user (we’ll need them to run scripts)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, that should be fine

OScar avatar

So in this scenario for root, anyone in the cloud_admin group, including say my individual account can run the Geodesic stuff right?

OScar avatar

assuming my individual account has keys and enabled with MFA of course

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yep, that should work

OScar avatar

Thanks @Erik Osterman (Cloud Posse)!!

OScar avatar

this clarification helps me tremendously in explaining what account is needed to run this stuff

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

np.. yea, in the end, you just need some kind of “administrator” access. then you can go about provisioning all the other stuff.

1

2018-11-21

OScar avatar

It looks like this error is only happening when I am inside the Geodesic shell…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think maybe your AWS_DEFAULT_PROFILE is not set to use that profile?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Or try to remove the default profile from aws/config, if you have one

OScar avatar

I don’t have a default profile in ~/.aws/config

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ok one other idea

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Run “hwclock -s” in your container

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It doesn’t seem to like your MFA token

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So maybe clock is out of sync

OScar avatar

oh you know what? i think you are right

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have seen that happen a lot in Docker for MAC

OScar avatar

one thing about this account is that the guy who set this up, also setup the mfa so i had to use his barcode in order to add this to my mobile

OScar avatar

hwclock yields this 2018-11-22 01:41:54.996656+00:00

OScar avatar

so that time seems to be different than my time

OScar avatar

its tomorrow

OScar avatar

for one thing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Though it’s GMT :)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Are the minutes and seconds similar?

OScar avatar

✗ (none) ~ ⨠ hwclock -v hwclock from util-linux 2.32 System Time: 1542851097.345229 Trying to open: /dev/rtc0 Using the rtc interface to the clock. Last drift adjustment done at 1542851026 seconds after 1969 Last calibration done at 1542851026 seconds after 1969 Hardware clock is on UTC time Assuming hardware clock is kept in UTC time. Waiting for clock tick... ...got clock tick Time read from Hardware Clock: 2018/11/22 01:44:58 Hw clock time : 2018/11/22 01:44:58 = 1542851098 seconds since 1969 Time since last adjustment is 72 seconds Calculated Hardware Clock drift is 0.000000 seconds 2018-11-22 01:44:56.995687+00:00

OScar avatar

but that does not seem like my local time

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Did you try hwclock -s?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Document Docker for Mac Time Drift Problem · Issue #203 · cloudposse/docs

what aws-vault: error: Failed to get credentials for example (source profile for example-staging-admin): SignatureDoesNotMatch: Signature expired: 20180806T044229Z is now earlier than 20180806T1916…

OScar avatar

i did, same issue, the date shows Nov 22 as well

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ok, can help dig in more when in front of a keyboard :)

OScar avatar

i need a drink!

OScar avatar

thanks @Erik Osterman (Cloud Posse)!

OScar avatar

i’ll keep trying things

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s officially o’clock in PST

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Maybe some tips in this issue

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Docker time outof sync with osx host · Issue #1260 · docker/for-mac

My osx system has IST time zone. When i run, docker info command, it shows a wrong system time which is few hours behind the osx host. docker info shows UTC time. I have properly configured time/ti…

OScar avatar
arunvelsriram/docker-time-sync-agent

docker-time-sync-agent is a tool to prevent time drift in Docker for Mac’s HyperKit VM. - arunvelsriram/docker-time-sync-agent

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Can try it out…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
hashicorp/terraform

Terraform is a tool for building, changing, and combining infrastructure safely and efficiently. - hashicorp/terraform

tamsky avatar

¿¿¿ “yet-another-config-file” ???

/me wishes it would have been able to use our existing *.tf -definitions of provider.[*].*.version

hashicorp/terraform

Terraform is a tool for building, changing, and combining infrastructure safely and efficiently. - hashicorp/terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also want this for modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

To satisfy client request for vendoring

OScar avatar

@Erik Osterman (Cloud Posse) i tried that agent, no bueno

OScar avatar
OScar
02:49:03 AM

I added this to the Dockerfile and now the date shows correctly, but it is like 2 seconds off!

2018-11-22

Jan avatar

Ola!

Jan avatar

quick question, is there a known cost to runt he smallest setup of the full reference archetecture

Jan avatar

with aws org

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan it depends on the project. There is no cost to provision the resources that are just ’metadata` like VPC, subnets, all IAM stuff (users, roles, groups), accounts, etc.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for Route53, you just pay for each DNS zone you have (I believe it’s $2 per zone per month)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then you deploy EC2, RDS, Aurora, Elasticsearch, Elacticache, for which you pay depending on the instance type and count

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you provision k8s using kops or EKS, you pay per instance type and count

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then, you pay for S3 storage

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and for network traffic

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so, if you deploy the initial setup - VPC, subnets, accounts, IAM, Route53, S3 (for TF and kops states) - you pay just for Route53 and S3 storage (which is almost nothing)

Jan avatar

yea thanks @Andriy Knysh (Cloud Posse)

Jan avatar

im doing an evaluation of full org using geodesic

Jan avatar

and figured I would do so on my own account to start with

Jan avatar

mmm there is something I am not groc’ing

Jan avatar

with aws-vault

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

What’s the error?

Jan avatar

ah nm

Jan avatar

im being stupid

Jan avatar
aws-vault login aws.tf-staging-admin
Enter passphrase to unlock /Users/xxxxx/.awsvault/keys/:
Enter token for arn:aws:iam::xxxxxxxxxxx:mfa/root-account-mfa-device: xxxx
aws-vault: error: Failed to get credentials for aws.tf-root (source profile for aws.tf-staging-admin): AccessDenied: Roles may not be assumed by root accounts.

Jan avatar

I had the staging account set to the root account id

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ah yes copy pasta get me every time

Jan avatar

yarp

Jan avatar

so I would almost do this with terraform local-exec and render templates

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Isn’t that a catch-22? Would love if we could automate it, but we need the profile configuration before we can run terraform

Jan avatar

yea totally, at some point bootstrapping has to start manually

Jan avatar

and pass the credentials for boot strap via a .tfvars file which the local exec kills after it runs

Jan avatar

also what the docs dont mention is that you should have 2 accounts setup to start with

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

2 AWS accounts ?

Jan avatar

root and staging

Jan avatar

for aws orgs

Jan avatar

maybe I have skipped ahead in the docs or something

Jan avatar
[profile example-staging-admin]
region=us-west-2
role_arn=arn:aws:iam::$aws_account_id_for_staging:role/OrganizationAccountAccessRole
mfa_serial=arn:aws:iam::$aws_account_id_for_root:mfa/[email protected]
source_profile=example
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrmmm so maybe doc issue. We programmatically create all accounts

Jan avatar

aws accounts?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Sec

Jan avatar

then I am in the wrong place on the doc for sure

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-root-modules

Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Jan avatar

brb, dinner

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you want, maybe open a “running issue” with all the confusing things you hit. Then we can address those.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hah, eating breakfast. Opposite ends of the word for sure.

Jan avatar

Where in the docs should I be starting from and how should I follow the guid end to end

Jan avatar

Guide *

Jan avatar

So I think I might understand how I ended up where I was in the docs. The quick start guide says to setup local env and tools. Linked in there is the page for aws-vault setup

Jan avatar

Which already has assumptions based on things not yet done

Jan avatar

Thank you @joshmyers

2
Jan avatar

“Create an IAM user with the name admin” I assume only CLI access?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrmmm can you send me a link?

joshmyers avatar
joshmyers

That admin user is another in the bootstrap process before you can assume roles as yourself.

2018-11-23

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan are you able to login to the root account from geodesic?

Jan avatar

So my current place is:

  1. root account is there
  2. aws-vault is setup with root admin user
  3. I can login or exec
  4. its when I then want to do the IAM make init I get promoted for the unset account id var
Jan avatar

I am

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so before #4, let’s provision the other accounts

Jan avatar

Ah I see

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Collection of Terraform root module invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

cd ...

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. init-terraform
Jan avatar

ok 2 secs

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. terraform plan <whatever account you need>
Jan avatar

Should this be doable without cloning down the repo?

Jan avatar

cd accounts && terraform plan testing

Jan avatar

@Erik Osterman (Cloud Posse) are you maybe able to help me on this?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan sorry missed your reply

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

can you give more details?

Jan avatar

Gimme a few minutes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you look at the Dockerfiles for each environment, e.g. prod https://github.com/cloudposse/prod.cloudposse.co/blob/master/Dockerfile

cloudposse/prod.cloudposse.co

Example Terraform/Kubernetes Reference Infrastructure for Cloud Posse Production Organization in AWS - cloudposse/prod.cloudposse.co

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we copy the root modules from terraform-root-modules repo into geodesic container https://github.com/cloudposse/prod.cloudposse.co/blob/master/Dockerfile#L35

cloudposse/prod.cloudposse.co

Example Terraform/Kubernetes Reference Infrastructure for Cloud Posse Production Organization in AWS - cloudposse/prod.cloudposse.co

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

copy whatever modules you need for that particular env

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

in the root account/env, we copy the accounts module https://github.com/cloudposse/root.cloudposse.co/blob/master/Dockerfile#L61

cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

because we create all the accounts in root (billing/master)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then, you run the geodesic container, then cd into the folder, and run terraform plan/apply

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hey @Jan schedule is crazy today. can we maybe setup some time this week? https://calendly.com/cloudposse

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i’ll help ya out

Jan avatar

That would be awesome

Jan avatar

Done, thanks mate

Jan avatar

@Andriy Knysh (Cloud Posse) thanks for the info, will explore that this evening

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan please give more details, I could help with that

Jan avatar

I will do so in a few hours when home

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Jan avatar

Right lets take a look see

Jan avatar

So currently I have root/ and testing/ (following the cold start)

Jan avatar

root env run, assume role done. tfstate backend all done

Jan avatar

@Andriy Knysh (Cloud Posse) now is the point where I got lost

Jan avatar

after tfbackend is done

Jan avatar
 ✓   (aws.tf-root-admin) tfstate-backend ⨠  ls
Makefile  README.md  main.tf  outputs.tf  scripts  terraform.tfstate  terraform.tfstate.backup	terraform.tfvars.example
 ⧉  root.aws.tf
 ✓   (aws.tf-root-admin) tfstate-backend ⨠  cd ..
 ⧉  root.aws.tf
 ✓   (aws.tf-root-admin) ~ ⨠  ls
Makefile  README.md  account-settings  accounts  atlantis  atlantis-repos  cloudtrail  iam  organization  root-dns  root-iam  terraform.tfvars	tfstate-backend  users
 ⧉  root.aws.tf
 ✓   (aws.tf-root-admin) ~ ⨠  cd accounts
-> Run 'init-terraform' to use this project
 ⧉  root.aws.tf
 ✓   (aws.tf-root-admin) accounts ⨠  ls
audit.auto.tfvars.example  audit.tf  dev.auto.tfvars.example  dev.tf  main.tf  prod.auto.tfvars.example  prod.tf  staging.auto.tfvars.example  staging.tf  testing.auto.tfvars.example	testing.tf
-> Run 'init-terraform' to use this project
 ⧉  root.aws.tf
Jan avatar

so I guess now the idea, based on your description would be to jump into the accounts dir and TF plan / apply

Jan avatar

but given that I will try run all the .tf’s that doesnt make sense

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea, so terraform-root-modules contains everything that could be used in any project. Three possible ways of doing it:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. You don’t have to copy all folders and all files if you don’t need them. In Dockerfile, copy just what you need, e.g. accounts/main.tf + accounts/testing.tf + …
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Fork/copy our repo and update it to suit your needs
Jan avatar

ok I see

Jan avatar

so I will just rm for now while testing

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. If you want to use our repo AND you copied all files, then use terraform plan -target=... to provision just the resources you need
Jan avatar

ok lets go with 3

Jan avatar
 ⧉  root.aws.tf
 ✓   (aws.tf-root-admin) accounts ⨠  terraform plan -target=testiing.tf
var.audit_account_email
  Audit account email

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so to provision just the testing account, you’d do: terraform plan -target=aws_organizations_account.testing

Jan avatar

ah ok

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so yes, if you copied all the files, it will ask you for the missing vars

Jan avatar
✓   (aws.tf-root-admin) accounts ⨠  terraform plan -target=aws_organizations_account.testing
var.audit_account_email
  Audit account email
Jan avatar

auditing is not a dependant of the testing env is it?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you can add them to the Dockerfile and make them empty (for now) so it would not ask it everytime

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

no, it just asks for all missing vars

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Terraform asks

Jan avatar

ah alright

Jan avatar

so you just unset them all in the Dockerfile

Jan avatar

So lets say I want to create ALL the envs

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

those that you don’t want/need, make an empty string

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

later when you want to create more envs, update the empty values with real ones and provision the nevs

Jan avatar
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Jan avatar

So maybe I missunderstood

Jan avatar

I was under the thinking that at this point in the cold start I would have geodesic create the other AWS accounts

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you need real values for those envs that you are creating (testing in your case)

Jan avatar

how would I have ID or NS servers for the other envs

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for those envs that you don’t need right now, you can make them empty strings for now so TF will not ask for the values

Jan avatar

unless the account and zone is a prerequisite

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

update the email

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

provision the account -> get the ID

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

update the Dockerfile with the account ID

Jan avatar

“provision the account -> get the ID” using gedesic?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes, root geodesic

Jan avatar

ok, as a suggestion then I would have these run as differnt layers or something

Jan avatar

or as make targets

Jan avatar

sec lemme try

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it’s cold start, so it’s not just one or a few commands to run

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you have to provision some resources, then update the Dockerfile, restart geodesic, then provision the next resource

Jan avatar

so then maybe its needed to have the description about unsetting and setting etc

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we have that in the docs (except for the accounts, which were added after the docs was created)

Jan avatar

could you point me to the place in the docs, for my own sanity?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

sorry, we have added the accounts to the doc already

Jan avatar

Update the TF_VAR_root_account_admin_user_names variable in Dockerfile for the root account with your own values.

Jan avatar

I was reading it as following top to bottom which doesnt work in this flow

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea sorry, the doc was not updated for that. The user names are here https://github.com/cloudposse/root.cloudposse.co/blob/master/conf/root-iam/terraform.tfvars

cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

but you can add them to the Dockerfile as well

Jan avatar

So I am not seeing the env vars in env

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

TF will read those env vars from the .tfvars file or from TF_VAR_ vars

Jan avatar
⧉  root.aws.tf
 ✓   (aws.tf-root-admin) accounts ⨠  pwd
/conf/accounts
 ⧉  root.aws.tf
 ✓   (aws.tf-root-admin) accounts ⨠  env | grep -i TF_VAR
TF_VAR_region=eu-central-1
TF_VAR_root_domain_name=root.aws.tf
TF_VAR_stage=root
TF_VAR_namespace=aws.tf
TF_VAR_parent_domain_name=aws.tf
TF_VAR_account_id=xxxxxxxxx
TF_VAR_aws_assume_role_arn=arn:aws:iam::xxxxxxxx:user/admin
TF_VAR_local_name_servers=["", "", "", ""]
 ⧉  root.aws.tf
Jan avatar

I did drop out and do a make init / make docker/build make install

Jan avatar

and re run

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you need to add them to either the .tfvars file or to Dockerfile

Jan avatar

they are in the dockerfile

Jan avatar

let me just remove all local copies of this docker file

Jan avatar

been too much debugging that im in a weird place

Jan avatar

think I spotted it

Jan avatar

the two env’s in the root Makefile need to be updated

Jan avatar
export CLUSTER ?= 
export DOCKER_ORG ?= 
Jan avatar

then make docker/build then make push

Jan avatar

after which make install

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Jan yes, ENV DOCKER_IMAGE="cloudposse/root.cloudposse.co" needs to be updated

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

you don’t need to push it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

make/install installs it locally

Jan avatar

I did

Jan avatar

something else is going on on my env

Jan avatar

locally then

Jan avatar

cause this is not playing as expected

Jan avatar

ENV DOCKER_IMAGE="cloudposse/root.cloudposse.co" was always updated

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(even if it was not, should still work since it’s a local docker image)

Jan avatar

yea, something else is up

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

what’s the issue?

Jan avatar

sorry kids were waking up

Jan avatar

I have cloned all the repos down clean

Jan avatar

starting at 0

Jan avatar

Change the IAM user names for the accounts

Jan avatar
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and they could be added to the .tfvars file OR as env vars to the Dockerfile

Jan avatar

cool thanks

Jan avatar

I would stick with 1 method, not both honestly

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Agree with you. It was changed recently for some other reasons. You can open issues in docs and root modules, and we’ll review. Thanks again

Jan avatar

cool cool, again im making notes as I go

Jan avatar

I honestly might just put in place gotemplates and have terraform render them

Jan avatar

or some glue code on top

Jan avatar

I will run some of the guys at the office through this all sooner rather than later too

Jan avatar

so will do a docs PR first

Jan avatar

so…

Jan avatar

in root

Jan avatar
make init
.......

Make docker/build
...
Status: Downloaded newer image for cloudposse/geodesic:0.38.0
 ---> edc6e5ea3362
Step 3/46 : ENV DOCKER_IMAGE="jdnza/root.aws.tf"
 ---> Running in d378f33e07fe
Removing intermediate container d378f33e07fe
 ---> ce3e38018eaa
Step 4/46 : ENV DOCKER_TAG="latest"
 ---> Running in 1ece15da606a
......
....
Successfully built 76c97523011f
Successfully tagged cloudposse/root.cloudposse.co:latest
(⎈ N/A:N/A)   ~/dev/aws.tf/root.aws.tf   master ●  make install
Password:
# Installing root.aws.tf from jdnza/root.aws.tf:latest...
Unable to find image 'jdnza/root.aws.tf:latest' locally
docker: Error response from daemon: manifest for jdnza/root.aws.tf:latest not found.
Jan avatar
(⎈ N/A:N/A)   ~/dev/aws.tf/root.aws.tf   master ●  cat Dockerfile | grep  -i DOCKER_IMAGE
ENV DOCKER_IMAGE="jdnza/root.aws.tf"
Jan avatar

updated makefile and its happy

Jan avatar

after make docker/build

Jan avatar

finally on track

Jan avatar
 aws_organizations_account.audit: Error creating account: ConcurrentModificationException: AWS Organizations can't complete your request because it conflicts with another attempt to modify the same entity. Try again later.
	status code: 400, request id: 83cad504-f29f-11e8-ba07-3d9eed4b75b4
* aws_organizations_account.dev: 1 error(s) occurred:

* aws_organizations_account.dev: Error creating account: ConcurrentModificationException: AWS Organizations can't complete your request because it conflicts with another attempt to modify the same entity. Try again later.
	status code: 400, request id: 83cad503-f29f-11e8-ba07-3d9eed4b75b4

Jan avatar

hahahah

Jan avatar

plan & apply again and its done

Jan avatar

“PROVISION IAM PROJECT TO CREATE ROOT IAM ROLE “

Jan avatar
cd iam

Comment out the `assume_role` section in `iam/main.tf`

should be

Jan avatar
cd root-iam

Comment out the `assume_role` section in `root-iam/main.tf`
Jan avatar

thinking about it now I would also have expandable example output from these commands

Jan avatar

or glue code replace all the copy pasta

Jan avatar

well as far as r53 zones

Jan avatar

gonna sleep, just past 2am here

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Let me know how it goes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

The docs need to be updated to reflect the latest changes to the modules and project structures

Jan avatar

Yea I see that, hopefully I can contribute some of those updates

Jan avatar

I have 2 dir’s

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

it will output the account ID

Jan avatar

root and testing

Jan avatar

I am using the root bin now yea?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then add that account ID to the Dockerfile in the other account’s repo (testing in your case)

Jan avatar

let me quickly do a git reset

Jan avatar

–hard

Jan avatar

I have been making all sorts of changes debugging this

Jan avatar

on a train so internet is slow , gimme a minute

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ok

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

so in shoer, you go to the root account and provision all other accounts, they will be added to the org automatically since the root account is the master/billing in this case

Jan avatar

that makes sense

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

then you update all Dockerfile(s) with the account IDs

Jan avatar

I knew there must be something I was missing

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

sorry for the confusion, we’ll update the docs and ref architectures for that

Jan avatar

alright, let me get back to that point

Jan avatar

hehe all good

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

next iteration will be better

Jan avatar

I have read most of the docs and loads of the TF / kops / k8s code now

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

nice

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

if you have any questions or concerns, please just open issues, we’ll get to them ASAP

Jan avatar

btw is there any reason I cant use the same value for parent domain name and namespace?

Jan avatar

Ideally I will send a pull request from fork with the docs stuff

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

we use the same namespace for all envs

Jan avatar

some English corrections, missed word etc

Jan avatar

NS yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

ah yes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

two cases here:

Jan avatar

im using aws.tf as domain and plan to use aws.tf for NS too

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Use the same parent domain for all envs, e.g. [testing.aws.tf](http://testing.aws.tf), [prod.aws.tf](http://prod.aws.tf), etc.
Jan avatar

yea totally with you there

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Use diff TLS for the envs/accounts, e.g. [example.com](http://example.com) for prod, [example.qa](http://example.qa) for staging etc.
Jan avatar

tld’s or tls?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

top-level domains, TLDs

Jan avatar

figured

Jan avatar

im like a human english linter

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

and since those DNS zones are in diff AWS accounts, we use DNS zone delegations - add Name servers for [prod.aws.tf](http://prod.aws.tf) to the [aws.tf](http://aws.tf) DNS zone

Jan avatar

with you there too

Jan avatar

so I essentially have done e2e tooling similar to what you have

Jan avatar

over the years many time in TF

Jan avatar

just with less abstraction and less nice docker exec env glue code

Jan avatar

so the entire design idea fits sooooo perfectly with my likes

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yea i see

Jan avatar

really excited to help contribute back

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

thanks for testing and using it

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

please, any issues or improvements, let us know

Jan avatar

for sure

Jan avatar

have a list I have built up which I will go voer again when there is a new tag for docs

Jan avatar

as it hasnt been deployed since last update

Jan avatar
-> Run 'assume-role' to login to AWS
 ⧉  root.aws.tf
 ✗   (none) ~ ⨠  assume-role
Enter passphrase to unlock /conf/.awsvault/keys/:
Enter token for arn:aws:iam::xxxxxxxxxx:mfa/admin: 387674
2018/11/23 14:40:40 Request body type has been overwritten. May cause race conditions
* Assumed role arn:aws:iam::xxxxxxxx:user/admin
* Found SSH agent config
* syslog-ng is already running
* Screen resized to 59x165
 ⧉  root.aws.tf
 ✓   (aws.tf-root-admin) ~ ⨠
Jan avatar

2018/11/23 14:40:40 Request body type has been overwritten. May cause race conditions

Jan avatar

might be from my train internet

Jan avatar

haha fighting internet here

Jan avatar
Jan
03:07:13 PM

Jan avatar

I might just give up and do this when im home in a few hours

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

We never tested that on high speed trains :)

Jan avatar

well I am

Jan avatar

yea will pick this up later

2018-11-25

2018-11-27

2018-11-30

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
12:02:59 AM
    keyboard_arrow_up