#geodesic (2021-04)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2021-04-30

2021-04-29

Joe Hosteny avatar
Joe Hosteny

I am working through some of the terraform-aws-components currently. I understand there are changes coming, both from looking at the branch activity and from comments made here. Which branch would currently be best to work off of - master, all-new-components or the per module upstream-<name> branches when copying down a single component? I’m not too concerned with things not being 100% stabilized - just looking to minimize disruptions later as things are rolled out to master.

Matt Gowie avatar
Matt Gowie

@Joe Hosteny all-new-components I believe would be the spot. That is where all the various upstream branches have been merged in the short term.

Joe Hosteny avatar
Joe Hosteny

Yup, thanks @Matt Gowie. I’ve found a number of minor issues that I’ve been able to work around. I haven’t filed tickets, since I am not sure if I am doing something wrong, or where to best put some of them (e.g., one could be a variant2 issue, or the way it is being used by atmos). Additionally, some look to be due to changes between that branch and master. If you would like, I can send you a list when I am done (or perhaps do a quick zoom) to see where to best route some of those issues.

Joe Hosteny avatar
Joe Hosteny

I’ve been able to import my old ref arch org and sub accounts so far, create dns and identity accounts via atmos, create the account and account-map state, and am working through iam-primary-roles.

Matt Gowie avatar
Matt Gowie

Good stuff @Joe Hosteny! That’s awesome. Yeah I’d definitely love to chat about what you’ve found and how you’re liking it so far. As I’m sure @Erik Osterman (Cloud Posse) would as well.

Matt Gowie avatar
Matt Gowie

As to where to file issues… it does depend. Collecting them and then chatting through it makes sense — Let’s do that as I’m sure I can direct you where to go

Matt Gowie avatar
Matt Gowie

I’ll send you a link to grab some time on my calendar whenever is convenient for you.

1
Joe Hosteny avatar
Joe Hosteny

I can already tell this will make my life about 100x easier…once the migration is done

10001
Matt Gowie avatar
Matt Gowie

Yeah, I think this will be a very impressive and sustainable way to build terraform projects once it gains a bit more community steam. I’ve definitely benefited from it a ton already.

2021-04-28

2021-04-23

Joe Hosteny avatar
Joe Hosteny

Hi all. I am running through the tutorial for atmos to understand how to port our old multi-account structure the new workflow. Am I correct in understanding all of the state files will now go into the root account?

jose.amengual avatar
jose.amengual

yes, that is how we use it

jose.amengual avatar
jose.amengual

on bucket with subfolders basically

Joe Hosteny avatar
Joe Hosteny

Thanks @jose.amengual. Right, that’s what I inferred from running through the tutorial. Just wanted to make sure that is the intended setup.

Joe Hosteny avatar
Joe Hosteny

Does this path look correct to you, for a component state file in an account / env? –

Joe Hosteny avatar
Joe Hosteny

<s3://acme-ue2-tfstate-key-hookworm/env:/ue2-root/>

Joe Hosteny avatar
Joe Hosteny

Seems env: is incorrect?

jose.amengual avatar
jose.amengual

mmmmm

Joe Hosteny avatar
Joe Hosteny

I’m actually on my way out, so I will look into it later. But it seemed like a missing interpolation for some var.

jose.amengual avatar
jose.amengual

yes I think you are missing something

jose.amengual avatar
jose.amengual

we use : “workspace_key_prefix”

Joe Hosteny avatar
Joe Hosteny

ok, i ran the stock tutorial. I found one issue, so perhaps this is another minor one. I will put a PR up once i try that change.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Igor Rodionov has written a script to help migrate it for one of our customers

Joe Hosteny avatar
Joe Hosteny

Is that something you could share @Erik Osterman (Cloud Posse) ?

Igor Rodionov avatar
Igor Rodionov

@Joe Hosteny Hello. I can share the script on monday - I have to remove project specific settings and edge cases

Joe Hosteny avatar
Joe Hosteny

Thanks @Igor Rodionov. That would be great. I am working on trying to port one of my orgs (built with reference arch) to the new structure, starting with the account module. I am taking notes, and am happy to contribute those back as well if it is helpful to others.

marc slayton avatar
marc slayton

I ran into the above issue as well (with “env:” leaking into the s3 bucket path) – defining workspace_key_prefix, and making sure there was an ‘environment’ and ‘stage’ set in the top level yaml files seemed to do the trick.

marc slayton avatar
marc slayton

fwiw, I also have a bunch of tutorial notes which may be helpful to some – not sure where I should upload them, however, so they do not get in the way.

Joe Hosteny avatar
Joe Hosteny

@ thanks for your help yesterday. I have some notes too - happy to set up a shared repo if you would like to record those somewhere in the interim, and I can add mine as well

marc slayton avatar
marc slayton

I have an example set of configs that got me to a successful multi-account spin up the first time (beginner’s luck, I expect). I am hoping to finish a procedure which works reproducibly, even if the initial tfstate-backend gets overwritten. This is a common gotcha, I think – an easy mistake to make in the initial config.

marc slayton avatar
marc slayton

On my second multi-account spin up, I made the mistake of using the same bucket and workspace definitions for the tfstate-backend initialization and for the root account spin up. This had the effect of overwriting my initial tfstate and left me needing to reimport the root account.

marc slayton avatar
marc slayton

I think the problem can be corrected by importing the root account definition – but as you know, Atmos has a bug in the terraform-core.variant file, which is blocking me from testing the workaround.

marc slayton avatar
marc slayton

It’s a little embarrassing, but learning how to recompile atmos has taken longer than I thought. Not much of a roadmap there just yet, and there are lots of unfamiliar tools (for me anyway), and fiddly details. As soon as I have a fresh compile of Atmos, I will forward you the procedure that worked, and also the workaround for the gotcha. If I can’t figure that out today, then I will forward what I have and cross my fingers that it works for you.

Joe Hosteny avatar
Joe Hosteny

Hi Marc, I was able to do this with the Dockerfile in the examples directory in the atmos repo. I then just forked the repo for the module changes and pointed it to my branch with the fix for the region issue. Let me see if there are any other changes from that.

Joe Hosteny avatar
Joe Hosteny
diff --git a/atmos/modules/terraform/terraform-core.variant b/atmos/modules/terraform/terraform-core.variant
index e0f0556..fdb9321 100644
--- a/atmos/modules/terraform/terraform-core.variant
+++ b/atmos/modules/terraform/terraform-core.variant
@@ -220,8 +220,7 @@ job "terraform provider override" {
       <<-EOS
       provider:
         aws:
-          - region: "${opt.region}"
-            assume_role:
+          - assume_role:
               role_arn: "${opt.role}"
       EOS
     ,
Joe Hosteny avatar
Joe Hosteny
02:29:11 PM
Joe Hosteny avatar
Joe Hosteny
02:29:45 PM
Joe Hosteny avatar
Joe Hosteny

The first patch is (obviously) my copy of the dockerfile. Second is main.variant in the cli subdir. You can import with these changes.

Joe Hosteny avatar
Joe Hosteny

The make docker/build will export a binary with the new options, so atmos will have the fix when run in the built container

Joe Hosteny avatar
Joe Hosteny

The tfstate-backend component also has bucket versioning enabled. It saved me once when I manually deleted the wrong workspace’s statefile going through this.

marc slayton avatar
marc slayton

Hey Joe – all the above makes sense to me, but there is still something obvious I seem to be missing. I have a similar repository to the one you rmention, only mine is called ‘gangofnuns’. I use the same name for my forked repository on github:

https://github.com/gangofnuns/atmos/releases/tag/v0.17.2-beta.1

I can build a binary and upload it, given a few extra variables on the command line, however, when I use Geodesic to pull down my version of the docker image, it contains no atmos at all. So clearly, I have the flow incorrect in some way. This is a good exercise though, as I’m sure this is something other people are wondering about. It’s complicated, and there’s no roadmap – I’ll keep digging this afternoon/eve.

marc slayton avatar
marc slayton

Problem appears to be here, when I run the build (NOTE: had to add a variable to get it working with the docker registry.)

DOCKER_IMAGE_NAME=gangofnuns/atmos make docker/build
marc slayton avatar
marc slayton
 => ERROR [5/8] RUN apk add [email protected]                                                                                                                                   1.1s
------
 > [5/8] RUN apk add [email protected]:
#9 0.983 WARNING: The repository tag for world dependency '[email protected]' does not exist
#9 0.983 ERROR: Not committing changes due to missing repository tags. Use --force-broken-world to override.
------
executor failed running [/bin/sh -c apk add [email protected]]: exit code: 99
make: *** [docker/build] Error 1
Joe Hosteny avatar
Joe Hosteny

What do you mean by “use Geodesic to pull down my version of the docker image”?

marc slayton avatar
marc slayton

The flow is a little complicated, but here’s how I understand it right now:

marc slayton avatar
marc slayton

1.) pull down the github repository for cloudposse/atmos.

marc slayton avatar
marc slayton

2.) make the code changes we talked about.

marc slayton avatar
marc slayton

3.) Create a new docker container with the code changes. Upload to a different docker registry.

marc slayton avatar
marc slayton

4.) Change the reference in the Geodesic container build to pull in the new container you just uploaded to the registry.

marc slayton avatar
marc slayton

5.) Run ‘make all’ from within a repo such as the ‘multi account reference’ repo.

marc slayton avatar
marc slayton

With an edit to the Dockerfile, this should pull down the atmos container I built above, and use it to create the Geodesic container.

marc slayton avatar
marc slayton

Even as I write this, it seems way too complicated. I must be missing something obvious about the intended flow.

Joe Hosteny avatar
Joe Hosteny

I’m a bit confused by what your new docker container is, vs. the reference in the “Geodesic container build”. I am only using one container - the one in the example repo. The line RUN variant2 export binary $PWD $CLI_NAME creates atmos, or whatever you call it if you change that name. So I do a make docker/build, sudo make install, and then run [atmos.mydomain.com](http://atmos.mydomain.com), and if you do which atmos when that runs, you will see one in /usr/local/bin/. CLI_NAME in the dockerfile controls the exe name, whereas the installable container name is set by APP_NAME in the Makefile.

Joe Hosteny avatar
Joe Hosteny

That is, I set APP_NAME in the Makefile to atmos.<somedomain>.com, since I will have multiple versions of the geodesic container, one for each set of accounts managed (since we have multiple segregated projects)

Joe Hosteny avatar
Joe Hosteny

So, from my bash command line, I run [atmos.crl.com](http://atmos.crl.com), which starts the container built from that Dockerfile, and then atmos from the command prompt inside that running container

Joe Hosteny avatar
Joe Hosteny

It sounds like you might be using two dockerfiles, and having one based on another? If so, is it possible you didn’t copy the binary to the latter stage?

Joe Hosteny avatar
Joe Hosteny

I’ve got to run for the evening, but if you are around tomorrow, send me a PM and perhaps we can do a quick zoom?

marc slayton avatar
marc slayton

RE: Geodesic container build – I’ve been following the instructions for building a multi-account stack. The instructions say: “Go get this repository, with these specific components, and stack definitions”. That repsitory comes with a Makefile and a Dockerfile for building Geodesic. That’s the container I’m using, and where I’ve been assuming I’d need to install the newly recompiled atmos.

marc slayton avatar
marc slayton

Yes, I will have worked through my confusion by then. (So embarrassed.) And in any case, I will send you what I have in terms of procedures for multi-account spin up. RE: Zoom - have you tried Gather Town instead? If not, no problem – I can do Zoom too. The password for the Gather space in the link is “so-foo-bar”. Works in any Chrome browser. Cheers –

Joe Hosteny avatar
Joe Hosteny

Thanks, that call cleared up what you are doing

marc slayton avatar
marc slayton

Appreciate the extra time, Joe – many thanks.

marc slayton avatar
marc slayton

Sorry I didn’t get to show you Gather Town, but maybe some other day. It is a fun step beyond Zoom, imho. Will send some preliminary spin up notes this eve. I am in California, btw, so it might be late. Cheers –

marc slayton avatar
marc slayton
08:01:01 AM

@Joe Hosteny - here’s my original template for a multi-account run. Notice that initially, I set the tfstate-backend to local, not s3. I later copied it to S3 in the master account. after accounts were spun up. I have more procedural notes, but it’s a bit late. I’ll have to finish organizing those for tomorrow. Enjoy!

marc slayton avatar
marc slayton

Hey Joe – here’s what I have thus far from a notes perspective. Hope to add more this evening after some other weekly chores. Still very much a work in progress. Cheers –

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

cc: @Andriy Knysh (Cloud Posse)

marc slayton avatar
marc slayton

^^^ quick caveat – these are embarrassingly incomplete. Slowly trying to improve. Very sorry for delays.

marc slayton avatar
marc slayton

Quick update: With a change to terraform-core.variant and a rebuild of atmos, I was able to import the master account, and thus get my second multi-account reference architecture up and running, despite having botched the tfstate-backend setup. So now I have two procedures that seem to work. This is using terraform 0.14.10.

marc slayton avatar
marc slayton

Important note: I ran into this bug after importing my master account. The workaround was to add:

  # There is no AWS Organizations API for reading role_name
  lifecycle {
    ignore_changes = ["iam_user_access_to_billing"]
  }

to

components/terraform/accounts/main.tf 

seems to have worked without issue. I’ve voted up the bug fix. Hopefully that one gets looked at soon.

marc slayton avatar
marc slayton

Still recording my jolly antics here.

2
Joe Hosteny avatar
Joe Hosteny

Hi @ - made a lot of progress over the weekend, including finishing deploying core iam / dns / cloudtrail across all accounts, and importing a number of pre-existing resources created via the old ref arch. Happy to sync at some point if it is helpful.

marc slayton avatar
marc slayton

I would most definitely be interested. I’m looking for sort of a ‘hit list’ of things I would need to address while back porting existing modules.

marc slayton avatar
marc slayton

With such a list, or a condensed series of liner notes, I could probably make decent progress on backporting further modules.

2021-04-10

2021-04-08

Neeraj Mittal avatar
Neeraj Mittal

I am new to sweetops and trying too get my way around reference-architecture repo. I can see mention about accounts, repos, assets subfolder and couldn’t understand how they are suppose to be used. Any help will be appreciated

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

please don’t start there - that repo is in limbo

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

https://docs.cloudposse.com is the best place to start for now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

multi-account documentation is not available

Neeraj Mittal avatar
Neeraj Mittal

figured out, how to use reference architecture with atmos, and components. And looks good so far

    keyboard_arrow_up