#geodesic (2019-06)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2019-06-27

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/packages

Cloud Posse installer and distribution of native apps, binaries and alpine packages - cloudposse/packages

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
remind101/assume-role

Easily assume AWS roles in your terminal. Contribute to remind101/assume-role development by creating an account on GitHub.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

probably about the same.

2019-06-26

oscarsullivan_old avatar
oscarsullivan_old

Hi guys, how do I upgrade Ansible to 2.8.1 on Geodesic 0.112.0

oscarsullivan_old avatar
oscarsullivan_old

have tried apk add ansible apk add --upgrade ansible apk add ansible-2.8.1 and apk add ansible-2.8.1-r0 (https://pkgs.alpinelinux.org/package/edge/main/x86/ansible)

oscarsullivan_old avatar
oscarsullivan_old

Also pip isn’t in the image by default so I figure it is not pip that installs ansible

oscarsullivan_old avatar
oscarsullivan_old
build(deps): bump ansible from 2.7.10 to 2.8.1 by dependabot-preview · Pull Request #493 · cloudposse/geodesic

Bumps ansible from 2.7.10 to 2.8.1. Commits See full diff in compare view Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a…

oscarsullivan_old avatar
oscarsullivan_old

so it is pip

oscarsullivan_old avatar
oscarsullivan_old
❌ . (none) ~ ➤ pip install
bash: pip: command not found
oscarsullivan_old avatar
oscarsullivan_old

but why isn’t it in my shell, especially when it isn;t removed in https://github.com/cloudposse/geodesic/blob/master/Dockerfile

oscarsullivan_old avatar
oscarsullivan_old

ohhh it’s a different stage of the build FROM alpine:3.9.3 as python dang it

oscarsullivan_old avatar
oscarsullivan_old

Solution for your docker file

apk add py-pip
pip install --upgrade ansible==2.8.1
sweetops avatar
sweetops

Erik, Jeremy, thanks for the help yesterday getting the reference architecture up and running. I was able to finish things up this morning and have it all built. Really impressive stuff. Going through it all this morning trying to get a firm grasp on how it all works.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

2019-06-25

sweetops avatar
sweetops

Hey everyone, following the quick start docs at https://docs.cloudposse.com/geodesic/module/quickstart/ and i’m running into:

sweetops avatar
sweetops
docker run -e CLUSTER_NAME \ -e DOCKER_IMAGE=cloudposse/${CLUSTER_NAME} \ -e DOCKER_TAG=dev \ cloudposse/geodesic:latest -c new-project | tar -xv -C .
docker: invalid reference format.
See 'docker run --help'.
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@sweetops the quick start docs are out of date and not functional. Use the github.com/cloudposse/reference-architectures instead

sweetops avatar
sweetops

ah okay. thanks Erik!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also, archives are here: https://archive.sweetops.com/geodesic/

geodesic

SweetOps is a collaborative DevOps community. We welcome engineers from around the world of all skill levels, backgrounds, and experience to join us! This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build sweet infrastructure.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you get stuck, maybe some nuggets in there.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dalekurt has been recently working through these

sweetops avatar
sweetops

So, I pulled the repo, edited configs/root.tfvars, and exported the aws account’s root master keys to ENV vars, I’m getting:

sweetops avatar
sweetops
terraform init -from-module=modules/root accounts/root
Copying configuration from "modules/root"...
Error: Target directory does not exist
Cannot initialize non-existent directory accounts/root.
make: *** [root/init] Error 1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Not sure. @Jeremy (Cloud Posse) provisioned these this week. Any ideas?

sweetops avatar
sweetops

oh, i was running tf 0.12

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

aha, yes, not updated for 0.12

sweetops avatar
sweetops

yeah, that’s my bad haha

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Yes, you need to have terraform version 0.11 installed on your workstation.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

I will be pushing some updates to the Reference Architecture sometime in the next few days.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

The main thing is updating the baseline version of Geodesic, and fixing the race condition in making the Docker images. Currently, Terraform often tries to build the Docker images before all the files are in place.

The other big things are to update Kubernetes to 1.12.9, switch from kube-dns to coredns, and to pin the versions of terraform and helm installed in the Docker images.

sweetops avatar
sweetops

@Jeremy (Cloud Posse) I’m guessing this is the race condition you mentioned?

sweetops avatar
sweetops
Error: Error applying plan:

1 error occurred:
	* module.account.module.docker_build.null_resource.docker_build: Error running command 'docker build -t [root.blvd.co](http://root\.blvd\.co) -f Dockerfile .': exit status 1. Output:

\#2 [internal] load .dockerignore

\#2       digest: sha256:c8c62ec01c2e58b7ca35e6a8231270186f80ab4c83633dace3b2a61f6e9dc939

\#2         name: "[internal] load .dockerignore"

\#2      started: 2019-06-25 19:16:05.8271816 +0000 UTC

\#2    completed: 2019-06-25 19:16:05.8272689 +0000 UTC

\#2     duration: 87.3µs

\#2      started: 2019-06-25 19:16:05.8274642 +0000 UTC

\#2    completed: 2019-06-25 19:16:05.8712445 +0000 UTC

\#2     duration: 43.7803ms

\#2 transferring context: 2B 0.0s done



\#1 [internal] load build definition from Dockerfile

\#1       digest: sha256:045540caaa44e0ec4d861b43e9328ac90843e9d94c485db1703c3e559ed7dc07

\#1         name: "[internal] load build definition from Dockerfile"

\#1      started: 2019-06-25 19:16:05.8264853 +0000 UTC

\#1    completed: 2019-06-25 19:16:05.8265771 +0000 UTC

\#1     duration: 91.8µs

\#1      started: 2019-06-25 19:16:05.8272773 +0000 UTC

\#1    completed: 2019-06-25 19:16:05.8602995 +0000 UTC

\#1     duration: 33.0222ms

\#1 transferring dockerfile: 2B 0.0s done

failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount930443153/Dockerfile: no such file or directory
Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

@sweetops Yes, that is the race condition. You can just run make root again. When it comes time to make the children, the make children command is safe to run multiple times, but to save time, I recommend you make each child one at a time. Or you can wait a couple of days for the next release of the reference architecture.

sweetops avatar
sweetops

Okay. I’ve still got some conceptual work to do on my end so I’ll probably just hold.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Since you are waiting on it, I will make an effort to get the release out today.

sweetops avatar
sweetops

oh, cool. I mean, no rush really, I don’t want to divert your focus for your day haha.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

No worries, it’s one of the things I’m currently working on for a new client.

sweetops avatar
sweetops

awesome. I appreciate the help.

sweetops avatar
sweetops

@Jeremy (Cloud Posse) Question for you, When spinning these accounts up, I want to rename the dev account to sandbox. Is that as simple as s/dev/sandbox/ in accounts_enabled[] in root.tfvars, renaming dev.tfvars and then stage=sandbox in that file?

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Honestly I’m not sure. I think it would be best to copy rather than rename /configs/dev.tfvars -> /configs/sandbox.tfvars and then customize what you want installed in the sandbox. Keep in mind that by default the dev environment does NOT include a Kubernetes cluster.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Yes, you also need to change stage = "dev" to stage = "sandbox" inside sandbox.tfvars and replace dev with sandbox in accounts_enabled[] in root.tfvars

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

I expect that is all you need to do, but I’m not positive.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Also keep in mind that the “stage” name shows up as a part of nearly every label there is, so we try to keep it short in order to avoid running into issues with names getting too long. So I suggest you pick a 3 or 4 letter name instead of a 7 letter name like “sandbox”.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

@sweetops We have pushed out a new reference-architecture release for you. Skimped a tiny bit on the testing, so please let me know if you find any issues. https://github.com/cloudposse/reference-architectures/releases/tag/0.14.0

cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

sweetops avatar
sweetops

oh awesome. pulling now

sweetops avatar
sweetops

Ran into some terraform errors

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

I was afraid of that. Please paste

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

in this thread

sweetops avatar
sweetops

Okay, sending you a log of the run. It’s a bit verbose so I’ll send as a file.

sweetops avatar
sweetops

Sent you the full log, here’s the actual errors, for this thread:

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

I got the log, that’s not actually a Terraform error. Your AWS access key is lacking permissions.

sweetops avatar
sweetops

oh, crap you’re right

sweetops avatar
sweetops

oohh, i’m in the new account waiting period on this new root account I spun up.

sweetops avatar
sweetops

okay, fixed that.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

BTW, how did you get out of the waiting period so quickly?

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Not Terraform. You need to set environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to static (not sesson) keys with a lot of privileges. Typically they are the root keys of the root account.

sweetops avatar
sweetops

yeah, this new aws root account was in the ‘waiting period’, I fixed that now

sweetops avatar
sweetops

should have checked that after I spun the account up heh

sweetops avatar
sweetops

so, will failing where it did cause any problems, or will make root pick up where it left off?

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

It is safe to run make root again, but I added a make root/init-resume just for this sort of thing.

sweetops avatar
sweetops

okay, I’ll give make root/init-resume a go then

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

After make root/init-resume (but not after make root) you need to run make root/provision

sweetops avatar
sweetops

okay, init-resume finished super fast

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Yes, it’s mainly to get you to a viable docker image. I now realize you were already past that. So make root/provision

sweetops avatar
sweetops

okay

sweetops avatar
sweetops

running root/provision

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

When that finishes, that will be the equivalent of having run make root successfully and you can proceed from there.

Cloud Posse avatar
Cloud Posse
01:26:30 AM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Jun 26, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2019-06-24

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:08 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:03:37 PM

Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7).

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Jul 03, 2019 11:30AM.
Add it to your calendar
zoom https://zoom.us/j/684901853
slack #office-hours (our channel)

2019-06-21

2019-06-20

2019-06-19

chrism avatar
chrism

was there something specific to fix this assume-role (win10/wsl/ubuntu18lts) ?

chrism avatar
chrism

chrism avatar
chrism

all good; found the file from the last time I updated geodesic ENV ASSUME_ROLE_INTERACTIVE=false ftw

chrism avatar
chrism

How are you supposed to use the legacy s3 storage? https://github.com/cloudposse/geodesic/commit/4170a58766fa925800c4293886b32da8d254bff9

I tried adding the following to docker

ENV TF_BUCKET_PREFIX=
ENV TF_BUCKET_PREFIX_FORMAT="basename-pwd"

getting the feeling I’ll have to clear the TF_BUCKET_PREFIX in the .envrc every folder as it still populates it with path depth I dont want

[direnv] use new TF bucket prefix method (#402) · cloudposse/[email protected]
  • [direnv] use new TF bucket prefix method TF_BUCKET_PREFIX_FORMAT selects the format to use for setting the TF remote state bucket prefix/key: the original $(basename $(pwd)) leaf-only form…
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ENV TF_BUCKET_PREFIX_FORMAT="basename-pwd"

chrism avatar
chrism

yup; works. I was trying to cheat and use the envrc file in a folder higher up (i.e. /conf/frankfurt/nginx/ (I put the file in frankfurt) to set it to use TF11 while i migrate some of the easier bits in my control first. Because it changes the env var as use terraform is initialised it was screwing with what I expected

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrmmm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

something like that should work, but maybe there’s a bug somewhere in what we have

chrism avatar
chrism

its just because the old one was root based so it gave no trucks about /{this folder/nginx I got around the region issue using workspaces

then it was fixed recently

chrism avatar
chrism

Is there a way to run multiple geodesics at the same time. it always seems to boot into whichever is running first

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So you would like multiple sessions of the same image?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think we could add an option for that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Right now it gives the Docker container the name of the image so it doesn’t work with concurrent sessions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It always execs into the running image if one is found

chrism avatar
chrism

timezone diff

So i have root.xxx and prod.xxx If i make all on root it boots into that container if i then do the same on prod I end up in roots container

chrism avatar
chrism

ideally should be able to have both open.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That’s not right! Have you installed the wrapper lately?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Try reinstalling it

chrism avatar
chrism

i tend to use make all habitually. seemed odd tbh

chrism avatar
chrism

geodesics up-to-date (hence all the oh fudge that assume role thing i’d been avoiding that breaks in wsl) I’ll dig deeper if its not expected to do that as its probably something stupid

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think this is what yoou want

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we have that in many dockerfiles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

#office-hours starting now! https://zoom.us/j/684901853

Have a demo of using Codefresh for ETL

Mat Geist avatar
Mat Geist

question regarding geodesic in CICD / automated environments. looking at https://github.com/cloudposse/testing.cloudposse.co/blob/master/codefresh/terraform/pipeline.yml i think im missing how the assume-role actually gets executed. as far as i can tell, theres no way to setup aws-vault to be completely non interactive (it always asks for the passphrase prompt). so, in a sentence: how are roles getting assumed in CICD environments

cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

aws-vault is for humans

cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in the CI/CD context, the credentials are provided via alternative means

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

For example, one way is to update a Codefresh shared secret with temporary credentials

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

E.g. if you don’t like the idea of long-lived creds stored in codefresh, this is one way

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

\#!/bin/bash 

set -e

eval "$(aws-vault exec cpco-testing-admin --assume-role-ttl=1h --session-ttl=12h -- sh -c 'export -p')"

output="/dev/shm/codefresh.yaml"
cat<<__EOF__>$output
apiVersion: "v1"
kind: "context"
owner: "account"
metadata:
  name: "aws-assume-role"
spec:
  type: "secret"
  data:
    AWS_SESSION_TOKEN: "${AWS_SESSION_TOKEN}"
    AWS_ACCESS_KEY_ID: "${AWS_ACCESS_KEY_ID}"
    AWS_SECRET_ACCESS_KEY: "${AWS_SECRET_ACCESS_KEY}"
    AWS_SECURITY_TOKEN: "${AWS_SECURITY_TOKEN}"
    AWS_PROFILE: "default"
    AWS_DEFAULT_PROFILE: "default"
    AWS_VAULT_SERVER_ENABLED: "false"
__EOF__

codefresh auth create-context --api-key $CF_API_KEY
codefresh patch context -f $output
rm -f ${output}

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@dustinvb nice little trick

superfresh1
Mat Geist avatar
Mat Geist

how are you able to use aws-vault without the manual passphrase input in that script?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Set the AWS_VAULT_FILE_PASSPHRASE env var

Mat Geist avatar
Mat Geist

oh wow thanks! been looking all over and never found that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think I found it looking through the code at some point

:--1:1
Mat Geist avatar
Mat Geist

i ended up writing a little tool, since working with aws-vault in ci pipelines was a bit too clunky for my tastes https://github.com/BetterWorks/go-assume its a quick and dirty script i threw together this afternoon but it works

:--1:1
dustinvb avatar
dustinvb
10:05:15 PM

@dustinvb has joined the channel

2019-06-17

Cloud Posse avatar
Cloud Posse
04:00:52 PM

Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7).

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Mar 20, 2019 11:30AM.
Add it to your calendar
zoom https://zoom.us/j/684901853
slack #office-hours (our channel)

2019-06-13

jober avatar
jober

Sorry another noob question:

How do I get domain resolutions to work for the member accounts, lets say [app.dev.example.com](http://app\.dev\.example\.com) in the dev account just being a static s3 site

jober avatar
jober

I have been digging around the root modules trying to figure this out and so far no luck

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so a few things are going on

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

first you need to delegate [dev.example.com](http://dev\.example\.com) to the dev account

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the account-dns root module handles creating the zone and is invoked in each child account

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then the root-dns module delegates the DNS to each child account

jober avatar
jober

So I went through the setup of the reference architectures, I have the root account with the NS records set for the dev account. In the dev account the NS records are setup as well and then I created an A record in the dev account to point to the bucket

jober avatar
jober

@Erik Osterman (Cloud Posse) would the original hosted zone I had setup for the root domain be interfering with it?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

?

jober avatar
jober

@Erik Osterman (Cloud Posse) yes

jober avatar
jober

Everything is working as far as the account shells and such. Just having the issue with Route53. I have a suspicion that the original hosted zone setup on the root account is affecting the reference-architecture setup

jober avatar
jober

I moved the registrar to point to the new name servers, and move over any legacy record sets, still no luck

jober avatar
jober

Got it to work

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Great job!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

What was it in the end?

jober avatar
jober

Forgot to update the registrar to the new nameservers

jober avatar
jober

knew it was going to be a noob mistake, thanks for the patience

2019-06-12

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Public #office-hours starting now! Join us on Zoom if you have any questions. https://zoom.us/j/684901853

jober avatar
jober

is it possible to make changes and not have to rebuild the shell everytime?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Use /localhost

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also, we have this PR pending for docs: https://github.com/cloudposse/docs/pull/460

Document workflow for developing terraform modules locally by Nuru · Pull Request #460 · cloudposse/docs

what Document workflow for developing terraform modules locally why Existing documentation does not cover the workflow

jober avatar
jober

Amazing

jober avatar
jober

Thanks so much, that provided a ton of clarity

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(@Jeremy (Cloud Posse) )

jober avatar
jober

When I follow these instructions I get an error:

Error copying source module: error downloading `file:///Users/justin/infrastructure/terraform-root-modules/aws/vpc` : source path error: stat /Users/justin/infrastructure/terraform-root-modules/aws/vpc: no such file or directory
jober avatar
jober

I followed the exact folder structures and everything

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But the users folder is not mounted - not by us

1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Somewhere that is referenced

jober avatar
jober
As a convenience, Geodesic mounts your home directory into the Geodesic container and creates a symbolic link so that you can reach your home directory using the same absolute path inside Geodesic that you would use on your workstation. This means that as long as you do your development in directories under your home directory (and on the same disk device), your workstation's absolute paths to your development files will work inside Geodesic just as well as outside it.
jober avatar
jober

Sorry I must be missing something

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Aha, that was some thing new Jeremy added

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Haven’t tested that myself

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I would verify that you have a current version of geodesic

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

And that you see the symlinks in your shell

1
jober avatar
jober

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Mapping of Home directory was added in Geodesic 0.94.0 https://github.com/cloudposse/geodesic/releases/tag/0.94.0

cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

:--1:1

2019-06-11

JeroenK avatar
JeroenK

in the cold start instruction accounts are provisioned, but in the process a e-mail account like [[email protected]] is needed. Is there a workaround because want to use our general department e-mail address.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Use plus addressing. By default the reference architectures in the repo above do that. See root.tfvars

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Each AWS account requires a unique email address because that is how AWS identifies an account.

JeroenK avatar
JeroenK

How can we use geodesic with for example an mgmt vpc that is connected to a staging vpc and a prod vpc. We use bitbucket server througout the organization. How does this work with the different accounts. Are the examples of custom (terraform)modules?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Think of geodesic as just a preconfigured shell with all the tools required for cloud automation

:--1:1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

What you describe is a configuration not a tool

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So you would add the configuration to geodesic and run it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This is where our root modules come in

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Those provide blueprints for typical configurations like the ones you described

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@JeroenK in https://github.com/cloudposse/terraform-root-modules, there are a few examples of VPC peering:

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

EKS - backing services (where you run things like RDS, ElastiCache etc.) VPC peering: https://github.com/cloudposse/terraform-root-modules/blob/master/aws/eks-backing-services-peering/main.tf

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

as @Erik Osterman (Cloud Posse) mentioned, geodesic has nothing to do with configuration (code, data, settings), it’s a cloud automation shell with many tools inside, used to secure access to AWS (assume role or enterprise auth like Okta) and orchestration of cloud operations

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

configuration usually consists of code (terraform, helm, helmfile, etc.) and data (variables, NEV variables, other settings)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

for code, we use module hierarchy: root modules (catalog of module invocations to provision entire infrastructure) - infrastructure modules (e.g. RDS, EKS, ECS - these are usually combination of other low-level modules) - low-level modules (usually to provision one or a few AWS resources, e.g. IAM role, S3 bucket with permissions, VPC with subnets, etc.)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

all those modules are usually “identity-less”, meaning they don’t care where and how they will be provisioned, all configuration is provided from TF variables, ENV variables, SSM param store, Vault, etc.)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

to directly answer your question, what we do is this:

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Create low-level modules (e.g. VPC, IAM, S3, etc.)
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Create infrastructure modules (e.g. EKS, ECS, RDS, Aurora), using the low-level modules
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Create a reusable catalog of module invocations (we call it root modules) that uses all other modules from the above
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. Provide configuration to the modules (usually using TF vars from files or Dockerfile, ENV vars, and SSM param store using chamber - depends on use case and whether the data are secrets or not)
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
  1. And finally, from geodesic, login to the AWS account (by assuming IAM role), all configuration gets populated from the sources described in #4, and provision infrastructure for the particular account using the root modules invocations (which, once inside the geodesic shell for the particular AWS account, already know how and where they will be provisioned since they got all the configuration)
Josh Larsen avatar
Josh Larsen

@Erik Osterman (Cloud Posse) do you have any docs or advice for upgrading to the most recent geodesic with terraform 0.12 with the purpose of upgrading to 0.12 wholly? i just noticed when i do make deps now terraform says the directory is not totally empty (before it would just ignore the envrc tfvars). also, should i be concerned that it may distort my remote state file?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Josh Larsen - we ran into this too

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s aggravating.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I can give you a temporary workaround (haven’t tested it), but I think it hsould work

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

basically, run terraform init blah and it should init the files to the blah folder

Josh Larsen avatar
Josh Larsen

ok, but that might mess with the tfstate pathing… new state file would for /account-dns might change to /blah/account-dns no?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then set export TF_DATA_DIR=$(pwd)/.terraform

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i see what you mean.

Josh Larsen avatar
Josh Larsen

guess i could copy it all up one folder after init, just clunky

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for now, I suggest overloading deps target until we have a cleaner workaround

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e..g do doing the extra copy step

Josh Larsen avatar
Josh Larsen

ok, then its safe to assume geodesic is not really fully in line with 0.12 quite yet?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s fair to say our strategy of terraform init -from-module=.... does not work as-is with 0.12

Josh Larsen avatar
Josh Larsen

ok, fair enough. we will try working around it. i do like that adding the version to .envrc changes the terraform version. nifty.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, happy with that part

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so there’ss a -force-copy arg now, but I wish it applied force to the “right” copy operation

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so all the terraform commands support specifying the path

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that path can be added to the TF_CLI* envs

2019-06-10

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:07 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:03:01 PM

Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7).

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Mar 20, 2019 11:30AM.
Add it to your calendar
zoom https://zoom.us/j/684901853
slack #office-hours (our channel)

jober avatar
jober

@Erik Osterman (Cloud Posse) quick question:

Is this https://docs.cloudposse.com/reference-architectures/cold-start/ still pretty much up to date?

jober avatar
jober

it looks like it may be out of date?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It’s mstly out of date

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

2019-06-06

mmuehlberger avatar
mmuehlberger

Hi folks, it’s been a while! I’ve got a tiny question regarding geodesic and direnv: I’d like to automate the execution of chamber to fetch a stored GitHub token and private key after assuming a role. I thought that having a .envrc file in /conf that does that would be a good idea, but it seems, that direnv is not running after assume-role. Any pointers on how to achieve that?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrmmm it should definitely operate even after assume role

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Are you running a current version of geodesic?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ohhhhhhh here’s what maybe is happening. You want it to rerun after assume role, however it runs only once

mmuehlberger avatar
mmuehlberger

Exactly!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

You would need to flush the direnv cache so it triggers again

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I forget how to do that

mmuehlberger avatar
mmuehlberger

What would be the easiest way to run a post-asssume-role command? Doesn’t need to be direnv, I would just want to execute some shell commands. Is there any way?

2019-06-05

JeroenK avatar
JeroenK

Error configuring the backend “s3”: Not a valid region: eu-north-1 I get this error while trying to create tfstate-backend Is eu-north-1 not allowed?

nutellinoit avatar
nutellinoit
backend/s3: Support New eu-north-1 Region Automatic Validation · Issue #19632 · hashicorp/terraform

Current Terraform Version terraform 0.11.10 Use-cases AWS has just publicly announced the availability of the eu-north-1 (Stockholm) region: https://aws.amazon.com/blogs/aws/now-open-aws-europe-sto

JeroenK avatar
JeroenK

Thanks the skip region validation workaround is the trick

fast_parrot1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

office hours starting now: https://zoom.us/j/684901853

2019-06-03

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:09 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:01:59 PM

Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7).

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Mar 20, 2019 11:30AM.
Add it to your calendar
zoom https://zoom.us/j/684901853
slack #office-hours (our channel)

    keyboard_arrow_up