#geodesic (2019-2)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2019-02-28

Erik Osterman avatar
Erik Osterman

for those of you in Los Angeles (#lax), I’ll be presenting on Geodesic at this meetup: https://www.meetup.com/Los-Angeles-Kubernetes-Meetup/events/258976627/

Benefits of Multi-Cloud & Evolution of a Build Process & Automate Everything attachment image

Thu, Mar 7, 2019, 6:00 PM: Hello Los Angeles Kubernauts!We are going to mix things up slightly by having not just two, but three K8s presentations!First, Kevaughn is going to discuss how Steelhouse de

:--1:1

2019-02-27

Chris avatar
Chris

howdy

Chris avatar
Chris

@Erik Osterman was asked to join the channel, unable to get the make docker/build to do anything other than error out: DOCKER not defined in docker/build make: *** [docker/build] Error 1

Chris avatar
Chris
07:36:21 PM
Chris avatar
Chris

am I doing something wrong here?

aknysh avatar
aknysh

hey @Chris

aknysh avatar
aknysh

can you run make init first and then make docker/build

Chris avatar
Chris

did

Chris avatar
Chris

make init runs fine

Chris avatar
Chris
07:38:43 PM
Chris avatar
Chris
07:40:35 PM
aknysh avatar
aknysh

can you try to use the latest geodesic https://github.com/cloudposse/geodesic/tree/0.71.3

cloudposse/geodesic

Geodesic is a cloud automation shell. It's the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Chris avatar
Chris

clone the repo, make init, make?

aknysh avatar
aknysh

do you have Docker installed ?

aknysh avatar
aknysh
cloudposse/build-harness

Collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more - cloudposse/build-harness

Chris avatar
Chris

docker desktop wasn’t running

Chris avatar
Chris

workingnow

aknysh avatar
aknysh

good

aknysh avatar
aknysh
cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

aknysh avatar
aknysh

(and the Dockerfile)

aknysh avatar
aknysh

then

aknysh avatar
aknysh

make init

aknysh avatar
aknysh

make docker/build

aknysh avatar
aknysh

make install

aknysh avatar
aknysh

[root.cloudvirga.com](http://root.cloudvirga.com)

Chris avatar
Chris

Was able to build the image, now trying to login and assume-role but getting AccessDenied: MultiFactorAuthentication failed, unable to validate MFA code. Please verify your MFA serial number is valid and associated with this user. Status code 403

Chris avatar
Chris

I’ll try a few more times

aknysh avatar
aknysh

Did you enable and configure MFA on the account for the user you are trying to login with?

2019-02-26

keen avatar

so, back to ref-arch - after the first pass through - make root make children make finalize with a defined set of children in accounts_enabled - is there a way to reuse the tooling to create a new child? (I may have asked before, but it’s been a really long last few weeks) I’ve tried a few things (adding a new entry to accounts_enabled, subing out the list to just a new acct, etc), but there’s apparently enough state retained to keep that from happening. using the root generated repo, I can make the tooling create a new account, but that doesnt generate a new repo framework…..

keen avatar

pointers to places that might lead me to a path to be able to do that are similarly welcome… (since there’s pretty much no way to know what sub accounts we’d need right now - I’ve got 3 prods and their required test/dev/stage envs on the books so far…)

Erik Osterman avatar
Erik Osterman

So you will want to fork the terraform root modules

Erik Osterman avatar
Erik Osterman

Then define all the different account types there

keen avatar

yeah saw that

Erik Osterman avatar
Erik Osterman

You can truly have any number of accounts

Erik Osterman avatar
Erik Osterman

I think best would be to do a quick call to see if I can help unblock you

Erik Osterman avatar
Erik Osterman

Sometime later this week..

keen avatar

just hoping to find a way to generate them, config them, and build the repo for each new account as it comes along (it seems like the ref-arch as it sits now is more focused on onetime use than long term generation)

Erik Osterman avatar
Erik Osterman

Yes totally correct

keen avatar

ok cool - I was hoping I was missing something. I can work with that.

Erik Osterman avatar
Erik Osterman

The current use-case solved was a one time cold start

thumbsup_all1
Erik Osterman avatar
Erik Osterman

We don’t try to maintain terraform state for the generation

Erik Osterman avatar
Erik Osterman

I think it’s doable, but the dns zone delegation or interaccount linking is complicated

keen avatar

are the discovery dns zones used for anything outside of k8s and friends?

Erik Osterman avatar
Erik Osterman

It’s used for discovery things like RDS, elastic ache, elastic search etc

keen avatar

gotcha

keen avatar

I’ll have to look a bit further at how you’re using it with rds in that case

Erik Osterman avatar
Erik Osterman

Check out our RDS module

keen avatar

will do

keen avatar

one other clarification - in the current setup, for each repo - root straight out of the ref-arch for example - there’s no converge the entire repo's state right? just cd dir init-t t plan t apply yes; cd ../dir repeat ?

keen avatar

(noting that I completely and totally understand why you’re splitting tfstate.)

keen avatar

I built a chef-solo framework around tf for similar reasons

joshmyers avatar
joshmyers

@keen no converge all.

keen avatar

thanks

Erik Osterman avatar
Erik Osterman

yes, which (as you mention) is by design. it’s very easy to manage across accounts in terraform.

Erik Osterman avatar
Erik Osterman

however, we made the conscious design decision to not use that pattern. instead, enforcing due process on how changes are introduced to environments using git workflow.

keen avatar

yep

Erik Osterman avatar
Erik Osterman

i think that the approaches aren’t mutually exclusive

Erik Osterman avatar
Erik Osterman

so for example, a certain set of accounts can be delegated out to BUs

Erik Osterman avatar
Erik Osterman

then BUs can have their own accounts that they converge

:100:1
keen avatar

would be nice to be able to have a single converge prod option (vs manual workflow steps in test, for example)

keen avatar

but that’s easy enough to achieve

keen avatar

(eventually)

Erik Osterman avatar
Erik Osterman
tmrts/boilr

boilerplate template manager that generates files or directories from template repositories - tmrts/boilr

Erik Osterman avatar
Erik Osterman

(and I know there are others like this)

joshmyers avatar
joshmyers

Not really, you have no idea what you are going to get via a plan if there are dependencies between the states. which almost all places will have.

Erik Osterman avatar
Erik Osterman

but this is a small precompiled go binary which is nice

Erik Osterman avatar
Erik Osterman

and could be an interesting notion to leverage with geodesic

keen avatar

yeah - looks like it might be slightly saner than some of the other generators I’ve worked with. (at quick glance)

2019-02-21

keen avatar
cloudposse/reference-architectures(master)%% make children
make: *** No rule to make target `dev/validate', needed by `children/validate'.  Stop.

so running into this, after a make root. looking at the makefiles, I’d tend to agree with make. which leads me to suspect there’s a missing step somewhere to generate more makefiles?

Erik Osterman avatar
Erik Osterman
Fix Make Error · Issue #10 · cloudposse/reference-architectures

what When trying to hack my way through using the reference architecture with an existing root account, I got prod and audit accounts created. When I run make root, it errors ➜ make children make[1…

Erik Osterman avatar
Erik Osterman

does this fix your issue?

Erik Osterman avatar
Erik Osterman

(sorry, we haven’t had a time to open the PR for this fix albeit very simple)

Erik Osterman avatar
Erik Osterman
05:16:36 PM
Erik Osterman avatar
Erik Osterman

this should be enough to fix it

keen avatar
 make children
tasks/Makefile.child:56: *** missing separator.  Stop.
keen avatar

\# Define a macro (`child`) for a child init & provisioner
define child


\## Validate the child configuration exists
keen avatar

that’s where I removed the = (child =)

Erik Osterman avatar
Erik Osterman

ok

Erik Osterman avatar
Erik Osterman
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

keen avatar

\# List of accounts to enable
accounts_enabled = [
  "dev",
  # "staging",
  # "prod",
  # "testing",
  # "data",
  # "corp",
  # "audit",
]
keen avatar

really odd - removing the = results in the missing sep. adding it back doesnt fit it. reverting the file does.

draistrick-mini:~/.../terraform-prototypes/cloudposse/reference-architectures(master)%% cat -ev tasks/Makefile.child |grep "define child"
define child =$
draistrick-mini:~/.../terraform-prototypes/cloudposse/reference-architectures(master)%% make clean
rm -rf //build-harness
rm -rf repos accounts .terraform *.tfstate* artifacts/*
draistrick-mini:~/.../terraform-prototypes/cloudposse/reference-architectures(master)%% make clean
tasks/Makefile.child:61: *** missing separator.  Stop.
draistrick-mini:~/.../terraform-prototypes/cloudposse/reference-architectures(master)%% cat -ev tasks/Makefile.child |grep "define child"
define child$
draistrick-mini:~/.../terraform-prototypes/cloudposse/reference-architectures(master)%% make clean
tasks/Makefile.child:61: *** missing separator.  Stop.
draistrick-mini:~/.../terraform-prototypes/cloudposse/reference-architectures(master)%% cat -ev tasks/Makefile.child |grep "define child"
define child =$
draistrick-mini:~/.../terraform-prototypes/cloudposse/reference-architectures(master)%% gco tasks/Makefile.child
draistrick-mini:~/.../terraform-prototypes/cloudposse/reference-architectures(master)%% make clean
rm -rf //build-harness
keen avatar

ah, something sublime is doing on save…

keen avatar

why dont I remember that make is tab sensitive? only change I see is tab > space

keen avatar

nope, same thing if I edit with nano. odd.

keen avatar

(and much smaller diff. aka, none)

keen avatar

oh, no, it’s reading my copy dammit

Erik Osterman avatar
Erik Osterman

haha yea, whitespace issues suck

keen avatar

ok let me reset everything and try again

keen avatar

a make root/reset would be useful. (though the world still breaks if a sub account already exists, so I also have to bump my email prefix)

Erik Osterman avatar
Erik Osterman

keep in mind, each time yuo do that, you create a new AWS subaccount

Erik Osterman avatar
Erik Osterman

subaccounts cannot be easily deleted

keen avatar

yep

keen avatar

that’s why I have a 1k account limit

Erik Osterman avatar
Erik Osterman

HAHHA

Erik Osterman avatar
Erik Osterman

wow, didn’t even know that was possible

Erik Osterman avatar
Erik Osterman

“Accounts as Cattle”

Thibault avatar
Thibault

hey everyone, is there any way to import the aws_zone53_zone.parent_dns_zone and aws_zone53_record.parent_dns_zone_soa when using reference-architectures and/or root.cloudposse.co repository directly?

Erik Osterman avatar
Erik Osterman

Not easily

Erik Osterman avatar
Erik Osterman

The reference architectures (as they stand) are basically designed to take a someone from a coldstart of zero (nothing in an account) to a full fledged architecture

Erik Osterman avatar
Erik Osterman

Basically, this was born out of our own needs to provision architectures for our clients

Thibault avatar
Thibault

fair enough, I ended up commenting out the auto-apply section of the Makefile.child and scripts/provision.sh (TF_CLI_ARGS_apply="") and -auto-approve in Makefile.root

Thibault avatar
Thibault

so it wouldn’t create automatically resources in our root account (which unfortunately already had resources in it)

Thibault avatar
Thibault

so now I’ve got geodesic containers for my different stage and i’m kind of provisioning them manually

Thibault avatar
Thibault

but i’m hitting a road block at the dns config, trying to import our parent (already existing) zone

keen avatar

yeah, it took a few weeks to get approved, but my justification of “working on automation tooling to manage creation of accounts” didnt result in any followup questions

keen avatar

at some point, I’d like to have a better idea of how to add new accounts to an existing structure - is there a sane way to do that with what exists now?

Erik Osterman avatar
Erik Osterman

yes, and no

Erik Osterman avatar
Erik Osterman

@Jan has done this and I know others as well

Erik Osterman avatar
Erik Osterman

But before doing that, best to get familiar with the overall strategy

Jan avatar

Heya

Erik Osterman avatar
Erik Osterman

If you have existing accounts, I suggest just using the base images and writing your own dockerfiles from hand.

Erik Osterman avatar
Erik Osterman

[testing.cloudposse.co](http://testing.cloudposse.co) is a good place to look for inspiration

Jan avatar

Yea I can give you loads of advice on that

Jan avatar

Can also tell you where I would like to start from if I had time

Jan avatar

Coming to grips with the reference architecture was a great way to learn

Jan avatar

But ultimately it would ha e been better to start with empty dockerfile

Jan avatar

And add

Jan avatar

Rather than adapt

Jan avatar

Once you understand the pattern it’s trivial

keen avatar

in this case, the plan is to start from scratch - but new products, teams, environments, will roll in and out all the time - so a straightforward way to provision a base new sub account is important

Jan avatar

Yep

Jan avatar

Optimize for change!

keen avatar

indeed.

Jan avatar

We have done loads of things that are custom

Jan avatar

Busy on a few cool ones

Jan avatar

We have heavy deadlines for end of April so have not been able to contribute much back yet

Jan avatar

But plan is to do so

keen avatar

in general, the overall framework is not far off from what I would build if I were scratch building again (and using tf as the basis, instead of as a component tool like I built last time.)

Jan avatar

If you have more specific sorta constraints / ideas / needs I am happy to help

keen avatar

ok, make children is running! (minus equal)

Jan avatar

For example on ours we had existing aws accounts, in an existing org where we setup our “root” in a sub aws org account

Jan avatar

With totally different network cidr schemes

Jan avatar

On premise vpn, hashicorp vault

Jan avatar

We are a gitlab / Jenkins shop

Jan avatar

In the end it’s just terraform

keen avatar

fun. did you try to import the existing accounts into the scheme, or migrate away from them? (the only thing I hate more than manually fixing tfstate is trying to craft it for existing resources)

Jan avatar

With a nice layer of convention on top

Jan avatar

They were empty accounts

Jan avatar

Just crated already

keen avatar

ah cool

keen avatar

easier there

Jan avatar

Well to an extent

Jan avatar

Honestly the virvana state would have been starting a full aws org from scratch

Jan avatar

My biggest pain so far with taking the geodesic reference architecture and adapting it has been the env names

Jan avatar

Try do a search and replace on “data” in terraform

Jan avatar

I dare you

keen avatar

hahahaha

Erik Osterman avatar
Erik Osterman

haha

Erik Osterman avatar
Erik Osterman

so for the record, the point there is you do not renmae

Erik Osterman avatar
Erik Osterman

you create additional ones

Erik Osterman avatar
Erik Osterman

since we have accounts_enabled, you can then choose the flavors you want

keen avatar

so make children failed out with auth problems - a number of similar errors scattered through the log

panic: SharedConfigAssumeRoleError: failed to load assume role for arn:aws:iam::894200800587:role/OrganizationAccountAccessRole, source profile has no shared credentials
...
The source_profile "brasstack" must specify either static credentials or an assume role configuration
....
Error: Error running plan: 1 error(s) occurred:

* provider.aws: No valid credential sources found for AWS Provider.
	Please see <https://terraform.io/docs/providers/aws/index.html> for more information on
	providing credentials for the AWS Provider
Erik Osterman avatar
Erik Osterman

did you enable the bootstrap module?

Erik Osterman avatar
Erik Osterman

the children will only be accessible via the bootstrap role

keen avatar

it’s in root.tfvars yes

keen avatar

only thing I removed was cloudtrail

keen avatar

arn:aws:iam::665716774983:role/brasstack-root-bootstrap

Erik Osterman avatar
Erik Osterman

in the artifacts/ folder you should see the temp aws config

keen avatar

yeah

keen avatar

\# Temporary configuration for AWS bootstrapping
[profile brasstack-root-admin]
region = us-west-2
role_arn = arn:aws:iam::665716774983:role/brasstack-root-bootstrap
source_profile = brasstack



\# Temporary configuration for AWS bootstrapping
[profile brasstack-dev-admin]
region = us-west-2
role_arn = arn:aws:iam::894200800587:role/OrganizationAccountAccessRole
source_profile = brasstack
Erik Osterman avatar
Erik Osterman

and [profile brasstack]

Erik Osterman avatar
Erik Osterman

?

Erik Osterman avatar
Erik Osterman

i don’t see that in that output

Erik Osterman avatar
Erik Osterman

(jumping on a call)

keen avatar

that’s in credentials with keys

Erik Osterman avatar
Erik Osterman

it should exist in the config file

keen avatar

but no `[profile brasstack]’

Erik Osterman avatar
Erik Osterman

just an empty def

keen avatar

mm, something didnt put it there.

keen avatar

oh no

keen avatar

it’s at the top

keen avatar

first line blindness

keen avatar

the full config/credentials stack looks correct - keys, iam roles, policies, between the two accounts. I can assume the role on the dev account (using a root account user not created by tf. guess I can try using the tf user)

keen avatar

yep, no issues there either.

keen avatar

failed to load assume role for arn:aws:iam::894200800587:role/OrganizationAccountAccessRole, source profile has no shared credentials seems to be from the go sdk, so I’m presuming tf

keen avatar

this seems to be the origin of the failure (and unhandled)

keen avatar
fetch <https://alpine.global.ssl.fastly.net/alpine/edge/community/x86_64/APKINDEX.tar.gz>
(1/1) Installing [email protected] (0.3.2-r1)
Executing busybox-1.28.4-r2.trigger
OK: 709 MiB in 120 packages
panic: SharedConfigAssumeRoleError: failed to load assume role for arn:aws:iam::894200800587:role/OrganizationAccountAccessRole, source profile has no shared credentials

goroutine 1 [running]:
[github.com/remind101/assume-role/vendor/github.com/aws/aws-sdk-go/aws/session.Must(0x0](http://github.com/remind101/assume-role/vendor/github.com/aws/aws-sdk-go/aws/session.Must\(0x0), 0x8e8d80, 0xc420019c90, 0x0)
	/Users/ejholmes/src/github.com/remind101/assume-role/vendor/github.com/aws/aws-sdk-go/aws/session/session.go:265 +0x54
main.assumeProfile(0x7ffd32d66db9, 0x13, 0x0, 0x0, 0x8e8340)
	/Users/ejholmes/src/github.com/remind101/assume-role/main.go:148 +0xf7
main.main()
	/Users/ejholmes/src/github.com/remind101/assume-role/main.go:77 +0x178
Handling [--apply-modules]...
Processing tfstate-backend...
keen avatar

then it keeps going until tf fails

keen avatar
Assume Role panic. · Issue #8 · cloudposse/reference-architectures

Command make children Error panic: SharedConfigAssumeRoleError: failed to load assume role for arnawsiam:role/OrganizationAccountAccessRole, source profile has no shared credentials Stat…

keen avatar
That looks like the right fix. Basically, when using the bootstrap user we need to use `AWS_SHARED_CREDENTIALS_FILE=/artifacts/.aws/credentials
keen avatar

looking much happier. I’m gonna PR that

keen avatar

well, next… Failed to save state: truncate terraform.tfstate: read-only file system

Erik Osterman avatar
Erik Osterman

that one is new! haven’t seen that

keen avatar

race condition on s3 create maybe?

module.tfstate_backend.aws_s3_bucket.default: Creation complete after 10s (ID: brasstack-dev-terraform-state)
module.tfstate_backend.aws_dynamodb_table.with_server_side_encryption: Still creating... (10s elapsed)
module.tfstate_backend.aws_dynamodb_table.with_server_side_encryption: Still creating... (20s elapsed)
module.tfstate_backend.aws_dynamodb_table.with_server_side_encryption: Still creating... (30s elapsed)
module.tfstate_backend.aws_dynamodb_table.with_server_side_encryption: Still creating... (40s elapsed)
module.tfstate_backend.aws_dynamodb_table.with_server_side_encryption: Still creating... (50s elapsed)
module.tfstate_backend.aws_dynamodb_table.with_server_side_encryption: Creation complete after 59s (ID: brasstack-dev-terraform-state-lock)
Failed to save state: truncate terraform.tfstate: read-only file system
keen avatar

though I’m speculating - actually that’s probably still saving to local state, not to remote

Erik Osterman avatar
Erik Osterman

so tfstate backend initialization is tricky

Erik Osterman avatar
Erik Osterman

it first initializes locally

Erik Osterman avatar
Erik Osterman

then imports it

Erik Osterman avatar
Erik Osterman
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

keen avatar

appearance is this was terraform apply -auto-approve -input=false not the import (first tf run in the container)

keen avatar

so, then, why would the container be read-only. hrm

loren avatar
loren

Which channel we’re we talking about test frameworks? https://opensource.com/article/19/2/testing-bash-bats

Testing Bash with BATS

The Bash Automated Testing System puts Bash code through the same types of testing processes used by Java, Ruby, and Python developers.

Erik Osterman avatar
Erik Osterman

Hrmmmmmm can’t recall.

Erik Osterman avatar
Erik Osterman

Think it recently was in #terraform

keen avatar

maybe my docker has taken a dive. removed the s3/ddb, reran…and docker error.

module.account.module.docker_build.null_resource.docker_build (local-exec): Sending build context to Docker daemon  27.65kB
module.account.module.docker_build.null_resource.docker_build (local-exec): Error response from daemon: mkdir /mnt/sda1/var/lib/docker/tmp/docker-builder348479862: input/output error

Erik Osterman avatar
Erik Osterman

yea, looks like it totally crapped out

Erik Osterman avatar
Erik Osterman

@loren are you using bats?

loren avatar
loren

i’m not no, pytest/testinfra, exploring go test some with terratest

loren avatar
loren

but i knew you were, and thought it would be a good read for other folks here…

Erik Osterman avatar
Erik Osterman

yep, thanks for the heads up

Erik Osterman avatar
Erik Osterman

looks like a good read

keen avatar

I’ve used bats before (in something I PRed. forget what. heard about it a few years before that. then later experimented with it for another project. also forget what)….it wasn’t bad. like anything, it has it’s own concepts

keen avatar

restarted docker-machine, and now children ran ok. so $chaos

Erik Osterman avatar
Erik Osterman

are you not using docker for mac?

keen avatar

nope, docker for mac panics 9 out of 10 startups (after 5-10m of complete resource sucking - the clock slows down…), the 10th just sucks all resources until I give up and kill it

keen avatar

go mohave.

2019-02-20

chrism avatar
chrism

yeah the submodules for just that

chrism avatar
chrism

think the point is that you’re supposed to configure every region (you have enabled) or someone with access via a liberally create iam key could simply spawn shit outside of the region you normally use

chrism avatar
chrism

Can you restrict regions at the SCP level? seems to all be around feature restriction

loren avatar
loren

Yes, IIRC, looks not unlike restricting regions via IAM policy…

chrism avatar
chrism
03:25:35 PM

Not sure this is baked properly in cloudposse/geodesic:0.71.0

joshmyers avatar
joshmyers

@chrism do you have anything in /localhost/.aws/config ?

chrism avatar
chrism

works fine on rel 70

Erik Osterman avatar
Erik Osterman

I can take a look as well in a few hours

joshmyers avatar
joshmyers
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

chrism avatar
chrism

yup; changing to 70 at least stopped me veering off down a rabbit hole to work out that one

joshmyers avatar
joshmyers

@chrism what does your $AWS_CONFIG_FILE look like?

chrism avatar
chrism

im assuming its something in the function choose_role_interactive() { in aws-vault but it seems sane enough (though ive no idea what the fzf bit tis)

chrism avatar
chrism
 ✓   (test-prod-admin) vpc ⨠  echo $AWS_CONFIG_FILE
/localhost/.aws//config
chrism avatar
chrism

thats running 70

chrism avatar
chrism

ill give you 71 in about a minute or so when its built again

joshmyers avatar
joshmyers

Cool, it should be the same, works for me (tm) - see https://github.com/cloudposse/geodesic/pull/376

Better support for aws-iam-authenticator by joshmyers · Pull Request #376 · cloudposse/geodesic

what This commit adds a command kopsctl login to build a kubecfg that optionally can use aws-iam-authenticator, given we have the KOPS_AWS_IAM_AUTHENTICATOR_ENABLED set to true why We want to enabl…

joshmyers avatar
joshmyers

what does your config look like is what I’m asking, is it in the same format as can be seen on that PR?

chrism avatar
chrism

✓ (ivendi-prod-admin) vpc ⨠ echo $AWS_CONFIG_FILE /localhost/.aws//config looks the same

chrism avatar
chrism

yeah its pretty much the same

chrism avatar
chrism

even though i can cat out my config

chrism avatar
chrism
05:19:54 PM
chrism avatar
chrism
 ✗   (none) ~ ⨠  ls -lsa /localhost/.aws//config4 -rwxr-xr-x 1 root root 1019 Feb  6 13:54 /localhost/.aws//config
chrism avatar
chrism

no diff if I change the path to /localhost/.aws/config rather than double slash

chrism avatar
chrism

Is this going to show a prompt everytime? because that kinda takes away from having N repos /org split

joshmyers avatar
joshmyers

Everytime you assume a role, or you can pass in a specific role as before assume-role role_name

joshmyers avatar
joshmyers

Needed a way to assume different IAM roles for k8s rbac

chrism avatar
chrism

i just typed assume-role previously and im assuming it defaults to the one in the env

joshmyers avatar
joshmyers

yeah, it did that if you didn’t give a name.

Erik Osterman avatar
Erik Osterman

so both @joshmyers and @chrism are right

Erik Osterman avatar
Erik Osterman

@joshmyers is alluding to the fact we support now multiple roles per account; not just one admin role

Erik Osterman avatar
Erik Osterman

e.g. one role to become a kubernetes admin, one role for read-only, etc

Erik Osterman avatar
Erik Osterman

however, the “one role policy” is probably going to continue to common

Erik Osterman avatar
Erik Osterman

so maybe we should still honor AWS_DEFAULT_PROFILE if it is set

Erik Osterman avatar
Erik Osterman

unset it for the selector

Erik Osterman avatar
Erik Osterman

thoughts @joshmyers?

joshmyers avatar
joshmyers

Seems fair, AWS_DEFAULT_PROFILE is still set

Erik Osterman avatar
Erik Osterman

hrm

Erik Osterman avatar
Erik Osterman

but not honored by the selector?

Erik Osterman avatar
Erik Osterman

(i haven’t looked into the code yet)

joshmyers avatar
joshmyers

AWS_DEFAULT_PROFILE is set everywhere IIRC so you will never get to selector

joshmyers avatar
joshmyers
Still unsure why you don’t have that populated @chrism - ` crudini –get “${AWS_CONFIG_FILE}” awk -F ‘ ‘ ‘{print $2}’`
Erik Osterman avatar
Erik Osterman
05:32:19 PM
Erik Osterman avatar
Erik Osterman

previously it worked like this

Erik Osterman avatar
Erik Osterman

but now, AWS_DEFAULT_PROFILE is gone

joshmyers avatar
joshmyers

yes, as I alluded to

Erik Osterman avatar
Erik Osterman

i guess what I mean, is we (cloudposse) should probably remove AWS_DEFAULT_PROFILE from our dockerfile (*.cloudposse.co)

Erik Osterman avatar
Erik Osterman

then we get the benefit we want

Erik Osterman avatar
Erik Osterman

but if AWS_DEFAULT_PROFILE is set, the original functionality persists that @chrism is using

1
chrism avatar
chrism
05:34:47 PM

crudini --get "${AWS_CONFIG_FILE}" | awk -F ' ' '{print $2}' fails with the same error but the variable is set.

chrism avatar
chrism

and the file exists

joshmyers avatar
joshmyers

hah, looks like it

chrism avatar
chrism

even giving it the direct path fails

joshmyers avatar
joshmyers

crudini --version ?

chrism avatar
chrism

0.9

joshmyers avatar
joshmyers
05:38:18 PM
chrism avatar
chrism

if i copy the file it works

Erik Osterman avatar
Erik Osterman

chrism, remind me: are you on WSL?

chrism avatar
chrism

yerp

Erik Osterman avatar
Erik Osterman

hrm… ok, maybe related

joshmyers avatar
joshmyers

ah ha

chrism avatar
chrism
05:39:00 PM
chrism avatar
chrism

file system permission based issue; i’d expect aws-vault to fail though

Erik Osterman avatar
Erik Osterman

does cat ${AWS_CONFIG_FILE} >/dev/null

Erik Osterman avatar
Erik Osterman

work?

chrism avatar
chrism

work as in do nothing yes

chrism avatar
chrism

Erik Osterman avatar
Erik Osterman

ok, no errors

chrism avatar
chrism

yeah it cats out fine

Erik Osterman avatar
Erik Osterman

i wonder if we run crudini with config from stdin instead then

joshmyers avatar
joshmyers
Default to AWS_DEFAULT_PROFILE for IAM assume-role by joshmyers · Pull Request #380 · cloudposse/geodesic

what Default to AWS_DEFAULT_PROFILE for IAM assume-role why If AWS_IAM_ROLE_INTERACTIVE is not set, we default to previous behaviour of using AWS_DEFAULT_PROFILE and not prompting via selector. Rat…

2019-02-19

chrism avatar
chrism

@Erik Osterman not sure if you’ve tripped over this; https://github.com/nozaq/terraform-aws-secure-baseline

nozaq/terraform-aws-secure-baseline

Terraform module to set up your AWS account with the secure baseline configuration based on CIS Amazon Web Services Foundations. - nozaq/terraform-aws-secure-baseline

:--1:2
chrism avatar
chrism

the alarm base is pretty similar to yours (bar you use loops, they’ve used individual entries)

Erik Osterman avatar
Erik Osterman

Will take a look. I know ours also needs some love.

chrism avatar
chrism

the layouts a bit shit in there but such is the joy of having to use N providers

loren avatar
loren

I feel like I’d prefer a module that took a single provider, and let me call it as many times as needed for each region I needed to configure

:--1:1
Erik Osterman avatar
Erik Osterman

Yea, I think this would be a better interface

loren avatar
loren

Looks alright if I use the submodules

2019-02-18

chrism avatar
chrism

In the ref architecture world is it better to pump flowlogs to cloudwatch or into the audit account S3 bucket.

loren avatar
loren
How to Facilitate Data Analysis and Fulfill Security Requirements by Using Centralized Flow Log Data | Amazon Web Services attachment image

I am an AWS Professional Services consultant, which has me working directly with AWS customers on a daily basis. One of my customers recently asked me to provide a solution to help them fulfill their security requirements by having the flow log data from VPC Flow Logs sent to a central AWS account. This is […]

chrism avatar
chrism

Ta. Need logging of my loggings logging

chrism avatar
chrism

Terraform network ACLS are crap. Nothing like having to use counts because someone decided subnet CIDR should be a single entry

2019-02-14

Erik Osterman avatar
Erik Osterman
05:19:30 AM

@Erik Osterman set the channel purpose: Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2019-02-13

daveyu avatar
daveyu

now that I’m used to chamber exec in geodesic, I’m looking for a clean way to export secrets for developers. I’d love to put something like this in an app’s .direnvrc: aws-vault exec myco-dev-admin -- chamber exec myapp -- sh -c 'export -p'. I haven’t been able to butcher my way through bash to get this to work, I think because aws-vault needs user input

Erik Osterman avatar
Erik Osterman

this looks good

Erik Osterman avatar
Erik Osterman
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman avatar
Erik Osterman

just need to add source<(...)

daveyu avatar
daveyu

When I try that, it hangs, expecting input for passphrase to unlock awsvault

daveyu avatar
daveyu

is it possible?

Erik Osterman avatar
Erik Osterman

yep, it’s possible

Erik Osterman avatar
Erik Osterman

i’m on a call

Erik Osterman avatar
Erik Osterman

we do it though

Erik Osterman avatar
Erik Osterman

also, they just added shell export compatibility

Erik Osterman avatar
Erik Osterman

not sure if a release has been cut

Erik Osterman avatar
Erik Osterman
Add env command to export secrets by jradtilbrook · Pull Request #184 · segmentio/chamber

This resolves #94 by adding a new command that will print the secrets in a format that can be eval’ed to export them as environment variables.

:--1:1

2019-02-12

Jan avatar

The easy way to automate everything

troll2
1
Erik Osterman avatar
Erik Osterman

beauty is in the eye of the beholder

2019-02-11

Abel Luck avatar
Abel Luck

the null-label module is just fantastic

joshmyers avatar
joshmyers

Mind boggling.

Abel Luck avatar
Abel Luck

who knew string concatenation could be so great

Abel Luck avatar
Abel Luck

Do you all generally create one log bucket per-thing that ships logs (e.g, an ALB) or do use one bucket to receive multiple services’ logs?

Abel Luck avatar
Abel Luck

The latter seems more user friendly, but managing multiple policies on a bucket is a PITA

joshmyers avatar
joshmyers

Genercally I try and and <S3://region-account-logs-bucket/$service>

:--1:1
Erik Osterman avatar
Erik Osterman

I think we should adopt this convention on our buckets for logs

Erik Osterman avatar
Erik Osterman

isolating buckets by region is smart.

Erik Osterman avatar
Erik Osterman

hey all!

Erik Osterman avatar
Erik Osterman

i’m giving a talk on Geodesic in the beginning of March here in Los Angeles

Erik Osterman avatar
Erik Osterman

I don’t have all the details yet.

Erik Osterman avatar
Erik Osterman

I’m putting together an abstract for the meetup.

Erik Osterman avatar
Erik Osterman

Would love to get your feedback if I’ve captured what we have with #geodesic

Erik Osterman avatar
Erik Osterman


Title: Geodesic Cloud Automation Shell
Subtitle: The easy way to automate everything
>
Geodesic is a cloud automation shell. It’s the superset of all other tools including (terraform, terragrunt, chamber, aws-vault, aws-okta, kops, gomplate, helm, helmfile, aws cli, variant, etc) that we use to automate workflows. You can think of it like a swiss army knife for creating and building consistent platforms to be shared across team environments. It easily versions staging/production/dev environments in a repeatable manner that can be followed by any team member with only a single dependency: docker. Because of this, it works with Mac OSX, Linux, and Windows 10. Learn how you can use the geodesic shell to improve your DevOps workflows! There will be a live demo with Q&A.

:--1:4
me1249 avatar
me1249

Mention Atlantis in the tool list

Jan avatar

“it’s a super set… “ rather than the?

Jan avatar

Across teams, rather than across team environments?

Jan avatar

Make is also a depedancy

Jan avatar

Otherwise cool

Erik Osterman avatar
Erik Osterman


Make is also a depedancy

Erik Osterman avatar
Erik Osterman

not technically

Erik Osterman avatar
Erik Osterman
docker run cloudposse/testing.cloudposse.com | bash 
Erik Osterman avatar
Erik Osterman

Please add any suggestions as a thread to

Erik Osterman avatar
Erik Osterman

@Jan @daveyu @Adam @joshmyers @Dombo @chrism @me1249 @loren @aknysh @Igor Rodionov

me1249 avatar
me1249

2019-02-09

Abel Luck avatar
Abel Luck

of course you do

Erik Osterman avatar
Erik Osterman

:)

2019-02-08

loren avatar
loren

took me a little bit to duplicate… the config to end up here is pretty specific

Abel Luck avatar
Abel Luck

What is the bootstrap module for exactly? Where is it used?

Abel Luck avatar
Abel Luck

the pr says “ * This is needed by the reference-architectures to provision the subaccounts because it’s not possible to assume-role using master account”

Abel Luck avatar
Abel Luck

but aren’t the accounts already provisioned by the time the bootstrap module is run?

Abel Luck avatar
Abel Luck
cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

Erik Osterman avatar
Erik Osterman

Our reference architecture assumes you start with a brand spanning new AWS account

Erik Osterman avatar
Erik Osterman

In fact that account is so new there are no existing IAM users

Erik Osterman avatar
Erik Osterman

So the role of the bootstrap module is to provision a user and role which can be used by the coldstart process to assume role into the sub accounts that get created

Erik Osterman avatar
Erik Osterman

Then at the very end we can destroy this module

Abel Luck avatar
Abel Luck

gotcha.

Abel Luck avatar
Abel Luck

i’ve been working through the cold start code by following https://docs.cloudposse.com/reference-architectures/cold-start/ as well as looking at the repos

Abel Luck avatar
Abel Luck

i’ve noticed that that doc page has diverged from the code, but i didn’t realize it was that much

Erik Osterman avatar
Erik Osterman

yes, almost night and day

Erik Osterman avatar
Erik Osterman

we just added support for “archived” pages

Erik Osterman avatar
Erik Osterman

i’m going to need to do that for a bit of our documenation until we can update it.

Abel Luck avatar
Abel Luck

Following on from my question yesterday about how users get created

Abel Luck avatar
Abel Luck

The user is added to the admin groups by the users root module, but later in the iam root module the organization-access-group module will REMOVE the user from the group, because the [prod|dev|...]_account_user_names list is empty

Erik Osterman avatar
Erik Osterman

yes, so users should get created in the [root.org.com](http://root.org.com) account

Erik Osterman avatar
Erik Osterman

and not added to root-modules directly

Erik Osterman avatar
Erik Osterman

an old pattern we had, was (1) adding users, then (2) adding users to the group list

Erik Osterman avatar
Erik Osterman

that didn’t work well

Abel Luck avatar
Abel Luck

makes sense yea

Erik Osterman avatar
Erik Osterman

the new pattern is (1) adding users and adding them to the groups at the same time

Erik Osterman avatar
Erik Osterman

so when you delete a user, they are deleted from the groups too

Erik Osterman avatar
Erik Osterman

works well with atlantis

Abel Luck avatar
Abel Luck

indeed, yes, have the user creation also add them to the group

Abel Luck avatar
Abel Luck

so the iam root module is run before the users are added

:--1:1
Erik Osterman avatar
Erik Osterman

i am not sure if our terraform examples in [root.cloudposse.co](http://root.cloudposse.co) are up to date; what i am describing is how we’re doing it with current customers

Abel Luck avatar
Abel Luck

because if i run the iam root module after adding users, it removes them

Erik Osterman avatar
Erik Osterman

i think i have seen that behavior

Erik Osterman avatar
Erik Osterman

that’s a bug we should address.

Abel Luck avatar
Abel Luck

specifically if i run iam a second time after adding users

Erik Osterman avatar
Erik Osterman

i thought it was quite weird though; usually terraform will only try to delete things it created

Erik Osterman avatar
Erik Osterman

so i was not anticipating that behavior. i am not quite sure what the fix is other than deprecating the functionality in the upstream gorup module

Abel Luck avatar
Abel Luck


WARNING: Multiple aws_iam_group_membership resources with the same group name will produce inconsistent behavior!

Abel Luck avatar
Abel Luck

not in this case it seems

joshmyers avatar
joshmyers

yeah, have seen this before

Erik Osterman avatar
Erik Osterman

yea, so our root modules are mostly organized into separate “phases”

Erik Osterman avatar
Erik Osterman

@Abel Luck btw, feel free to schedule some time with me next week: https://calendly.com/cloudposse

Abel Luck avatar
Abel Luck

It seems to me that the organization-access-group module should only create the group and apply the policy

Abel Luck avatar
Abel Luck

not actually manage the members of the group

Erik Osterman avatar
Erik Osterman

yes

Erik Osterman avatar
Erik Osterman

agreed - this is what i think we should deprecate

Abel Luck avatar
Abel Luck

that works better when users are decoupled from the group

Erik Osterman avatar
Erik Osterman

:100:

Erik Osterman avatar
Erik Osterman

(btw, if you have time to submit a PR for that, we’ll promptly review)

Erik Osterman avatar
Erik Osterman

if not, we’ll probably fix this on our next customer rollout

Abel Luck avatar
Abel Luck

i think the same applies to the iam-assumed-roles module

Erik Osterman avatar
Erik Osterman

yep - indeed

Abel Luck avatar
Abel Luck

So i’ve been piecing together my own version of the reference architecture using ya’lls great building blocks

Erik Osterman avatar
Erik Osterman

haha, yes, but i think that’s the spirit!

Abel Luck avatar
Abel Luck

Abel Luck avatar
Abel Luck

Geodesic is really cool and I totally see the problem its solving, its just I can’t introduce it into our workflow yet. So I’m back in the land of special directory layouts and copy/pasted HCL

Abel Luck avatar
Abel Luck

using docker image layers to maintain root modules is just brilliant

Erik Osterman avatar
Erik Osterman

so the root modules just helps us be more DRY

Erik Osterman avatar
Erik Osterman

definitely doesn’t preclude doing things the traditional terraform way

Abel Luck avatar
Abel Luck

Since I started with the outdated docs, I created the accounts by hand, and also created the root admin user by hand

Abel Luck avatar
Abel Luck

that’s no problem..

Abel Luck avatar
Abel Luck

Now I have the root account bootstrapping complete as far : tfstate backend bootrap, root-iam policies, account-settings and creating users

Abel Luck avatar
Abel Luck

however the switch role url for the non-root accounts isn’t working

Abel Luck avatar
Abel Luck

rather, the url works, but i get a perm denied error

Abel Luck avatar
Abel Luck

now given what I know about AWS roles, the sub-account (aka the trusting account) needs to be involved, but I haven’t involved the root user of the sub-account at all

Erik Osterman avatar
Erik Osterman

how were the child accounts created?

Abel Luck avatar
Abel Luck

i created them manually and invited them to the root acount

Abel Luck avatar
Abel Luck

maybe the invited accounts don’t have the OrganizationAccountAccessRole?

:100:1
Abel Luck avatar
Abel Luck

ding ding

Abel Luck avatar
Abel Luck
Accessing and Administering the Member Accounts in Your Organization - AWS Organizations

Access the accounts that are part of your organization in AWS Organizations. Use the root user or an AWS Identity and Access Management (IAM) role to access the resources of a member account as a user in the organization’s master account.

Erik Osterman avatar
Erik Osterman

correct

Erik Osterman avatar
Erik Osterman

they do not

Erik Osterman avatar
Erik Osterman

we do have a module you can use to provision that though

Erik Osterman avatar
Erik Osterman
cloudposse/terraform-aws-organization-access-role

Terraform module to create an IAM Role to grant permissions to delegated IAM users in the master account to access an invited member account - cloudposse/terraform-aws-organization-access-role

2019-02-07

Micah Martin avatar
Micah Martin

Is there an architectural design for how the reference architecture is setup?

joshmyers avatar
joshmyers

As in a diagram?

Micah Martin avatar
Micah Martin

@joshmyers yes

joshmyers avatar
joshmyers

Not as far as I’m aware, no.

joshmyers avatar
joshmyers

Do you have any specific questions?

loren avatar
loren

hmm, wonder what it would look like in cloudcraft… maybe sign up for a free trial, spin out the reference architecture, capture it in cloudcraft, and export a diagram?

joshmyers avatar
joshmyers

I think that would be an excellent idea

joshmyers avatar
joshmyers

Have to say I think CloudCraft diagrams look meltingly good, but actually not that easy to follow

Erik Osterman avatar
Erik Osterman

I struggled with cloudcraft diagrams where if what I wanted to add something which was not isomorphic

Erik Osterman avatar
Erik Osterman

But yes, diagrams are long overdue

loren avatar
loren

there are a couple other similar services i think, could try them all i imagine

Erik Osterman avatar
Erik Osterman

We use Lucid Charts

Erik Osterman avatar
Erik Osterman

Have a lot of diagrams that we use in our SOWs, just haven’t had the chance to open source something.

chrism avatar
chrism

FYI The export cloudcraft from aws is a tad iffy. We’re currently paying for it though, its nice to have docs that don’t look arse

Abel Luck avatar
Abel Luck

hey folks :slightly_smiling_face: I’m checking out the reference architecture setup. great work, it’s really nice. I can’t seem to locate the code where the admin IAM users are created The root-iam module takes a list of user names root_account_admin_user_names, but looking through all the repos, I can’t find where those users are actually created before being passed to root-iam

Abel Luck avatar
Abel Luck

i think the module that creates the users is cloudposse/terraform-aws-iam-user, but where is this called?

joshmyers avatar
joshmyers
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

aknysh avatar
aknysh

@Abel Luck https://github.com/cloudposse/reference-architectures is our latest approach. We changed how the users are provisioned. Before, we were adding users to root_account_admin_user_names, but this was not scalable (e.g. when using CI/CD and need to add a user, we’d have to update the user list)

cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

aknysh avatar
aknysh

so now, for each new user we create a separate file in https://github.com/cloudposse/root.cloudposse.co/blob/master/conf/users

cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

aknysh avatar
aknysh
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

aknysh avatar
aknysh
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

aknysh avatar
aknysh

@Abel Luck welcome to the community

Abel Luck avatar
Abel Luck

Ah interesting.

Abel Luck avatar
Abel Luck

that clears that up!

Abel Luck avatar
Abel Luck

where does the local.admin_groups come from?

aknysh avatar
aknysh

share the URL where you see it

aknysh avatar
aknysh

\# Fetch the OrganizationAccountAccessRole ARNs from SSM
module "admin_groups" {
  source         = "git::<https://github.com/cloudposse/terraform-aws-ssm-parameter-store?ref=tags/0.1.5>"
  parameter_read = "${formatlist("/${var.namespace}/%s/admin_group", local.accounts_enabled)}"
}
aknysh avatar
aknysh

it’s read from SSM

joshmyers avatar
joshmyers
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

joshmyers avatar
joshmyers

terraform-root-modules/aws/users gets merged with [root.cloudposse.co/aws/users](http://root.cloudposse.co/aws/users)

Abel Luck avatar
Abel Luck

gotcha

Abel Luck avatar
Abel Luck

nice use of the ssm

joshmyers avatar
joshmyers

Logic to add users is common to all stages, so goes in your terraform-root-modules but your users maybe different per stage, so we put those per stage

Abel Luck avatar
Abel Luck

thanks

joshmyers avatar
joshmyers

well, all users get added to the root account in our case, but that is the general idea behind your stage codebase (like [testing.cloudposse.co](http://testing.cloudposse.co)) and terraform-root-modules

2019-02-06

loren avatar
loren

Hmm, I don’t think terragrunt itself has any internal logic for injecting -no-color… What does your terraform.tfvars file look like?

loren avatar
loren

I’ll look at the terragrunt code when I get to a computer to confirm…

loren avatar
loren
gruntwork-io/terragrunt

Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules. - gruntwork-io/terragrunt

loren avatar
loren

huh. usually slack expands the code with those urls

loren avatar
loren

that was done relatively recently… pretty sure related to how terragrunt checks the auto-init output for a particular error message that is not an error (to auto-init) so it knows to ignore the non-zero return code

loren avatar
loren
Speed up init usage · gruntwork-io/[email protected]

When we use init to download modules, set -get=false, -get-plugins=false, and -backend=false so that all of those can be handled in the second call to init (if necessary). I tried this befo…

loren avatar
loren

curious if this works: TF_CLI_ARGS_init=-no-color terragrunt ...

loren avatar
loren

either way, i think you have enough here to report as a bug… terragrunt may need to be a little smarter about how/when it appends -no-color in auto-init

joshmyers avatar
joshmyers

There is already an issue for it

loren avatar
loren

i couldn’t find one

loren avatar
loren

there is one saying that -no-color is not being passed to init

joshmyers avatar
joshmyers
Terraform "-no-color" option not passed to terraform init · Issue #390 · gruntwork-io/terragrunt

In a command-line such as terragrunt plan –terragrunt-source-update -no-color the -no-color argument does not get passed to the terraform init call, only to the terraform plan call.

joshmyers avatar
joshmyers

yeah

loren avatar
loren

that’s kinda the opposite though

joshmyers avatar
joshmyers

ah right, hadn’t quite followed all the convo

chrism avatar
chrism

adding nano to the image is one of those little things that makes life less painful

1
vim1
Erik Osterman avatar
Erik Osterman
Add env command to export secrets by jradtilbrook · Pull Request #184 · segmentio/chamber

This resolves #94 by adding a new command that will print the secrets in a format that can be eval’ed to export them as environment variables.

Erik Osterman avatar
Erik Osterman

Chamber now supports env export suitable for eval

me1249 avatar
me1249

Nice

aknysh avatar
aknysh

@me1249 can you please rebuild README

aknysh avatar
aknysh

make init

aknysh avatar
aknysh

make readme/deps

aknysh avatar
aknysh

make readme

me1249 avatar
me1249

Done

aknysh avatar
aknysh
0.72.0 [terragrunt] upgrade to 0.17.4
me1249 avatar
me1249

Thanks

2019-02-05

me1249 avatar
me1249

Are you guys doing any authentication to private git repositories for terraform modules in geodesic? If so, how are you handling authentication?

joshmyers avatar
joshmyers

@me1249 yes, SSH keys

me1249 avatar
me1249

how are you making the SSH keys available to the docker container?

joshmyers avatar
joshmyers

When running geodesic locally your agent gets mounted in, when running via Atlantis, it is writing out an SSH key from chamber

me1249 avatar
me1249

ah

joshmyers avatar
joshmyers

Which docker container exactly?

me1249 avatar
me1249

I totally missed ATLANTIS_SSH_PRIVATE_KEY

me1249 avatar
me1249

Thanks

Erik Osterman avatar
Erik Osterman

@rohit.verma I forget, are you using GitLab?

rohit.verma avatar
rohit.verma

yes we are using gitlab

me1249 avatar
me1249

Still having issues after settings ATLANTIS_SSH_PRIVATE_KEY - it seems like the atlantis user doesn’t have access to the SSH auth socket - anyone come across this issue?

Erik Osterman avatar
Erik Osterman

hrmmmm

Erik Osterman avatar
Erik Osterman
01:17:27 AM

confesses, i only tested that the key got added and not I could use it

Erik Osterman avatar
Erik Osterman

sec

Erik Osterman avatar
Erik Osterman
cloudposse/geodesic

Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! https://slack.cloudposse.com/ - clou…

Erik Osterman avatar
Erik Osterman

yes, we should chmod the sock as well.

Erik Osterman avatar
Erik Osterman

on line 89, we gosu to the ATLANTIS_USER; prior to that we’re root

Erik Osterman avatar
Erik Osterman

@me1249 btw, have you seen the tmate-session util we added to geodesic

Erik Osterman avatar
Erik Osterman

if you install tmate in your container, you can call tmate-session and it will output a URL you can use for debugging

me1249 avatar
me1249

Yep - no luck with chmoding the socket either

me1249 avatar
me1249

even tried chowning it

me1249 avatar
me1249

Have to execute the ssh-agent under the atlantis user using gosu

Erik Osterman avatar
Erik Osterman

hrmmmm

Erik Osterman avatar
Erik Osterman

sec

Erik Osterman avatar
Erik Osterman

what did you chown it?

Erik Osterman avatar
Erik Osterman

what was the mode?

Erik Osterman avatar
Erik Osterman

past experience tells me that it’s not required to run the agent under atlantis.

Erik Osterman avatar
Erik Osterman

ohh

Erik Osterman avatar
Erik Osterman

wait

Erik Osterman avatar
Erik Osterman

yes, the pid probably needs to be same uid

me1249 avatar
me1249

So changing to source <(gosu ${ATLANTIS_USER} ssh-agent -s)

:--1:1
Erik Osterman avatar
Erik Osterman

no - that’s not true, since on Linux we bind mount in the agent sock

me1249 avatar
me1249

fixes the issue

Erik Osterman avatar
Erik Osterman

that’s a nice fix

Erik Osterman avatar
Erik Osterman

however, <(...) won’t work without a TTY

Erik Osterman avatar
Erik Osterman

have you deployed that under ECS and it works?

me1249 avatar
me1249

Just deploying it now

Erik Osterman avatar
Erik Osterman

ok, lmk

Erik Osterman avatar
Erik Osterman

ohhhh we had source <(ssh-agent -s)

Erik Osterman avatar
Erik Osterman

hrmmm odd. i had trouble with that under certain circumstances.

Erik Osterman avatar
Erik Osterman

if that doesn’t work, try eval $(gosu ${ATLANTIS_USER} ssh-agent -s)

me1249 avatar
me1249

It worked

cool-doge1
Erik Osterman avatar
Erik Osterman

woohoo!

Erik Osterman avatar
Erik Osterman

can you open a PR for that?

Erik Osterman avatar
Erik Osterman

we’ll release that right away

me1249 avatar
me1249

Sure can

Erik Osterman avatar
Erik Osterman

thanks @me1249!!

Erik Osterman avatar
Erik Osterman

merged and released as 0.69.1

me1249 avatar
me1249

No problem

me1249 avatar
me1249

Got another fix coming for you soon regarding timezones

Erik Osterman avatar
Erik Osterman

haha, cool

Erik Osterman avatar
Erik Osterman
[kopsctl] add commands to facilitate management of cluster by osterman · Pull Request #378 · cloudposse/geodesic

what Add commands to easily rotate a kops cluster&#39;s ssh keys Add command to easily connect to a kops cluster Add command to see a kops plan why This are routine operations that are complicat…

Erik Osterman avatar
Erik Osterman

here’s an example of using the #variant tool for exposing a clean cli UI

Erik Osterman avatar
Erik Osterman

much more powerful than make

me1249 avatar
me1249

will have to check it out

Erik Osterman avatar
Erik Osterman
04:15:10 AM
Erik Osterman avatar
Erik Osterman
04:15:28 AM
me1249 avatar
me1249

nice

Erik Osterman avatar
Erik Osterman

so even if not using kops, this pattern can be extended to other things

Erik Osterman avatar
Erik Osterman

like myorgctl

Erik Osterman avatar
Erik Osterman

Erik Osterman avatar
Erik Osterman

that ties together all the tools “my org” uses

me1249 avatar
me1249

In case anyone comes searching in here that’s trying to use atlantis with tfenv and terragrunt - do not set TF_CLI_DEFAULT_NO_COLOR=true

me1249 avatar
me1249

It breaks terraform init

me1249 avatar
me1249

terragrunt passes -no-color by default so it ends up passing it twice - terraform doesn’t like that

Erik Osterman avatar
Erik Osterman

I found that terragrunt inconsistently passes -no-color

loren avatar
loren

i went ahead and opened an issue for this: https://github.com/gruntwork-io/terragrunt/issues/648

Using extra_args on init to pass -no-color results in duplicate -no-color (auto-init) · Issue #648 · gruntwork-io/terragrunt

Terragrunt&#39;s auto-init is adding -no-color to its terraform command, so when extra_args is used to pass -no-color to terraform init, we end up with two -no-color args on the command. Terraform …

2
:--1:1
Erik Osterman avatar
Erik Osterman

Thanks!

Erik Osterman avatar
Erik Osterman

yes, with terragrunt that causes problems

Erik Osterman avatar
Erik Osterman

I had all kinds of problems with terragrunt and -no-color irrespective of atlantis

Erik Osterman avatar
Erik Osterman
06:16:41 AM
Erik Osterman avatar
Erik Osterman

(from our soon to be released slack archives!!)

2019-02-03

me1249 avatar
me1249

@Erik Osterman - how often do you update your APK repository?

Erik Osterman avatar
Erik Osterman

on every merge

Erik Osterman avatar
Erik Osterman

we cut a release on every merge to master

Erik Osterman avatar
Erik Osterman

in other words, if a PR is merged, it’s been deployed

Erik Osterman avatar
Erik Osterman

when installing packages, recommend doing somethin glike

Erik Osterman avatar
Erik Osterman

apk add --update [email protected]=0.11.1-r0

Erik Osterman avatar
Erik Osterman

for strict pinning, plus updating the apk cache

me1249 avatar
me1249

odd - when I’m building I’m still getting terraform 0.11.10 and i get this if I try to update

ERROR: unsatisfiable constraints: terraform-0.11.7-r0: breaks: world[terraform=0.11.11]

me1249 avatar
me1249

Running this got it to work apk add terraform –update-cache –repository https://apk.cloudposse.com/3.8/vendor –allow-untrusted

Erik Osterman avatar
Erik Osterman

hrm

Erik Osterman avatar
Erik Osterman

not good.

me1249 avatar
me1249

ah i see why i still get 0.11.10 - layer caching

Erik Osterman avatar
Erik Osterman
apk add [email protected]==0.11.11-r0
fetch <https://alpine.global.ssl.fastly.net/alpine/v3.8/main/x86_64/APKINDEX.tar.gz>
fetch <https://alpine.global.ssl.fastly.net/alpine/v3.8/community/x86_64/APKINDEX.tar.gz>
fetch <https://apk.cloudposse.com/3.8/vendor/x86_64/APKINDEX.tar.gz>
fetch <https://alpine.global.ssl.fastly.net/alpine/edge/testing/x86_64/APKINDEX.tar.gz>
fetch <https://alpine.global.ssl.fastly.net/alpine/edge/community/x86_64/APKINDEX.tar.gz>
(1/1) Upgrading [email protected] (0.11.10-r0 -> 0.11.11-r0)
Executing busybox-1.28.4-r2.trigger
OK: 702 MiB in 119 packages
-> Run 'assume-role' to login to AWS
Erik Osterman avatar
Erik Osterman

here’s what I get in geodesic

me1249 avatar
me1249

ah you need the -r0

me1249 avatar
me1249

All good now, thanks

Erik Osterman avatar
Erik Osterman

yea, I haven’t dug deeper into why the release is required

Erik Osterman avatar
Erik Osterman

we are going to do some more work on the alpine repo over the coming month (time permitting)

Erik Osterman avatar
Erik Osterman

among other things, improve the build process so that it only builds changed paths

Erik Osterman avatar
Erik Osterman

also, considering rendering the APKBUILD template and committing so it’s more obvious what’s going on. currently, the APKBUILD template wraps make

Erik Osterman avatar
Erik Osterman

Also, --allow-untrusted should not be required (we don’t use it); the packages are signed

me1249 avatar
me1249

    keyboard_arrow_up