#geodesic (2019-03)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic

Archive: https://archive.sweetops.com/geodesic/

2019-03-01

oscarsullivan_old avatar
oscarsullivan_old
07:54:48 PM

Did I set something up wrong? 1) What’s with the unescaped characters 2) How come none of my vault profiles appear? Thanks

oscarsullivan_old avatar
oscarsullivan_old
07:58:55 PM

Ah! Nvm! For some reason a character kept being placed.. backspace solved it

1

2019-03-02

oscarsullivan_old avatar
oscarsullivan_old

^ I would consider that as unexpected behaviour though. I’m not placing the - character as well as the glitchy character before assume-role on the CLI

oscarsullivan_old avatar
oscarsullivan_old

https://github.com/cloudposse/reference-architectures#1-provision-root-account make root errors due to permission errors… I’ve noticed artifacts is created under root:root.. any reason why?

cloudposse/reference-architectures

Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

oscarsullivan_old avatar
oscarsullivan_old

Ok seems to be working with some hacky solution of running make root having it fail then chmod 777 on reference-architectures dir lol

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrm… Not sure - didn’t see this when I ran it on Linux, but sounds entirely plausible that we have some permissions issue.

oscarsullivan_old avatar
oscarsullivan_old

@Erik Osterman (Cloud Posse) happened again btw exact same steps…

clone ref-arc change configs make root permission error bc artefacts under root:root

oscarsullivan_old avatar
oscarsullivan_old
Error: Error applying plan:

2 error(s) occurred:

* module.tfstate_backend.aws_s3_bucket.default: 1 error(s) occurred:

* aws_s3_bucket.default: Error creating S3 bucket: IllegalLocationConstraintException: The us-west-2 location constraint is incompatible for the region specific endpoint this request was sent to.
	status code: 400, request id: C43BDB5AF4DC7779, host id: V/wPm7gDiU5Dic9bDogXcAunrYnK5Y2l5g9StldhV17/dtjo5t4+PD5JAztIubtGZ1iNCHHeKgE=
* module.tfstate_backend.aws_dynamodb_table.with_server_side_encryption: 1 error(s) occurred:

root.tfvars:

# The default region for this account
aws_region = "eu-west-2"

Any thoughts? Did a grep of reference-architecture and no mention of us-west-2.. must be on the S3 Backend module right?

oscarsullivan_old avatar
oscarsullivan_old

^ solutions to this are in an existing issue on github :] just gotta wait for my sub-account limit to increase now…

2019-03-03

oscarsullivan_old avatar
oscarsullivan_old

When in root.tfvars of reference infra it specifics really old versions of root modules and geodesic.. any reason I shouldn’t put them both to the latest releases or does it eventually pull them down?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’ve done some refactoring of environment variables that is probably incompatible with the current version of the reference architectures

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

In our next customer engagement that involves a cold-start we’ll clean this up.

oscarsullivan_old avatar
oscarsullivan_old

Awesome thanks.

oscarsullivan_old avatar
oscarsullivan_old

If I use reference architectures now am I committing to some old setup? Could I have run it again to update it? Would I want to? I still haven’t established to full reach and effect of it. Based on the breakdown of each step it shouldn’t matter want version of geodesic is used for setup. I do wonder how to update my geodesic though for a stage?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

not that old. I think we have a call tomorrow - I can fill you in. The main change that happened after the ref arch is the introduction of tfenv

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

also, a move towards using direnv to define local project settings rather than using globals

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Interface-wise this should be stable now for a while. We hadn’t “solved” how to manage envs in a scaleable fashion until recently

2019-03-04

oscarsullivan_old avatar
oscarsullivan_old
02:02:10 PM

.. if they’re not created already with reference-architectures

oscarsullivan_old avatar
oscarsullivan_old

How do you guys ‘hotswap’ backends in geodesic env?

oscarsullivan_old avatar
oscarsullivan_old

I feel like it has something to do with direnv but I don’t really get it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Are you referring to S3 backends?

oscarsullivan_old avatar
oscarsullivan_old

Yes!

oscarsullivan_old avatar
oscarsullivan_old

Let’s say I have an api project with s3 as the backend. This project exists on dev staging prod. Since we don’t share buckets and buckets are uniquely named and you can’t interpolate in the backend config, how do you not duplicate code?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

aha yes

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so this is solved with environment variables

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we use direnv to define them for terraform consumption

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so in a project, you’ll define a .envrc

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and in the Dockerfile you’ll define some globals

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/testing.cloudposse.co

Example Terraform Reference Architecture that implements a Geodesic Module for an Automated Testing Organization in AWS - cloudposse/testing.cloudposse.co

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Here’s a sample .envrc

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

You’ll see, this is very DRY

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We specify an ENV for the remote module, which will be normalized with tfenv to a “terraform compatible” env

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

use terraform + use tfenv setups up the TF_CLI_ARGS_init

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

using the the cloudposse direnv stdlib is totally optional.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@rohit.verma for example, decided not to use our helpers and just defines his TF_CLI_ARGS_init variables explicitly.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It's the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

note, the use terraform helper will do this:

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
export TF_BUCKET_PREFIX=${TF_BUCKET_PREFIX:-$(basename $(pwd))}
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is so that each of your project folders gets it’s own state folder inside the state bucket

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

by default it uses the current folder name

2019-03-06

oscarsullivan_old avatar
oscarsullivan_old
cloudposse/prod.cloudposse.co

Example Terraform/Kubernetes Reference Infrastructure for Cloud Posse Production Organization in AWS - cloudposse/prod.cloudposse.co

loren avatar

Long as it’s consistent throughout a project, life is good! Thank you editorconfig!

oscarsullivan_old avatar
oscarsullivan_old

I’ve changed to it to spaces in mine

loren avatar

If it’s your project, that’s your prerogative

loren avatar

I like to lint for editorconfig violations in CI, for those few editors that don’t honor its settings by default, https://github.com/jedmao/eclint

jedmao/eclint

Validate or fix code that doesn’t adhere to EditorConfig settings or infer settings from existing code. - jedmao/eclint

1
oscarsullivan_old avatar
oscarsullivan_old

Thanks I’ll bookmark that.. don’t yet have a linter in my CI

oscarsullivan_old avatar
oscarsullivan_old

@Erik Osterman (Cloud Posse) what steps should one take to change the template used for make readme? The cloudposse template (wherever that is located) adds all the logos etc to the top of my README. Would like to be able to change it so it corresponds to my team.. thoughts?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

certainly…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/build-harness

Collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more - cloudposse/build-harness

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here are the ENVs you can play with

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

README_TEMPLATE_FILE is what you want

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s the template

oscarsullivan_old avatar
oscarsullivan_old

Thanks! That looks like I’d therefore have to fork build harness entirely? Or is there a super neat way this can be a geodesic conf setting

oscarsullivan_old avatar
oscarsullivan_old

@Erik Osterman (Cloud Posse).. as promised..

@everyoneelse https://github.com/osulli/geodesic-getting-started hope this helps those starting Geodesic.

osulli/geodesic-getting-started

A getting-started guide for Cloud Posse’s Geodesic. - osulli/geodesic-getting-started

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@oscarsullivan_old this is phenomenal! thanks so much for putting this together. hearing it from your hands-on perspective is invaluable.

osulli/geodesic-getting-started

A getting-started guide for Cloud Posse’s Geodesic. - osulli/geodesic-getting-started

chrism avatar

What OS was https://github.com/cloudposse/github-authorized-keys tested on (im assuming in aws)?

cloudposse/github-authorized-keys

Use GitHub teams to manage system user accounts and authorized_keys - cloudposse/github-authorized-keys

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It was tested on quite a few Linux distos

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

However the problem is usually that useradd varies by distro so you always need to update the template

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It defaults to alpine

chrism avatar

I was running it on ub18 in docker (just following the readme tbh) Not wholly sure why it failed tbh, said it couldnt talk to github which seems self explanatory; I was just wondering if the pathing / commands its expecting to execute on the host are correct

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

can you share the error @chrism

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

devil is in the details

chrism avatar

yep 1 mo

chrism avatar

the instructions seem wonky as well; --expose is an override of EXPOSE in docker but the docs show it like port -p 301:301

chrism avatar
{"level":"info","msg":"Run syncUsers job on start","time":"2019-03-06T17:06:42Z"}
{"job":"syncUsers","level":"error","msg":"Connection to github.com failed","subsystem":"jobs","time":"2019-03-06T17:07:12Z"}
{"level":"info","msg":"Run ssh integration job on start","time":"2019-03-06T17:07:12Z"}
{"job":"sshIntegrate","level":"info","msg":"Ensure file /usr/bin/github-authorized-keys","subsystem":"jobs","time":"2019-03-06T17:07:12Z"}
{"job":"sshIntegrate","level":"info","msg":"Ensure exec mode for file /usr/bin/github-authorized-keys","subsystem":"jobs","time":"2019-03-06T17:07:12Z"}
{"job":"sshIntegrate","level":"info","msg":"Ensure AuthorizedKeysCommand line in sshd_config","subsystem":"jobs","time":"2019-03-06T17:07:12Z"}
{"job":"sshIntegrate","level":"info","msg":"Ensure AuthorizedKeysCommandUser line in sshd_config","subsystem":"jobs","time":"2019-03-06T17:07:12Z"}
{"job":"sshIntegrate","level":"info","msg":"Restart ssh","subsystem":"jobs","time":"2019-03-06T17:07:12Z"}
{"job":"sshIntegrate","level":"info","msg":"Output: ","subsystem":"jobs","time":"2019-03-06T17:07:12Z"}
{"job":"sshIntegrate","level":"error","msg":"Error: fork/exec /usr/sbin/service: no such file or directory","subsystem":"jobs","time":"2019-03-06T17:07:12Z"}
{"level":"info","msg":"Start jobs scheduler","time":"2019-03-06T17:07:12Z"}
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /user/:name/authorized_keys --> github.com/cloudposse/github-authorized-keys/server.Run.func1 (3 handlers)
[GIN-debug] Listening and serving HTTP on :301
chrism avatar
docker run   -v /:/host   --expose "301"   -p 127.0.0.1:301:301  -e GITHUB_API_TOKEN=x   -e GITHUB_ORGANIZATION=x
  -e GITHUB_TEAM=ops   -e SYNC_USERS_INTERVAL=200   -e LISTEN=:301   -e INTEGRATE_SSH=true        cloudposse/github-authorized-keys
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


{“job”<i class=”em em-“sshIntegrate”,”level””></i>“error”,”msg”<i class=”em em-“Error”></i> fork/exec /usr/sbin/service: no such file or directory”,”subsystem”<i class=”em em-“jobs”,”time””></i>“2019-03-06T1712Z”}

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is assuming a systemd setup

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sounds like you don’t have systemd

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

try INTEGRATE_SSH=false

chrism avatar

Nah still failed

{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: GithubOrganization - ******","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: GithubTeamName - D*******s","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: GithubTeamID - *","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: EtcdEndpoints - []","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: EtcdPrefix - /github-authorized-keys","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: EtcdTTL - 24h0m0s seconds","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: UserGID - ","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: UserGroups - []","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: UserShell - /bin/bash","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: Root - /","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: Interval - 200 seconds","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: IntegrateWithSSH - false","time":"2019-03-07T13:25:01Z"}
{"class":"RootCmd","level":"info","method":"RunE","msg":"Config: Listen - :301","time":"2019-03-07T13:25:01Z"}
{"level":"info","msg":"Run syncUsers job on start","time":"2019-03-07T13:25:01Z"}
{"job":"syncUsers","level":"error","msg":"Connection to github.com failed","subsystem":"jobs","time":"2019-03-07T13:25:06Z"}
{"level":"info","msg":"Start jobs scheduler","time":"2019-03-07T13:25:06Z"}
[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:   export GIN_MODE=release
 - using code:  gin.SetMode(gin.ReleaseMode)

[GIN-debug] GET    /user/:name/authorized_keys --> github.com/cloudposse/github-authorized-keys/server.Run.func1 (3 handlers)
[GIN-debug] Listening and serving HTTP on :301
chrism avatar

bleh docker networking issue

chrism avatar
{"level":"info","msg":"Run syncUsers job on start","time":"2019-03-07T13:39:56Z"}
adduser: unrecognized option: disabled-password
BusyBox v1.25.1 (2016-10-26 16:15:20 GMT) multi-call binary.

Usage: adduser [OPTIONS] USER [GROUP]

Create new user, or add USER to GROUP

        -h DIR          Home directory
        -g GECOS        GECOS field
        -s SHELL        Login shell
        -G GRP          Add user to existing group
        -S              Create a system user
        -D              Don't assign a password
        -H              Don't create home directory
        -u UID          User id
        -k SKEL         Skeleton directory (/etc/skel)

{"job":"syncUsers","level":"error","msg":"exit status 1","subsystem":"jobs","time":"2019-03-07T13:39:57Z"}
adduser: unrecognized option: disabled-password

lol

chrism avatar

The INTEGRATE_SSH works as expected but as you’d expect it shits a brick. Meh. Kinda wish this was an apt package or cron job lol

chrism avatar

Need to jiggle the linux env vars

chrism avatar

think ive got it now even if its only added 1 of 3 users Getting there

chrism avatar

Got to the point where its added a user to match my github name, but no keys and no errors I’d missed SYNC_USERS_ROOT=/host

chrism avatar

woot got it

chrism avatar

Now back to crying into my packer script.

chrism avatar

Started looking at getting bastion running along side it. This would probably work better as a docker compose; or after much alcohol.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

have you seen the cloud formation?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

someone else submitted that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you want to share your working configuration we can add it to examples maybe

chrism avatar

not noticed the cloudformation. Tbh I was tripping up over a) not reading further down the damn page (rtfm fail) And not not realising team names had to be lowercase (org names are case sensitive) (which is githubs fault)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I want to incorporate this in our docs.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
06:27:42 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

even explaining this is a big win as a way to show how the abstraction works.

oscarsullivan_old avatar
oscarsullivan_old

No prob

oscarsullivan_old avatar
oscarsullivan_old

Sure is useful

oscarsullivan_old avatar
oscarsullivan_old

I was wondering actually if you need to specify the key and encrypt variables or if they’re done automatically..

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Ideally if you open that as a PR we can review it and leave comments

2019-03-07

2019-03-08

oscarsullivan_old avatar
oscarsullivan_old

Guys any thoughts on the following:

oscarsullivan_old avatar
oscarsullivan_old
#terraform.tf
terraform {
 backend "s3" {}
}
#variables.tf
variable "stage" {}

variable "namespace" {}

variable "aws_region" {}

variable "tf_bucket_region" {}

variable "tf_bucket" {}

variable "tf_dynamodb_table" {}

variable "TF_VAR_tf_bucket_region" {}

variable "TF_VAR_tf_bucket" {}

variable "TF_VAR_tf_dynamodb_table" {}

#Dockerfile
# Terraform vars
ENV TF_VAR_region="${AWS_REGION}"
ENV TF_VAR_account_id="${AWS_ACCOUNT}"
ENV TF_VAR_namespace="${NAMESPACE}"
ENV TF_VAR_stage="${STAGE}"
ENV TF_VAR_domain_name="${DOMAIN_NAME}"
ENV TF_VAR_zone_name="${DOMAIN_NAME}"

# chamber KMS config
ENV CHAMBER_KMS_KEY_ALIAS="alias/${TF_VAR_namespace}-${TF_VAR_stage}-chamber"

# Terraform State Bucket
ENV TF_BUCKET_REGION="${AWS_REGION}"
ENV TF_BUCKET="${TF_VAR_namespace}-${TF_VAR_stage}-terraform-state"
ENV TF_DYNAMODB_TABLE="${TF_VAR_namespace}-${TF_VAR_stage}-terraform-state-lock"

CLI:

 ✓   (healthera-sandbox-admin) backend ⨠ terraform init
Initializing modules...
- module.terraform_state_backend
- module.terraform_state_backend.base_label
- module.terraform_state_backend.s3_bucket_label
- module.terraform_state_backend.dynamodb_table_label

Initializing the backend...
bucket
  The name of the S3 bucket

Any thoughts on why it would be asking for the bucket name? AKA -backend-config="bucket=my-state-bucket"

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Something must be wrong with the environment variables

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you pm me your TF_CLI envs I can evaluate

oscarsullivan_old avatar
oscarsullivan_old

Ah just seen this. 2 secs will send over.

oscarsullivan_old avatar
oscarsullivan_old

# Terraform vars
ENV TF_VAR_region="${AWS_REGION}"
ENV TF_VAR_account_id="${AWS_ACCOUNT}"
ENV TF_VAR_namespace="${NAMESPACE}"
ENV TF_VAR_stage="${STAGE}"
ENV TF_VAR_domain_name="${DOMAIN_NAME}"
ENV TF_VAR_zone_name="${DOMAIN_NAME}"

# chamber KMS config
ENV CHAMBER_KMS_KEY_ALIAS="alias/${TF_VAR_namespace}-${TF_VAR_stage}-chamber"

# Terraform State Bucket
ENV TF_BUCKET_REGION="${AWS_REGION}"
ENV TF_BUCKET="${TF_VAR_namespace}-${TF_VAR_stage}-terraform-state"
ENV TF_DYNAMODB_TABLE="${TF_VAR_namespace}-${TF_VAR_stage}-terraform-state-lock"

That’s my dockefifle for tf stuff

oscarsullivan_old avatar
oscarsullivan_old

If I so an env | grep -i bucket I can see them

Alex Siegman avatar
Alex Siegman

is the domain just for display, or do we actually modify resources on the TF_VAR_domain_name?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So this is an older dockerfile. Having the tf envs in the dockerfile led to sprawl. e.g. domain_name defined as a global does not make sense. It was used by one module. This is why we moved to direnv.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Do you have “use terraform” and “use tfenv” in your .envrc

oscarsullivan_old avatar
oscarsullivan_old

Yeh but I have no backend .envrc

oscarsullivan_old avatar
oscarsullivan_old

I think I tried it but it changes nothing

oscarsullivan_old avatar
oscarsullivan_old

Is there a project with backend interpolation working along with newest dockefifle?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It will definitely not work to call terraform init if you are not setting up the environment with tfenv

oscarsullivan_old avatar
oscarsullivan_old

Would be fab If you get a chance to send me bash history or stdinout of files you edit to tfenv the backend variables and then how they’re used

oscarsullivan_old avatar
oscarsullivan_old

There s just too much guess work. Need to see how someone does it

oscarsullivan_old avatar
oscarsullivan_old

What’s in their conf/backend/.envrc And what do they run to In It project What’s in their terraform backend block and variables file

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/root.cloudposse.co

Example Terraform Reference Architecture for Geodesic Module Parent (“Root” or “Identity”) Organization in AWS. - cloudposse/root.cloudposse.co

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This works

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Have a demo of this at a meetup lastnight

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Please work backwards from terraform init

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Understand how to pass environment variables to terraform init

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Then try setting those explicitly without using our env mapping

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Tfenv is what we use to map envs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/tfenv

Transform environment variables for use with Terraform (e.g. HOSTNAMETF_VAR_hostname) - cloudposse/tfenv

oscarsullivan_old avatar
oscarsullivan_old

Thanks.. I feel like I’m getting closer and following what you’re saying but

oscarsullivan_old avatar
oscarsullivan_old
export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=${TF_BUCKET}
export TF_CLI_INIT_BACKEND_CONFIG_REGION=${TF_BUCKET_REGION}
export TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=${TF_DYNAMODB_TABLE}

use tfenv
source <(tfenv)

oscarsullivan_old avatar
oscarsullivan_old
terraform init

tfenv
terraform init

tfenv terraform init
oscarsullivan_old avatar
oscarsullivan_old

none of those work

oscarsullivan_old avatar
oscarsullivan_old


Understand how to pass environment variables to terraform init
This is exactly what I’m trying to figure out.

At the moment I’m manually running terraform init -backend-config="bucket=${TF_BUCKET}" -backend-config="region=${TF_BUCKET_REGION}" -backend-config="dynamodb_table=${TF_DYNAMODB_TABLE}"

oscarsullivan_old avatar
oscarsullivan_old

Also tried this as my .envrc and rebuild:

# Terraform State Bucket
export BUCKET_REGION="${AWS_REGION}"
export BUCKET="${VAR_namespace}-${VAR_stage}-terraform-state"
export DYNAMODB_TABLE="${VAR_namespace}-${VAR_stage}-terraform-state-lock"

#export CLI_INIT_BACKEND_CONFIG_REGION=${BUCKET_REGION}
#export CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=${DYNAMODB_TABLE}


###

export TF_CLI_INIT_FROM_MODULE=git::<https://github.com/cloudposse/terraform-root-modules.git//aws/tfstate-backend?ref=tags/0.53.4>
export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=${BUCKET}
source <(tfenv)
terraform init
oscarsullivan_old avatar
oscarsullivan_old

as per readme for tfenv

oscarsullivan_old avatar
oscarsullivan_old

Man this makes me so frustrated! Every readme is just 2 lines away from being comprehensible. Never is there a final usage example

oscarsullivan_old avatar
oscarsullivan_old

“Here’s what it looks like top to bottom”

oscarsullivan_old avatar
oscarsullivan_old

oscarsullivan_old avatar
oscarsullivan_old
   31  export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=terraform-state-bucket
   32  tfenv
   33  tfenv terraform init

So this works.. but now how do I get my .envrc to automatically be doing this

oscarsullivan_old avatar
oscarsullivan_old

` 28 direnv allow /conf/backend/.envrc` I tried this however but that didn’t do anything

oscarsullivan_old avatar
oscarsullivan_old

hang on think Im onto something

oscarsullivan_old avatar
oscarsullivan_old
# Terraform State Bucket
export BUCKET_REGION="${AWS_REGION}"
export BUCKET="${VAR_namespace}-${VAR_stage}-terraform-state"
export DYNAMODB_TABLE="${VAR_namespace}-${VAR_stage}-terraform-state-lock"

#export CLI_INIT_BACKEND_CONFIG_REGION=${BUCKET_REGION}
#export CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=${DYNAMODB_TABLE}


###

export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=${BUCKET}
source <(tfenv)
use tfenv

Nope after a rebuild none of the following work

    2  terraform init
    3  tfenv terraform init
    4  tfenv
oscarsullivan_old avatar
oscarsullivan_old

It can’t be due to interpolation in envrc surely

oscarsullivan_old avatar
oscarsullivan_old

Man I’m fuming at this point. This is so easy to doc

oscarsullivan_old avatar
oscarsullivan_old

1) Example .envrc 2) Example using .envrc

oscarsullivan_old avatar
oscarsullivan_old

Wait what I have to GO INTO the conf dir to activate these variables?

 ✓   (healthera-sandbox-admin) ~ ⨠ cd /conf/backend/
direnv: loading .envrc
direnv: using terraform
direnv: using atlantis
direnv: using tfenv
direnv: export +BUCKET +BUCKET_REGION +DYNAMODB_TABLE +TF_BUCKET_PREFIX +TF_CLI_ARGS_init +TF_CLI_INIT_BACKEND_CONFIG_BUCKET +TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE +TF_CLI_INIT_BACKEND_CONFIG_KEY +TF_CLI_INIT_BACKEND_CONFIG_REGION +TF_STATE_FILE +TF_VAR_bucket +TF_VAR_bucket_region +TF_VAR_direnv_diff +TF_VAR_direnv_watches +TF_VAR_dynamodb_table +TF_VAR_oldpwd +TF_VAR_tf_bucket_prefix +TF_VAR_tf_cli_args_init +TF_VAR_tf_state_file ~TF_VAR_pwd ~TF_VAR_shlvl
oscarsullivan_old avatar
oscarsullivan_old

No idea. Feel like I’ve tried every permutation that could possibly be inferred from the README

oscarsullivan_old avatar
oscarsullivan_old

wow I think I got it this time

oscarsullivan_old avatar
oscarsullivan_old

holy shit I did

oscarsullivan_old avatar
oscarsullivan_old

.envrc goes into the terraform module

oscarsullivan_old avatar
oscarsullivan_old

you cd into the terraform module dir

oscarsullivan_old avatar
oscarsullivan_old

you type direnv allow if it prompts

oscarsullivan_old avatar
oscarsullivan_old

and bam

oscarsullivan_old avatar
oscarsullivan_old

that was stupidly hard

oscarsullivan_old avatar
oscarsullivan_old

amazing

oscarsullivan_old avatar
oscarsullivan_old

even better

oscarsullivan_old avatar
oscarsullivan_old

you don’t even need it in the directory of the project

oscarsullivan_old avatar
oscarsullivan_old

can be in the root terraform dir

oscarsullivan_old avatar
oscarsullivan_old

i.e. devops/terraform/providers/aws/.envrc instead of devops/terraform/provdiers/aws/vpc/.envrc

oscarsullivan_old avatar
oscarsullivan_old

OK when you’re next around, since I get it now, we should cover best practices so I can then doc this, please.

oscarsullivan_old avatar
oscarsullivan_old

And also is encrypt required to be defined?


terraform {
 backend "s3" {
 encrypt = true
 }
}
 ✓   (healthera-sandbox-admin) aws ⨠ direnv allow
direnv: loading .envrc
direnv: using terraform
direnv: using atlantis
direnv: using tfenv
direnv: export +BUCKET +BUCKET_REGION +CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE +CLI_INIT_BACKEND_CONFIG_REGION +DYNAMODB_TABLE +TF_BUCKET_PREFIX +TF_CLI_ARGS_init +TF_CLI_INIT_BACKEND_CONFIG_BUCKET +TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE +TF_CLI_INIT_BACKEND_CONFIG_KEY +TF_CLI_INIT_BACKEND_CONFIG_REGION +TF_STATE_FILE +TF_VAR_bucket +TF_VAR_bucket_region +TF_VAR_cli_init_backend_config_dynamodb_table +TF_VAR_cli_init_backend_config_region +TF_VAR_direnv_diff +TF_VAR_direnv_watches +TF_VAR_dynamodb_table +TF_VAR_oldpwd +TF_VAR_tf_bucket_prefix +TF_VAR_tf_cli_args_init +TF_VAR_tf_state_file ~TF_VAR_pwd ~TF_VAR_shlvl

Can’t see it there

oscarsullivan_old avatar
oscarsullivan_old

Ok so:

Current setup:

terraform/aws/.envrc

# Terraform State Bucket
export BUCKET="${NAMESPACE}-${STAGE}-terraform-state"
export BUCKET_REGION="${AWS_REGION}"
export DYNAMODB_TABLE="${NAMESPACE}-${STAGE}-terraform-state-lock"

export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=${BUCKET}
export TF_CLI_INIT_BACKEND_CONFIG_REGION=${BUCKET_REGION}
export TF_CLI_INIT_BACKEND_CONFIG_DYNAMODB_TABLE=${DYNAMODB_TABLE}

use terraform
use atlantis
use tfenv

terraform/aws/vpc/.envrc

# Terraform State Bucket
export BUCKET_KEY="backend"
#export BUCKET_REGION="${AWS_REGION}"x

# Terraform init bucket settings
#export TF_CLI_INIT_BACKEND_CONFIG_KEY=${BUCKET_KEY}

use terraform
use atlantis
use tfenv

Note the commented out BUCKET_REGION…

Two commands:

BUCKET_REGION commented out:

 ✓   (healthera-sandbox-admin) backend ⨠ terraform plan
var.bucket_region
  Enter a value: 

BUCKET_REGION not commented out:
TF works as expected

#####

The question:
How do I manage regions with .envrc… its as though I can only have one .envrc at a time. This suggests I should define the region in the Dockerfile as a global, however that would mean I need a geodesic module per region. Ideally I have a ‘prod’ account and use the regions inside it.
Like wise for state Key where should I set this.

This is all in the context of one bucket per AWS account and each AWS account has infra running on multiple data centers…

oscarsullivan_old avatar
oscarsullivan_old

The ultimate goal is so that I only have one .envrc for all my terraform projects (DRY; not one per TF project) and eventually TFENV spits out something like

export TF_VAR_tf_cli_args_init='-backend-config=region=eu-west-2 -backend-config=dynamodb_table=acme-sandbox-terraform-state-lock -backend-config=bucket=acme-sandbox-terraform-state -backend-config=key=backend/terraform.tfstate'

oscarsullivan_old avatar
oscarsullivan_old

although I think key needs to have backend/eu-west-2/terraform.tfstate tbf

oscarsullivan_old avatar
oscarsullivan_old

Hang on I think I’ve got it…

oscarsullivan_old avatar
oscarsullivan_old

Nope

oscarsullivan_old avatar
oscarsullivan_old

Thought maybe having the main .envrc in the tf root and then another .envrc in the indiivudla project with source <(tfenv) inside would allow two

oscarsullivan_old avatar
oscarsullivan_old

Yeh really feel like to deploy a TF project across regions you’d need to have another geodesic module..

oscarsullivan_old avatar
oscarsullivan_old

According to direnv it should be possible to load two .envrcs with source_env .. or source_up but those commands aren’t found

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That’s odd. The source_env works for us as we used it this week. Something is wrong with the direnv integration in your shell.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Did you upgrade geodesic? The earlier version in the ref arch perhaps did not support direnv

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We rolled out a lot of enhancements during January that did not percolate through to docs and ref arch due to a very aggressive 2 week sprint for a customer.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I suggest to try updating the geodesic base image to the latest one

oscarsullivan_old avatar
oscarsullivan_old

Thanks @Erik Osterman (Cloud Posse) Will confirm whether I am on latest geodesic vase and root module base

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The fact that source env is not working I think is a good hint to why you are having a lot of grief :-)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

To deploy across regions what you want to do is use a directory approach in conf to support that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Something like conf/us-west-2/backing-services

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Then in the us-west-2 folder set the AWS_DEFAULT_REGION to us-west-2 in the envrc

oscarsullivan_old avatar
oscarsullivan_old
FROM cloudposse/terraform-root-modules:0.53.0 as terraform-root-modules

FROM cloudposse/helmfiles:0.19.1 as helmfiles

FROM cloudposse/geodesic:0.72.2

Damn

oscarsullivan_old avatar
oscarsullivan_old

Already on latest

oscarsullivan_old avatar
oscarsullivan_old

Haha Erik I’m such a muppet. source up is a thing to go into .envrc not a CLI command

1
oscarsullivan_old avatar
oscarsullivan_old

wasn’t clear to me ¯_(ツ)_/¯

oscarsullivan_old avatar
oscarsullivan_old

aws/.envrc

# DRY variables - not changed per project

# Terraform State Bucket
export BUCKET="${NAMESPACE}-${STAGE}-terraform-state"
export BUCKET_REGION="${AWS_REGION}"
export DYNAMODB_TABLE="${NAMESPACE}-${STAGE}-terraform-state-lock"

aws/backend/.envrc

source_env ..

# Terraform State Bucket
export BUCKET_KEY="backend"

# Terraform init bucket settings
export TF_CLI_INIT_BACKEND_CONFIG_KEY=${BUCKET_KEY}

use terraform
use atlantis
use tfenv
oscarsullivan_old avatar
oscarsullivan_old

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha, yea, glad you got to the bottom of it. definitely worth reading up on https://direnv.net the full capabilities of direnv

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

looks like you’re getting it though …

oscarsullivan_old avatar
oscarsullivan_old

https://archive.sweetops.com/geodesic/#b3fc9758-0fad-4afe-a80b-2873fdef4907 so close here last time @Erik Osterman (Cloud Posse) lols

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hey @oscarsullivan_old sorry been so focused on the conference. Back to life as normal next week.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If I find some time today will take a look.

oscarsullivan_old avatar
oscarsullivan_old

Thanks back to trying reference-architectures on another account but that’s erroring a few times lool

oscarsullivan_old avatar
oscarsullivan_old

in a rush to get it working this weeekend before I lose hold of the company credit card

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

will reach out this afternoon

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

in the middle of a meeting atm

Alex Siegman avatar
Alex Siegman

@Erik Osterman (Cloud Posse) I think I found the docs you’re referring to here: https://github.com/osulli/geodesic-getting-started If those are they, I’ll read through this and ping back if I have any issues

osulli/geodesic-getting-started

A getting-started guide for Cloud Posse’s Geodesic. - osulli/geodesic-getting-started

oscarsullivan_old avatar
oscarsullivan_old

Let me know if you spot errors I’m gonna tidy it tomorrow. Noticed a few issues. Also the backend example is wrong

oscarsullivan_old avatar
oscarsullivan_old

Need @Erik Osterman (Cloud Posse) input on how to get it to stop prompting me for a bucket name. For now I’m manually setting the bucket via CLI art but ideally that’s part of geodesic.. which I know it is just for some reason I can’t get it to work.

Alex Siegman avatar
Alex Siegman

Well, I’m working through making a root account, which it looks like you might have skipped?

oscarsullivan_old avatar
oscarsullivan_old

Reference architecture was a real ball ache for mw

oscarsullivan_old avatar
oscarsullivan_old

I did not get it working

oscarsullivan_old avatar
oscarsullivan_old

Tried again this afternoon and faced a few errors and ditched it againnafyer a few hours

Alex Siegman avatar
Alex Siegman

Ah, yeah, my company is at the point we want to build out something similar to the ref architecture, so I was just going to spin it up quick and see how these tools handle it, and maybe just use them

oscarsullivan_old avatar
oscarsullivan_old

Also I don’t like using stuff I don’t understand and therefore can’t debug

1
oscarsullivan_old avatar
oscarsullivan_old

Sometimes if automation goes wrong you can manually do a step then continue say

oscarsullivan_old avatar
oscarsullivan_old

But with ref arch I am flying blind

Alex Siegman avatar
Alex Siegman

So, we also have an existing account I could use as our root account, and that might make the most sense, but I wanted to test this on a blank canvas to be sure I didn’t trash anything there.

oscarsullivan_old avatar
oscarsullivan_old

Don’t run it on your current account

oscarsullivan_old avatar
oscarsullivan_old

No one knows what would happen lol

oscarsullivan_old avatar
oscarsullivan_old

That’s not been done before

Alex Siegman avatar
Alex Siegman

Exactly. I’m just thinking forward, because eventually once I get these k8s clusters built inside the proper account arch, I’m going to have to deal with moving data

Alex Siegman avatar
Alex Siegman

and that’s not going to be a small project

oscarsullivan_old avatar
oscarsullivan_old

So I had two eoutwd

oscarsullivan_old avatar
oscarsullivan_old

Routes

oscarsullivan_old avatar
oscarsullivan_old

Use existing root account and manually create sub accounts their VPCs, do I am roles and vpc peering

oscarsullivan_old avatar
oscarsullivan_old

Or new root account and ref arch

oscarsullivan_old avatar
oscarsullivan_old

Well new root accounts need an AWS sub account limit increase and that took my 9 days to get it

oscarsullivan_old avatar
oscarsullivan_old

Furthermore every time I’ve run it I get some sort of errors

oscarsullivan_old avatar
oscarsullivan_old

Several were my fault this time tbf

oscarsullivan_old avatar
oscarsullivan_old

But it’s not very uh

oscarsullivan_old avatar
oscarsullivan_old

Idempotent

oscarsullivan_old avatar
oscarsullivan_old

And if a bootstrap tool isn’t idempotent it’s stressful when it errors midway

Alex Siegman avatar
Alex Siegman

Yeah, that’s my concern also

Alex Siegman avatar
Alex Siegman

But I just started looking a couple hours ago, not really a fair shake :)

oscarsullivan_old avatar
oscarsullivan_old

If you follow all 3 guides on my gitjub you’ll have

oscarsullivan_old avatar
oscarsullivan_old

Your existing root account

oscarsullivan_old avatar
oscarsullivan_old

Aws SSO for console access

oscarsullivan_old avatar
oscarsullivan_old

Sub accounts with aws organisations

oscarsullivan_old avatar
oscarsullivan_old

And a geodesic module for your sandbox account

oscarsullivan_old avatar
oscarsullivan_old

This weekend I’m working on VPC peering

2019-03-09

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@oscarsullivan_old are you around?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I can help jump on a zoom quickly and see if I can get you unstuck

oscarsullivan_old avatar
oscarsullivan_old

No sorry Erik not for the rest of the night!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ah bummer

oscarsullivan_old avatar
oscarsullivan_old

Sorry thank you though

2019-03-11

oscarsullivan_old avatar
oscarsullivan_old

Where in conf/ or rootfs/ should I put bash aliases? Getting a bit dull typing out my full path to my terraform projects

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

stick those in rootfs/etc/profile.d/mycompanyname.sh

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or some filename like that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

profile.d is loaded by the shell when you start up

raehik avatar

Hey all, question that’s been on my mind. AFAIK Geodesic sets remote state S3 key according to the directory basename/leaf directory

raehik avatar

has the possibility of using the whole directory come up? e.g. (/conf/)cluster1/{vpc,kops}, (/conf/)cluster2/{vpc,kops}

oscarsullivan_old avatar
oscarsullivan_old


AFAIK Geodesic sets remote state S3 key according to the directory basename/leaf directory
I’ve had to make changes for this ability.. it wasn’t doing it automatically for me

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It would be nice to have to generate the path w.r.t. /conf

oscarsullivan_old avatar
oscarsullivan_old

aws/.envrc

# DRY variables - not changed per project

# Terraform State Bucket
export BUCKET="${NAMESPACE}-${STAGE}-terraform-state"
export BUCKET_REGION="${AWS_REGION}"
export DYNAMODB_TABLE="${NAMESPACE}-${STAGE}-terraform-state-lock"

aws/backend/.envrc

source_env ..

# Terraform State Bucket
export BUCKET_KEY="backend"

# Terraform init bucket settings
export TF_CLI_INIT_BACKEND_CONFIG_KEY=${BUCKET_KEY}

use terraform
use atlantis
use tfenv
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we generate it here

Thibault avatar
Thibault

since switching to .envrc files, I’m having the same issue

raehik avatar

@Erik Osterman (Cloud Posse) yeah it’s a really easy change, just curious if it came up

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just lack of time

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thibault avatar
Thibault

aka the bucket prefix doesn’t seem to be respected

raehik avatar

rc.d/terraform :

pwd_tmp=$(pwd)
export TF_BUCKET_PREFIX=${TF_BUCKET_PREFIX:-${pwd_tmp:1}}
raehik avatar

(or so. actually we need to remove /conf, oops)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
export TF_BUCKET_PREFIX=${TF_BUCKET_PREFIX:-$(pwd | cut -d/ -f2-)}
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this will strip off the first part (/conf)

raehik avatar

are you sure it’s not -f3-

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you’re totally right

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i had a brainfart

raehik avatar

that works perfect on my Geodesic and local

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

state folders should be relate to conf

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. ordinarily, we would have /conf/vpc, so the state bucket folder should be vpc

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or /conf/us-west-2/vpc should be us-west-2/vpc

raehik avatar

right, would be interested in having it as an option

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

where s3://$BUCKET/$PREFIX/terraform.tfstate

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@raehik if you want to open a PR for that change we’ll promptly review

https://github.com/cloudposse/geodesic/blob/master/rootfs/etc/direnv/rc.d/terraform#L16

raehik avatar

it’s a possible breaking change, right? so how would I make it an option, check for ${TF_USE_FULL_PWD}?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yea, it’s possibly breaking, though I suspect anyone who was using subfolders would have already encountered this problem.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We can add a flag. I’d default it to the new format.

oscarsullivan_old avatar
oscarsullivan_old

Is this for the s3 key?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

maybe we have TF_USE_CWD vs TF_USE_PWD

oscarsullivan_old avatar
oscarsullivan_old

It would be nice if it were a new var

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

CWD = current working directory

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Why a new var?

raehik avatar

PWD = present working directory?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh

raehik avatar

*print working directory, my b

oscarsullivan_old avatar
oscarsullivan_old

because BUCKET prefix is something else

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so for clarification, we’re talking about defaulting TF_BUCKET_PREFIX

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

to something more accurate

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok, how about this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

TF_BUCKET_PREFIX_FORMAT

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

and we can have one format be pwd and other format be basename or basename-pwd or something like that?

raehik avatar

that makes sense

raehik avatar

more descriptive than just a flag

oscarsullivan_old avatar
oscarsullivan_old

Sorry was getting confused. On another similar note, the key for the s3 key isn’t set automatically I don’t think. Can anyone else confirm or perhaps just my setup

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s where it’s set

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

export TF_CLI_INIT_BACKEND_CONFIG_KEY="${TF_BUCKET_PREFIX}/${TF_STATE_FILE}"

oscarsullivan_old avatar
oscarsullivan_old

thanka

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@oscarsullivan_old and @raehik this is fixed now for your guys right? https://github.com/cloudposse/reference-architectures/issues/13

changing root.tfvars us-west-2 causes failure · Issue #13 · cloudposse/reference-architectures

during a first-run of R-A, I changed in root.tfvars aws_region = &quot;us-west-2&quot; to aws_region = &quot;us-east-2&quot; during make root this generated an error - leading me to suspect there&#…

oscarsullivan_old avatar
oscarsullivan_old

In theory Yeh! Not tried it

Josh Larsen avatar
Josh Larsen

so, are [root.cloudposse.co](http://root.cloudposse.co) and [test.cloudposse.co](http://test.cloudposse.co) repos just examples or outdated in favor of the reference-arcitechtures repo?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the *.[cloudposse.co](http://cloudposse.co) are examples of how we use geodesic; these are what we use for presentations and demos

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the reference-architectures is a first stab at automating that process but makes a very strong assumption: start with a virgin AWS root account

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

what we implement for our customers tends to be a little bit ahead of what we have in *.[cloudposse.co](http://cloudposse.co) just b/c we don’t have the time to keep them updated

Josh Larsen avatar
Josh Larsen

so for a cold start you would recommend reference-architecture over using the examples?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it somewhat depends on your existing level of experience with AWS, docker, multi-account architectures, terraform… and also if you’re going to use k8s.

Josh Larsen avatar
Josh Larsen

ok assuming a good understanding and experience in all of those + planning to use k8s

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Then I suggest kicking the tires for reference architectures. Before you start, have a look at the open issues so you can be prepared of common issues.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I can also give you a review over zoom

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the first step is to request an increase in the number of AWS accounts

Josh Larsen avatar
Josh Larsen

yup

Josh Larsen avatar
Josh Larsen

ok thanks, i’ll hit your calendar

1
oscarsullivan_old avatar
oscarsullivan_old

Expect 7 days for increase @Josh Larsen I sat around for 1.5 business weeks waiting for the account increase lols

2019-03-12

Jan avatar

hey hey

Jan avatar

how is terraform module dependancy / execution order dealt with in geodesic ?

Jan avatar

having /conf/tfstate-backend /conf/vpc /conf/foo /conf/bar (other than knowing that tfstate needs to be fiorst then vpc then others)

Jan avatar

thinking about it more from a pipeline point of view

chrism avatar
yamamoto-febc/terraform-provider-rke

Terraform provider plugin for deploy kubernetes cluster by RKE(Rancher Kubernetes Engine) - yamamoto-febc/terraform-provider-rke

oscarsullivan_old avatar
oscarsullivan_old

How come Packer isn’t included in geodesic??

oscarsullivan_old avatar
oscarsullivan_old
---

- hosts: localhost
  become: yes
  pre_tasks:
    - name: Check if running Ubuntu
      fail: msg="DevOps Workstation can only be run on Ubuntu."
      when: ansible_distribution != "Ubuntu"
    - name: Update apt cache
      become: yes
      apt:
        cache_valid_time: 600
        update_cache: yes
  vars_files: 
    - vars/devops-workstation/requirements.yml
    - vars/devops-workstation/settings.yml
  roles:
    - fubarhouse.golang
    - geerlingguy.docker
  tasks:
    - name: Install pip3 requirements
      pip:
        chdir: vars/devops-workstation/
        executable: pip3
        requirements: requirements.txt
        extra_args: --no-cache-dir
    - name: Set symlink to code directory
      file:
        src: "{{ sd }}"
        dest: /devops
        owner: root
        group: root
        state: link
    - name: Install Terraform
      unarchive:
        src: <https://releases.hashicorp.com/terraform/{{> terraform }}/terraform_{{ terraform }}_linux_amd64.zip
        dest: /usr/local/bin
        remote_src: yes
        mode: 775
        owner: root
        group: root
    - name: Install Packer
      unarchive:
        src: <https://releases.hashicorp.com/packer/{{> packer }}/packer_{{ packer }}_linux_amd64.zip
        dest: /usr/local/bin
        remote_src: yes
        mode: 775
        owner: root
        group: root
    - name: Set symlink for Go
      file:
        src: /usr/local/go/bin/go
        dest: /usr/local/bin/go
        mode: 775
        owner: root
        group: root
        state: link
    - name: Set symlink for Scenery
      file:
        src: /home/{{ user }}/go/bin/scenery
        dest: /usr/local/bin/scenery
        mode: 775
        owner: root
        group: root
        state: link
    - name: Set symlink for Dep
      file:
        src: /home/{{ user }}/go/bin/dep
        dest: /usr/local/bin/dep
        mode: 775
        owner: root
        group: root
        state: link
    - name: Install Terragrunt
      get_url:
        url: <https://github.com/gruntwork-io/terragrunt/releases/download/v{{> terragrunt }}/terragrunt_linux_amd64
        dest: /usr/local/bin/terragrunt
        mode: 775
        owner: root
        group: root
    - name: Install aws-vault
      get_url:
        url: <https://github.com/99designs/aws-vault/releases/download/v{{> aws_vault }}/aws-vault-linux-amd64
        dest: /usr/local/bin/aws-vault
        mode: 775
        owner: root
        group: root
    - name: Run dep to install Terratest
      shell: cd {{ GOPATH }} && dep init && dep ensure -add github.com/gruntwork-io/terratest/modules/terraform
oscarsullivan_old avatar
oscarsullivan_old

should use something like the playbook I use to setup my local

oscarsullivan_old avatar
oscarsullivan_old

Would be fab if we can get it included. Just checked out repo and looks like you host your own terraform image thats imported into Alpine so not sure to what extend I can do this for you and PR.

oscarsullivan_old avatar
oscarsullivan_old

Also pip is INCREDIBLY behind so not a viable source, which would have been an easy PR.

oscarsullivan_old avatar
oscarsullivan_old

but also my solution for now is running on my local outside of geodesic with: aws-vault exec healthera-mgmt-iac bash manage.sh build base.json

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
04:17:03 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

works for me

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/packages

Cloud Posse installer and distribution of native apps, binaries and alpine packages - cloudposse/packages

mmuehlberger avatar
mmuehlberger

@Erik Osterman (Cloud Posse) Regarding terraform-root-modules#132 (chamber dependencies): I’ve forked the repo already to adapt it to my needs, I just wanted to point it out. Do you want me to continue to document issues like this for future reference or is that just clutter for you?

oscarsullivan_old avatar
oscarsullivan_old


f it’s out of date, submit PR here:
Sorry I meant pip’s version is out of date.

oscarsullivan_old avatar
oscarsullivan_old

do you include packer in the base Geodesic image though? I couldn’t see it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we cannot ship everything in the base image b/c it gets toooooooo big

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s why we have the cloudposse/packages distribution

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

just run: RUN apk add --update packer@cloudposse to your Dockerfile

oscarsullivan_old avatar
oscarsullivan_old

thaaaaanks

oscarsullivan_old avatar
oscarsullivan_old

perfect

Josh Larsen avatar
Josh Larsen

@oscarsullivan_old oh you are correct… i got stonewalled by tech support saying my account was too new to increase that limit. i guess this is a new policy now. he wouldn’t even give me a solid timeframe at all… just said the account has to be around for a “few weeks” before they will up the limit.

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

WOW

oscarsullivan_old avatar
oscarsullivan_old

Wow, didn’t encounter that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that is lame.

oscarsullivan_old avatar
oscarsullivan_old

that’s amazing

oscarsullivan_old avatar
oscarsullivan_old

…-ly lame yes

Josh Larsen avatar
Josh Larsen

he said it was a new policy

oscarsullivan_old avatar
oscarsullivan_old

haha wow just did mine last week

Thibault avatar
Thibault

could be due to the amount of people trying Cloudposse’s ref arch?

mmuehlberger avatar
mmuehlberger

Don’t they have their own thing Launchpad, since they acknowledge, that multi-account setups are kinda best practice nowadays?

Thibault avatar
Thibault

¯_(ツ)_/¯

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha

mmuehlberger avatar
mmuehlberger

So basically, you need to have some spare accounts lying around to “be prepared”.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha man, there’s going to be a black market now for “aged” AWS accounts

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

like there is for domains, email addresses, instagram accounts, facebook accounts, etc

oscarsullivan_old avatar
oscarsullivan_old

Well I have instructions on how to do it maunally

oscarsullivan_old avatar
oscarsullivan_old

just, shit.. that’s all

oscarsullivan_old avatar
oscarsullivan_old

I mean its not

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

or people like @Dombo who I think requested account limits of 1000

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oscarsullivan_old avatar
oscarsullivan_old

like I’ve said before I like knowing what’s happening… when I use refarchitecture no idea what is happening REALLY and when it errors its a ballache to diagnose

Thibault avatar
Thibault

i’ve followed the ref arch, and continued manually when encountered errors

Thibault avatar
Thibault

got there in the end

oscarsullivan_old avatar
oscarsullivan_old

I hit a billing error or something

oscarsullivan_old avatar
oscarsullivan_old

Idk it just got a bit frustrating

Josh Larsen avatar
Josh Larsen

@oscarsullivan_old are you refering to just the setup piece or to geodesic in general

oscarsullivan_old avatar
oscarsullivan_old

all

oscarsullivan_old avatar
oscarsullivan_old

aws setup and geodesic seutp

oscarsullivan_old avatar
oscarsullivan_old

I am the doc master

oscarsullivan_old avatar
oscarsullivan_old
osulli - Overview

DevOps Engineer, passionate teacher, investigative philomath. - osulli

oscarsullivan_old avatar
oscarsullivan_old

see aws* and geodesic-getting-started

oscarsullivan_old avatar
oscarsullivan_old

Can help as been super busy at work so a few bits not quite up to scrap.. that’s its weakness as well as not being automated

Josh Larsen avatar
Josh Larsen

thanks @oscarsullivan_old for that getting started doc… it helps a lot just in understanding how all the env vars are utilized now. very helpful.

1
Josh Larsen avatar
Josh Larsen

though i am curious on your reasoning to put the vpn and in general have a management account separate from the root.

oscarsullivan_old avatar
oscarsullivan_old

In the same way I don’t install nginx as root or use sudo to solve my problems

oscarsullivan_old avatar
oscarsullivan_old

Root is root and isn’t meant for anything but being the root. Applies to permissions and accounts

Josh Larsen avatar
Josh Larsen
05:08:20 PM

here is the reply from tech support.. you might add this into the cold start docs.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @Josh Larsen! will open an issue incase it helps others

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
AWS Account Limits · Issue #18 · cloudposse/reference-architectures

We’re receiving reports from community members that requesting account limits is taking longer. For fresh AWS root accounts, it’s even more delayed.

Alex Siegman avatar
Alex Siegman

Is there a way to easily “unmake” the root account bootstrapping to test from a fresh place on the reference architectures? I feel like I should be able to do a terraform destroy in here somewhere and accomplish that, but not seeing built in make targets to make it easy

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so, it depends.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

basically, the problem is AWS makes it programatically impossible to destroy accounts created using the API without first logging in and accepting T&Cs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

this is a big reason we haven’t yet tried to tackle automated e2e testing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so since it means we’d need to start from point X, it’s not clear where point X is

Alex Siegman avatar
Alex Siegman

Well, with the dynamodb problems AWS had this afternoon, the bootstrapping on my root account broke

Alex Siegman avatar
Alex Siegman

So I was just hoping I could “terraform destroy” the right way

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

unfortunately no make target that implements it right now

Alex Siegman avatar
Alex Siegman

I can fix the state mismatch by hand

Alex Siegman avatar
Alex Siegman

Was just curious

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but we’ve been talking more about this internally

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ultimately, we want it to be as easy as docker-compose up

Alex Siegman avatar
Alex Siegman

It’d be nice if we could clear out an account with a terraform destroy, even if it doesn’t kill the account itself, just for testing purposes.

oscarsullivan_old avatar
oscarsullivan_old

@Erik Osterman (Cloud Posse) @Alex Siegman AWS purge tool

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ya, thought this is where it gets complicated

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

b/c one of the steps is to create the accounts

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so now to bring it back up it needs to skip that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

tf is not good at skipping

Alex Siegman avatar
Alex Siegman

tf certainly is not. Where is the state stored for the account that is brought up? in the root account? or in the subaccount?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

not if 99% of everything is done using local-exec

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’re using terraform to setup the reference architectures, but perhaps it was not the ideal tool for the job. we’re basically using terraform to generate terraform code.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then executing that terraform code inside of docker.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

because we’re using local-exec a lot to call docker, there’s no easy way to recover state. terraform cannot compute an accurate plan.

loren avatar

There’s also cloudnuke… Don’t need to use terraform if you really want to just destroy everything

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hahaha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

too true

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

loren avatar
gruntwork-io/cloud-nuke

A tool for cleaning up your cloud accounts by nuking (deleting) all resources within it - gruntwork-io/cloud-nuke

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

~too bad it doesn’t delete ECS too~

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

a number of other tools though

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
rebuy-de/aws-nuke

Nuke a whole AWS account and delete all its resources. - rebuy-de/aws-nuke

1
loren avatar

Yeah, it doesn’t do everything, for sure, but if the idea is right, and the design is right, open a PR

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

~though that’s the thing; they have a PR for that, but won’t merge it b/c it won’t selectively nuke~i class=”em em-smiley”></i>

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Nuke ECS services and ECS clusters by yorinasub17 · Pull Request #36 · gruntwork-io/cloud-nuke

First attempt at addressing #32 This implements nuking of: ECS tasks (indirectly, by draining ECS services) ECS services ECS clusters This does NOT implement nuking of: ECS task definitions Targ…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh, maybe this is out of date now

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

nm

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

haha, it was @sarkis who opened the issue

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
ECS resources are not nuked · Issue #32 · gruntwork-io/cloud-nuke

Resources like ECSCluster, Tasks, Services should all be nuked.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @loren this was a great suggestion. i think we can work with something like this.

1
loren avatar

Yeah, they merged it, but removed some functionality due to limitations they couldn’t figure out at the time

2019-03-13

chrism avatar

Anyone used terraform for helm + k8 without wondering why they f’king bothered and didn’t just use the 5 or 6 lines of shell they usually used to set helm up

chrism avatar

some of this stuffs like self-flagellation

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@chrism we have been using Helmfile

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I am curious to learn what problems you have run into using helm with terraform.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(Also in our experience it’s never been 5-6 lines! So many configuration options.)

chrism avatar

I have it wired up to deploy a cluster in a private network via RKE; so that bits fine; there’s a terraform plugin which does most of what’s needed. I scripted out the terraform to setup helm to deploy rancher to it. But its flaky as hell

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

RKE?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Rancher?

chrism avatar

All the “workarounds” to setting up tiller basically end in people posting commands to execute manually. And for extra fun because all this junks then in the tf state file when you destroy it locks up trying to undo helm which doesnt work very well

chrism avatar

and yeah 6 or 7 lined is a little off 12 is more accurate

chrism avatar
chrism
04:27:20 PM
chrism avatar

Ultimately though we’re using rancher, then we launch clusters from rancher (cant script that yet)

chrism avatar

I’ve a growing hatred for acls and security groups

chrism avatar

There should be a training mode where you just run shit for a while and they spit out by tag which group needs to talk to what on which ports

chrism avatar

TF should have just gone with helm file and avoided the mass of extra types it has to support and the usual code rot / delayed releases

chrism avatar

You know you’re starting to lose the plot when you have helpers called fuckingssh to ssh via a bastion but making sure the stupid keys written out by terraform are chmod 400 because too many shits were given

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Have you seen our kopsctl cli using variant? It handles SSH using key stored in SSM

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Haha

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@mumoshu once proposed a Helmfile provider for terraform

chrism avatar

just seems more logical for something that doesn’t “undo” very well

chrism avatar

running helm reset only works half the time and when it does the bugger hangs around thinking about it

chrism avatar

I hate helm

chrism avatar

I get the idea

chrism avatar

But the implementation makes me want to throw shit

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yea, the current implementation leaves a lot to be desired

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

apparently helm 3 is available for tire kicking

chrism avatar

Rancher switching to helm from rke-addons was also a total pita

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I would like to see your rancher setup

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you’re using the rancher cli with geodesic?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If so, we should probably add a package for it

chrism avatar

Sorry totally missed that (bloody threading notifications are just a white dot on the channel) Yeah I’ve customised my image to include a few extra tools and add a few non-repository terraform plugins

chrism avatar

RKE being KOPS by Rancher basically

chrism avatar

I’ve set folders in my conf for regions which I’m pulling our internal modules into (using make file + env files to override the region on build etal)

chrism avatar
RUN apk add nano && \
    apk add gnupg

RUN wget <https://github.com/mozilla/sops/releases/download/3.2.0/sops-3.2.0.linux> -O /usr/bin/sops
RUN mkdir -p ~/.terraform.d/plugins/ && wget <https://github.com/yamamoto-febc/terraform-provider-rke/releases/download/0.9.0/terraform-provider-rke_0.9.0_linux-amd64.zip> -O ~/.terraform.d/plugins/trke.zip
RUN unzip ~/.terraform.d/plugins/trke.zip -d ~/.terraform.d/plugins/ && rm  ~/.terraform.d/plugins/*.zip -f


RUN wget <https://github.com/stefansundin/terraform-provider-ssh/releases/download/v0.0.2/terraform-provider-ssh_v0.0.2_linux_amd64.zip> -O ~/.terraform.d/plugins/ssh.zip
RUN unzip ~/.terraform.d/plugins/ssh.zip -d ~/.terraform.d/plugins/ && rm  ~/.terraform.d/plugins/*.zip -f

RUN wget <https://github.com/kubernauts/tk8/releases/download/v0.6.0/tk8-linux-amd64> -O /usr/bin/tk8
RUN wget <https://github.com/rancher/rke/releases/download/v0.1.17/rke_linux-amd64> -O /usr/bin/rke

These are the main things i pull in that arent in the default image

chrism avatar

I’m using SSM in much the same way you are for passing around extra things. With the rancher stuff I was using local files to maintain what tiny portion of my sanity remains

chrism avatar

The ssh terraform module handles the ssh tunnelling for the stuff that follows after it (local_exec) The ticket for making connection{} handle that in terraform as you’d expect is still open/under discussion etal but this works fine, its blunt but fine.

chrism avatar
mumoshu/variant

Write modern CLIs in YAML. Bash + Workflows + Dataflows + Dependency Injection, JSON Schema for inputs validation - mumoshu/variant

chrism avatar

Looks quote nice even if the idea of more yaml in my life isn’t hugely appealing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I like the simple DSL

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it’s the “glue” for combining tools and presenting a common interface that your team can use

mumoshu avatar
mumoshu

@chrism i’ve been looking for a migration path from yaml+bash to something more maintainable, as an additional feature to variant

e.g. bash + yaml –> (yaml OR starlarkOR tengo OR Lua) + (bash OR starlark OR tengo OR Lua)

https://github.com/d5/tengo https://github.com/google/starlark-go

d5/tengo

A fast script language for Go. Contribute to d5/tengo development by creating an account on GitHub.

google/starlark-go

Starlark in Go: the Starlark configuration language, implemented in Go - google/starlark-go

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

wow, neat!

mumoshu avatar
mumoshu

or maybe just export yaml+bash as golanga sources s such the interop with other golang code becomes easier..

chrism avatar

they took a process that reliably worked, and swapped it for one that can fail magically

chrism avatar

and added 2 more pages of instructions to the setup

chrism avatar

the Rancher K2 stuff looked interesting (or was it k3) if for nothing more than them reducing the amount of shit required

chrism avatar

(something I liked about nomad)

chrism avatar

Quickly shoved a aws_security_group in my config; now to spend 20 minutes changing to separate entries because terraform thinks it needs to delete it to update

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I am thinking of maybe setting up a weekly recurring “office hours” that would be free for anyone to join

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

it would be specifically to help those using geodesic and our modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thinking 11am PST on Wednesdays.

oscarsullivan_old avatar
oscarsullivan_old

Sounds great. Would love to both take part in a Q and an A perspective. Is 11:30am-12pm PST feasible? Allows me to get home and unpack after work before joining

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Yes that works

2019-03-14

chrism avatar

anyone got a TLDR price comparison of running a cluster in EKS vs running it manually with partially reserved nodes.

chrism avatar

Saw an article by cloudhealth but you know when you really cba reading pages of shit just for a THIS COSTS MORE

mmuehlberger avatar
mmuehlberger

I don’t have a cost comparison, but it basically replaces your master nodes for a $144 per month fee.

mmuehlberger avatar
mmuehlberger

Too expensive for small clusters, but better when sizing up and desiring HA.

1
mmuehlberger avatar
mmuehlberger

Update on AWS account limit increase: They approved an account I created last Thursday for 10 sub-accounts. (The account is completely independent, but same company had another account previously though, which shared the same billing address, not sure if that matters).

chrism avatar

Fun fact; when you setup rancher you have to give it a hostname for the cluster to use. Fair enough you might think. But if you’re setting it up on a private network with an internal lb and planning to expose it from the public subnet via nginx, IF you use a real domain name all the k8 clusters you make HAVE to be able to talk to that dns entry.

There’s no x-host override to pass down (so say you could set rancher to use myinternallb.local and nginx could have mycluster.domain.com)

chrism avatar

If you set nginx to pass the lb name as host header down because of the way they’ve written the damn UI it fails

chrism avatar

the ui loads

chrism avatar

but all the calls are fully resolved by URL so it tries calling into nginx with the local lb name

chrism avatar

Other things have a similar issue like identity server; but they had the foresight to allow you to pass additional headers in.

2019-03-18

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
12:09:06 AM

added an integration to this channel: Google Calendar for Team Events

 avatar
12:12:19 AM
fast_parrot2
1
1

2019-03-19

Alex Siegman avatar
Alex Siegman

Curious, does the calendar figure out the right time zone to show me based on my settings in slack? 1:30 PM with no TZ info is basically a number without units, useless! (Pet peeve, I work from 3 timezones in the US plus Japan regularly, so I have to deal with TZ translation all the time)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks I will update the title to include the TZ

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

11:30am PST

oscarsullivan_old avatar
oscarsullivan_old

I correctly get 6:30 PM

oscarsullivan_old avatar
oscarsullivan_old

v cool

oscarsullivan_old avatar
oscarsullivan_old

I’ve just opened a PR to my getting started guide with an example project showing Geodesic and Terraform interactions (A project I actually use, though anonymised so I’ve not tested it with that anonymisation). Updated the docs as well for clarity and accuracy. https://github.com/osulli/geodesic-getting-started/pull/1

Update getting-started guide by osulli · Pull Request #1 · osulli/geodesic-getting-started

What Update the guides with clearer examples Add Example project that I actually use with Geodesic and Terraform Why Several more weeks worth of experience using the tools Some clear errors in t…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Thanks @oscarsullivan_old! Will review that

oscarsullivan_old avatar
oscarsullivan_old

The example I had given before on how to use .envrc was totally wrong and its been bothering me for over a week that I might be leading others the wrong way!

oscarsullivan_old avatar
oscarsullivan_old

I need to get me some of that make readme magic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@tamsky has also been working on some .envrc docs

oscarsullivan_old avatar
oscarsullivan_old

https://github.com/osulli/geodesic-getting-started/pull/1/files#r267123117 tis an interesting point you raised. Left my thoughts on it… I still don’t have a solution

oscarsullivan_old avatar
oscarsullivan_old

Been thinking about it over the last few days

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you can make it on the call on wednesday, we can review the multi-region discussion

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

should be quite straight forward actually

oscarsullivan_old avatar
oscarsullivan_old

Sure can. Would be happy to answer any Qs as well from you or others on how I recently setup Geodesic

oscarsullivan_old avatar
oscarsullivan_old

6:30 PM London time, tomorrow

2019-03-20

joshmyers avatar
joshmyers

multi region is interesting, where does your state live?

joshmyers avatar
joshmyers

hope ya’ll are namespacing things by region!

SweetOps #geodesic avatar
SweetOps #geodesic
06:00:01 PM
Alex Siegman avatar
Alex Siegman

Dangit, I meant to show up to that so I could listen in. I need to add these to my calendar!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Alex Siegman we’ll have another one next week

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

same time

1

2019-03-21

oscarsullivan_old avatar
oscarsullivan_old

How are you abstracting lists to .envrcs?

export AVAILABILITY_ZONES=["eu-west-2a", "eu-west-2b", "eu-west-2c"]
export AVAILABILITY_ZONES="eu-west-2a", "eu-west-2b", "eu-west-2c"
mmuehlberger avatar
mmuehlberger

Still want to give you an answer for the future: export AVAILABILITY_ZONES='["eu-west-2a", "eu-west-2b", "eu-west-2c"]' should work.

mmuehlberger avatar
mmuehlberger

Ah, @loren answered already, my bad.

1
oscarsullivan_old avatar
oscarsullivan_old

@mmuehlberger ended up coming back to this for elsewhere. How do you then use this? ${var.availability_zones” doesn’t do the tirck

oscarsullivan_old avatar
oscarsullivan_old

Oh hold on. I’m doing this ["${var.readonly}"]

mmuehlberger avatar
mmuehlberger

"${var.readonly}" should be fine.

mmuehlberger avatar
mmuehlberger

Wait, I confused myself already.

oscarsullivan_old avatar
oscarsullivan_old

Haha

oscarsullivan_old avatar
oscarsullivan_old
export AVAILABILITY_ZONES='["eu-west-2a", "eu-west-2b", "eu-west-2c"]'
variable "readonly" {}
  principals_readonly_access = "${var.readonly}"
oscarsullivan_old avatar
oscarsullivan_old
* module.ecr.var.principals_readonly_access: variable principals_readonly_access in module ecr should be type list, got string
mmuehlberger avatar
mmuehlberger

Yeah, readonly is of type string, if you don’t specify otherwise.

oscarsullivan_old avatar
oscarsullivan_old

thanks

mmuehlberger avatar
mmuehlberger
variable "readonly" {
  type = "list"
}
oscarsullivan_old avatar
oscarsullivan_old

mmuehlberger avatar
mmuehlberger

That would make readonly a list.

oscarsullivan_old avatar
oscarsullivan_old

Perfect. Was missing that type declaration

oscarsullivan_old avatar
oscarsullivan_old

Much abstraction. Very wow doge cool-doge

mmuehlberger avatar
mmuehlberger

If you give it a default of [] it automatically infers it.

oscarsullivan_old avatar
oscarsullivan_old

so variable "readonly" [] instaed of {}

oscarsullivan_old avatar
oscarsullivan_old

or

variable "readonly" {
         []
}
mmuehlberger avatar
mmuehlberger
variable "readonly" {
  default = []
}
oscarsullivan_old avatar
oscarsullivan_old

cool thanks

oscarsullivan_old avatar
oscarsullivan_old

I’m getting invalid inputs

oscarsullivan_old avatar
oscarsullivan_old

Nvm I’ll just use a tfvars file with direnv

availability_zones=["eu-west-2a", "eu-west-2b", "eu-west-2c"]
loren avatar

if you do choose to use envs, you need to quote the assignment for complex var-types…

1
loren avatar
Input Variables - Configuration Language - Terraform by HashiCorp

Input variables are parameters for Terraform modules. This page covers configuration syntax for variables.

loren avatar

for a map:

export TEST='{bar="baz", foo="zzz"}'
oscarsullivan_old avatar
oscarsullivan_old

Thanks both. I should be using tfvars file anyway since it is TF specific variable.

chrism avatar

When you type yes outside of a question and /usr/bin/yes goes into an infinite loop that ctrl+c wont break

oscarsullivan_old avatar
oscarsullivan_old

omg I do that all the time with terraform plan

oscarsullivan_old avatar
oscarsullivan_old

terraform plan yes y y y y y y y y y...

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

With terraform it supports a flag to default to yes on all prompts

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

No need to script it

oscarsullivan_old avatar
oscarsullivan_old

it’s just a mistake hahaha

oscarsullivan_old avatar
oscarsullivan_old

I go to accept it thinking it was apply

loren avatar

whew, tks for reminding me! i went to apply a config and got distracted, forgot it was waiting on my input

loren avatar

of course my temp cred had expired. sigh

loren avatar

another thing i’m looking forward to in 0.12, support for credential_process providers

oscarsullivan_old avatar
oscarsullivan_old

Haha. Can’t you increase the max time the creds last inside of IAM portal though?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

aws-vault exec --server is to address this problem provided a sufficiently long session duration (max 12 h)

loren avatar

i unfortunately do not have permissions to do that in this particular environment. it’s a federated identity, 1hr max. only way to get a cred is with custom utilities. i might have to steal that –server idea though… i can easily write my own tool…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, there are some IAM metadata proxies out there that emulate the AWS behavior

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

could be a good place to start @loren

loren avatar

Yep, on it, thanks!

2019-03-22

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

of those of you using geodesic (recent release), does your terminal look like this when you use assume-role?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
09:40:26 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s an animated gif (slightly washed out) https://sweetops.com/wp-content/uploads/2019/03/geodesic-demo-1.gif

attachment image
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Basically, what I want to understand if it renders “normally” for you, or if you have some ugliness

2019-03-25

joshmyers avatar
joshmyers

Is that not working as expected?

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:10 PM

There is 1 event this week

Office Hours

March 27th, 2019 from 11:30 AM to 12:20 PM GMT-0700 at https://zoom.us/j/684901853

Abel Luck avatar
Abel Luck

which terraform module is responsible for creating a billing user/group such that a user can access the billing section of the console

Abel Luck avatar
Abel Luck

it seems to me that the admin users created have admin perms on the sub accounts, but zero perms on the root account. i’d like to be able to add an additional group policy to allow some of these users to access billing

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We don’t have any turnkey groups like that right now. Just haven’t gotten around to it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

Abel Luck avatar
Abel Luck

I ended up adding a new module that creates the group policy, like terraform-aws-organization-access-group, calling that module in root-iam, and then adding the group to the list of groups used in the users root module (on a per user basis)

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” invocations for provisioning reference architectures - cloudposse/terraform-root-modules

1
casey avatar

has anyone been able to port forward from the geodesic shell? Im trying to view kubernetes-dashboard on my host machine but unable to get port forwarding working.

casey avatar
casey
07:26:05 PM
casey avatar

i have tried that but saying site cannot be reached when going to localhost:54515

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Document Kubectl Proxy in Geodesic · Issue #428 · cloudposse/docs

what We port map a random port into the geodesic container This port is what should be used for proxying kubectl proxy –port=${KUBERNETES_API_PORT} –address=0.0.0.0 –accept-hosts=&#39;.*&#39; wh…

1
casey avatar

thank you, that worked!

2019-03-26

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Awesome @Abel Luck ! Sounds spot on

2019-03-27

oscarsullivan_old avatar
oscarsullivan_old
02:04:22 PM

@Erik Osterman (Cloud Posse) did you ask about this before

oscarsullivan_old avatar
oscarsullivan_old
02:05:23 PM
SweetOps #geodesic avatar
SweetOps #geodesic
06:00:02 PM
Office Hours

March 27th, 2019 from 11:30 AM to 12:20 PM GMT-0700 at https://zoom.us/j/684901853

2019-03-28

chrism avatar

What’s the deal with teleport (noticed it slid into geodesic)

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Teleport is not (or at least should not be) in the released Geodesic. Where did you “notice it slid in”?

chrism avatar

In the commit history… Unless Im mistaken

chrism avatar

Which is wholly possible

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

We have published a helmfile and chart for Teleport recently, but that should not have affected Geodesic.

chrism avatar

Its listed on 0.76 release

chrism avatar

I hadn’t checked the diff. Mobile GitHub is nightmare fuel

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Oh that, yes, sorry, my mistake. Geodesic includes our kops manifest template, which we updated to support installing Teleport ssh agents on the instances.

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

It is the companion to our charts and helmfiles which deploy the Teleport proxy and auth daemons.

chrism avatar

Cool. Teleport looks pretty neat if not more complicated than the usual bastion stuff

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Teleport is a lot more complicated to set up, but it is indeed very neat. And in High Availability mode, very robust.

casey avatar

Has anyone had this problem with ubuntu 16.04? I updated Geodesic to 0.85.0 and I am now getting the following when I go to assume role:

casey avatar
casey
04:20:36 PM
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy G (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Can you try the most recent previous release and see if it works? If so, then we will know it was a recent change

casey avatar

0.86.0?

casey avatar

oh miss read sorry

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

0.84.1

casey avatar

yeah one sec

casey avatar

another note is that it is working on mac for 0.85.0

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhh interesting

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

is this issue on your WSL machine?

casey avatar

no im on ubuntu 16.04

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ok

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@oscarsullivan_old reported some problem too (he’s also on ubuntu)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

i’ll be free this afternoon to take a look

casey avatar

yeah, confirming 0.84.1 is not working either

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ohhh interesting

casey avatar

I think it has something to do with the interactive prompt that shows, I found this issue but not sure what the resolution was

casey avatar
`assume-role` interactive does not works on linux · Issue #391 · cloudposse/geodesic

what assume-role interactive does not works on linux ✗ (none) ~ ⨠ assume-role Failed to read /dev/tty Usage: assume-role [role]

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

maybe ` 0.80.0` broke it?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

when we went from alpine 3.8 -> 3.9

casey avatar

ill try 0.79.0 one sec

casey avatar

nope 0.79.0 same issue

casey avatar

0.56.0 working, but non-interactive

casey avatar

*0.57.0

oscarsullivan_old avatar
oscarsullivan_old

The solution as per comments was https://github.com/cloudposse/geodesic/pull/390

[fzf] Fix for non-interactive mode by osterman · Pull Request #390 · cloudposse/geodesic

what Disable bash completion for fzf if non-interactive terminal why The upstream bash completion script has a bug where it doesn’t handle this use-case

oscarsullivan_old avatar
oscarsullivan_old

but I added noise to the issue because I thought my issue was the same one, but it wasn’t

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@casey I don’t have access to an Ubuntu machine running Docker running Geodesic to test, but I am responsible for most of the recent changes to Geodesic and am happy to help. Do you see any problems before trying to assume-role?

casey avatar

no

casey avatar

assume-role is working on geodesic 0.57.0

casey avatar

i bumped to 0.85.0 working on my mac, and everything was fine on osx

casey avatar

went back ubuntu 16.04 at home and it was not working

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Please try echo foo | fzf --height 30% --preview 'echo bar' and LMK if that works in 0.8[56].0 on Ubuntu

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

You should see an interactive prompt, and then just hit return and the prompt should clear.

casey avatar

getting

casey avatar

Failed to read /dev/tty

casey avatar
casey
04:47:43 PM
Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

OK, the problem is fzf. What does env | grep TERM print?

casey avatar

TERM=xterm-256color

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Are you in fact running some kind of xterm or are you running a script without a tty?

casey avatar

I dont know

casey avatar

im just using the default terminal on ubuntu

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Can you change the terminal window size?

casey avatar

yeah

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

When you resize the window, does echo ${COLUMNS}x${LINES} change?

casey avatar

yeah

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Let’s simplify a bit and unset FZF_DEFAULT_OPTS and then try the above fzf command again.

casey avatar

same

casey avatar

Failed to read /dev/tty

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

OK. Let’s back up. (If you have time to work through this now).

casey avatar

i can go at it for another 10 min

casey avatar

then we can resume when I am back if its not enough time

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

great. Try ([[ -t 1 ]] && echo true) || echo false

casey avatar

im doing this inside geodesic container correct?

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Do it inside the Geodesic container and then open a second window on the Ubuntu machine (not Geodesic) and run that same command line.

casey avatar

saying bash syntax error near `true)’

casey avatar

bash: conditional binary operator expected bash: syntax error near true)’ `

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Sorry, typo, fixed

casey avatar

both are saying true

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Let me do a little research. When can you pick this up again?

casey avatar

just ping me

casey avatar

ill be back in like an hour or 2

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

OK

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Ping me with @Jeremy G (Cloud Posse) 6 hours from now if you haven’t heard from me before then.

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

As a workaround until this is fixed, setting export ASSUME_ROLE_INTERACTIVE=false should get you back to work. You can use our new customization features to set this up automatically each time you run the container. See https://github.com/cloudposse/geodesic/pull/422 for how to do that.

Enable run-time customization by Nuru · Pull Request #422 · cloudposse/geodesic

what In addition to some small cleanups and additions, provide a capability for users to customize Geodesic at runtime. why Because people vary, their computers vary, what they are trying to accomp…

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@casey The assume-role failure is due to a bug in the Ubuntu kernel. See https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1813873

Bug #1813873 “Userspace break as a result of missing patch backp...” : Bugs : linux package : Ubuntu

Hi, The most recent set of Ubuntu kernels applied a variety of tty patches including: https://github.com/torvalds/linux/commit/c96cf923a98d1b094df9f0cf97a83e118817e31b But have not applied the more recent https://github.com/torvalds/linux/commit/d3736d82e8169768218ee0ef68718875918091a0 patch. This second patch is required to prevent a rather serious regression where userspace applications reading from stdin can receive EAGAIN when they should not. I will try to link correspondence from th…

1
Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

Solution is to upgrade your Ubuntu kernel to 4.4.0-143.169 or 4.15.0-46. See https://github.com/cloudposse/geodesic/issues/427#issuecomment-477805621

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Awesome sleuthing

1
casey avatar

that was it!

casey avatar

how did you figure that out?! thank you though

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy G (Cloud Posse) knows this kind of stuff inside and out

casey avatar

clearly. That was impressive

    keyboard_arrow_up