#geodesic (2019-09)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2019-09-30

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:04 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:07:03 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Oct 09, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2019-09-27

Fred Light avatar
Fred Light

hi guys ! tnaks for the SweetWorks ! I don’t really understand how you can override the policy or roles attributes using terraform-aws-ecr (0.6.1 tag) module. The documentation seems a bit outdated because the provided samples raise error in geodesic shell. Do you have an up to date usage sample ?

Alex Siegman avatar
Alex Siegman

I’m using this regularly. Here’s an example that I’m using:

module "example_django_app" {
  source       = "git::<https://github.com/cloudposse/terraform-aws-ecr.git?ref=tags/0.6.1>"
  namespace    = "${var.namespace}"
  stage        = "${var.stage}"
  name         = "example-django-app"
  use_fullname = "false"

  max_image_count = "800"

  principals_full_access     = ["${local.principals_full_access}"]
  principals_readonly_access = ["${local.principals_readonly_access}"]

  tags = "${module.label.tags}"
}

Those locals are processed and provided in the module like this: https://github.com/cloudposse/terraform-aws-ecr/blob/0.11/master/main.tf#L1-L8 And I use pass in what it wants like this, via tfvars

external_principals_full_access=[
  "arn:aws:sts::1234567:role/OrganizationAccountAccessRole"
]
external_principals_readonly_access=[
  "arn:aws:iam::987654321:role/nodes.us-east-1.staging.example"
]
cloudposse/terraform-aws-ecr

Terraform Module to manage Docker Container Registries on AWS ECR - cloudposse/terraform-aws-ecr

oscar avatar
oscar

You would want to extend it (fork and PR) so that it can default to a policy doc that has CP’s default but also support taking from a users input https://github.com/cloudposse/terraform-aws-ecr/blob/master/main.tf#L28

cloudposse/terraform-aws-ecr

Terraform Module to manage Docker Container Registries on AWS ECR - cloudposse/terraform-aws-ecr

Fred Light avatar
Fred Light

thxs @oscar

oscar avatar
oscar

(but as of yet doesn’t look like you can hence why you’d need to develop it yourself)

Fred Light avatar
Fred Light

ok got it clear. i was wondering this because of the error but in the meantime i understood it was linked to the IAM role not being specified as data. So no need to PR at the end but good to know it work this way.

oscar avatar
oscar

What’s the error?

Fred Light avatar
Fred Light
module.ecr.aws_ecr_lifecycle_policy.default: Creation complete after 0s (ID: goalgo-dev)
module.ecr.aws_ecr_repository_policy.default: Still creating... (10s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (20s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (30s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (40s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (50s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (1m0s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (1m10s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (1m20s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (1m30s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (1m40s elapsed)
module.ecr.aws_ecr_repository_policy.default: Still creating... (1m50s elapsed)
Releasing state lock. This may take a few moments...

Error: Error applying plan:

1 error(s) occurred:

* module.ecr.aws_ecr_repository_policy.default: 1 error(s) occurred:

* aws_ecr_repository_policy.default: Error creating ECR Repository Policy: InvalidParameterException: Invalid parameter at 'PolicyText' failed to satisfy constraint: 'Invalid repository policy provided'
	status code: 400, request id: cd97a9c3-1065-474f-8169-51235bd0ebf7
oscar avatar
oscar

Also send your usage of the module

oscar avatar
oscar

Might show something

Fred Light avatar
Fred Light
Fred Light avatar
Fred Light

the error was when principals_full_access was defined actually

oscar avatar
oscar

So you’ve taken that out and it is fine?

Fred Light avatar
Fred Light

no i added it and seems fine

oscar avatar
oscar

FYI

oscar avatar
oscar

0.6.1 is TF 11

Fred Light avatar
Fred Light

yes perfect

oscar avatar
oscar

0.7.0 is TF 12

oscar avatar
oscar

So you want TF 11? Okie that’s fine. Is your Geodesic using TF 11?

Fred Light avatar
Fred Light

terraform version Terraform v0.11.7

  • provider.aws v2.30.0
  • provider.null v2.1.2
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

@Fred Light here is the latest example on using aws-ecr module https://github.com/cloudposse/terraform-root-modules/blob/master/aws/ecr/kops_ecr_app.tf#L11

cloudposse/terraform-root-modules

Example Terraform service catalog of “root module” blueprints for provisioning reference architectures - cloudposse/terraform-root-modules

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
cloudposse/terraform-aws-ecr

Terraform Module to manage Docker Container Registries on AWS ECR - cloudposse/terraform-aws-ecr

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

thanks @oscar !

2019-09-25

oscar avatar
oscar

We have some people using AWS Vault and other AWS auth mechanics. We’ve disabled aws-vault in the Dockerfile due to the UI prompts it causes that aren’t needed for ‘other auth’ users (90% of users). For the 10% using AWS Vault, how can we have them use the same Geodesic/Dockerfile, but toggle AWS_VAULT_ENABLED ? I notice if you set this to true inside the container nothing happens, so it is clearly a build arg (could be wrong).

Any alts?

oscar avatar
oscar

(Or is it possible to have it enabled but disable -> Run 'assume-role' to login to AWS with aws-vault that is printed on every command )

joshmyers avatar
joshmyers

@oscar Have you looked through the geodesic codebase for how to do this? It is all pretty readable. I’m sure PRs welcome if it doesn’t do what you require

joshmyers avatar
joshmyers
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

oscar avatar
oscar

I have not. I will take a look and see if I can put a PR through

oscar avatar
oscar

Thx

oscar avatar
oscar

I imagine that runs at build time only?

oscar avatar
oscar

Or rather container start

:--1:1
joshmyers avatar
joshmyers

So, off the top of my head you could use build args here, which geodesic and build harness already supports

joshmyers avatar
joshmyers
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

joshmyers avatar
joshmyers
cloudposse/build-harness

Collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more - cloudposse/build-harness

joshmyers avatar
joshmyers

So you could an an ARG to the dockerfile, and change https://github.com/cloudposse/geodesic/blob/master/Makefile#L20 to make --no-print-directory docker:build ARGS="AWS_VAULT_ENABLED"

cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

oscar avatar
oscar

I’m actually leaning towards…

oscar avatar
oscar
	PROMPT_HOOKS+=("aws_vault_prompt")
	function aws_vault_prompt() {
		if [ -z "${AWS_VAULT}" ] && [ "${AWS_VAULT_PROMPT" == "true"]; then
			echo -e "-> Run '$(green assume-role)' to login to AWS with aws-vaultdsfdsfdssf"
		fi
	}
oscar avatar
oscar

joshmyers avatar
joshmyers

and add some makefile default that users can set an env var to turn off at build time

oscar avatar
oscar

Would you accept the above?

joshmyers avatar
joshmyers

It’s not me

oscar avatar
oscar

Jeremy and Erik??

joshmyers avatar
joshmyers

Not sure who atm but best open a PR and see

joshmyers avatar
joshmyers

The above just turns off the prompt though, why not use the ENV var that is already there to turn the whole lot off, which is actually what you want

joshmyers avatar
joshmyers

Not just the prompt.

oscar avatar
oscar

I want AWS VAULT on, but prompt off

oscar avatar
oscar

that way everyone can use the same Dockerfile no extra config

joshmyers avatar
joshmyers

We have some people using AWS Vault and other AWS auth mechanics. We’ve disabled aws-vault in the Dockerfile due to the UI prompts it causes that aren’t needed for ‘other auth’ users (90% of users). For the 10% using AWS Vault, how can we have them use the same Geodesic/Dockerfile, but toggle AWS_VAULT_ENABLED ? I notice if you set this to true inside the container nothing happens, so it is clearly a build arg (could be wrong).

Any alts?

oscar avatar
oscar

L142

	PROMPT_HOOKS+=("aws_vault_prompt")
	function aws_vault_prompt() {
		if [[ -z "${AWS_VAULT}" ]] && [[ "${AWS_VAULT_PROMPT" != "false"]]; then
			echo -e "-> Run '$(green assume-role)' to login to AWS with aws-vault"
		fi
	}
joshmyers avatar
joshmyers

The above would also allow the same Dockerfile etc to be used

oscar avatar
oscar

Yeh think I had some development time between then and went out for a lunch

oscar avatar
oscar

That snippet wouldn’t affect anyone else

joshmyers avatar
joshmyers

but the folks who do want to use aws-vault just set some var in their bash profile or whatever which will be used when they make build

oscar avatar
oscar

but would allow others to disable prompt

joshmyers avatar
joshmyers

What you want to do is already supported with some tweaks to the dockerfile/Makefile, which wouldn’t need to effect all of you.

joshmyers avatar
joshmyers

Open a PR and I’m sure someone will get to it.

oscar avatar
oscar
Feature: Toggle AWS Vault Helptext by osulli · Pull Request #525 · cloudposse/geodesic

What Adds ability to pass an ENV in individual Geodesic module&#39;s Dockerfiles to toggle the prompt. By default this will not affect anyone. Why I am not a fan of the help text that is prompted o…

joshmyers avatar
joshmyers

@oscar How will you set AWS_VAULT_PROMPT ?

oscar avatar
oscar

Geodesic Dockerfile

joshmyers avatar
joshmyers

Right, so you still need to add something to the Dockerfile

oscar avatar
oscar

Yep, I am ok with that, is that a problem generally?

oscar avatar
oscar

Seems like an OK thing. It’s a per geodesic module config

joshmyers avatar
joshmyers

No but I don’t see how it differs from the suggestion above which doesn’t involve a PR as it is already supported. Also still not sure why you want AWS_VAULT_ENABLED true, but no prompt….

oscar avatar
oscar

Other people can benefit

oscar avatar
oscar

Ok I can explain

oscar avatar
oscar

Maybe context will help understand my rationale

joshmyers avatar
joshmyers

Maybe best off to add that to the PR.

oscar avatar
oscar

We have 9 users on Azure AD authing with AWS

oscar avatar
oscar

we have to use this NPM package called aws-azure-login

oscar avatar
oscar

we don’t need aws-vault

oscar avatar
oscar

but we have 1 user using IAM users with keypair

oscar avatar
oscar

they need aws-vault

oscar avatar
oscar

We all use the same geodesic repo + module + files

oscar avatar
oscar

I understand the geodesic local config

oscar avatar
oscar

that is fine in principle

oscar avatar
oscar

but I don’t want to have to ask people to do such things

joshmyers avatar
joshmyers

So how can all users use the same dockerfile now if passing AWS_VAULT_PROMPT = false ?

oscar avatar
oscar

Because we don’t use aws-vault, but now the prompt is gone

oscar avatar
oscar

So it’s there

oscar avatar
oscar

just not in our faces

joshmyers avatar
joshmyers

sigh

joshmyers avatar
joshmyers

nvm

oscar avatar
oscar

Plus even when using aws-vault it’s annoying

oscar avatar
oscar

You don’t need the prompt to use vault

oscar avatar
oscar

or am I misunderstanding

joshmyers avatar
joshmyers

You don’t need the prompt no.

oscar avatar
oscar

I am missing why you are frustrated then

oscar avatar
oscar

Your solution is good. Have a .bashrc variable and pass it in the build target, but I don’t want to have to ask people to modify their .bashrc

joshmyers avatar
joshmyers

From experience working with devs who may not be used to AWS or geodesic, telling them to assume-role is useful. Rather than not, expecting people to know to run the thing before anything else

joshmyers avatar
joshmyers

and what you actually want to do is not enable AWS_VAULT for those users, not just hide the prompt for all…

joshmyers avatar
joshmyers

anyway, nvm, I see your use case.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

public #office-hours starting now! join us to talk shop https://zoom.us/j/508587304

party_parrot1

2019-09-24

oscar avatar
oscar

I’ve placed a file in /rootfs/usr/bin with the hopes it’ll copy to bin but no luck on make build. Any guidance available please?

joshmyers avatar
joshmyers

did you make install your new container? Sure you are running the newly built version?

joshmyers avatar
joshmyers

Putting into rootfs/usr/local/bin/foo worked for me with a make all

oscar avatar
oscar

This is an existing geodesic module. I have a new script : rootfs/usr/bin/lint and run make build and then start my Geodesic module, but the script is not in the /usr/bin/ dir in the Geodesic module container

oscar avatar
oscar

Doesn’t seem any different to my rootfs/etc/profile.d/aliases.sh script, yet doesn’t copy over

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

In your Dockerfile, you’ll need something like COPY rootfs/ /

oscar avatar
oscar

COPY rootfs/ / Already present

oscar avatar
oscar
oscar avatar
oscar

so alises.sh works just fine

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I suspect it’s attaching to a running container

oscar avatar
oscar

but lint does not

oscar avatar
oscar

ah ok

oscar avatar
oscar

Ah yes

oscar avatar
oscar

I did have it open in a nother window

oscar avatar
oscar

lets see

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oscar avatar
oscar

Spot on

oscar avatar
oscar

Anyway to give a /usr/bin/ file chmod +x easily?

oscar avatar
oscar
cloudposse/geodesic

Geodesic is a cloud automation shell. It&#39;s the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…

oscar avatar
oscar

For now I’ve popped

COPY rootfs/ /
RUN chmod -R +x /usr/bin/
oscar avatar
oscar

in Dockerfile which will do, but let me know if there’s a better solution

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Hrmmm modes should be carried over from what is in git

:100:3
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

According to that link, it’s an executable

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Would need to take a deeper look

joshmyers avatar
joshmyers

lol.

dalekurt avatar
dalekurt

@Erik Osterman (Cloud Posse) That is correct, the mode of the file (in git) would be the same when copied to the Docker image.

oscar avatar
oscar
05:41:35 PM

Excellent thanks

Hrmmm modes should be carried over from what is in git

oscar avatar
oscar

Missed that part.

oscar avatar
oscar

2019-09-23

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:04 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:01:12 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Oct 02, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2019-09-20

2019-09-18

dalekurt avatar
dalekurt

Hey guys, I’m using the reference architecture to setup my AWS landing zone and having an issue with make finalize with the following output - https://gist.github.com/dalekurt/7c451ba3914f066bf16b42392904aca1

dalekurt avatar
dalekurt

I believe from the output some s3 bucket are missing from other accounts.

russell.t.sherman avatar
russell.t.sherman

from the output it looks like all your child account state buckets are missing.. maybe you missed steps when provisioning the children.. child account should be provisioned and finalized before finalizing the root account..

dalekurt avatar
dalekurt

Yes, I had a successful completion of the make children but that my very well be true. I will review the children stage of the deployment

dalekurt avatar
dalekurt

I confirmed that the s3 buckets do exist in one of the accounts the error is complaining about.

dalekurt avatar
dalekurt

I’m assuming that this should be the mydomain-com-dev-terraform-state

2019-09-16

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:06 PM

There are no events this week

1
Cloud Posse avatar
Cloud Posse
04:02:47 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Sep 25, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2019-09-09

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:09 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:03:44 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Sep 18, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2019-09-04

2019-09-02

SweetOps #geodesic avatar
SweetOps #geodesic
04:00:04 PM

There are no events this week

Cloud Posse avatar
Cloud Posse
04:01:48 PM

:zoom: Join us for “Office Hours” every Wednesday 11:30AM (PST, GMT-7) via Zoom.

This is an opportunity to ask us questions about geodesic, get live demos and learn from others using it. Next one is Sep 11, 2019 11:30AM.
Register for Webinar
slack #office-hours (our channel)

2019-09-01

    keyboard_arrow_up