#geodesic (2021-09)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic

Archive: https://archive.sweetops.com/geodesic/

2021-09-02

hugo976 avatar
hugo976

I’m looking to enable access to the docker socket on the container, I found the with_docker == true setting on the wrapper file but how would I pass that when starting geodesic?

hugo976 avatar
hugo976

Or rather make it so I don’t have to export it before running geodesic

2021-09-03

chrism avatar

Is there anything like a migration guide for going from geodesic to new geodesic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

unfortunately, no, however, remember it’s just a toolbox. so if any tools go away in the base image, just re-install. if any script/profile.d functionality goes away, just copy it in.

chrism avatar

Aye we pretty much maintain the current version tooling wise anyway. Good to know for definite though

chrism avatar

we’re kinda rolling on a custom build of 0.136.1 (as we mostly use it for the tfenv / structure / tools and keep terraform up to date via the docker file)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we’ve moved away from tfenv/direnv b/c it was not very conducive for gitops.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we document our current approach here https://docs.cloudposse.com/

1
chrism avatar

later versions assume-role vanishes etc guessing there was a major change of direction

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’ve moved to Leapp b/c we work predominantly with various SSO providers and this is the best tool we’ve found so far

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Leapp - One step away from your Cloudattachment image

Leapp grants to the users the generation of temporary credentials only for accessing the Cloud programmatically.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

as for assuming roles, we’re doing that in the terraform directly, so we don’t need to have the command itself

chrism avatar

See what you mean about the sso stuff; we use aws sso (backed by them) I totally missed the cli support coming in and how much less mess it is than dealing with assuming roles across accounts. Makes messing with boto simpler too

2021-09-06

2021-09-07

ChristianF avatar
ChristianF

hi there. Was wondering how people use chamber and atmos together. For example, how to get parameters from terraform (urls, keys) into helmfile or other tools. From what I saw it’s purely static config and does not support variable substituation, e.g to inject env variables dynamically. The same goes for secrets management. Or is the current understanding to just leave the env variable substitution logic inside TF and helmfile?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

the design philosophy there to not make anything dynamic in the stack configs. otherwise we’re just creating a language on top of a language.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

instead, it’s the responsibility for your terraform components (root modules) to read from SSM

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s why chamber is no longer needed, since terraform reads from SSM flawlessly.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

make your component as declarative as possible. if you need a secret, the path to the secret in SSM should be the parameter, not the value.

ChristianF avatar
ChristianF

thanks for the quick reply. this sounds like the way to go to read a secret stored in SSM from terraform. my challenge is in reading a secret stored in SSM from helm/helmfile. for example previously I had terraform write the secret and then chamber injecting for helmfile to consume through env vars to create a k8s secret.

ChristianF avatar
ChristianF

i haven’t checked yet but maybe there is a way already to consume secrets from ssm in helmfile

ChristianF avatar
ChristianF

ok. I think I found what I was looking for here https://github.com/roboll/helmfile/pull/906.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

yes, you can read from SSM directly in helmfile

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

you can also use the terraform-provider-helmfile which we are using for customers that have a lot of complicated helmfiles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

for customers that have easier helm deployments, we’ve started using the native helm provider in terraform.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
GitHub - cloudposse/terraform-aws-helm-release: Create helm release and common aws resources like an eks iam roleattachment image

Create helm release and common aws resources like an eks iam role - GitHub - cloudposse/terraform-aws-helm-release: Create helm release and common aws resources like an eks iam role

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so if you use any of the terraform methods, you can access SSM that way, or as you mentioned you can read from SSM natively in helmfile

ChristianF avatar
ChristianF

thanks for the pointers and generally for the help. I would like to use atmos in the short to midterm and execute helmfile straight from there. In particular the stack concept would fit well in my use case.

2021-09-08

2021-09-09

2021-09-24

2021-09-29

Brian Ojeda avatar
Brian Ojeda

Congrats on the atmos v1 golang rewrite release! :tada: Huge improvement over the variant based version. Those new ATMOS_ envs configs are great additions. @Erik Osterman (Cloud Posse) @Andriy Knysh (Cloud Posse)

https://github.com/cloudposse/atmos/releases/tag/1.0.0

Release 1.0.0 · cloudposse/atmosattachment image

what Rewrite atmos in Go Add more generic CLI config in YAML Allow running CLI commands from any folder Allow using stack folders (add examples of this) Allow using component folders (add examples…

4
4
1
Brian Ojeda avatar
Brian Ojeda

This is means the stack yaml parser are the same for atmos and cloudposse/utils TF provider. Right?

Release 1.0.0 · cloudposse/atmosattachment image

what Rewrite atmos in Go Add more generic CLI config in YAML Allow running CLI commands from any folder Allow using stack folders (add examples of this) Allow using component folders (add examples…

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

It’s the same

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

The new atmos has a few improvements

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

So we need to use the code from atmos in the provider and not repeat it in two places

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Update to the provider is coming

Brian Ojeda avatar
Brian Ojeda

That is great! Love the work.

Brian Ojeda avatar
Brian Ojeda

I need to figure out why the TF workspace names are different the two version. The new v1, is naming the workspace as {environment}-{stage}-{base-component-name} but the older atmos would name it {environment}-{stage}-{component-name} . The prefix works the same with both. Only the workspace name is affected.

Once I can figure it out, I am moving everything to v1.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

oh, the new one uses ``{environment}-{stage}-{base-component-name}` as the prefix?

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

this is an oversight, will fix for backwards compatibility (although the name does not matter for new deployments)

Brian Ojeda avatar
Brian Ojeda

Yes. I tried to re-deploy something I had previously deployed. I noticed it was trying to create all the resources because it was using different workspace (s3) name.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

A new release is coming today

1
1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

other nice enhancements are: • nested component folders • nested stack configs • new top level commands like atmos describe component

• fixed backend.tf generation

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) I do think we should update the main README with all the new commands and features

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

yes we should (will do it in the course of a few days)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)
Release 1.1.0 · cloudposse/atmosattachment image

Add terraform generate backend command @aknysh (#57) what Add terraform generate backend command Fix terraform workspace calculation for derived components why Generate backend files for compo…

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(fixed what you found with TF workspace for the derived components, not it works as the old atmos)

Brian Ojeda avatar
Brian Ojeda

Thank you. I’ll be checking it out later today.

Brian Ojeda avatar
Brian Ojeda

Works perfectly sir.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Also, we fixed our packages repository (deb, rpm, apk) and it now has the latest versions

1
Michael Dizon avatar
Michael Dizon

are the modules terraform-aws-components going to be updated for 1.0?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

someday

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

first we want to solve vendoring with atmos. once we do that I think we’re closer to supporting it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we want to make it easier to upstream and downstream components via git

Michael Dizon avatar
Michael Dizon

makes sense

Michael Dizon avatar
Michael Dizon

what’s the issue with vendoring now?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

internally, there’s a huge desire to support vendering. it comes up every single week. it’s been held up on the atmos rewrite, which is now done. So I hope it unlocks that functionality sooner rather than later.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we want to support vendoring natively in atmos as a core functionality.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so stack configs can specify an upstream origin and version

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then run an atmos command to pull from there (and eventually push via a PR)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

basically, when you run terraform init that is vendoring the providers locally

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

we want something like that in atmos for components

Michael Dizon avatar
Michael Dizon

slick

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

then imagine things like atmos terraform upgrade eks (not sure the interface yet… maybe atmos upgrade component eks

Michael Dizon avatar
Michael Dizon

yeah, is there a way to test any of the interdependencies?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

no, i don’t see that ever being in scope

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(too complex to merit the investment)

Michael Dizon avatar
Michael Dizon

yeah

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I do see us adding terratests to components

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

also, i see us adding support to import any module and make it a component dynamically

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

e.g. atmos terraform generate component vpn <https://github.com/cloudposse/terraform-aws-ec2-client-vpn> --version 0.2.0

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(terraform components are opinionated terraform root modules that wrap other modules and have a state backend defined, which child modules do not have)

Michael Dizon avatar
Michael Dizon

would that set up the providers as well

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

that’s already handled by terraform, so we don’t need to do anything about that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(we wouldn’t advise committing providers to source control)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

our objective here is to support gitops best practices, vendoring and committing the source into the repo

Michael Dizon avatar
Michael Dizon

for providers, i was referencing the way the modules are set up in terrafrom-aws-components

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I’m not following

Michael Dizon avatar
Michael Dizon
terraform-aws-components/providers.tf at master · cloudposse/terraform-aws-componentsattachment image

Opinionated, self-contained Terraform root modules that each solve one, specific problem - terraform-aws-components/providers.tf at master · cloudposse/terraform-aws-components

Brian Ojeda avatar
Brian Ojeda

You guys are on point. Love the addition with v1.2; especially ATMOS_LOGS_VERBOSE . I have been using it daily since v1.1. I think the only thing missing is atmos workflow . Once it is available, assuming it will be, I can push it out to all our CI and teams.

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

Workflows are coming soon, thanks again for testing

2021-09-30

    keyboard_arrow_up