#atmos (2024-11)

Interesting. So we are working on supporting an enabled flag for components.

https://github.com/cloudposse/atmos/pull/756

The scope is not currently to destroy. However, it might be worth considering that.

In the current implementation, enabled would make the component “invisible”

an alternative to commenting it out.

I can see a component as having 3 states

• enabled

• disabled

• destroyed

I think if the components are commented out or deleted from the stack file, describe affected should know what to do with it or output a destroyed flag or something like that for another job to use that matrix to destroy it

Ok, that makes sense.

So enabled = true/false affects the visibility, but removal from the configuration is visible via atmos describe affected

@Andriy Knysh (Cloud Posse) what happens today in describe affected if the component is removed? is that surfaced?

I renamed a component from pepetest to pepetest1, describe affected saw the new pepetest1 component got deployed, but the old one is still there in the cloud environment

the key is do we have the information in describe affected JSON output

The right behavior might not be implemented, but maybe we have the data there to act on

if a component is removed, Atmos does not see it (it will not consider it affected) - this is the current implementation. This is b/c Atmos compares the current branch with a remote branch/tag/sha - if the current branch does not have the component, then it’s “not affected”.

Setting enabled: false is not “removal”, so describe affected sees that

but with that, you need a two-step approach, one to say enable: false and then another PR to remove the yaml from the stack file

yes

i would say we didn’t consider a complete component removal with describe affected - we need to revisit this

Added a task to the backlog

@jose.amengual another one for you

this feels like Xmas

Something is wrong with your name_pattern or name_template

i didn’t change those.

Share your atmos config, if you can

https://atmos.tools/cli/configuration/stacks/

Ok, I think I initially misunderstood.

Are you setting workspace key prefix anywhere?

nope

@Andriy Knysh (Cloud Posse) any ideas

is the workspaces are port of the terraform state right?

i’ve tried to clean all tf files and deleted this workspace, but still got back named as dev so maybe the default workspace holds some information?

Yes, in atmos we use one workspace for each instance of a component deployed.

Have you updated to the latest atmos? We just fixed a problem related to workspaces and changing backends

yes, i’m using the very latest

but i bet it’s atmos what is switching to the workspace so the name dev comes from atmos not from the state

Yes, atmos dynamically computes the workspace name and switches to it

i see.

this is my dev stack file

and interestingly for the project and the cluster the names are generated correctly

i have a component named nats and stack named dev and instead of dev-nats the workspace is simply dev

this is the correct behavior for Atmos components that don’t inherit from other components

in this case, the TF workspace is simply the stack name

only if you have a derived component (inherited from a base component), then TF workspace will be <stack>+<component>

hmm. what you mean by “inherit” the do cluster and project name is correct and didn’t inherit from anything. or i miss something here.

components:
  terraform:
    nats:
      metadata:
        component: nats
      vars:
        ...

vs

components:
  terraform:
    cluster:
      metadata:
        component: do/doks

regarding inheritance, please see https://atmos.tools/core-concepts/stacks/inheritance/

ok i read that before. but i didn’t us that in any catalog so far.

in the two examples above, both nats and cluster Atmos components do not inherit from any other Atmos components, so the TF workspaces for both of them will be dev

so what you have is 100% correct, the workspaces for these two components are just dev - the stack name

maybe those were generated wrongly because i made a lot changes back and forth since.

yes, looks like it

anyhow i’ fine with a single dev workspace named after the stack. as long as the states are correctly separated.

i’m not fully familiar with tf workspaces, but that means if all these components are in the same workspace they are sharing the state or not ?

each component has workspace_key_prefix - it’s usually generated by Atmos, but you can override it per component

workspace_key_prefix is, if you look at the backend s3 bucket, the top-level folder

so each component will have it’s own top-level folder in the bucket, and each stack, in a separate TF worksapce, will have its own subfolder in the folder

so yes, each component state is separated from any other component state (diff folders in the state bucket)

note that it’s still in the same backend (same S3 bucket). If you want to separate backends (e.g. per tenant/OU, per account, etc.), you need to create and configure multiple backends

sure. but i don’t see workspace_key_prefix generated anywhere.

it’s gcs, not s3 actually.

_defaults.yaml:

terraform:
  backend_type: gcs
  backend:
    gcs:
      bucket: mw-tf-state
      prefix: platform/infra
      encryption_key: '{{ env "GCS_ENCRYPTION_KEY" }}'

GCP has prefix

which is the same

https://atmos.tools/core-concepts/components/terraform/backends/#google-cloud-storage-backend

you don’t need to hardcode it here

backend:
    gcs:
      bucket: mw-tf-state
      prefix: platform/infra

in this case, all components will use the same prefix

hmm ok, let me check that

please review the doc, if you don’t specify prefix, Atmos will auto-generate it

thank you!

so if i understand it correctly, without setting the prefix atmos will generate it and because of that my component states will end up in separate folders in the gcs bucket, so even they share a workspace states are separated.

good to know that.:)

only problem is that i prefer to store them in some folder instead of the root of the bucket but i can leave with that

note that you can specify the prefix per component, in which case Atmos will just use it

i can review your config, let me know

it’s fine. this bucket is solely for tf states so it’s ok even in the root. i prefer to leave it to atmos to generate.

on a different topic while we chat.. any chance to add support for command like this in atmos.yaml :

components:
  terraform:
    command: xy command -- tofu

so support command with double dash

it would be perfect that way i could load secrets as env vars. and i won’t need custom commands.

i don’t remember if command: xy command -- tofu is supported now (need to look at the code). Did you test it?

yes. i just tested unfortunately it’s not working.

I have done that with asdf and it works

ok, we’ll create a task for this, thank you

did you try to quote it?

thanks a lot!!

@Erik Osterman (Cloud Posse) do you have an example maybe with asdf?

i’ve just tried with quote but not working.

trying with a small shell script:

command: ./optofu.sh

but still not working

i’ve created a PR: https://github.com/cloudposse/atmos/pull/762

thanks, i’ll review it today. Did you test it?

roughly. is there any go test(s) i can run?

strange but for some reason it’s not working with my atmos config. it’s working fine with atmos.yaml in the repository. i will dig into it.

problem is that when the template executed it uses tfexec

ah, yes, tfexec doesn’t understand those commands, it needs terraform

Of course… this is what we frequently do

you can create a catalog stack file for a solution.

E.g. here’s how we do “EKS” and all related components

ok. is there any related example in the repo?

Here you can see we created a default cluster config, that imports a bunch of other componetns

Those could be inline, but we chose to import them

thanks!

yes, please don’t use globals

there are a few ways to share vars b/w components

e.g. create a base abstract component (with the default values) and inherit it in the other components

see this doc for ref https://atmos.tools/design-patterns/abstract-component

And multiple inheritance

https://atmos.tools/core-concepts/stacks/inheritance#multiple-inheritance

thx!