@Dan Miller (Cloud Posse)

• If I want to have a seperate state bucket/dynamodb table for each account, where should I deploy the tfstate-backend component? Currently it is deployed at core-gbl-root but it kinda does not make sense to me as the region of the dynamodb table is at eu-west-1. The default reference architecture has a single bucket/table, but it’s very common to add additional deployments to separate state. However, I’d recommend keeping all of the state component deployments in the same stack, core-gbl-root. For example, something like this:

# stacks/orgs/acme/core/root/us-east-1/baseline.yaml
...

components:
  terraform:
    #
    # Each tenant has a dedicated Terraform State backend
    #

    tfstate-backend/core:
      metadata:
        component: tfstate-backend
        inherits:
          - tfstate-backend
      vars:
        attributes:
          - "core"

    tfstate-backend/plat:
      metadata:
        component: tfstate-backend
        inherits:
          - tfstate-backend
      vars:
        attributes:
          - "plat"

then in the defaults for each tenant, you can specify the given backend configuration like this:

# stacks/orgs/acme/plat/_defaults.yaml
terraform:
  # Valid options: s3, remote, vault, etc.
  backend_type: s3
  backend:
    s3:
      bucket: acme-core-use1-root-tfstate-plat
      dynamodb_table: acme-core-use1-root-tfstate-plat-lock
      role_arn: arn:aws:iam::xxxxxxxxxx:role/acme-core-gbl-root-tfstate-plat
      encrypt: true
      key: terraform.tfstate
      acl: bucket-owner-full-control
      region: us-east-1
  remote_state_backend:
    s3:
      # This ensures that remote-state uses the role_arn even when
      # the backend has role_arn overridden to be set to `null`.
      role_arn: arn:aws:iam::xxxxxxxxxx:role/acme-core-gbl-root-tfstate-plat

• Where should I deploy the account and account-map components? The management account? If so what is the tenant, environment and stage for the management account? Yup the management account, specifically the core-root account. The root of the organization needs to manage accounts in that org

thanks for the reply, it will help a lot. 2 follow-up questions:

• any method for for making the bucket/table name dynamic?

• if I am based in europe and eu-west-1 is the closest region, should I still deploy all this to us-east-1?

• any method for for making the bucket/table name dynamic? Yup in my example see the inherits option. that way I can define a default, abstract tfstate-backend component configuration, and then only specify what’s different with the instantiated tfstate-backend/plat component

See this page: https://atmos.tools/design-patterns/multiple-component-instances

• if I am based in europe and eu-west-1 is the closest region, should I still deploy all this to us-east-1? No not at all! Choose the region that makes sense for you

Anything we can add here? https://atmos.tools/install#other-ways-to-install

…some examples of your configuration maybe

No need I think as I added asdf and Mise tabs a while ago. They are available in the link you provided. Renovate will take care of updating .tool-versions/.mise.toml respectively.

currently we show all the components in the Atmos UI. i agree we don’t need to show the abstract components to execute the commands like atmos terraform plan/apply, but they can be used in the commands like atmos describe component we’ll probably disable the abstract components in the UI

@Andriy Knysh (Cloud Posse) we display abstract in the UI? I think we should remove that.

But add it to atmos list components

As one of the columns

A big +1 to this. gets really confusing when you follow the pattern below, but all components from configuration.yaml are shown even though not enabled in deploy.yaml

configuration.yaml

components:
  terraform:
    sample/component1:
      metadata:
        component: component1
        type: abstract
      vars:
        enabled: false

    sample/component2:
      metadata:
        component: component2
        type: abstract
      vars:
        enabled: false

    sample/component3:
      metadata:
        component: component3
        type: abstract
      vars:
        enabled: false

deploy.yaml

import:
  - catalog/configuration.yaml

components:
  terraform:
    sample/component1:
      metadata:
        component: componen1
        type: real
      vars:
        enabled: true
        

Great, I’ll create a task for this

https://github.com/cloudposse/atmos/releases/tag/v1.125.0

This is now resolved

Thanks!!

Works beautifully thanks for this!

@Erik Osterman (Cloud Posse)

@Andrew Chemis, great questions. Here’s what we’ve realized about the limitations of overusing atmos.Component in template functions:

1. Performance Issues: a. Calling Terraform outputs is inherently slow. It requires initializing the root module, including downloading the providers. b. Template functions are always processed, even if the section of the stack configuration they reference isn’t used. This can significantly impact performance.
2. Permission Requirements: a. Running Terraform outputs requires sufficient permissions to access the outputs, which also means full access to the Terraform state. b. This can become a bottleneck, especially in environments with hundreds of VPCs or deeply nested stacks.
3. Provisioning Dependency: Terraform outputs will throw an error if the root module hasn’t been provisioned yet.
4. Interpolation issues: Working with lists and maps is cumbersome. In short, while atmos.Component can be useful, its current implementation introduces challenges around performance, permissions, and reliability. A better solution is needed to address these issues effectively.

We’re working introducing improvements to this, that vastly expand the power of Atmos.

First, we’re introducing native support for Atmos Functions using YAML explicit types (tags). These are not evaluated by the Go template engine, and therefore, we have full control over when/how they work.

Then, we’re implementing the ability to efficiently store/retrieve values from a pluggable backend. This will solve the performance issues, since we will not require terraform init it solves the permissions requirements, since it doesn’t require the same permissions context for the backend, it solves provisioning dependencies, since it will not error if not defined and supports default values, and it doesn’t rely on interpolation with gotemplates, and instead uses explicit functions.

https://github.com/cloudposse/atmos/pull/810

Hopefully this will be ready in about 1-2 weeks time.

Thank you! Thats great news to see those changes! I think I will use data calls for now and implement those fixes later! Now I dont need to troubleshoot the failures any further

@Josh Simmonds yes, correct

by using the static backend, you can “mimic” the component as if it was provisioned w/o actually provisioning it

Fantastic! I’ll keep playing with that and see if I can make it work for my use case!

Thanks, Andriy!

we made it work already in some infras, so if you have any issues, let us know, and we’ll help you

I’d like to jump on this thread as I noticed an issue trying to use the remote_state_backend* params.

I’ve followed the same example in “Brownfield Considerations” doc, but I get an error:

invalid ‘components.terraform.remote_state_backend_type’ section @Josh Simmonds and myself looked through the schema and found that backend_type nested under the component name seems to be the correct format.

So the stacks/catalog/vpc-flow-logs-bucket.yaml example:

components:
  terraform:
    vpc-flow-logs-bucket/defaults:
      metadata:
        type: abstract
    remote_state_backend_type: static
    remote_state_backend:
      static:
        vpc_flow_logs_bucket_arn: "arn:aws:s3::/my-vpc-flow-logs-bucket"

should actually look like this?

components:
  terraform:
    vpc-flow-logs-bucket/defaults:
      metadata:
        type: abstract
      backend_type: static
      backend:
        static:
          vpc_flow_logs_bucket_arn: "arn:aws:s3::/my-vpc-flow-logs-bucket"

yes, the indent is wrong, thanks for pointing it out, we’ll fix it

Is the change remote_state_backend_type -> to backend_type also correct?

should be remote_state_backend_type and remote_state_backend

backend is used to work with TF state

remote_state_backend is used to read the remote state of other components (as you are doing with the account-map)

@Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/pull/818

thanks

you’ll have to go through the CodeRabbit AI first

(in this case, it’s not correct)

coderabbit flagged the same thing I did

in this case, the AI is correct (needs to be fixed)

vpc_flow_logs_bucket_arn: "arn:aws:s3::/my-vpc-flow-logs-bucket"

Thats a neat tool!

merged to main, thanks @Matt Schmidt

Thanks @Andriy Knysh (Cloud Posse)!!

In my case, i’m working around this by producing an additional output (and will submit a PR to the module itself), but am still curious what the prescriptive guidance is

@Josh Simmonds there are a few diff topics in the question:

1. concat lists in the templates Use the Sprig concat function

https://masterminds.github.io/sprig/lists.html

1. The expression above will return a Go list. You can convert it to JSON by using the toJson Sprig function

   '{{ toJson (concat (atmos.Component "vpc" .stack).outputs.private_subnet_ids (atmos.Component "vpc" .stack).outputs.public_subnet_ids) }}'.
   

https://masterminds.github.io/sprig/defaults.html

1. The expression above will output a JSON-encoded string, not a list. You can send the string to the terraform module, but the variable type must be a string, and you’ll have to decode the string into TF list by using the jsondecode function

1. If you don’t want or can’t make the variable type a string, we are releasing a new version of Atmos today/tomorrow with the functionality to convert the JSON-encoded strings from templates into native YAML types (maps and lists), so they can be send directly to the terraform module

@Josh Simmonds we just published the new Atmos release https://github.com/cloudposse/atmos/releases/tag/v1.111.0 it will allow you to do what you described (by using either the !template or !terraform.output Atmos YAML function)

@Andriy Knysh (Cloud Posse) You’re a hero, this worked perfectly and the example in the docs is nearly identical to what I needed to make my TGW module work. Thanks for releasing that today and for all the efforts around it. Ya’ll are crushing it with this tool, so thank you!

@Yonatan Koren

Might help add more popularity (such as raising the star count)

I had some push back a couple months back when other devs complained star count was lower than other competitors

Yea, would like to do that. Though there seems to be a trick to doing it well.

Note, as an org, we have 18K+ stars, while none of the competitors maintain as many projects

Oh i meant for atmos itself

Yes the org star count is high, thats nice. How did you get the overall org star count number? Is there a way to see that on github itself to showcase for non believers ?

https://play.clickhouse.com/play?user=play#U0VMRUNUIAogICAgYWN0aW9uLCAKICAgIGNvdW50KCkgCkZST00[…]0UgJ2Nsb3VkcG9zc2UlJyAKR1JPVVAgQlkgCiAgICBhY3Rpb247

I thought there was a way to pass to the CLI something

the env section can be used to define ENV vars globally, per account, per component etc. It supports all the inheritance and deep-merging and is similar to the vars section

the ENV vars defined in the env section will be used in the process that runs atmos terraform and all other Atmos commands

you can see the ENV vars in the logs if you set

ATMOS_LOGS_LEVEL=Trace atmos terraform plan ...

in the env section, you can use templates to get the values for each ENV var

and you can use the new Atmos YAML functions like !exec and !template

https://atmos.tools/core-concepts/stacks/yaml-functions/#examples

ok, so I still need to use templates to grab from value from the ENV var that comes outside atmos

if you are not hardcoding the ENV vars (e.g. some of them are secrets or are dynamic), yes, you can use templates, datasources, or YAML functions to get the values for the ENV vars

ok, cool, I will use the new functions

so I added this to atmos.yaml

gomplate:
      enabled: true
      timeout: 5
      # <https://docs.gomplate.ca/datasources>
      datasources:
        envs:
          prnumber: "env:///PR_NUMBER"

and in my component I added a var

  pr: '{{ (datasource "envs").prnumber }}'

but I got

template: all-atmos-sections:265:41: executing "all-atmos-sections" at <"envs">: can't evaluate field prnumber in type []interface {}

I figure it out

why do you need outputs in there?

outputs is the value of var.name

I need to append something to it

I do not know if there is a better way or cleaner way to add that to the name

We are not using the context.tf the way cloudposse does

if you need to add something (static string) to the template value, that’s the way

since this is azure

@Jeremy G (Cloud Posse) heads up

@Erik Osterman (Cloud Posse) Necessary but not sufficient for Geodesic Prompt support of atmos terraform shell.

the vendor command could use some refactoring

specially filters

stacks can solve part of the issue for me

Atmos does not support vendoring stacks yet

I have to do all this to get one folder

 targets:
          - "./"
        included_paths:
          - "**/components/**"
          - "**/*.md"
          - "**/stacks/**"
        excluded_paths: 
          - "**/production/**"
          - "**/qa/**"
          - "**/development/**"
          - "**/staging/**"
          - "**/management/**"
         

could I use something like

targets:
          - "./"
        included_paths:
          - "**/components/**"
          - "**/*.md"
          - "**/stacks/{{env "STACK"}}**"
        

is this in vendor.yaml?

yes

currently, in vendor.yaml templates are supported only in source and target

but… can be impooved to support templates in all other sections like excluded_paths

to be honest, all I want is catalog, stack/mystack, components

in the easiest way possible

can you give more details about why are you using this config (what exactly do you want and don’t want to vendor)

included_paths:
          - "**/components/**"
          - "**/*.md"
          - "**/stacks/**"
        excluded_paths: 
          - "**/production/**"
          - "**/qa/**"
          - "**/development/**"
          - "**/staging/**"
          - "**/management/**"

I basically want vendor specific stacks depending on how my github workflows decides the changes are only for one stack or another

for example:

if: github.event_name == 'pull_request' then the stack is sandbox

so then I want to vendor sandbox only

(does something bad happen if you vendor (download) everything, or you just don’t want to download it to save time)

well, after you asked that, I was looking at the reasons why I needed this

this is related to when decribe affected did not support –stack

but now it does

so I can just vendor all the stacks and pass to describe affected the stack I want to execute on

oh i see, yes you can use —stack now

this is still a valid concern to be able to use more descriptive expressions to vendor to not download unnecessary components and stacks

I agree, the fact that the excluded_paths can’t match subdirs with ** it is very limiting

@jose.amengual can you open the issue regarding excluded paths behavior

And a separate issue with the filtering behavior you would like

Also, are you familiar with using tags for filtering?

That is supported today

mmm not much, familiar with the tags filtering

I can create tickets tomorrow

https://github.com/cloudposse/atmos/issues/835

https://atmos.tools/core-concepts/components/terraform/workspaces/#terraform-workspace-override-in-atmos

@Jesus Fernandez , terraform_workspace_pattern does not support Go templates (yet), it only supports the context tokens above

instaed of using {domain}, you can prob use {namespace} (or {environment})

uhm so I guess it’s tightly coupled with null-label right? The problem is that I’m using both as vars for many modules and there might be collisions right?

Nono, instead use name_template which is not tightly coupled

name_pattern is the original way, which is tightly coupled and our examples should be updated to use name_template. Since we have many user still using the original way, we are not deprecating it .

so then if I use name_pattern I don’t need to play with terraform_workspace_pattern?

Erik, this is about terraform workspace pattern, not related to stack names

@Jesus Fernandez we also support this

	// Run `terraform workspace` before executing other terraform commands
	// only if the `TF_WORKSPACE` environment variable is not set by the caller

so you can set TF_WORKSPACE env variable and then execute atmos terraform ... commands

The terraform workspace pattern doesn’t support go templates? That will be a problem for anyone using our context provider.

@Andriy Knysh (Cloud Posse) please confirm, otherwise I will create a task to address this

we have two attributes in the metadata section for each component:

components:
  terraform:
    vpc:
      metadata:
        terraform_workspace: 
        terraform_workspace_pattern:  

terraform_workspace_pattern supports the context tokens (those are not Go templates)

https://atmos.tools/tutorials/atmos-component-migrations-in-yaml/#overriding-the-workspace-name

but the metadata section supports Go templates

so we can use someting like this

components:
  terraform:
    vpc:
      metadata:
        terraform_workspace: '{{ .vars.tenant}}-{{ .vars.stage}}'

or even this (using atmos.Component template function to get the values from a different component)

components:
  terraform:
    vpc:
      metadata:
        terraform_workspace: '{{ (atmos.Component <component> <stack>).settings.settings1 }}-{{ (atmos.Component <component> <stack>).settings.settings2 }}'

Atmos YAML functions can be used as well

https://atmos.tools/core-concepts/stacks/yaml-functions/

components:
  terraform:
    vpc:
      metadata:
        terraform_workspace: !exec <shell script>

Ok, great, so @Jesus Fernandez you should be able to do what you want using simply terraform_workspace

@Andriy Knysh (Cloud Posse) did I get that last one right?

for the values, a dot . needs to be used

where’s the missing dot?

thanks, let me try

leading one I guess

the dots should be before the vars

.vars.tenant

(w/o a dot, it’s a function call in Go templates)

uhm I get an error:

The workspace name "{{ .var.tenant }}-{{ .var.account }}-{{ .var.region }}-{{ .var.domain }}-{component}" is not allowed. The name must contain only URL safe
characters, and no path separators.

Looks like atmos might be validating before processing the template

{component} is not a template token

it’s a “legacy” context token and can be used only in terraform_workspace_pattern

@Jesus Fernandez what component name do you want in the TF namespace, terraform component or Atmos component (which can be diff from the TF component)

you can use the following in the Go templates

atmos_component: Atmos component
atmos_manifest: Atmos stack manifest file
atmos_stack: Atmos stack
atmos_stack_file: Same as atmos_manifest
component: Terraform component

@Jesus Fernandez please try this ^

I tried with the most simple one:

components:
  terraform:
    network:
      metadata:
        component: network
        terraform_workspace: '{{ .var.tenant }}'

and still get the same error

btw for the stacks.name_template I saw that it should be {{ .vars.tenant }} (vars rather than var)

yes, should be {{ .vars.xxx }}

The workspace name "{{ .vars.tenant }}" is not allowed. The name must contain only URL safe
characters, and no path separators.

it doesn’t seem to realize that’s a go template

did you enable Go templates in atmos.yaml ?

oh.. that was it, I had gotemplate rather than gomplate

thanks, now it works flawlessly

if you have that in atmos.yaml, templates should work in metadata.terraform_workspace

sorry to ask again, but I’m confused now… I was testing your solution with atmos describe component and it properly renders the workspace name :white_check_mark: but when I do atmos terraform plan it doesn’t render it

it doesn’t render it what does terraform show for the workspace?

@Jesus Fernandez i’ll check templates in metadata for terraform plan/apply and will get back to you

@Jesus Fernandez please try this release https://github.com/cloudposse/atmos/releases/tag/v1.118.0

sorry for the delay, confirmed that now it works! Wondering if there’re any plans to extend this logic to a terraform_workspace_template alike field under atmos.yaml, so that the workspace can follow the same pattern without the need to set per-component metadata.terraform?

yes, this would be a useful features. @Gabriela Campana (Cloud Posse) please create a task for Atmos:

Add the ability to configure terraform_workspace and terraform_workspace_template in atmos.yaml to be applied to all components

@Jesus Fernandez for now, you can accomplish this in a very DRY way using inheritance

(just want to make sure you are aware of that solution)

Erik, unfortunately, those are part of the metadata section, and the metadata section is not inherited since it has other attributes related to a single component only

we can probably inherit some parts of the metadata section

or add another a different section

let’s discuss this

I’m confused. Don’t we often enable things like spacelift by setting it enabled to true in the metadata section? and doing that via imports.

it’s in the settings section

and settings is in metadata

no, settings is a separate first class section

aha, my fault

Perhaps we should move some things out of metadata

For example, the backend config.

(we can continue to support it in both places, but only settings would work with inheritance)

we now want to inherit some parts of the metadata section, but not inherit the other parts like “inherits:” :)

Well, I think that gets complicated to explain.

Maybe things should just not be in metadata.

yes

in metadata we should have things only relevant to one component

for example, we don’t want to allow inheriting this

components:
  terraform:
    derived-component-2:
      metadata:
        inherits:
          - base-component-2
          - derived-component-1

b/c it would be a huge inheritance mess

@Jesus Fernandez we are going to discuss it internally and implement a solution to be able to inherit TF workspaces and workspace templates

I think we should implement this as part of a follow-on PR from https://github.com/cloudposse/atmos/pull/834

This is the first-class way we want to support it using YAML functions, rather than templating.

@Andriy Knysh (Cloud Posse) do we still need the new task below or this will be included with @Matt Calhoun’s PR?

• Add the ability to configure terraform_workspace and terraform_workspace_template in atmos.yaml to be applied to all components

If anyone is interested in development of any of these features, reach out to me and join our atmos-dev channel

@Erik Osterman (Cloud Posse) what are the plan for atmos pro service? If I want to use dependency in components is it my only option? Spacelift is too expensive, what will be your pricing model?

We haven’t formalized the pricing model. There will likely be a free tier like with AWS which enables light usage. At this time, it’s wide open, free and unmetered.

@Roman Orlovskiy this looks like correct behavior: all (or almost all) TF components depend on the account-map/modules/iam-roles module, and once it changes, tmos detects it

but I see what you are saying about triggering all the stacks

on the one hand, all the components are affected now since iam-roles changed. On the other hand, you know that the changes should not affect all the other components

for cases like this, we can prob add a filter flag to atmos describe affected to allow you to select the triggers

Thanks for the reply! For my use case in the PR, I would not want to send different filters per different execution, I would prefer something more static that would work in any PR/pipeline. So, maybe, it could be possible to provide some kind of –ignore parameter to atmos describe affected with a list of component path’s to ignore? Or, if the output of the describe affected command included a list of components that triggered/affected it, then it would be possible to write some jq command to filter out such items too.

Actually, I would love to be able to ignore/filter out components like account/account-map/aws-teams/aws-team-roles/tfstate-backend components for atmos describe affected globally, as those components require SuperUser permissions in any case, so the CI/CD agent will not be able to execute them at all. But, I can do this with jq already, so this is not a big issue.

yes, something like --ignore flag would do it

you can also do it using jq

to filter out those that have affected: component.module (since it means that a TF module was changed)

Right, but if the PR includes changes for valid submodules (not triggered from account-map/modules/…), then they will be ignored too in this case, as there is no way for me to separate them right now based on command’s output.

is this the reason why the rootfs directory is provided as an alternative to running commands directly?

So, first off, we have a task assigned to improve the DX on this. There’s no reason atmos cannot set these environment variables, if they aren’t already set.

if I do not set the export ATMOS_CLI_CONFIG_PATH=/Users/denizgokcin/codes/infrastructure and ATMOS_BASE_PATH=/Users/denizgokcin/codes/infrastructure

The reason these are required, is the cloudposse/terraform-provider-utils reads the stack configurations. It’s used inside many of our opinionated components.

When terraform executes a provider, it runs somewhere inside the .terraform directory

It doesn’t have the “context” of where the stacks are or the atmos.yaml

So it works when you set those environment variables, because it will then use to those to locate everything.

We have a task to infer these values and set the environment variables, if they are not set, ensuring that modules/providers can find the configs.

thanks for the reply! would be a really nice addition if the absolute path of the directory where atmos.yaml is in in setted for these variable. is there a link to the issue/task for this so that I can follow it and maybe even contribute

and also is it possiblr to work around this limitation for all commands through the custom cli commands?

@Erik Osterman (Cloud Posse)

@haitham911eg should be picking this up this week

This is our task in Linear.

@deniz gökçin we’re definitely looking for help, so if you’re interested in contributing, join us in atmos-dev and we can find some issues that are mutually beneficial

hello again I joined the dev-channel and am definitely interested in contributing. I will follow the channel and see if I can be helpful. Thanks!

but now that I think about it, this will force a “One PR per stack change”

Yea, I think it’s possible, but with the aforementioned side effect. The trick is to separate deployment from release.

Some use multiple PRs for that.

The workaround we have in place, is to create the issues tracking the releases that are required (with GH issues)

Then labeling those issues for release.

With Atmos Pro, it’s all managed with dispatching workflows and workflows (jobs) can have environment protection rules.

That enables the multi-staged deployment.

I see, this is getting conplicated

There’s always atlantis style chat ops

lol

And applying before merge (I say that since I know you’re favorable to it)

https://github.com/peter-evans/slash-command-dispatch

https://atmos.tools/core-concepts/stacks/define-components/#disabling-components-with-metadataenabled

Disabling a component does not cause deletion. It just signals that it’s no longer managed by Atmos.

That said, we have an issue planned to support deletions. @jose.amengual has been asking for it.

Related thread https://sweetops.slack.com/archives/C031919U8A0/p1730497920640769?thread_ts=1730488929.176399&channel=C031919U8A0&message_ts=1730497920.640769

Hi Erik thanks a lot!

Additionally, I’ve heard about Atmos Pro from various sources and even tried signing up for it, but it seems incomplete at the moment. Is it not available for use yet?

@Igor Rodionov

It’s rare that the dispatch workflow is used other than for instrumenting it with other systems like atmos pro.

Can we take a step back and first understand what you want to accomplish with a dispatch workflow rather than a workflow that triggers on conventional commit events?

Before creating a workflow that runs on commit, the purpose was to test the workflows one by one. I also plan to apply the workflow execution by commit or PR that you mentioned.

When I push changes and execute the dispatch workflow, it runs successfully. However, the workflow doesn’t seem to track changes from the plan step or produce any meaningful summary of those changes. Ok, I think I understand now, based on your description.

The dispatch workflow is not intended to work the way you are using it.

It don’t believe it calls describe affected.

@Stephan Helas then using templates, you can ref any subkey in any object (including lists and maps) in the template itself).

for example:

get any value from embedded maps at any level ^

to get an item from a list, use the template index function

{{ index .MyList 0 }} 

i want to pass the structure, no just one key, this is why i used the toJson function. i tried to use {{ index }} with number and name, but both failed

for example

❯ kn_atmos_vault describe component wms-cluster/1 -s wms-xe02-sandbox | yq .vars
ami_filters:
  name: KN-RHEL8-*
  owner_id: "146727266814"

if i use index

        ami_filters: '{{ .ami_filters }}'
        ami_filter_owner: '{{ index .ami_filters 0 }}'

i’ll get

❯ atmos describe component wms-cluster/1 -s wms-xe02-sandbox | yq .vars
ami_filter_owner: "33"
ami_filters:
  name: KN-RHEL8-*
  owner_id: "146727266814"

if i use {{ index .aws_filters “owner_id” }}, i’ll get

at <index .ami_filters “owner_id”>: error calling index: cannot index slice/array with type string

if i use index

        ami_filters: '{{ .ami_filters }}'
        ami_filter_owner: !template '{{ .ami_filters }}'

it shows a string?? (i guess)

❯ atmos describe component wms-cluster/1 -s wms-xe02-sandbox | yq .vars
ami_filter_owner: '!template {"name":"KN-RHEL8-*","owner_id":"146727266814"}'
ami_filters: 
  name: KN-RHEL8-*
  owner_id: "146727266814"

so i think, the !template function is executed somehow too late to use it that way.

Is there any way to pass not only a single value but a subkey (i don’t know if thats the correct term) to a abstract component and then use all or part of the keys to pass variables.

you use the !template function when you want it to convert lists and maps to YAML lists and maps. if the output is a string, you can use “plain” Go templates to get any key from any map

try this

ami_filters: '{{ .settings.ami_filters.owner_id.name }}'

oh, you mean this

ami_filter_owner: "33"

shows as a string

that’s how templates work

it’s ok, you can send it to Terraform var even if the var type is a number

also I think you are asking many diff questions in one

please post you config again, and we’ll start from the beginning

for example, this is not a correct usage of templates

catalog/high_level.yaml

import:
- path: catalog/components/low_level
  context:
    ami_filters: !template '{{ .settings.ami_filters | toJson }}'

which i then can use in catalog/components/low_level.yaml

components:
  terraform:
    foo:
      vars:
        ami_filters: '{{ .ami_filters }}'           <--- this will work
        ami_owner: '{{ .ami_filters.owner_id }}'    <--- this will fail, at <.ami_filters.owner_id>: can't evaluate field owner_id in type interface {}

b/c all those templates get evaluated at the first pass

and this '{{ .ami_filters }}' does not exist yet

see https://atmos.tools/core-concepts/stacks/templates/#excluding-templates-in-imports

but 33 is neither the name nor the owner_id. i don’t know where the index is pointing at.

i try to explain my problem at bit wider. so i uses (i call them) high-level catalog components, which include other components. now, if i define any key to be passed to the component, i can never remove it again. But, if the key doesn’t contain anything, i’ll get <no value> which will be passed on to terraform.

so, i try to only set keys, that are actually defined in the stack. to not set every subkey, i’d like to pass subkeys from my high level component down to the component. but sometimes i don’t want to pass everyhing. for example one module might need a.b.c the other might need a.b.f. so if i could reference sub keys in my component, everything would be very nice

you need to set evaluations: 2

i have, evalutations set to 2

and at least use this

ami_filters: '{{`{{ .ami_filters }}`}}'

so it will be evaluated at the second pass

ok, one second

hm, now it shows no error, but <no value>

oh

you don’t need to use such a complex config

imports with templates then templates again

show me the final config that you are using

this is my high level catalog item

import:
  - path: catalog/components/remote-state/v0_1
    context:
      name: wms-account-region
      component: wms-account-region
      version: v0.2
      tenant: '{{ .settings.account.tenant }}'
      tier: 'wms'
      instance: '{{ .settings.account.instance }}'
      stage: '{{ .settings.account.stage }}'
      workspace_tier: account
      region: '{{ .settings.region }}'

  - path: catalog/components/wms-base/v1_3

  - path: catalog/components/wms-cluster/v1_5
    context:
      automation_account_id: '{{ .settings.aws_automation_account_id }}'
      automation_user_name: '{{ .settings.aws_automation_user_name }}'
      ex_environment_id: '{{ .settings.wms.ex_environment_id }}'
      subnet_index: '{{ .settings.wms.subnet_index }}'
      ami_filters: !template '{{ .settings.wms.ami_filters | toJson }}'

this is the component wms-cluster/v1_5

this is the stack component settings (redacted)

❯ atmos describe component wms-cluster/1 -s wms-xe02-sandbox | yq .settings
account:
  instance: "2"
  stage: sandbox
  tenant: accounts
  tier: wms
account_id: "xxxxxxx"
aws:
  ami_image: ami-xxxxxx
aws_automation_account_id: "xxxxxxx"

depends_on:
  1:
    component: wms-base
instance: xe02
version: wms-cluster/v1.5
wms:
  ami_filters:
    name: KN-RHEL8-*
    owner_id: "xxxxx"
  ex_environment_id: xxxxxx
  subnet_index: 0

and the vars

ok, thanks, I’ll give it another look in an hour

Thx!

@Stephan Helas please change this

ami_filter_owner: '{{ `{{.ami_filters.name }}` }}'

to this

ami_filter_owner: '{{ .ami_filters.name }}'

and see what you get

also,

f your map key is a complex string, contains special characters, or is not a string, you can use the index function:

KN-RHEL8-* has special chars

so maybe try this

ami_filter_owner: '{{ index .ami_filters "name" }}'

i already did I’l try to create a minimal repo where you can reproduce the problem

please do

it’s not easy to look and work with templates, and find issues by just looking at it

i understand

Not yet, but I can create a task for that.

We should only be running it on atmos version and any --help command.

Can you confirm why you want to disable it, so we fix the right thing.

yes please. i’d like it to be optional or at least only check on atmos version

Looks like it runs on Terraform commands as well

 √ . [devops] (HOST) infra ⨠ atmos terraform plan eks/app -s plat-use1
╭──────────────────────────────────────────────────────────────╮
│             Update available! 1.119.0 » 1.121.0              │
│ Atmos Releases: <https://github.com/cloudposse/atmos/releases> │
│          Install Atmos: <https://atmos.tools/install>          │
╰──────────────────────────────────────────────────────────────╯

Initializing the backend...

Grr ok, that should not be happening

Well fix that and add a flag to disable it

not passing a component to the get-settings?

nope

is supposed to be required: false but I get a Error: invalid input

I think no

ok, so the docs are wrong

https://github.com/cloudposse/github-action-atmos-get-setting/blob/v2/action.yml#L10

is there a default component?

I resolved it with adding the
assume_role:
role_arn:

@Alcp you added that under terraform > providers > aws ?

in a stack?

yes in the stack for the backend and remote state section

Can you share the corresponding YAML

thanks @Stephan Helas, it will be added in the next release

@Andriy Knysh (Cloud Posse) should I create a task?

Or did you fix it in 1.122?

no, i missed it in 1.22.0

will add in another release

@Stephan Helas the validation schema has been updated (and merged to main)

https://github.com/cloudposse/atmos/pull/842

Here is the stack config:

vars:
  stage: static

components:
  terraform:
    weather-report-abstract:
      metadata:
        type: abstract
        component: wttr
      remote_state_backend_type: static
      remote_state_backend:
        static:
          location: disneyland
      vars:
        location: Seattle
        language: en
        format: ''
        options: ''
        units: m
    weather-report-disneyland:
      metadata:
        component: wttr
        inherits:
          - weather-report-abstract
      vars: {}
    hello-world:
      vars:
        #name: !terraform.output weather-report-abstract static location
        name: !terraform.output weather-report-disneyland static location

The wttr terraform component get the weather The hello-world just concats the var with a message.

Here is the output I am seeing:

Changes to Outputs:
  + text = "Seattle says hello world"

@Matt Schmidt you are correct, the !terraform.output func does not take into account a static backend (it needs to be improved)

the static backend is currently used by this TF module https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/remote-state which uses this TF provider https://github.com/cloudposse/terraform-provider-utils

we will add that to the yaml function in one of the new releases

Great, let me see if I can get it working without the !terraform.output. I’ll post back here if I need more help

what’s the data type of records?

instead of using range , take a look at the Atmos YAML functions

https://atmos.tools/core-concepts/stacks/yaml-functions/

!template '{{ toJson (atmos.Component will give you back the correct type

if fqdns is a list of strings, you will get back a list of strings

same with using !terraform.output - it will return the correct types (complex types like maps and lists)

so you don’t need to construct all of that using templates

Hey Andriy, thanks for replying. I’m looking to take a few different outputs from a module and stitch them into a json object for another module.

It was hoping to keep the outputs of Module1 general, but it seems the least annoying path forward here might be to just output the object already constructed

I have my fqdns and their associated IPs in two different outputs (list of strings) from Module1. I was hoping to stitch them together with some templating to send them into Module2 as a complex object

Given

fqdns = ["host1","host2"]
ips = ["x.x.x.x", "y.y.y.y"]

I was hoping to construct something like

records:
  - name: host1.example.com
    type: A
    values:
      - x.x.x.x
  - name: host2.example.com
...

this is a complicated case :slightly_smiling_face: I don’t know why it throws the error (I can test it later).

Even if it did not throw the error, it would not work (in principle) b/c '{{ range $i, $fqdn := ((atmos.Component "nproxy" .stack).outputs.fqdns) }}{{$fqdn}}{{end}}' will return a string with the content generated by the template, not a list of objects

please try

records: !template '{{ (atmos.Component "nproxy" .stack).outputs.fqdns }}'

and see what it generates

Appreciate your time on this. I’m aware that my current attempted template wouldn’t return the object Im trying to build. I had written it all out and I kept removing pieces to try to debug the } error. This was the simplest statement I could think to test.

run atmos describe component to see the result after template evaluation

That works as expected:

records:
  - host1 host2 host3

tho, I needed the toRawJson to get terraform to pass the object. I went with the pre-building approach:

records: !template '{{ toRawJson ((atmos.Component "nproxy" .stack).outputs.r53_private_record_obj) }}'

yes, use the new version

now it supports --stack

tyty

same with the github action

@Andriy Knysh (Cloud Posse) @Erik Osterman (Cloud Posse)

in the atmos.yaml it looks like I have schemas: {}

the latest version of Atmos will automatically use the remote scema https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json to validate stacks if you don’t specify it in atmos.yaml

i don’t know what’s happening in your case, maybe since you did not have validation before, your stack configs are not valid?

can you share the error here?

@Andriy Knysh (Cloud Posse) is there a way to turn off validation?

currently, the only way is to copy the schema locally and point atmos.yaml to it

you can update the local schema to not “validate” some sections

but I think the problem is, some of your stacks are not “valid” - I don’t know if it’s true, but you can check the errors

if it’s true, then it has to be fixed

the errors seem so fundamental that it seems odd that they’ve been working if they truly are invalid

can you show the exact error?

so one is something like:

...
{ "keywordLocation": "properties/components/$ref",
  "absoluteKeywordLocation": "<schema store url ending in /components/$ref>",
  "instanceLocation": "/components",
  "error": "doesn't validate with '/definitions/components'" }
...

Just ran a terraform plan. So I guess the errors aren’t getting in the way (well of at least that stack / component combo).

do you have other parts of the error? (the last part usually shows the location)

the error format is basically a list of the entries matching the following pattern :

Atmos manifest JSON Schema validation error in the file '<file name>':
{ "valid": false, "errors": [<structured like the previous message>...] }

would the location be in the first map I sent? looks to me like they all have the same 4 keys

appreciate the help!

i don’t understand what’s going on here. Can you clone https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json to a local file, and then in atmos.yaml point to it?

sure I will give that a shot

ty for you help! the interesting/actionable parts of those sequences often seems to be at the end or maybe spread out in a cyclical fashion.

did you use the local file?

yeah though I don’t know if that was the issue. why have that be the path for the local file? my inclination would be to put it somewhere like ./schemas/atmos-manifest-1.json or something like that

i just wanted to know if there’s a problem on your system downloading the remote schema. You can download the file, place it in any location you want (inside or outside the repo), point atmos.yaml to it, and run atmos validate stacks again

@Andriy Knysh (Cloud Posse) yeah i was more asking from a conceptual point of view why that location. but if it’s just preference that’s a fine answer for me. Thanks again for all your help !

no problem

but what was the result of that? What atmos validate stacks shows?

@Andriy Knysh (Cloud Posse) nothing now, I worked through the errors. There were some that were actionable and dealing with those also cleared up the rest.

Do you mean Cloud Posse module? If so, then it should be fine. It aims to manage all Terraform/OpenTofu supported backends.

That should work, but we don’t have an example

ok. i’ll give it a try

thx

That sounds like an oversight

I will create a task to fix that

@Shubham Tholiya took a look at this and could not reproduce it

Can you share what order of commands you ran?

I first run atmos terraform plan -s <stack_name> argocd and then atmos terraform show -s <stack_name> argocd --json <stack_name>-argocd.planfile --skip-init

The same behaviour happens if I run atmos terraform show -s <stack_name> argocd --skip-init -- --json <stack_name>-argocd.planfile instead

If I run terraform show --json <stack_name>-argocd.planfile inside the component directory, the planfile remains, so I was suspecting atmos to delete the file

And can you confirm that version of atmos you are using?

I’m running version 1.123.0

https://github.com/cloudposse/atmos/pull/855 This pr should fix the issue

https://sweetops.slack.com/archives/C031919U8A0/p1734148920713719

@Jeremy G (Cloud Posse) can you share the example static backend for account map

I’d rather not share the static backend for account map we are using for testing, because it is sort of an anti-pattern, very customized for our testing. Let me put together an example of a static backend for account that is more aligned with our plans for brownfield support.

@Igor Rodionov

See discussion here: https://sweetops.slack.com/archives/CB6GHNLG0/p1732309900582849

I think that is more of terraform question, no?

Atmos runs terraform basically

well in terraform you create a resource and run import, but I am struggling to do the mapping between creating a resource in a terraform file and the yaml definition in a stack in atmos

mmm can you do this in vanilla terraform ?

I mean, does it work?

or do you have to do a two-stage deployment to get it done?

yeah and it’s the suggested approach to move from vanilla TF to ATMOS , just struggling to get my head around how you define a resource manually in atmos https://docs.cloudposse.com/learn/maintenance/tutorials/how-to-use-atmos-with-existing-terraform/#part-5-importing-state

atmos will pass the inputs to terraform

so, if the import block needs some var.resource_id

you will need to pass that in the stack yaml

for example :

keyvault/infra:
      metadata:
        component: keyvault
      settings:
        depends_on:
          1:
            component: "vnet"
          2:
            component: "loganalytics"
      vars:
        name: "outputs"
        public_network_access_enabled: true
        network_acls:
          bypass: "None"
          default_action: "Allow"
          import_from: /resource/myresource/id/azure

and you can use var.import_from in the import block if you need to

FOr anyone following on later on this you can use: zone_enabled : false to not create a zone

Documentation is incorrect

Also, I don’t think we will invest much more into helmfile support, without community contirbutions. The fix here is to correct the documentation.

That is sad. How do you handle helm deployments? Also with terraform? How did you overcome the inconsistencies terraform sometimes has with helm?

We predominantly use either a) terraform+helm or b) argocd

I think the inconsistencies with terraform+helm have largely been resolved.

Historically, it was a big problem

DEV-2860: Fix documentation of Atmos for helmfile states, that deploy runs diff and then apply, but the code looks like it runs just sync

which version of atmos?

1.123.0

have you run atmos clean?

and try again?

I hadn’t, but doing so produces the same results. Steps:

1. Switch back to tf 1.9.8
2. Run atmos terraform clean <component> -s stackname
3. Accept deletions
4. Switch back to tf 1.10.2
5. Attempt to deploy, which fails with the same error as before re: state config changes

I had this happen to me, and I remember running atmos clean

which basically removed the .terraform folder and the backend.json config

Yeah, unfortunately that didn’t do the trick, sadly., but thanks for the idea

Could be my local setup, actually. Looks like 1.10 removed some role deprecated role assumption I might be relying on: https://github.com/hashicorp/terraform/pull/35721/files

ohhhh interesting

@Erik Osterman (Cloud Posse) // @Andriy Knysh (Cloud Posse) https://github.com/cloudposse/atmos/issues/850 issue filed. I think it’s in atmos, given the lack of the assume_role block inside the backend.tf.json

i can’t look at this right now (will be able to look tomorrow).

In Atmos, you can configure any backend properties in the backend section. Did you try that?

Checking to see if I can configure that myself!

you need to update the assume role section, terraform deprecated the old way some time ago, and now looks like they removed it completely

in Atmos, we support adding any attributes to the backend section

backend:
  s3:
    ......

please make sure if you are using the latest versions of Atmos and the remote-state, you are using the latest version of the utils provider

https://github.com/cloudposse/terraform-provider-utils/releases/tag/v1.27.0 https://github.com/cloudposse/atmos/releases/tag/v1.122.0

both Atmos and utils provider use the same code to process remote-state

(if still not working, ping us again and we’ll take a look)

i’ve ensure i’m using latest version of atmos, version 1.123, and the utils is at least version 1.27.

Downloading registry.terraform.io/cloudposse/stack-config/yaml 1.8.0 for alb...
- alb in .terraform/modules/alb/modules/remote-state

.
.
.
- Installing cloudposse/utils v1.27.0...
- Installed cloudposse/utils v1.27.0 (self-signed, key ID 7B22D099488F3D11)

still getting the error message.

for the utils provider to work correctly, make sure your atmos.yaml is in the correct location, please see https://atmos.tools/core-concepts/components/terraform/remote-state/#atmos-configuration

@Andriy Knysh (Cloud Posse), i think may have found where the issue is. You’re right in sense i should ensure my configure for atmos is correctly done, which i have.

I think the atmos cli don’t handle well if a user’s home directory have a dot in the name of their directory. Example, my home directory actual name is /Users/samuel.than And i store my atmos components, config and stacks in directory ~/cloud-ops, which specifically is /Users/samuel.than/cloud-ops initially, setting environment $ATMOS_BASE_PATH=/Users/samuel.than/cloud-ops, don’t seem to work (gives the error that i’ve been getting, plugin crashed)

Once i’ve move the atmos repo to something like /usr/local/etc/cloud-ops and setup my $ATMOS_BASE_PATH to /usr/local/etc/cloud-ops. It starts to work

oh, nice find, thanks

we should review this and fix it in Atmos

in the utils provider, in fact

yeah, that’ll probably explains the crazy errors i used to encounter previously when it can’t find remote modules…

looking forward to see fix

thank u

That is so odd. Why would the dot have any affect on atmos behavior? @Andriy Knysh (Cloud Posse) do you have any ideas?

not yet, need to test it

I’ve encountered this error as well, even on projects that do not make use of remote-state.

We don’t have a . in the username, and have set ATMOS_BASE_PATH. Running atmos v1.47.0 (i know ) and Terraform v1.5.7

We have pinned utils back to v1.26.0 on a few projects as a workaround

@Tyler Rankin can you elaborate? It sounds like if you don’t have ATMOS_BASE_PATH set then you have this problem. (this is a known issue, and we will open a PR for this soon)

But I don’t see why pinning versions of atmos changes that behavior

1.26.0 is ancient

Apologies, I mean to say we are setting ATMOS_BASE_PATH as expected and don’t use a . in the username (like OP originally thought the issue was). Everything worked for us up until terraform-provider-utils latest release (1.27.0).

We are pinning terraform-provider-utils back to the previous 1.26.0, not atmos. This is issue also came up here. I’m not sure the intricacies between remote-state / atmos / utils, but utils 1.27.0 seems to have caused us issues

@Andriy Knysh (Cloud Posse) @Erik Osterman (Cloud Posse) do we need a task to track this?

@Andriy Knysh (Cloud Posse) could this be related to that other report we had this week on the provider? …where you said to pin to 1.26.0?

i don’t know yet what the issue is with the 1.27.0 utils provider, but one more person reported an issue. I’ll check it and fix. For now, please pin the provider to 1.26.0

oh, I see, this error

panic: assignment to entry in nil map

is fixed in a new PR and will be released as a new Atmos version today

we’ll then update the utils provider

btw @Tyler Rankin, the error above happens if you are using a non-existing component in Atmos commands like atmos terraform plan. You should take a look at this in your stack configs. The new Atmos version will display a correct error message instead of panicking, but the issue with invalid component names will not be fixed (it’s a configuration issue, if any)

but let’s not jump to any conclusion, we’ll release a new Atmos version and utils provider version today, and you can test again

Thanks for the additional details @Andriy Knysh (Cloud Posse). I was beginning to recognize a similar pattern. We have stacks that may or may not make use of certain components, and as such we have utilized var.ignore_errors in a few places since we only care about a specific component if it is actually deployed.

Looking forward to running tests on the updated versions

@Andriy Knysh (Cloud Posse) I built a local utils using the new atmos 1.130.0 and was able to see the panic has been removed when calling utils_component_config. Is terraform-provider-utils scheduled for a release today?

yes we will release it today, thank you for testing

More context on the specific command I am running,

# run from within a Geodesic container using atmos 1.123.0 

$ atmos terraform plan rds-key --stack ue2-stage

Initializing the backend...
Initializing modules...

Initializing provider plugins...
- terraform.io/builtin/terraform is built in to OpenTofu
- Reusing previous version of cloudposse/utils from the dependency lock file
- Reusing previous version of datadog/datadog from the dependency lock file
- Reusing previous version of hashicorp/aws from the dependency lock file
- Reusing previous version of hashicorp/local from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Using previously-installed datadog/datadog v3.49.0
- Using previously-installed hashicorp/aws v3.76.1
- Using previously-installed hashicorp/local v2.5.2
- Using previously-installed hashicorp/external v2.3.4
- Using previously-installed cloudposse/utils v1.7.1

OpenTofu has been successfully initialized!
Usage: tofu [global options] workspace select NAME

  Select a different OpenTofu workspace.

Options:

    -or-create=false    Create the OpenTofu workspace if it doesn't exist.
Error parsing command-line flags: flag provided but not defined: -rds-key

Usage: tofu [global options] workspace new [OPTIONS] NAME

  Create a new OpenTofu workspace.

Options:

    -lock=false         Don't hold a state lock during the operation. This is
                        dangerous if others might concurrently run commands
                        against the same workspace.

    -lock-timeout=0s    Duration to retry a state lock.

    -state=path         Copy an existing state file into the new workspace.
Error parsing command-line flags: flag provided but not defined: -rds-key

exit status 1

In atmos.yaml, I am trying to conditionally use environment in the name pattern via name_template for a few new resources I am adding in a different region.

# atmos.yaml

stacks:
  base_path: "stacks"
  included_paths:
    - "org/**/*"
  name_pattern: "{stage}" # old format for v1 infra
  # name_pattern: "{environment}-{stage}" # new format for v2 infra, but I don't want to rename everything
  name_template: "{{- if .vars.environment }}{{.vars.environment}}-{{end}}{{.vars.stage}}"
  excluded_paths:
    - "**/_*.yaml" # Do not consider any file beginning with "_" as a stack file

I’ll bet @Andriy Knysh (Cloud Posse) know what’s going on here.

Looks like the the name pattern is working when finding the stack but then the workspace name creation isn’t working?

@Andriy Knysh (Cloud Posse) bumping this up

@Andriy Knysh (Cloud Posse) do you have any insight on this? If not, I’m planning to hack a bit on it to see if I can resolve the component name being parsed as a cli option/flag

@Weston Platter what CLI command did you run?

(also note, if both name_template and name_pattern are defined in atmos.yaml, name_template takes precedence and overrides name_pattern - both are not supported at the same time (b/c it’s not possible to know which one to use)

i’ll test what you defined here

name_template: "{{- if .vars.environment }}{{.vars.environment}}-{{end}}{{.vars.stage}}"

here’s the specific cli command I ran,

atmos terraform plan rds-key --stack ue2-stage

where rds-key was the atmos component name

Thanks for taking a look!

It seems to me like that field is used in documentation examples.

Looks like failures when being set to null and being set to an ARN

what terraform version are you using?

seems 1.10.2 doesn’t work and 1.9.2 does. I would like to upgrade though, any suggestions on what config to change to enable that?

Terraform deprecated assume_role block some time ago in favor of just using role_arn some time ago, and in 1.10 they removed it completely

the new backend config should look like this in Atmos

terraform:
  backend:
    s3:
      bucket: xxxxxxxx-tfstate
      dynamodb_table: xxxxxxx-tfstate-lock
      role_arn: xxxxxxxxxx
      encrypt: true
      key: terraform.tfstate
      acl: bucket-owner-full-control
      region: us-east-2

(it also works in older TF versions like 1.9)

Yes, so if you add it to defaults, it gets inherited by everything

I would first master inheritance, before using templating.

If you pass a variable to terraform, that it is not expecting, it will give you that message.

if you define short_region in your variables.tf of the component that warning will go away

whats the best way to share vars between some components but not all of them? https://atmos.tools/core-concepts/stacks/inheritance/#multiple-inheritance

Now with multiple inheritance, the assumption is that you use the same variables names, which is a convention that we follow. This reduces the cognitive load on developers.

Now, if you don’t have that option, then templating is more appropriate. That said, I think the underlying problem is the lack of consistency that comes from components that defined multiple variables that fulfill the same purpose, but with different names.

Appreciate the insights, my intent here is to maintain existing naming conventions and reduce the cognitive load that would come from having to redefine (from an org perspective) what environment means to us. This would translate to what atmos considers stage and we use region where it uses environment. From my reading of the docs, templating seems to be the only way to workaround this difference, and since I want to maintain this naming convention, that feels like a framework decision that I have to square away at a baseline.

I understand that its an opinionated choice that atmos makes, just hoping there’s some means of working with it that doesn’t require defining variables in my terraform components that won’t be used (and then having to deal with the linting issues that would bring).

Historically, we only supported name_pattern. Today we support any convention. But you need to use name_template instead

Also, make sure you saw https://sweetops.slack.com/archives/C031919U8A0/p1734178991069519

Oh, awesome, this looks like what i was missing! I’ll kick the tires on this today. Thanks!

Great! Let me know how it goes.

what do you mean by? Since it is a provider, however, context is automatically available to all child modules of the root module. you mean inside the same component? bc you will still have to pass in namespace, tenant, stage, ect. to the provider on each component where you use this instead of passing them just on variables, right?

this is great but I prefer the module null-label but maybe I’m missing something