#atmos (2024-07)

Yes, so in this case I recommend a hierarchical backend architecture.

We use the following workflow to initialize the root tfstate backend.

init/tfstate:
    description: Provision Terraform State Backend for initial deployment.
    steps:
      - command: terraform deploy tfstate-backend -var=access_roles_enabled=false --stack core-use1-root --auto-generate-backend-file=false
      - command: until aws s3 ls acme-core-use1-root-tfstate; do sleep 5; done
        type: shell
      - command: terraform deploy tfstate-backend -var=access_roles_enabled=false --stack core-use1-root --init-run-reconfigure=false

Then you can initialize any number of additional backends, with the state of each of those backends the root backend.

global env to get the git commit number so I can set it as a tag? In the stack configurations, you can use any Sprig or Gomplate function.

https://masterminds.github.io/sprig/

https://docs.gomplate.ca/config/

So if you are running in a GHA, you can read the environment variables using the env function

Or if. you want to read it from the .git/HEAD file you can use a file data source

Ah I see and then do you just download the tfstate-backend as a vendor but not actually use it or even define it in stack/catalog? or you will still do this to move the state of this later to the bucket it created?

also thanks for the info on the template I will look into this so I can add this it looks like this will do the trick

@Erik Osterman (Cloud Posse) the cmd on the workflow:

- command: until aws s3 ls acme-core-use1-root-tfstate; do sleep 5; done
        type: shell

dose not work for me so I change it to the default wait from aws cli:

- command: aws s3api wait bucket-exists --bucket acme-core-use1-root-tfstate

is there a way to also use template here? like if I want to use profile on this cmd?

Yes, see https://atmos.tools/core-concepts/custom-commands/ for examples

I see I will try it out thanks!

atmos version ?

https://atmos.tools/cli/commands/version

I mean to set it like a .terraform-version

or to validate that we are using the correct atmos version

do you want to validate it from the command-line, or from a CI/CD?

I will like to validate this in command-line for CI/CD I guess I can just create a workflow to run but a team mate ask if we have a way to set the atmos version like we do terraform with like a file or something like that

@Andriy Knysh (Cloud Posse)

@Miguel Zablah i guess we need more details on what you wnat to do here. By validating Atmos version, do you mean you want to install a particular Atmos version in your CI/CD workflow?

Sounds like maybe you’re looking for a version manager, like https://github.com/tofuutils/tenv

oh, it’s a nice tool, thanks @burnzy

@Miguel Zablah if you were asking about version management, tenv looks very nice for that

Oh yeah this sounds like what I was looking for

nice tool!! thanks @burnzy and @Andriy Knysh (Cloud Posse)!

Have you taken a look at some of these examples: https://github.com/cloudposse/atmos

Er… i meant to link to https://github.com/cloudposse/atmos/tree/main/examples

what Atmos command do you want to prevent the interactive mode for?

almost all commands don’t use UX, it’s just a few will open a TUI when called w/o parameters, e.g.

atmos atmos workflow

w/o parameters, they will not execute at all, hence we start an interactive TUI for the user to select the parameters for the commands

Also we have an open task to fix using interactive mode for things like help. It was not supposed to work that way

i mean basically if you just type $>atmos

its different behavior to almost (aptitude comes to mind) all other cmd line utilities.

just an idea. not too important

Yes, agree - it should not load interactive mode.

(I also find it very annoying when it does that)

we can add a setting to atmos.yaml to control that

I like the interactive mode -_- Keep it at least as an option pls

Yes definitely not going away, and we want to invest more into it

It’s just that things like help menus gain nothing by being interactive. And when mistyping a command it goes into an interactive menu, rather than just emit the help option, which is awkward.

we will overhaul atmos help and add atmos man command soon

can confirm the speed-up is significant with the atmos.Component caching now, thanks @Andriy Knysh (Cloud Posse)!

What is atmos pro ?

@Erik Osterman (Cloud Posse) @Matt Calhoun

We’re working on something that enhances GitHub Actions for continuous delivery

Does this mean there will be a free version and a pay version ?

It doesn’t change anything about how it works today. We are building new features that are not possible today in GHA, and incorporating those into the Atmos Pro GitHub App.

The GitHub App will be a paid offering.

For now, we’re working on enterprise=level features that work in concert with GitHub Enterprise. We’ll have more details in the coming months.

cool, looking forward to it

thanks for sharing!

@Matt Calhoun @Igor Rodionov @Jeremy G (Cloud Posse)

@Andriy Knysh (Cloud Posse)

Since the tests passed, I just released it as a patch release. @Matt Calhoun is out today.

@Andriy Knysh (Cloud Posse)

atmos terraform import <component> -s <stack> ADDRESS ID

@ameer

thank you!

this is not the first time people asked the same question. It’s not about “the correct way to fetch the list of subnet IDs”, it’s how to use lists data type, Go templates and YAML to correctly construct valid YAML files (since by uisng Go templates, you are just manipulating text files). We’ll test that and add some docs in the next Atmos release

So what I’m trying to accomplish won’t work?

it will work, let us find the correct way and we’ll show it here

@Patrick McDonald are you familiar with Helm by any chance?

See https://sweetops.slack.com/archives/C031919U8A0/p1721000839628849?thread_ts=1716399123.769359&cid=C031919U8A0

https://sweetops.slack.com/archives/C031919U8A0/p1719132460159829

@Andriy Knysh (Cloud Posse) I think we need to add a chapter about this, since it’s very easily misunderstood

are single quotes required? Its treating the list as a string.

subnet_ids: '{{ toJson (atmos.Component "vpc" .stack).outputs.private_subnet_ids }}'

subnet_ids: '["subnet-0f8ae0b62b933ab72","subnet-08551c9314d89f695","subnet-04f5a4e154ec39aa3"]'

so we have two diff things here:

1. The quotes are not needed for Go templates (if you use the quotes, the result will be quoted as well)
2. But w/o using the quotes, it’s not a valid YAML It’s a combination of Go templates and YAML. We’ll check how to do it correctly and let you know

( I realize I didn’t read your post carefully enough, @Patrick McDonald)

But w/o using the quotes, it’s not a valid YAML Well, that’s only because validation is happening before rendering instead of after rendering

I think we have a task to fix that, and move validation after rendering

it’s not about validation (we are not validating anything yet), the YAML Go lib we are using does not like that YAML. We are reading the files as YAML before rendering the templates

And we need to read the files as YAML to be able to parse them and get all the sections in order to get the final values for all the sections which are actually used in the Go templates.

So we read the files, parse the YAML, deep-merge all the sections, then use the final values as the context for all the Go templates.

It’s not the other way around - we don’t read the files and process the templates b/c the context for the templates are the final mvalues from all the files and sections.

It’s reading all the files first, then deep-merging all the sections, then finding the final values for the component in the stack which are the context for all the Go templates.

If we did evaluate the Go templates before reading the files as YAML and processing the stacks, then context would have to be provided separately (we don’t know how, and it would not work anyway b/c we want to use the final values for the component in the stack, which we can know only after processing everything).

we’ll look into that

@Patrick McDonald did you found a way to do this?

Okay I think I got it this works for me:

vpc_private_subnets: ['{{ range $index, $element := (atmos.Component "vpc" .stack).outputs.vpc_private_subnets }}{{ if $index }}, {{ end }}{{ $element }}{{ end }}']

@Andriy Knysh (Cloud Posse)

@Andrew Chemis terraform_workspace_pattern attribute is not in atmos.yaml, it should be defined in component stack manifests in metadata section, e.g.

components:
  terraform:
    my-component:
      vars: {}
      env: {}
      metadata:
        # Override Terraform workspace
        # Note that by default, Terraform workspace is generated from the context, e.g. `<environment>-<stage>`
        terraform_workspace_pattern: "{tenant}-{environment}-{stage}-{component}"

I want to force all values to be lowercase. Is that possible?

if you want to force it at the component level, you can use Go templates and Gomplate functions https://docs.gomplate.ca/functions/strings/

for example:

refer to templating doc for more details https://atmos.tools/core-concepts/stacks/templates/

let me know if you have questions or need more help on that

Ah, I was in my root manifest file and said the wrong thing . Exactly what I was looking for - thanks!

@Andriy Knysh (Cloud Posse) hmm just now trying this, getting the error: Error: Error loading state: Failed to retrieve workspace {{ strings.ToLower "SharedServices-uw2-prod-vpc-ws" }}: invalid value for workspace

In the new backend config its creating

   "workspaces": {
   "name": "{{ strings.ToLower \"SharedServices-uw2-prod-vpc-ws\" }}"
}

I have it enabled in my atmos.yaml

templates:
  settings:
    enabled: true
    sprig:
      enabled: false
    gomplate:
      enabled: true
      timeout: 5

and in my stack manifest:

components:
  terraform:
    vpc:
      metadata:
        terraform_workspace_pattern: '{{ strings.ToLower "{tenant}-{environment}-{stage}-{component}-ws" }}'

Any thoughts? Thanks again!

@Andrew Chemis I’m sorry I showed you that, but Atmos does not support templates in terraform_workspace_pattern, it currently supports templates in these sections https://atmos.tools/core-concepts/stacks/templates/#atmos-sections-supporting-go-templates

templates are not supported in the entire metadata section for other reasons (related to inheritance). We will add templates to terraform_workspace_pattern in the next Atmos release

why do you have the tenant SharedServices in upper-case? Can you make it lower-case?

brownfield deployment that uses control tower and account factory, so a few accounts don’t match naming convention. Ill just adjust the casing

@Andriy Knysh (Cloud Posse) Sorry for repeated annoyances :sweat_smile:. When I try this config,

terraform:
  backend_type: cloud
  backend:
    cloud:
      organization: "ac_atmos_testing"
      hostname: "app.terraform.io"
      workspaces:
        name: "{terraform_workspace}-ws"

with a -ws at the end, I get the issue Workspace "sharedservices-uw2-prod-ws" doesn't exist.

I think this isnt working because the stack manifest is trying to create the workspace using a different naming convention. I dont want to change how im naming my stacks to include ws at the end.

I know i’m not using Atmos properly, but, when I change workspaces:name to anything except {terraform_workspaces} it simply errors out.

workspaces:
  name: "null"

Workspace "sharedservices-uw2-prod" doesn't exist.

So the only way to customize a workspace name is a component level override? End of the day, what im trying to do is set a default workspace convention that differs from the stack name. In my current environment, I will never reuse components in my organization. Adding {component} to my workspace is noise and complicates permission management. But, then for a weird reason we want to append -ws to the end of the workspace name.

Desired stack name: {tenant}-{environment}-{stage} desired workspace name: {tenant}-{environment}-{stage}-ws

In our env, {tenant} is basically the account name, so were also probably null-label wrong

with a -ws at the end, I get the issue Workspace "sharedservices-uw2-prod-ws" doesn't exist.

that’s because the selected TF workspace is what’s replaced in the {terraform_workspace } token

to understand how Atmos ctreates TF workspaces for components, please refer to this doc https://atmos.tools/core-concepts/components/terraform/workspaces/

whatever workspace is calculated for a component in a stack (automatically by using the stack name, or by overriding it in metadata.terraform_worspace_pattern), the token {terraform_workspace} gets replaced with it

in the above case,

      workspaces:
        name: "{terraform_workspace}-ws"

is not the same as the selected TF workspace (it adds -ws )

execute atmos describe component <component> -s <stack> to see what TF workspace is used, it should be in the workspace field in the output

in any case, this will be always wrong

      workspaces:
        name: "{terraform_workspace}-ws"

b/c the selected TF workspace will be inserted instead of the {terraform_workspace} token. Adding -ws will make the workspace in TF Cloud different from the selected TF workspace

use this instead

terraform:
  backend_type: cloud
  backend:
    cloud:
      organization: "ac_atmos_testing"
      hostname: "app.terraform.io"
      workspaces:
        name: "{terraform_workspace}"

and then you can add -ws at the end for a component:

components:
  terraform:
    vpc:
      metadata:
        terraform_workspace_pattern: '{tenant}-{environment}-{stage}-ws'

It does not. The back story is how you work with state largely depends on the backend (e.g. TFE, AWS/S3, GCP, Azure, etc). It’s hard to do a generic implementation.

got it

but it supports all standard Terraform commands for state management atmos terraform state list/mv <component> -s <stack>

also terraform import

atmos terraform import <component> -s <stack> ADDRESS ID 

The import or move blocks should work too…

Actually come to think of it, if you added those blocks, it would only work for a single workspace and not all workspaces.

Like andriy said, you can use the atmos terraform import command. If you have multiple imports, you can also use the shell to run the raw commands

atmos terraform shell <component> -s <stack>
terraform import ADDRESS ID

Am playing with the new version today and trying to use .atmos_stack in an import. In the following config collection stack is the var and stack_test is hard coded to be the same value. In the output stack processes fine showing it’s a string. Then stack2 works fine changing the hard coded value to upper.

But I cannot change stack to upper and with the two enabled outputs _enabled doesn’t process properly, but that same syntax does fine with the hard coded value in _enabled2.

Am I trying to do magic again or is there something I need to learn about the handling of .atmos_stack when going down this path?

And cause it’s usually you called out @Andriy Knysh (Cloud Posse)

we have two diff ways of using Go templates, and they are executed at completely diff times:

1. The templates in imports use the static context, the vars in the context should be static b/c we need to process all the imports first before we find the final values for the component in the stacks
2. The templates in other sections in Atmos stack manifests. These are processed as the last step in the processing pipeline and they use the final values for the component in the stacks In short, you can’t use Go templates in imports with random context or context from the component in the stack. The context in imports is static

refer to this for more detaiols https://atmos.tools/core-concepts/stacks/templates/#template-evaluations

So the same stuff we covered. I misunderstood the change notes as potentially unlocking something that just doesn’t.

• `Allow Gomplate, Sprig and Atmos template functions in imports in Atmos stack manifests

the release allows the functions in imports (so you can use all Gomplate, Sprig and Atmos template functions)

template data (context) is not the same as template functions

this is template data

stack: "{{ .atmos_stack }}"

it starts with a dot

this is template functions

atmos.Component
strings.Lower

Gotcha, thank you. I managed to repeat my misunderstanding by working on a different problem. Sorry to interrupt your day.

not a problem

we can improve Atmos in many ways, but using templates in imports can’t be changed - the reason beign that we need to import everything first in order to find the component in the stacks and then find the final vars and config for it. We can’t use the final values in imports (in templates)

I just gotta get hit with the clue stick a few more times. Quite a few of the problems we’re dealing with in our re-work of Datadog monitor management can benefit from keeping Terraform dumber - so I keep repeating mistakes with Atmos’ features.

I’ll get there.

How do you folks currently do it ?

some people have OPA policy files in stacks/schemas/opa, and for each policy file there is a test file ending with _test.rego

does atmos run the tests automagically ?

can atmos also run conftest to collect coverage ?

and then in GHA, something like

jobs:
  opa-tests:
    steps:
      - name: Setup OPA
        uses: open-policy-agent/setup-opa
        with:
          version: latest

      - name: Run OPA Tests
        run: |
          for policy in stacks/schemas/opa/*; do
            opa test "${policy}" -v
          done

(Atmos does not embed conftest or other binaries, but you can execute any of them on the command line or from GHA, or from Atmos workflows, or from Atmos custom commands)

ah ok got it. we’ll use a similar action and run conftest

out of curiosity, any customers using conftest to test their atmos opa policies yet ?

btw, thanks a lot Andriy for your insight and guidance here

i’m not aware of people using conftest, let us know if you know anybody. BTW, conftest looks like a very nice framework to write tests

not related to OPA

please read this doc that explains your use-case

https://atmos.tools/core-concepts/stacks/templates/#excluding-templates-in-imports-from-processing-by-atmos

Templates in imports and templates in stack manifests are diff things, and they get evaluateed at diff states of the stack processing pipeline

since you have templates and imports and have enabled templates in stack manifests, Atmos tries to evaluate templates in imports

@Andriy Knysh (Cloud Posse) Thank you for your response.

Ultimately, I’m trying to error any missing keys when rendering map keys from context and not assign them <no value>. https://stackoverflow.com/questions/49933684/prevent-no-value-being-inserted-by-golang-text-template-library

In Go Templating the setting is missingkey=zero, which seems to be linked to ignore_missing_template_values

import:
  - path: "<path_to_atmos_manifest>"
    context: {}
    ignore_missing_template_values: false

However, it seems like this setting is being ignored completely for me. I verified I’m on the latest atmos version 1.85.0

Example stack

import:
  - path: catalog/defaults/mycatalog
    context: {}
    ignore_missing_template_values: false
vars:
  environment: "myenv" 

Example component

components:
  terraform:
    mycomponent:
      vars:
        FOO: "{{ .foo }}"

Shouldn’t this give me an error if I describe the stack, since I do not pass .foo?

atmos describe component mycomponent -s myenv

I found that the quotations DO matter in this context…

If I remove "" ,

components:
  terraform:
    mycomponent:
      vars:
        FOO: {{ .foo }}

Then I’ll get the error I was expecting:

invalid stack manifest 'catalog/defaults/mycatalog.yaml'
yaml: invalid map key: map[interface {}]interface {}{".foo":interface {}(nil)}

Interesting- I hadn’t faced this before.

Hi. Given my last message I am still facing an issue…

Say I have the following files setup as such: myenv.yaml

import:
  - path: "catalog/templates/stacks/stack_template"
    context:
      foo: "my_string"
      env_name: "myenv"

stack_template.yaml

import:
  - path: "catalog/templates/components/component_template"
    context:
      foo: "{{`{{ .foo }}`}}"

vars:
  environment: "{{ .env_name }}"

component_template.yaml

components:
  terraform:
    mycomponent:
      vars:
        foo: {{ .foo }}

Since foo is declared, it appears normally when I describe the component:

MacBook-Pro:atmos zainzahran$ atmos describe component mycomponent -s myenv | grep foo
    foo:
      name: foo
  foo: my_string

Now, if I remove foo from context in myenv.yaml

MacBook-Pro:atmos zainzahran$ atmos describe component mycomponent -s myenv | grep foo
invalid stack manifest 'catalog/templates/components/component_template.yaml'
yaml: invalid map key: map[interface {}]interface {}{".foo":interface {}(nil)}

^^This is what I expect and what I got.

Now, I noticed some abnormal behavior with missing contexts… Say, I change my component_template.yaml by adding -testing to foo

components:
  terraform:
    mycomponent:
      vars:
        foo: {{ .foo }}-testing

The result

MacBook-Pro:atmos zainzahran$ atmos describe component mycomponent -s myenv | grep foo
invalid stack manifest 'catalog/templates/components/component_template.yaml'
yaml: line 4: did not find expected key

That should be normal, however, if move testing- to the front of that string, like so

components:
  terraform:
    mycomponent:
      vars:
        foo: testing-{{ .foo }}

The result is

MacBook-Pro:atmos zainzahran$ atmos describe component mycomponent -s myenv | grep foo
    foo:
      name: foo
  foo: testing-<no value>

Lastly, I tried wrapping them in quotes, but it always results in in the final render

components:
  terraform:
    mycomponent:
      vars:
        foo: "testing-{{ .foo }}"

Still gives me

MacBook-Pro:atmos zainzahran$ atmos describe component mycomponent -s myenv | grep foo
    foo:
      name: foo
  foo: testing-<no value>

Additionally, adding ignore_missing_template_values: true does not error if there is a <no value> present when importing.

Any help with this would be appreciated.

@Zain Zahran this is the same question as https://sweetops.slack.com/archives/C031919U8A0/p1722420585991999

note that the templates in imports are completely different templates then in other sections.

Templates in imports are evaluated first (in order to be able to import all the files and process all the configs), and they use static context defined in the context var

this means that this is not correct

import:
  - path: "catalog/templates/components/component_template"
    context:
      foo: "{{`{{ .foo }}`}}"

b/c you are trying to define context using a template variable from the same context

context in imports should have static variables, and not use templates that use other Atmos sections for the reason explained in the other thread - we need to import all files first, and for that we need to know all the template variables

please take a look at these docs:

https://atmos.tools/core-concepts/stacks/templates/#template-evaluations

https://atmos.tools/core-concepts/stacks/templates/#excluding-templates-in-imports-from-processing-by-atmos

the first doc explains why the templates in import are not the same as templates in other Atmos sections

and they should be treated differently

@Zain Zahran I can help you more, please DM me your setup and I’ll take a look

The catalog is defined here stacks/catalog/aws-team-roles and imported here stacks/org/stage-1/global.yaml.

These 2 files have codeowners for ops teams, however, the global.yaml also imports files for IAM roles such as IRSAs for services. This file could technically be modified to misconfigure aws-team-roles or modify it in a way to add additional perms

import:
  - stacks/catalog/services/irsa.yaml
  # last imports
  - stacks/catalog/aws-team-roles

We could use (atmos or non-atmos) OPA for stacks/catalog/services/irsa.yaml but there must be a better way to go about this

Using Atmos OPA policies you can restrict which files may configure components. Then use CODEOWNERS with repository/org rulesets to prevent those files from being modified without approval from specific groups

but doesn’t atmos OPA work exclusively on deep merged yaml ?

i only want to block developers from contributing high priv components in their stack yaml

Got it. Yes, in that case you probably need something that operates at the HCL level.

The other thing you can do, is have different GitHub environments which have different OIDC scopes

Then, using a more privileged environment you could require additional reviews

or leverage the environment protection rules we previously mentioned

The hcl level is easy to gate with codeowners, you must mean the yaml level, no?

Hmm yes perhaps diff roles and reviewers would work codeowners cannot be dynamically set tho

i only want to block developers from contributing high priv components in their stack yaml There are 2 things at play here.

A) Developer tries to change the configuration for privileged components like aws-teams or aws-team-roles . For this, you can use Atmos OPA to restrict where those configurations can be defined, combined with CODEOWNERS on the paths for rego policy files and protected stack configurations.

B) Attacker decides to just create a new privileged component and copy aws-teams to foobar . To protect against this, you may need another level of defense.

(B) is what I’m talking about using either GitHub environments or HCL-level policies

Ah I see, B I’m not too worried about at the moment. You’re right that is another concern though for sure. More to think about it here… Maybe we can just codeowner all of components/terraform to ops to gate new terraform from entering.

I’m more concerned with A. We can easily protect the catalog with codeowners but we cannot easily protect the deep merged yaml catalogs that are imported which can alter the component prior to it getting into the root stack

With atmos opa, we only can run policies on the final deep merged output, but then we would need to know what specifically we are gating against.

We essentially want zero changes to X component yaml inputs, in any level of the hierarchy, and I’m probably too new, but i dont see an easy way to do this with opa

unless we took an md5 of all the yaml inputs of components X, Y, Z and if the md5 changed, it would disable the ability to merge the PR

With atmos opa, we only can run policies on the final deep merged output, but then we would need to know what specifically we are gating against. This is wrong. We built the OPA for a client specifically for this exact use case. It’s true that it’s deep merged, but we include the file it came from as part of the metadata, so you can rejcet untrusted files.

I’m looking for example

Here’s where you can see the data structure

https://atmos.tools/cli/commands/describe/component/#sources-of-component-variables

ohhhhh very cool

i didnt realize that, ty

sources.vars

so you want sources.vars.stack_dependencies.stack_file and only allow that to be certain paths

perfect, so we can allowlist the sources, use codeowners on each source, and then verify the sources using OPA with atmos

Yep, that’s it

please note that Go templates in imports are different from Go templates in stack manifests

please see this doc https://atmos.tools/core-concepts/stacks/templates/#template-evaluations

templates in imports are evaluated as the very first step and they require a static context which should not be templates depending on other sections (which are evalutaed as the very last step in the stack processing pipeline)

this is not a technical thing, this is fundamental (which means it’s not a bug and we can’t fix it).

in order to process a component in a stack, we need to import all manifest files (b/c we can have config for the component in any of the manifests), but to import all of the manifest files we need to know all the template values.

In other words, we can’t use template values in imports if those values depend on the final value from the component in the stack (which we can know only after we import and process everything)

also please see https://atmos.tools/core-concepts/stacks/templates/#excluding-templates-in-imports-from-processing-by-atmos

ok.

you can use templates in two cases at the same time: in imports and in stack manifests (please see the doc above). But those are completely diff templates and they use completely diff contexts and they are evaluated at diff stages