#atmos (2023-08)

(we have examples for GHA, but not GitLab)

https://atmos.tools/category/github-actions

This is a good question. I would like to know as well. I have detailed schemas for majority of my components, but only a “good enough one” for stack and workflow files. @Matt Gowie, this is my current one if you want a starting point. I have another one that extends that one with several conditional blocks (see below) that will reference JSONSchema for a component used by atmos validate too.

{
  "$schema": "<https://json-schema.org/draft/2020-12/schema>",
  "$id": "tf-component-stacks",
  "title": "component-stack",
  "type": "object",
  "properties": {
    "import": {
      "type": "array",
      "items": {
        "type": "string"
      }
    },
    "components": {
      "type": "object",
      "properties": {
        "helmfile": {},
        "terraform": {
          "type": "object",
          "patternProperties": {
            ".+": {
              "allOf": [
                {
                  "if": {
                    "properties": {
                      "component": {
                        "enum": [
                          "account-monitors"
                        ]
                      }
                    },
                    "required": [
                      "component"
                    ]
                  },
                  "then": {
                    "$ref": "account-monitors.json",
                    "additionalProperties": false
                  }
                },
                {
                  "if": {
                    "properties": {
                      "component": {
                        "enum": [
                          "account-settings"
                        ]
                      }
                    },
                    "required": [
                      "component"
                    ]
                  },
                  "then": {
                    "$ref": "account-settings.json",
                    "additionalProperties": false
                  }
                },
                // ...more if-then block for components with scheams 

                // below is a generic schema for components without scheams
                {
                  "if": {
                    "patternProperties": {
                      ".+": {}
                    }
                  },
                  "then": {
                    "properties": {
                      "remote_state_backend": {},
                      "remote_state_backend_type": {},
                      "settings": {},
                      "backend": {},
                      "backend_type": {},
                      "deps": {},
                      "command": {},
                      "env": {},
                      "metadata": {},
                      "workspace": {},
                      "inheritance": {},
                      "sources": {},
                      "component": {
                        "type": "string"
                      },
                      "vars": {
                        "type": "object",
                        "properties": {
                          "enabled": {
                            "type": "boolean"
                          },
                          "environment": {
                            "type": "string"
                          },
                          "label_key_case": {
                            "type": "string"
                          },
                          "label_value_case": {
                            "type": "string"
                          },
                          "localstack_enabled": {
                            "type": "boolean"
                          },
                          "namespace": {
                            "type": "string"
                          },
                          "region": {
                            "type": "string",
                            "enum": [
                              "us-east-1"
                            ]
                          },
                          "sandbox_mode_enabled": {
                            "type": "boolean"
                          },
                          "stage": {
                            "type": "string"
                          },
                          "tags": {
                            "type": "object",
                            "required": [
                              "env"
                            ],
                            "patternProperties": {
                              "[a-zA-Z0-9_-]+": {
                                "type": "string"
                              }
                            }
                          },
                          "name": {
                            "type": "string",
                            "pattern": ".+"
                          }
                        }
                      }
                    },
                    "additionalProperties": false
                  }
                }
              ]
            }
          }
        }
      },
      "additionalProperties": false
    },
    "env": {
      "type": "object",
      "patternProperties": {
        ".+": {
          "type": [
            "boolean",
            "number",
            "string"
          ]
        }
      }
    },
    "terraform": {
      "properties": {
        "remote_state_backend": {},
        "remote_state_backend_type": {},
        "settings": {},
        "backend": {},
        "backend_type": {},
        "deps": {},
        "command": {},
        "env": {},
        "metadata": {},
        "workspace": {},
        "inheritance": {},
        "sources": {},
        "s3": {}
      },
      "additionalProperties": false
    },
    "vars": {
      "type": "object",
      "properties": {
        "environment": {
          "type": "string"
        },
        "label_key_case": {
          "type": "string"
        },
        "label_value_case": {
          "type": "string"
        },
        "namespace": {
          "type": "string"
        },
        "stage": {
          "type": "string"
        },
        "tags": {
          "type": "object",
          "patternProperties": {
            ".+": {
              "type": [
                "boolean",
                "number",
                "string"
              ]
            }
          }
        }
      }
    }
  },
  "additionalProperties": false
}

Ah good stuff, will check that out.

@Andriy Knysh (Cloud Posse)

we have a task to add schema for all Atmos configs, and execute it in atmos validate stacks

Is that a GH issue? Got something to track?

internal task (but can be added to GH)

in the schema, we will not be checking each component’s variables as in the example above - this is not practical and not scalable. We’ll be checking the general structure and all the sections like vars, settings, env, etc.

to validate component’s vars, we already have atmos validate component with OPA policies and JSONSchema

I use the atmos validate componenet to check the component vars too, but use the schema above for dev-time type-hints.

Yeah those serve different purposes and I think that makes total sense

So largely looking for guidance as I update our architecture for our stage-specific updates

and it looks like this is going to cause problems with that regardless so we likely need a sortable filename that ensures our defaults are always first

https://github.com/cloudposse/atmos/blob/master/pkg/stack/stack_processor.go#L86

imports are processed in the order they are defined (the above code is not related to how imports are processed, it’s to output the imports field in atmos describe component command)

when using Globs, we don’t control how the files are processed since we are using a 3rd-party lib for that

so to control the imports, you have to use two diff imports, one for the defaults, and the other for de/prod/staging etc. settings

so how can i use 2 separate without it erroring if a file of that pattern doesn’t exist?

ex:

  - path: mixins/services/**/*-default
    context:
      stage: dev
      environment: ue1
  - path: mixins/services/**/*-dev
    context:
      stage: dev
      environment: ue1

if no files have -dev in them, it will error

and, technically, default is going to be for most envs anyway so there probably won’t be many overrides

if no files have -dev in them

then why do you import it? (in any language, if you import/include something that does not exist, it will error)

basically, we want to support an override pattern so when we want to override a particular stacks var we can

services/all has imports for all services

plat-ue1-prod (dev, etc) imports services/all, but that’s the default.

for instances where we want to override an SSM param’s value or ECS service ENV var, I’m thinking we have a filename-{{ .stage }}.yaml file pattern for consistency

seems a bit hacky, but this works. it would technically import the same default file twice:

import:
  - path: "mixins/services/**/*-default*"
    context:
      stage: "{{ .stage }}"
      environment: "{{ .environment }}"
  - path: "mixins/services/**/*-[default][{{ .stage }}]*"
    context:
      stage: "{{ .stage }}"
      environment: "{{ .environment }}"

the key is in **/*-[default][{{ .stage }}]* where it matches “default” (so that will never fail) and .stage (which would be dev, etc)

but yeah…that does botch the order for sure

if a file is created alphabetically below default, it will put default after that file

looks like a regular expression :)

i’ve tested a lot of patterns, but none of the fit perfectly.

going to just ensure all envs have at least one file for now.

yes, we know that. The command was not updated after the Go templates were introduced

Oh ok. I disabled it on CI since we’re already running affected anyway.

will update the command asap

No. Not directly.

This isn’t a future of atmos at this time.

Generally you would put the whitelist in my example in the same waf-acl component?

So, the only pure terraform way to do it is via the remote-state pattern that is used through the components in cloudposse/aws-terraform-components .

Generally you would put the whitelist in my example in the same waf-acl component? I do it the way you have in your example for most IPSets for WAFs. However, I also do it all within one component if the IPSets only is used by a single WAF.

Ok make sense, I like the way I did it I just was not sure how to pass the ARN to create the rule

My waf component is custom. If I pass a reference to a IPSet or another WAF object type, I do a data lookup within the component to get the arn.

Right, I think we just vendored the cloudposse one. Right now the rules are in yaml

ip_set_reference_statement_rules:
          - name: IP-Whitelist
            action: allow
            priority: 1

            statement:
              arn:

            visibility_config:
              cloudwatch_metrics_enabled: true
              metric_name: IP-Whitelist
              sampled_requests_enabled: true

Yep. You can modify so to check for <waf-object-type>.statement.name instead. If arn is not defined but name is, then get the ARN via a lookup.

right good idea

that is not possible

Well, it depends.

We just use YAML, so depending on how you want to reference it, you can use YAML anchors.

Please note, that anchors don’t work across YAML documents. Not a limitation of atmos, but a limitation of YAML.

https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/

I need the ARN created for the IPSet

Oh, but that’s not { reference waf-whitelist here } that’s { reference some terraform output of waf-whitelist here }

This should be implemented by your components.

Terraform can easily reference the ARN (e.g. in SSM or with remote state, data lookups, etc), if your components are written properly, which is why we don’t implement it in atmos.

I think our components might be too generic

I consider it generic to write outputs to SSM

It’s a consistent, generic convention

Like in case of the WAF, the rules are currently in yaml but they should probably be in terraform?

Aha I see

you could create and outputs/inputs module that do that for you to bridge the gap

Well, No, the rules should be in YAML

So, what you want is the corresponding rule ARN that was created for each of the inputs. Perhaps this should be in the outputs of the component.

Would it be worth adding IP support in the cloudposse waf component

instead of just the ARN an ipset

I think the easiest thing would be to propose what the change would look like in YAML or outputs

Then we can advise

Right now it looks like:

ip_set_reference_statement_rules:
          - name: IP-Whitelist
            action: allow
            priority: 1

            statement:
              arn: <SOME ARN>

            visibility_config:
              cloudwatch_metrics_enabled: true
              metric_name: IP-Whitelist
              sampled_requests_enabled: true

We could add:

ip_set_reference_statement_rules:
          - name: IP-Whitelist
            action: allow
            priority: 1

            statement:
              ips:
                addresses: ["17.0.0.0/8"]

            visibility_config:
              cloudwatch_metrics_enabled: true
              metric_name: IP-Whitelist
              sampled_requests_enabled: true

Yes, something like that makes sense to me. @Andriy Knysh (Cloud Posse) who would be best to chime in from our side?

(naming is bad, but its the idea)

the resource need the ARN of the IPSet https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl.html#ip_set_reference_statement-block

The idea would be to create it dynamically

at least in the current TF odule, we can’t use the IPs https://github.com/cloudposse/terraform-aws-waf/blob/main/rules.tf#L93

Yeah I propose to code the support for it!

If you guys think it makes sense

we deal with cases like that all the time. With Atmos, it’s not supported in YAML directly, but using the remote-state module. In YAML, we provide a component selector which consists of the component name, environment, tenant, stage (if the component is in a diff stack)

Yeah I propose to code the support for it!

you can open a PR for https://github.com/cloudposse/terraform-aws-waf and add code to get the IPSet ARN using terraform, yes

I dont think we are on the same page, basically in WAF rules are created in yaml. So the idea would be for the module to create the necessary IPSet based on the input as an alternative to the ARN as input.

Basically https://sweetops.slack.com/archives/C031919U8A0/p1692216132927799?thread_ts=1692209301.581579&cid=C031919U8A0 would create an IPSet and use its ARN in the rule

We might not want to do it, we might prefer to support a name import I was just proposing

we are on the same page What I’m saying is

• Atmos does not support custom functions in YAML, and the way to do it is to add some Terraform support which can be used in YAML by Atmos

• To add that Terraform support, there are two ways of doing it

1. Use remote-state to get the outputs from the waf-whitelist component, and use its ARN in the other component

1. Update the waf module with some TF code. Then, in YAML, you could provide the ARN of the IPSet, OR a list of IP addresses. Then here in the code https://github.com/cloudposse/terraform-aws-waf/blob/main/rules.tf#L423, check if the ARN is provided, use it. If the ARN is not provided, use the outputs from ` TF data source which looks up the IPSet ARN by IP (or other parameters). The IP (or other parameters) will be specified in YAML

once that support is aded to the waf module, then in YAML you will be able to use either ARN or IP

Agreed so you dont think the waf should create the IPSet resource, just reference it

in any case, some TF support/code needs to be added to support the functionality

Agreed so you dont think the waf should create the IPSet resource, just reference it

it can. It’s yet another way of doing it

we can add IPSet resource to the module, and if you provide the inputs for it and it gets created, we can use its ARN. Or the ARN can be provided separately as it is now

Perfect, I can work on that tomorrow and push a PR

thank you

I did a prototype https://github.com/cloudposse/terraform-aws-waf/compare/main...masterpointio:terraform-aws-waf:main?expand=1, but I am having a hard time with the dynamic block because the resource arn is not evaluated at runtime, terraform evaluates it a planning time. So it spits out aws_wafv2_ip_set.default is object with no attributes , I tried a few different ways to try to force it to understand the dependency but it just doesn’t seem to like it. Have you guys seen that before?

can you open a PR so we could review it? (there are many ways to deal with the issue, but all of them are different and we deal with them on a case by case basis)

Done: https://github.com/cloudposse/terraform-aws-waf/pull/49

I did a few experiment in the commits

This one being the best: https://github.com/cloudposse/terraform-aws-waf/pull/49/commits/574aab148e331b94c033ae6657a190f4063cf6f1

Found the bug, PR ready for review https://github.com/cloudposse/terraform-aws-waf/pull/49

we’ll review it, thanks

I fixed the readme

https://atmos.tools/cli/configuration/#logs

set the log level Debug or Trace

either in atmos.yaml or export ATMOS_LOGS_LEVEL=Trace

1. Globals are not ok in any language. Even if Terraform did not throw the warnings, by using a bunch of globals you would arrive at a big mess very soon. If you have a need to use variables that are common for a few components, you can create an abstract base component (mixin) and inherit from it in your components. By inheriting from the base component, all its variables and other settings will be available in the derived components. Atmos supports many types of component inheritance, see https://atmos.tools/core-concepts/components/inheritance

1. In the TF state S3 bucket, things are stored as component/stack b/c we use the component as the workspace key prefix and the stack as the workspace. So under the key prefix, you see the component name, and inside the folder you see all the workspaces (Atmos stacks) where the component is provisioned

@Zain Zahran if you need help with Atmos configurations, let us know, we could suggest some ideas and review your code

1. Got it. I figured as much. Glad to learn this. I had taken a brief look at the atmos’ abstract mixins before. I think that could prove useful, but also definitely the components and modules should be written in a way that enforces a specific set of vars. Passing in unused things would be bad practice.
2. Understood, that makes more sense. @Andriy Knysh (Cloud Posse) sounds good I’ll let y’all know. Thanks for your help!!

We have, what looks like, runs triggering based on dependencies (as expected), but the dependencies are not showing up in Spacelift.

Is this expected based on the latest Atmos version, @Andriy Knysh (Cloud Posse)?

check that the spacelift component uses the latest version of yam-stack-config which should use the latest version of the utils provider which supports the latest version of Atmos (since it uses Atmos Go modules)

there are many things involved here, not only the latest Atmos CLI version (which is just used locally)

no, it is 1.0.0 not 1.2.0 of https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/releases

we’re on this version of the refarch: https://github.com/cloudposse/terraform-aws-components/blob/main/modules/spacelift/admin-stack/child-stacks.tf#L47

there were so many diff updates to the Spacelift modules and components by many people to add many changes (some of them breaking), I can’t say from the top of my head which version needs to be used

ok

I know that the latest version of yaml-stack-config/spacelift module needs to be used which uses the latest version of the utils provider

https://github.com/cloudposse/terraform-yaml-stack-config/releases/tag/1.5.0

yeah, we’re stuck in a weird loop of the policy triggering dependencies

we need to audit our dependencies for sure

just an example