#atmos (2024-05)

Great question! I think we could/should add some guidance to our docs.

Let’ me take a stab at that now, and present some options.

First, make sure you’re familiar with the .gitattributes file which let’s you flag certain files/paths as auto generated, which collapses them in your Pull Requests, and reduces the eye strain.

Option 1: By default, Cloud Posse (in our engagements and refarch), vendor the everything in to the repositories.

Pros

• an immutable record of components / less reliance on the repo repositories

• super easy to test changes without a “fork bomb” and ensuring “PR storm” as you update multiple repos

• super easy to diverge when you want to

• super easy to detect changes and what’s affected

• much faster than cloning all dependencies

• super easy to “grep” (search) the repo to find where something is defined

• No need to dereference a bunch of URLs just to find where something is defined

• Easier for newcomers to understand what is going on

Cons

• Reviewing PRs containing tons of vendored files sucks

• …? I struggle to see them

Option 2: Vendoring components (or anything for that fact, which is supported by atmos) can be done “Just in time”, more or less like terraform init for provider and modules.

Pros:

• only things that change or are different are in the local repo

• PRs don’t contain a bunch of duplicated files

• It’smore “DRY” (but I’d argue this is not really any more DRY than committing them. Not in practice because vendoring is completely automated) Cons

• It’s slower to run, because everything must first be downloaded

• It’s not immutable. Remote refs can change, including tags

• Remote sources can go away, or suffer transient errors

• It’s harder to understand what something is doing, when you have to dereference dozens of URLs to look at the code

• Cannot just do a “code search” (grep) through the repo to see where something is defined

• In order to determine what is affected, you have to clone everything which is slower

• If you want to test out some change, you have to fork it and create branch with your changes, then update your pinning

• If you want to diverge, you also have to fork it, or vendor it in locally

Option 3: Hybrid - might make sense in some cirucmstances. Vendor & commit 3rd-party dependencies you do not control, and for everything else permit remote dependencies and vendor JIT.

I didn’t even give thought to someone making a cheeky ref update to a tag

Alright, this is supremely helpful, thanks a ton for the analysis. I’ll bring these points back to my team for discussion.

Great, would love to hear the feedback.

Also, not sure if this is just a coincidence, but check out this thread

https://sweetops.slack.com/archives/C031919U8A0/p1714484118739959

I really like the point of this making the configuration set immutable as well. We’re locked in with exactly what we have committed to the repository.

A lot of our “best practices” are inspired by the tools we’ve cut our teeth on. In this case, our recommendation is inspired by ArgoCD (~Flux) is the inspiration - at least the way we’ve implemented and used it.

Yeah, I was working with vendoring in different versions today to their own component path (component/1.2.1, component/1.2.2) and it took me a moment to realize this was changing the workspace prefix in the S3 bucket where the state for the stack was being stored.

it took me a moment to realize this was changing the workspace prefix in the S3 bucket where the state for the stack was being stored.

This is configurable

So now am vendoring in different versions to different environment paths to keep the component names the same as things are promoted. (1.2.1 => component/prod, 1.2.2 => component/dev)

This is a great way to support multiple concurrent versions, just make sure you configure the workspace key prefix in the atmos settings for the component.

So now am vendoring in different versions to different environment paths to keep the component names the same as things are promote This is another way. Think of them as release channels (e.g. alpha, beta, stable, etc)

Either way, its’ great to fix the component path in the state bucket, so you don’t encounter problems if you reorganize how you store components on disk.

So then my configs look sort of like,

components:
  terraform:
    egress_vpc/vpc:
      metadata:
        component: vendored/networking/vpc/dev
        inherits:
          - vpc/dev

    egress_vpc:
      metadata:
        component: vendored/networking/egress_vpc/dev
      settings:
        depends_on:
          1:
            component: egress_vpc/vpc
      vars:
        enabled: true
        vpc_configuration: egress_vpc/vpc

@Andriy Knysh (Cloud Posse) do we have docs on how to “pin” the component path in the state backend?

Note, I would personally invert the paths for release channels.

• vendored/dev/networking….

• vendored/prod/…. Since over time, some components might get deprecated and removed, or others never progressing past dev.

Yeah, that’s a very good point and something we’re still working through.

We also want to get to the point where we have “release waves”…..so, release a change to a “canary” group, and then carry out groupA, groupB, groupC, etc….

Which, version numbers would honestly help with that a bit more. How could I pin the workspace key prefix for a client if I did have the component version in the path?

Ahh, looks like towards the bottom of the page here: https://atmos.tools/quick-start/configure-terraform-backend/

Yea, I think our example could be better….

But this:

I can have my cake and eat it too. Delicious.

Yeah, just hugely beneficial, thank you so much for the help.

Our pleasure!

I have the remote-state stuff working that you were able to guide me through the other week…many thanks as well there, I think that’s really going to help us level up our IaC game.

Yes, exactly one of the use cases for custom sub commands. This way you can call it with atmos to make it feel more integrated, documented (e.g. with atmos help). You can then automate it with atmos workflows. Alternatively you can skip the sub command and only use a workflow too. Up to you.

You can also access the stack configuration from custom commands

any examples on how to access stack configuration through custom commands?

Have you seen this? https://atmos.tools/core-concepts/custom-commands/#show-component-info

Instead of running echo, just run your python script instead.

@Gabriela Campana (Cloud Posse) we need a task to add example command invocation for each example custom command. cc @Andriy Knysh (Cloud Posse)

E.g. the docs only show how to define it, not how to call it. Of course, it can be inferred, but “as a new user I want to quickly see how to run the command because it will help me connect the dots”

@Kubhera we’ll improve the docs. For an example on “any examples on how to access stack configuration through custom commands?”, please see this custom command (as Erik mentioned above):

or this one https://atmos.tools/core-concepts/custom-commands/#set-eks-cluster

DEV-2129 : Add example command invocation for each example custom command

Or would gomplate, if a random function exists, generate a new random string upon each atmos run?

I suppose uuid does exist here

https://docs.gomplate.ca/functions/uuid/

Unfortunately cannot effectively use the uuid function in gomplate for this because it will cause permadrift

Just to confirm, aesthetically the automatic checksum hash is not option you are considering? …because it solves exactly this problem with the bucket length, but will chop and checksum the tail end

Sorry, I didn’t read carefully. You are not asking about controlling the length.

Regarding the post you linked, we discussed this on office hours. @matt mentioned that Jeff Barr in response to the post, said AWS is not going to charge for unauthorized requests and they are making changes for that. To be clear, they refunded the person who made the unfortunate discovery, and are making more permanent fixes to prevent this in the future.

Oh i didn’t realize that jeff bar responded with that. That’s great news. Thanks Erik. Then i suppose it’s a non issue once they makes changes for it

Is there a doc available that points to jeff barrs response? That may calm some nerves

Found it

https://twitter.com/jeffbarr/status/1785386554372042890

https://youtu.be/QNcSZwHgBEg?t=1441

Haha, @Kubhera my ears are ringing from the number of times this is coming up in questions.

TL;DR: we don’t support this today

Design-wise, we’ve really wanted to stick with vanilla Terraform as much as possible. Our argument is that the more we stick into atmos that depend on Terraform behaviors, the more end-users are “vendor locked” into using it; also, we don’t want to re-invent “terraform” inside of Atmos and YAML. We want to use Terraform for what it’s good at, and atmos for where it’s lacking: configuration. However, that’s a self-imposed constraint, and it seems like one users are not so concerned about and a feature frequently asked for, albeit for different use cases. This channel has many such requests.

So we’ve been discussing a way of doing that. It’s tricky because there are a dozen types of backends. We’ll not want to reimplement the functionality in atmos to read the raw state but instead make terraform output a datasource in atmos.

We’ve recently introduced datasources into atmos, so this will slot in nicely there. Also, take a look at the other data sources, and see if one will work for you.

https://atmos.tools/core-concepts/stacks/templating#configuring-templating-in-atmos-stack-manifests

Here are the data sources we support today: https://docs.gomplate.ca/datasources/

So if you write you outputs to SSM, for example, atmos can read those.

Alternatively, since you’re using a workflow, you can do this workaround.
File file Files can be read in any of the supported formats, including by piping through standard input (Stdin). Directories are also supported.

So in one of your steps, save the terraform output as a json blob to a file.

Then define a datasource to read that file. All the values in the file will be available then in the stack configuration.

As always, atmos is a swiss army knife, so there’s away to do it. Just maybe not optimized yet for your use-case.

What about switching from terraform remote state to aws data sources instead for each component?

This way we don’t have to depend on the output of another component to deploy a dependent component.

Thoughts on this?