I think we would like to have more, haven’t standardized the interface

Could you elaborate on what you might want to integrate?

I’m considering integrating different tooling that would benefit from Atmos’ YAML deep merging. Right now, I’d like to add a new component type to atmos.yaml. For example:

components:
  ansible:
     base_path: "components/ansible"

I’d like to use the atmos describe stacks and components functionality to create my own version of tfvars.json using a custom wrapper script that is harnessed by a custom Atmos command. I ran into a schema failure because it couldn’t validate the structure, so I added new Ansible component definitions to stacks/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json , created a catalog default for the component, and everything validates but it can’t find the Ansible components. So I’m just looking into if there are any better ways to handle this edge case

Ah yes, I have been thinking about this some and how we could add support for custom component types. That’s interesting if atmos is not loading those other custom components. I don’t think we have tested that, and also the implication on schemas. I think to support multiple custom types, we would want to extend support for multiple json schemas that get deep merged

I think we would want to register custom types in atmos.yaml, and provide some common conventions like base_path, command, and so forth. This is also where we could define the schema validation to use for the component type.

It doesn’t work well, unless you have a role that can access all backends

So we are implementing a better long term solution

https://github.com/cloudposse/atmos/pull/874

Using this, you can store outputs outside of the terraform state, and read from the store

This simplifies the permissions model

ETA for this is end of next week

@Erik Osterman (Cloud Posse) neat, thanks for the info!

So this came up recently in separate thread for a different purpose

https://sweetops.slack.com/archives/C031919U8A0/p1731978354315109

We need to add a feature flag to toggle usage of workspaces. Relatively easy fix for us to make.

It would be great!

Should I open an issue on github to track the issue?

Sure, let’s do that, so we can notify you when it’s done

I think we can get to it in the next 2 weeks or so.

Sounds good to me, I will create the issue tomorrow then

https://github.com/cloudposse/atmos/issues/915

@Jeremy White (Cloud Posse)

(note this is an refarch topic, not atmos)

So, I’d just like to clarify something. Since the cloudfront cdn module can have multiple buckets involved, you are seeking to update the grants on the origin bucket? Or on the logs bucket? or on both?

On the logs bucket.

To be honest, I realized that for the logs bucket I only need the external account associated with the canonical ID for log delivery in AWS. But still wanted to know if it’s possible to have the current AWS account have permissions added as well

there doesn’t appear to be a way to update the grants for the logs bucket in the module. It’s set to only have one: https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/blob/main/main.tf#L435

to be clear here, you can always try adding more grants outside the module and see if terraform removes them. That is, you might find that additional ACL rules will not be scrubbed when terraform plans for drift/change. I only say that because it seems there’s a dynamic block on the ACL rules which might indicate additional rules will be ignored: https://github.com/cloudposse/terraform-aws-s3-bucket/blob/main/main.tf#L180

(note, the acl_grants local is formed on the var grants)

if it plays nice, you could just add more grants after running the module to establish the base ACL

Oh right, I recall we had discussed adding support for -- to custom commands.

This would be a good escape hatch.

Are you familiar with the -- convention in CLI commands?

I am not sure, for me, -- is for long arguments?

Aha, so when it’s not followed by any argument signifies end of arguments. That means that everything after it is to be interpreted as a literal string and not as flags.

I’ve created a task so that we can implement this. I don’t think it’s a lot of work and we had meant to do that before. I think we might get to it in the next few weeks.

Hummmm

My use case is to have a custom command atmos ansible run that is calling ansible-playbook

I would like to be able to add for example the argument -vvvvvvv to be passed to ansible-playbook without defining it in the custom CLI

Or for example -l flags to be sent too without specifying arguments

Because at the moment, every extra arguments given to the custom CLI that is not specify ends up in an error and cannot be further processed

Does it make more sense?

Yes, so if we implement this, you could do:

atmos ansible run —- -vvvvvvv

And then there would be an ENV that contains -vvvvvvv

So you could pass that literal string to ansible or parse it yourself

Note, it would contain all characters after the double hyphen

It would be fine!

Thanks!

Great! for now, you can define an env like ANSIBLE_ARGS and use that until we support it.

then use ANSIBLE_ARGS in your custom command

Indeed, that should work

Thanks for the workaround!

Y’all’s development pace is wild! Keep up the great work and great features and fixes

This was one of @Dan Miller (Cloud Posse)’s first contributions to the atmos core! Nice work

I’ve got 3 more queued

Were your thoughts to move it to atmos-dev?

No, it was too noisy in there. Maybe atmos-releases or atmos-notifications

(We had something similar in atmos-dev before we turned it off)

in your atmos.yaml, do you have this

components:
  terraform:
    # Can also be set using 'ATMOS_COMPONENTS_TERRAFORM_INIT_RUN_RECONFIGURE' ENV var, or '--init-run-reconfigure' command-line argument
    init_run_reconfigure: true

also, you can always execute atmos terraform shell msk -s staging-data-msk and then inside the shell use the native terraform commands

Ooo…okay.

https://atmos.tools/cli/commands/terraform/shell/

Shell doesn’t work either:

 atmos terraform shell msk -s staging-data-msk         
exit status 1

Error: Backend configuration changed

A change in the backend configuration has been detected, which may require
migrating existing state.

If you wish to attempt automatic migration of the state, use "terraform init
-migrate-state".
If you wish to store the current configuration with no changes to the state,
use "terraform init -reconfigure".

do you have init_run_reconfigure: true?

Yep:

components:
  terraform:
    base_path: "components/terraform"
    apply_auto_approve: false
    deploy_run_init: true
    init_run_reconfigure: true
    auto_generate_backend_file: true

try

atmos terraform init msk -s staging-data-msk -- -reconfigure

That is the first thing I tried. You can see it in the initial code I pasted.

Ah. Crumb. That says plan. One sec.

are you using atmos.Component or !terraform.output functions in your stacks?

Yep.

In this one, specifically !terraform.output.

ok, I prob know what the issue is - the !terraform.output functions does not take into account init_run_reconfigure: true (it calls a Hashi lib to execute terraform init) we’ll test it and release a fix in a day or two

for now, you can do the following:

the issue with the backend is not for this component msk -s staging-data-msk, but for the component in the stack in the !terraform.output func

so you can do

cd <path to that component>
terraform init -reconfigure

Makes sense. Thanks for the help!

let me know if it fixes the issue

@Gabriela Campana (Cloud Posse) please create a task for Atmos:

In atmos.Component and !terraform.output functions, detect the init_run_reconfigure: true setting in atmos yaml and execute terraform init -reconfigure

Turns out this was a bug with a separate module not providing expected outputs. Thanks for helping me debug, though!

(I found this by switching to atmos.Component values, which at least gave slightly more error:

$ atmos terraform init msk -s staging-data-msk                       
template: all-atmos-sections:205:35: executing "all-atmos-sections" at <atmos.Component>: error calling Component: exit status 1

Error: Backend configuration changed

A change in the backend configuration has been detected, which may require
migrating existing state.

If you wish to attempt automatic migration of the state, use "terraform init
-migrate-state".
If you wish to store the current configuration with no changes to the state,
use "terraform init -reconfigure".

But I’d say that surfacing the error more clearly would be nice. (I know that surfacing errors through the stack is a pain)

Turns out this was a bug with a separate module not providing expected outputs

the component in the !terraform.output func did not have the expected outputs?

(we def need to improve error handling here)

Yes. The !terraform.ouptut/atmos.Component was missing outputs. Adding those missing outputs (fortunately we control that module) fixed the error.

@Andriy Knysh (Cloud Posse) DEV-2921: In atmos.Component and !terraform.output functions, detect the init_run_reconfigure: true setting in atmos yaml and execute terraform init -reconfigure

@Andriy Knysh (Cloud Posse)

You should not use YAML as a replacement for HCL

https://atmos.tools/best-practices/components#split-components-by-lifecycle

See the warning there

Do you mean writing a component that uses multiple child modules?

or using “child modules” as components?

The former, writing a component that uses multiple child modules! I saw those notes in the docs, but was a little confused! It seems like its a balancing act.

I was planning to organize each of our different product surfaces into their own components so they can be managed/deployed independently.

I’m not sure there’d be a case in which we’d have a component that was general enough to re-use, but certainly I could foresee HCL modules that we’d reuse (e.g. database, internal api servers, storage buckets).

I noticed that a lot of the CloudPosse components are for specific pieces of infrastructure, for example aws-vpc or aws-aurora-postgres. Something like a Postgres instance is only used by an API server feels like it’d be too small to warrant splitting it into its own component, so we should just let Terraform handle the dependencies rather than splitting it into two lifecycles.

Thanks for your response, it clears a lot up!

But the lifecycle of the API and the database are almost entirely disjoint

That’s why they we argue they should be separate

True! Perhaps the stacks should be organized by product. Do end-users typically solely the utilize output atmos describe affected as opposed using workflows to apply/plan each component in a stack?

Describe affected was implemented mostly for our GitHub actions

We also plan to improve the command line ability to do the same, but it’s not available today

We have a task assigned to implement:

atmos terraform apply —-affected

This would under the hood run describe affected and iterate over each component in dependency order and apply it.

I also searched for a component with subnet in its name and no luck

i was imagining a component wrapper for this module https://github.com/cloudposse/terraform-aws-dynamic-subnets

or would you folks be open to setting up a for_each for the cloudposse/dynamic-subnets/aws module directly in the vpc component ?

https://github.com/cloudposse-terraform-components/aws-vpc/blob/385e26c4f8f7137efcf9dd1d46d05e166e9854ae/src/main.tf#L140-L142

What do you folks think?

You should/could be able to use this directly: https://github.com/cloudposse/terraform-aws-dynamic-subnets

since atmos supports provider & backend generation

https://atmos.tools/core-concepts/vendor/component-manifest/#vendoring-modules-as-components

oh very cool

and you can use vendor manifests the same way. doesn’t have to be a component manifest.

If i vendored the module as a component, I’d still need to hardcode the vpc unless i either

• modified the vendored module as a component to read from the remote state

• Or modified the dynamic-subnets module to allow for a data source to use tags to collect the vpc id, then vendor it in as a component, and then there is no hard coding

Heres a draft pr that would make it easier to vendor dynamic-subnets as a component without hard coding

https://github.com/cloudposse/terraform-aws-dynamic-subnets/pull/219

Or you can use !terraform.output to retrieve the vpc id

https://atmos.tools/core-concepts/stacks/yaml-functions/terraform.output/

Holy moly, that’s great

So something like this?

    vpc/example/dynamic-subnets/db:
      metadata:
        component: dynamic-subnets
      vars:
        vpc_id: !terraform.output vpc/example vpc_id

Any specific line items in the docs we can share to prevent this ?

Were they using the overrides pattern?

Do you mean using [_override.tf](http://_override.tf)?

https://atmos.tools/best-practices/terraform/#use-terraform-overrides-to-extend-monkey-patch-vendored-components

@RB does this count? https://atmos.tools/best-practices/components#use-parameterization-but-avoid-over-parameterization

No they were updating the component directly after downstreaming

Long and short, hard coding subnets entirely defeats the purpose of reusable components. I don’t think we explicitly point out not to do it.

Ah ok

If I were to contribute it, which page would it be best to call that out on ?

Component best practices?

Yes, please do!

Can you share the yaml instead?

Also, recommend using !terraform.output instead of {{ atmos.Component }}

https://atmos.tools/core-concepts/stacks/templates/functions/atmos.Component/#handling-outputs-containing-maps-or-lists

atmos.Component literally injects text into the document, not an object.

So if your text is not well formatted for YAML you’ll get these kinds of errors probably.

Using !terraform.output injects a well structured object automatically, avoiding the risks

Sure, let me get the YAML. Just a moment. The value I’m trying to get is a a string, not an object.

ok

Attempt #1 was:

components:
  terraform:
    application:
      metadata:
        component: organization/application
      vars:
        subnet: '{{ (atmos.Component "vpc" .stack).outputs.public.subnet-id }}'

Then I tried:

components:
  terraform:
    application:
      metadata:
        component: organization/application
      vars:
        subnet: '{{ index (atmos.Component "vpc" .stack) "outputs" "public" "subnet-id" }}'

I attempted to use !terraform.output as you suggested however the function isn’t variadic so attempting to do !terraform.output vpc public subnet-id gives invalid number of arguments in the Atmos YAML function: !terraform.output vpc public subnet-id

@Kyle Decot the correct expression would be (if oyu have the variable public which is a map with the key subnet-id

{{ (atmos.Component "vpc" .stack).outputs.public.subnet-id }}

however …

@Andriy Knysh (Cloud Posse) what about the !template.output syntax

use index function

{{index .person "last-name"}}

for !terraform.output, this is not correct syntax

!terraform.output vpc public subnet-id 

the function accepts a component and output name (2 params), or a component, stack,, and output name (3 params)

try this

'{{ index ((atmos.Component "vpc" .stack).outputs.public) "subnet-id" }}'

Yes! Using a combination of index and atmos.Component seems to do the trick.

@Andriy Knysh (Cloud Posse) is the limitation with !terraform.output that you cannot retrieve a attribute from the output?

@Andriy Knysh (Cloud Posse) bumping this up

we are finishing up some changes to improve the !terraform.output function, will be in the next release

yes, it applies only when using the remote-state module

and you specify the backend role in backend and remote state backend role in remote_state_backend

right, gotcha - It’s what I thought but just wanted to check

when executing !terraform.output, it just executes terrafrom output using the already assumed role or configured in the TF module - so this function does not know anything about roles and depends on the role config in the corresponding TF component (or, if not configured in the TF component, the caller role will be used by TF)

@David Elston a correction: the !terraform.output YAML function (and atmos.Component template function) does generate the backend file before executing terraform output

so the configured backend will be used

terraform:
  backend_type: s3
  backend:
    s3:
      acl: "bucket-owner-full-control"
      encrypt: true
      bucket: "your-s3-bucket-name"
      dynamodb_table: "your-dynamodb-table-name"
      key: "terraform.tfstate"
      region: "your-aws-region"
      role_arn: "arn:aws:iam::xxxxxxxx:role/terraform-backend-read-write"

We’re investigating why a binary was not created with this release. Stay tuned.

cc @Igor Rodionov

This release will be skipped as there was a problem in main at this commit. Please use 1.146.1 instead.

I’ve tried setting the various log level parameters (via atmos.yaml, environment variable, and --logs-level).

I’m attempting to vendor from a private OCI registry, so my guess would be that go-containerregistry is having trouble getting credentials

Dry run shows:

✓ cloud-sql-instance (latest)
                                                    
  Done! Dry run completed. No components vendored.  

I am a bit confused. With a dry run nothing should be vendored.

What happens with out dry run?

This is all the information I get with atmos vendor pull

$ atmos vendor pull
x cloud-sql-instance (latest)
                                                         
  Vendored 0 components. Failed to vendor 1 components.

Aha, ok - that’s helpful

Is this a public oci image?

It’s private- my brief look at go-containerregistry made it seem like my Docker-configured credHelpers should be used to authenticate

My guess is it’s Auth related and I don’t know if we’ve tested on private repo

That’d be my guess, too. Should these installPkgMsgs end up in the logs? https://github.com/cloudposse/atmos/blob/main/internal/exec/vendor_model.go#L289

If you have a chance to take a look and fix it, we’ll gladly merge the PR :-)

Otherwise I will add a task to fix it, but not sure when we can get to it

The error message is only printed if we’re moving on to install another component. In my case I only have one vendored component, so the completion message is printed instead

I’m also not sure I understand why all the logging is guarded by !isTTY checks. I would think we’d want logs even if we’re displaying the terminal ui?

here’s a pr- https://github.com/cloudposse/atmos/pull/936

Hrmmm, yes, that is probably because it messes up the UI, but that just means we need to find a better way to handle it.

Maybe we should force a log file, if UI, and direct the user to the log file in the event of errors

Or in the UI we should print it below the component before proceeding to the next component

{
  "terraform": {
    "required_providers": {
      { "test": {
        "source": "test"
      }
    }
  }
}

As it is done now for the backend block

Not at this time. Providers and Backends. Can you describe your use case?

Is it possible to generate code from atmos itself, we need an analog of the generate function of terragrunt to override the configuration?

@Petr Dondukov this is definitely a common request, especially from users migrating from other toolchains. I wouldn’t rule it out, and we’ve definitely losened our stance on this (we used to be starkly against it), but we have added some conventional ways for providers and backends. I think if we were to implement generation, we do not want to take the same approach as for backends and providers. We do not want to replace HCL with YAML.

Could you help share some of your use-cases?

Note, we have over 165 components that we’ve written and the only code generation we rely on is provider generation and backend generation. Everything else we’ve accomplished with atmos, without code generation.

https://github.com/cloudposse-terraform-components

Also, if we showed you how you could use terragrunt with atmos, would that be a good stop-gap?

Yes, I realize we’re using terragrunt right now and our infrastructure is complex and we’d like to simplify it. I am currently looking for a way to redefine provider versions (or define from scratch), as we have historically had a parent and child module approach and have had to completely remove the provider block from hundreds of our terraform modules. We generate the provider block via terragrunt and that’s how we have the infrastructure built, not the best solution of course. Also, even without terragrunt we over-utilize child modules in other parent modules, that’s why it’s necessary for us.

Also, I found that we need to define in terraform modules the configuration for name_template for atmos to work, or I’m throwing this through the cloudposse/context provider for the whole stack, like this: name_template: "{{{.providers.context.values.tenant}}-{{{.providers.context.values.environment}}-{{{.providers.context.values.resource.name}}-{{{.providers.context.values.region}}-{{{{.providers.context.values.resource.number}}}". But without specifying the source = cloudposse/context parameter in the required_providers block, terraform gives an error that it can’t find the context provider in the registry. An option would be to describe this block in each terraform module, but we want to retain versatility.

In general, the first one is not very critical. Now I’m more interested in how to throw a source for the cloudposse/context provider without fixing the terraform modules themselves.

@Erik Osterman (Cloud Posse)

Provider is definitely in the registry. Have you seen this demo? https://github.com/cloudposse/atmos/tree/main/examples/demo-context

Yes, I understand. However, in order to use the context provider we need to prescribe it in the terraform module, to pass the source = "cloudposse/context" parameter. We don’t want to add this to the terraform module itself, so as not to lose versatility, but to make do with the atmos tool.

We don’t want to add this to the terraform module itself, so as not to lose versatility, but to make do with the atmos tool. Just curious, is this because this context provider is perceived an atmos feature?

If the resources in the module do not use the resources provided by the context provider, then there’s no reason to use the context provider.

The context provider is a modern alternative to the terraform-null-label module (or any naming module, as others exist now)

Re: name_template this is how the “slug” or “id” is generated for atmos components, so you can refer to it with a simple, programmatically consistent name.

So this setting:

name_template: "{{.providers.context.values.tenant}}-{{.providers.context.values.environment}}-{{.providers.context.values.resource.name}}-{{.providers.context.values.region}}-{{.providers.context.values.resource.number}}"

you can configure that based on any vars instead.

So let’s say all of your root modules defined “name”, “location”, “account”, then you can define a name template liek this:

name_template: "{{.vars.account}}-{{{.vars.location}}-{{.vars.name}}-{{.component_name}}"

This assumes every root module has a HCL variable defined for account, location, name.

Not sure if i’m reading between the lines correctly :smiley: it sounds like maybe the concern was needing to use .providers.context and that’s not needed.

Unfortunately, we don’t use these variables. And if we write them at the stack level, terraform writes an error of unused variables

Ok, so I think we’re getting somewhere

So basically, you need to be able to pull these from somewhere else, that’s neither providers nor vars.

I think there’s a solution for this. The settings field of a component is a free-form map.

You should be able to use .settings instead of .providers

name_template: "{{.settings.context.account}}-{{{.settings.context.location}}-{{.settings.context.name}}-{{.component_name}}"

@Andriy Knysh (Cloud Posse) can you confirm

and by the way, .account and .location and .name were totally arbitrary. You can establish whatever convention you like.

yes, the settings section is a free-form map, so it can be used to “store” data/metadata and then use it in other configs. Since settings is a free-form map, you can use any sunsection under it (e.g. settings.context)

and in the name_template you can use any Atmos section, not only vars or providers

For the Go template tokens, and you can use any Atmos sections (e.g. vars, providers, settings)

so this is perfectly fine

name_template: "{{.settings.context.account}}-{{{.settings.context.location}}-{{.settings.context.name}}"

settings:
  context:
    account: xxxx
    location: yyyy
    name: zzzz

(it will not define any new vars and will not send them to the TF components)

@Petr Dondukov ^

let us know if we can help with this or you need more details

Thanx a lot! That is exactly what I need!

Looks like templating is disabled in your atmos.yaml. It’s disabled by default

I will get back to this later. In general, I recommend refactoring questions into individual messages as they are easier to answer individually with threads for each topic.

thanks Erik, that is right, I had a typo I had this in my config:

tempates:
  settings:
    enabled: true

Look into multiple inheritance and abstract components.

For detecting changes, we have atmos describe affected, which compiles the fully deep merged configurations, and then compare the them between two git refs.

Right now, this returns a large amount of json data and is predominantly concerned with supporting our GitHub actions. We intend to present a much simpler command with human friendly output :-)

I expect that we’ll get to it in Q1

I can see why explicitly referencing variables the way you are in Terragrunt is convenient for the use-case you described. I think what would be helpful is if we wrote up some docs specifically for users coming from Terragrunt because atmos is quite a different way of approach the problem. Terramate, for example, is very similar to Terragrunt and is simpler migration for Terragrunt users. Anyways, I will have to pause here and follow up through out the day.

Thanks Erik again for the answer. However, as far as I understand (but I might be wrong here) the diff of the deep merged config does not help, my concern is that we just pass all the vars that are merged in any way (through inheritance or import, etc.) to the component, but some of them might be not actually used by the component, they are there for others only. And maybe in such situation the imports and inheritance were not used properly, but you cannot guarantee that everyone will do such good job, so in my opinion explicit passing of inputs for the root modules is always better (not instead of the inheritance and imports, but together with them).

So as an example, I have var a, b and c defined somehow. component1 might be using a and b, component 2 might be using a, c and there might be a 3rd one not using any of these at all, but as I understand all of these vars will be passed to all 3 components in the stack. So how will I know (other then running a plan) if I change var c which component will be affected? And I understand that in many cases this is not going to be a problem, but once you have a complex system with a lots of components (20+ is a lot already I think) variables and environments, this can make the difference between having a big ball of mud and a system which you can actually understand.

Anyway it might be just some additional convention I might use, I would just like to see the options and how exactly this was intended to be used :)

running 80+ components, and a large number of atmos stacks:

• most variables are nicely scoped per component, so unless you put stuff in _defaults, it should not be a real issue

• we are keeping the atmos describe stacks (postprocessed by yq) in our repo, as well as the tfvars. That way we have full visibility on change impact. (we use the files also for various other reasons, incl more advanced opa policies)

Thanks Hans for the answer. I think the might be the key, to have the variables scoped to the component, thanks for pointing it out. I will still need to figure out exactly how to organize things to my liking, but at least I think I am starting to understand the idea.

Yes, what @Hans D points out is the right convention. I guess we haven’t really documented that somewhere. In a related way, I think it would be neat if atmos had an optional mode to validate variables passed to terraform are acceptable by the component. This would reduce the error prone nature.

@Andriy Knysh (Cloud Posse) would that be complicated? I know you already load the variables of the HCL component somewhere in atmos.

@Dave in Atmos, all variables are scoped to a particular component (inheritance or imports, should not matter)- at least this is how it should be configured. If you are using global vars (e.g. scoped to the entire stack), then yes, all of them will be sent to all TF components used in the stack - but it’s not how it should be configured - no globals (except the context variables like stage, environment etc. which are used in all (Cloud Posse’s ) components

you can share your config, we can take a look

@Andriy Knysh (Cloud Posse) not scoped to a particular it the global vars are used; then inherited by all components

So as an example, I have var a, b and c defined somehow. component1 might be using a and b, component 2 might be using a, c and there might be a 3rd one not using any of these at all, but as I understand all of these vars will be passed to all 3 components in the stack

i might be not understanding the problem completely, but it sounds like those vars are “global”

the question prob is: how do we prevent declaring vars in Atmos manifests for Atmos components that are not used by the corresponding TF components. we can add this to Atmos (it already has all the information about the configured vars in the Atmos component and all the vars used by the TF component/module)

we are going to add the validation to atmos validate stacks and atmos validate component commands: if a variable was configured in Atmos stack manifests, but it’s not used by the TF component(s), show an error will be in one of the new releases

@Dave ^

that sounds pretty nice, and thanks a lot for explaining all this

I think @Matt Calhoun is working on some updates to the docs

Everything related to sharing data, including remote state will be consolidated in the share data section

We should probably have a high level overview since there are different methods with different trade offs

so this is happening when the vpc-flow-log-bucket is already deployed, and I am trying to create a plan for the vpc component.

Just to confirm, are you trying the advanced quick start?

@Dave please review this doc https://atmos.tools/core-concepts/components/terraform/remote-state/#atmos-configuration

if the vpc-flow-log-bucket is already deployed, vpc uses the remote-state to get outputs from the vpc-flow-log-bucket component, which uses the utils provider.

and since terraform executes the provider code from the component/module directory, the provider can’t find atmos.yaml there - it should be in one of the known locations (or provided via the ENV vars)

Yes, I am trying to follow the advanced quick start, I just have it a little simplified, because I only have the stage in the name pattern (since I have a single level hierarchy).

unfortunately I did not find anything new in that guide, I think I followed everything described over there, but I still get this error

I already had the env vars set (without them the error was completely different)

Are you using absolute or relative paths?

(Also, this particular rough edge will be addressed in a forth coming update to atmos; Pr is in progress)

Fair question. It would be in the atmos schema.json

https://atmos.tools/schemas/atmos/atmos-manifest/1.0/atmos-manifest.json

We should have this list though. Let me see what we can do.

Thanks for the info @Erik Osterman (Cloud Posse)!

The use case is that I want to add some info to stacks but I want to be sure doing so won’t impact Atmos. Is there a designated path for such custom info?