#atmos (2023-10)

It’s not practical. Since you also need to be for in the component, it’s not as simple as bumping the version in the component.yaml

the vendor pull can be done via a github action

Also, the other problem we had with it is every single component.yaml is updated then when a new release is made even if no tf files were affected

should not be much of a problem when the grouping functionality can be used. Fro the human doing a visual check re the version in component.yaml and the repo version this would even be better (less explaining to do as why a lower version != behind)

I welcome you to try the experiment. I remain skeptical, or that different teams may have different preferences.

the latter: fully agree. Will see if I can make it work

@Erik Osterman (Cloud Posse) as i see it now, the component updater also updates a component if it is not affected by the repo version bump

I can explain on next office hours how it works. We are also working on a better way to distribute individually versioned components as proper artifacts, that will replace the current method

TL;DR, the way it’s working right now is that: it’s not if the component changed in the commit of that version, it’s relative to your local copy. If your local copy differs from the component in that version, it will be updated. With out using tags for each component in the monorepo there would be no efficient way to do it. Hence why we are moving to a registry model using OCI.

if you try setting log level to a higher setting like Warning does that help accomplish what you wanted?

Given “Atmos will not log any messages (note that this does not prevent other tools like Terraform from logging).” I doubt that will be the case, but I’ll try

you can export the ENV vars in .bashrc (or similar file in other shells)

https://devconnected.com/set-environment-variable-bash-how-to/

this is similar to what geodesic does

I use this in my .envrc in the root of my repo with direnv to auto load this

export ATMOS_CLI_CONFIG_PATH=$(git rev-parse --show-toplevel)
export ATMOS_BASE_PATH=$(git rev-parse --show-toplevel)

Went with direnv , satisfied my need for a clean solution.

atmos/examples/complete is actually to test ALL Atmos functionality (and also to test bad/wrong configs, not only the correct ones), we agree it’s a bad name for the folder. We are going to create another folder with a minimal setup to configure a few Atmos components and stacks, and make it easy to understand and provision

this is already described in https://atmos.tools/category/quick-start (but w/o actually having the correct config itself, which we are going to add)

My structure is:

.
├── atmos.yaml
├── components
│  └── terraform
│     └── infra
│        ├── vpc
│        │  └── component.yaml
│        └── vpc-flow-logs-bucket
│           └── component.yaml
└── stacks
   └── orgs
      └── cp
         ├── _defaults.yaml
         └── tenant1
            ├── _defaults.yaml
            └── dev
               └── _defaults.yaml

My atmos.yaml:

base_path: "."

components:
  terraform:
    base_path: "components/terraform"
    apply_auto_approve: false
    deploy_run_init: true
    init_run_reconfigure: true
    auto_generate_backend_file: false

stacks:
  base_path: "stacks"
  included_paths:
    - "orgs/**/*"
  excluded_paths:
    - "**/_defaults.yaml"
  name_pattern: "{tenant}-{environment}-{stage}"

workflows:
  base_path: "stacks/workflows"

logs:
  file: "/dev/stdout"
  level: Info

How do I troubleshoot exactly where is it failing?

atmos version: 1.45.3

apparently vendor pull does not work if the component is not referenced anywhere in the stack..

touch stacks/orgs/cp/tenant1/dev/my-stack.yaml will get rid of that error

you need at least one .yaml file in orgs/ that is not _defaults.yaml

thanks!

good luck

@jonjitsu thanks. Yes, this is an issue with cold start, we’ll fix it in Atmos (to better describe the error to the user), and in the docs

@Andriy Knysh (Cloud Posse) Do we need a task for that?

we have a bunch of examples for IRSA, see here https://github.com/cloudposse/terraform-aws-components/tree/main/modules/eks

for example:

https://github.com/cloudposse/terraform-aws-components/blob/main/modules/eks/actions-runner-controller/main.tf

"cloudposse/helm-release/aws" 1) creates a Helm release; 2) creates an IRSA with the required permissions and assigns it to the Helm release k8s Service Account

ALB controller is another example https://github.com/cloudposse/terraform-aws-components/blob/main/modules/eks/alb-controller/main.tf#L23

external-dns is yet another https://github.com/cloudposse/terraform-aws-components/blob/main/modules/eks/external-dns/main.tf#L44

those use remote-state to get the EKS component from the remote state https://github.com/cloudposse/terraform-aws-components/blob/main/modules/eks/external-dns/remote-state.tf

and then use the credentials from the EKS cluster in the helm and kubernetes providers` https://github.com/cloudposse/terraform-aws-components/blob/main/modules/eks/external-dns/provider-helm.tf

you just need to copy (verbatim) the [provider.helm.tf](http://provider.helm.tf) fle into your component folder

and use the remote state module

module "eks" {
  source  = "cloudposse/stack-config/yaml//modules/remote-state"
  version = "1.5.0"

  component = var.eks_component_name

  context = module.this.context
}

and enable the IAML role for IRSA iam_role_enabled = true

and provide the required permissions so the Pods will assume the role and will be able to access the AWS resources

everything here https://github.com/cloudposse/terraform-aws-components/tree/main/modules/eks is used in production by many companies, and many of those components create and use IRSA - take a look

Oh ok, yeah I looked at these. But they are very specifically tied to helm releases, and do not require you referencing a lot of different resources in your stack such as SQS, S3, SSM,… So was just wondering if there were any generic standards for solely the service roles for custom apps which are not deployed through terraform

https://github.com/cloudposse/terraform-aws-eks-iam-role

it’s also used by https://github.com/cloudposse/terraform-aws-helm-release

i.e. I want to run atmos vendor pull aurora-postgres

it’s the ergonomics i’d expect from the subcommand

we are planning to add support for stacks and other artifacts

so a parameter to specify the type is required

but the current behavior is basically error by default

but i agree, it should give a better error description

will improve the error handling in the next release, which will be today/tomorrow, which btw will allow pulling artifacts (including terraform components) from OCI registries, eg AWS public ECR

I agree though that atmos vendor pull [COMPONENT] would feel more natural.

but in that case, we’ll not be able to pull any other types of artifacts. And if we add other types, then the command will not be symmetric

just added this error checking, will be in the next release

atmos vendor pull infra/vpc2

maybe using https://atmos.tools/core-concepts/components/inheritance#multilevel-inheritance

you can have a set of abstract base components (abstract so nobody would provision them even if attempted)

then another set of abstract base components that inherit from the first set and add more things

so you can combine it in diff combinations

this just shows that the settings section is a free-form map, and you can put anything into it

then, it can be read in CI/CD (e.g. GHA) to make some decisions based on values

also, it’s present in the outputs of many Atmos commands, e.g. atmos describe component, which can be used in shell scripts, GHA, and Atmos custom commands

do you have an example of how you use it in GHA?

https://github.com/cloudposse/atmos/blob/master/examples/complete/rootfs/usr/local/etc/atmos/atmos.yaml#L163

https://github.com/cloudposse/atmos/blob/master/examples/complete/rootfs/usr/local/etc/atmos/atmos.yaml#L186

example in a custom command ^

this is a GHA that installs and uses Atmos https://github.com/cloudposse/github-action-atmos-affected-stacks/blob/main/action.yml

instead of atmos describe affected https://github.com/cloudposse/github-action-atmos-affected-stacks/blob/main/action.yml#L106, you can call any Atmos command including atmos describe component. The use jq or yq to select settings.config.xxxx (or whatever values in settings you have

this is another example of using Atmos commands in GHA https://github.com/cloudposse/github-action-atmos-terraform-select-components/blob/8be5c434fb10cf870ccabf94bda98d43d88971b0/src/filter.sh#L14

and this is an example of using the settings section as a free-form map https://github.com/cloudposse/github-action-atmos-terraform-select-components/blob/8be5c434fb10cf870ccabf94bda98d43d88971b0/src/filter.sh#L3

settings.github.actions_enabled was added to Atmos stacks, and the GHA reads it

Haha, yea, saw that! If anything, it’s vindication that we’ve been thinking about it the right way all the time.

The patent it self, is on something else, more about how their pipeline is implemented.

https://patents.google.com/patent/WO2019113216A1/en

It’s also pending, so not granted and highly likely to be contested.

What do you need to do? what steps you need to take?

well… i’m most worried about splitting up the state file into multiple state files…

ahhh that

there is some tools like tf-migrate , terraformer etc

now if you duplicate the state you could potentially just delete what you will not be using automagically

For example, if you were moving an RDS cluster over another stack/module, but the resource ids/names do not change and everything else in that state is not needed, you could move the code and the plan will only keep the RDS cluster and it will delete everything else since the code is not there.

that way if you screw up you just copy the state again and try it etc

you can use the move{} and import{} blocks in some cases too if you need to

This all depends on how the code is structured, obviously

yeah.. ok taht makes sense and gives me a few avenues to go down. I like the idea of copying state … also love the new move/import blocks. .have used those quite a bit already

Sprig function in component vendoring! Great stuff!

also the “schema” section of docs is not very comprehensive. https://atmos.tools/core-concepts/stacks/#schema

It would be nice to have a full reference page like github actions for their workflow files

we have a task for this, thank you for pointing it out again

and a task to integrate with IDEs

https://github.com/cloudposse/atmos/issues/431

if you want to separate the two environments completely, you can use a diff namespace (to make it a diff Org)

if you want to add tennant , you can do it on a set of the stacks w/o affecting the already existing stacks (which don’t use tenant). The label module has support for it, and you use those settings in Atmos YAML stack manifest for that part of the infra. In other words, you can have some part of the config using tenant and the other part not uisng it

depending on how you want to segregate your infra, you can use one way or another

here is the config w/o using tenant (you addd that to the _defauts.yaml stack manifest for the first part of the infra

terraform:
  vars:
    label_order:
      - namespace
      - environment
      - stage
      - name
      - attributes
    descriptor_formats:
      account_name:
        format: "%v"
        labels:
          - stage
      stack:
        format: "%v-%v"
        labels:
          - environment
          - stage

if you want to separate the two environments completely, you can use a diff namespace (to make it a diff Org) (edited) Unfortunately it will be part of the same org, so changing namespace will affect it own share of components

here is the config w/o using tenant (you addd that to the _defauts.yaml stack manifest for the first part of the infra Does that also affect the atmos stack names? it doesnt right? Then I would need a separate config for it?

and here is the config to use tenant in Atmos stacks, you add it to some other stack manifest for the new infra

terraform:
  vars:
    label_order:
      - namespace
      - tenant
      - environment
      - stage
      - name
      - attributes
    descriptor_formats:
      account_name:
        format: "%v"
        labels:
          - tenant
          - stage
      stack:
        format: "%v-%v-%v"
        labels:
          - tenant
          - environment
          - stage

Does that also affect the atmos stack names?

if you add tenant it will affect everything

Ah I see, theres a separate descriptor for stacks which gets picked up by atmos as the stack name config?

for the existing stacks that currently don’t use tenenat, you can specify a “virtual” tenant and use it in Atmos stack names but w/o updating anything in the existing resources

you can open a PR and DM me, I’ll review and improve

for example, one company did not use tenant before, and all resource names and Atmos stack names were w/o tenant. Then we added a “virtual” tenant for them to preserve all the existing resources

import:
  - orgs/<namespace>/_defaults

vars:
  tenant: ""

then we used this config for the old part of infra w/o tenant

terraform:
  vars:
    label_order:
      - namespace
      - environment
      - stage
      - name
      - attributes
    descriptor_formats:
      account_name:
        format: "%v"
        labels:
          - stage
      stack:
        format: "%v-%v"
        labels:
          - environment
          - stage

but in your case, I would add a “virtual” tenant for the existing part of the repo and start using it in the Atmos stacks (so everytging is consistent and symmetric)

I’d just don’t use tenant in the resource names for the existing part of the repo to not destroy everything that was already provisioned

so I would add

import:
  - orgs/<namespace>/_defaults

vars:
  tenant: core  # or other name to name the "virtual" tenant

the all Atmos stack names would be namepsace-tenant-environment-name-attributes, and for the virtual tenant I’d use

namepsace-core-environment-name-attributes

and then I’d configure the existing part of the repo in Atmos stack manifest to not use tenant just for that part by using this config

terraform:
  vars:
    label_order:
      - namespace
      - environment
      - stage
      - name
      - attributes
    descriptor_formats:
      account_name:
        format: "%v"
        labels:
          - stage
      stack:
        format: "%v-%v"
        labels:
          - environment
          - stage

and for the new part of the infra I’d add a new tenant, and used this config

terraform:
  vars:
    label_order:
      - namespace
      - tenant
      - environment
      - stage
      - name
      - attributes
    descriptor_formats:
      account_name:
        format: "%v"
        labels:
          - tenant
          - stage
      stack:
        format: "%v-%v-%v"
        labels:
          - tenant
          - environment
          - stage

so the existing resources would remain the same

So, does

stack:
        format: "%v-%v-%v"
        labels:
          - tenant
          - environment
          - stage

override name_patternthen?

the new rsources will have tenant in the names and IDs

and all atmos commands would use tenant in the stack names (‘core’ for the old “virtual” one)

override name_patternthen?

no, name pattern in atmos.yaml - you change it to use tenant as well

so when you execute Atmos commands, you always use tenant in the stack names for consistency

so all existing Atmos stacks names would change to use tenant in the stack names, but all existing resources remanin the same (w/o tenant in the names/Ids)

Ah ok, I think thats the main issue right now. Because my current name_pattern is {namespace}-{environment}-{stage} , but as I realised now tenant would be more suited than namespace

yes, try that

add tenant to name_pattern

add tenant to the existing infra (with a proper name)

add the label configs (shown above) to the old config (not use tenant to preserve the resources), and to the new config (using tenant in the resource names)

What is descriptor.stack used for exactly? is it for backend storage location?

no, not backend. it’s used in the new account-map/dynamic-roles.tf

you can set workspace_key_prefix per component in backend.s3 section, similar to https://github.com/cloudposse/atmos/blob/master/examples/complete/stacks/catalog/terraform/vpc.yaml#L8

this is what Atmos will use if you set it

I’m aware that its technically feasible, I’m curious about whether it has any downstream consequences with any of the other atmos cli commands (e.g. describe affected etc.)

Also conscious that CloudPosse explicitly did not take this approach with your own nested components (e.g. tgw/spoke , spacelift/worker-pool etc), and wondering why you preferenced a flat structure in S3?

I don’t want to deviate from the refarch if in doing so I break from some fundamental underlying assumption or principle that I might have missed in the documentation.

nothing is affected by the backend structure, atmos describe affected does not use it

it’s only for terraform

we don’t set workspace_key_prefix for all components and let Atmos generate it automatically

workspace_key_prefix is just when you want to override it for diff reasons, e.g. if you have already provisioned a component using some prefix, and want to keep it when you change Atmos component name w/o destroying the component first

I understand this does not directly answer your question. And the reason is that we did not consider anything else besides what is auto-generated by Atmos with the possibility of overriding it in some cases

I think that all makes sense, I appreciate my example represents a niche use-case of the tooling here.

It looks there’s nothing that prevents using workspace prefix to group components, whether to mirror their position in the monorepo or for any other purpose.

Thanks very much for the help.

yes, nothing prevents you from making your own structure. Atmos generates it and writes to backend.tf.json, and then TF uses it

it’s just that you’ll have to set workspace_key_prefix on all the components in the YAML stack manifests

backend.s3 section participates in all the inheritance chains and deep-merging

Atmos supports deep-merging of maps at any level, but currently does not support removing items from the maps

we use a Go YAML lib which did not support special YAML syntax to specify removal

they have a new version of the lib now, but that’s not in Atmos yet

I would create a bunch of base abstract components with different combination of the values, and inherit from them in the real components

Yeah, thats what I feared, makes sense

we will update and test the new YAML lib in Atmos that supports YAML syntax to remove attributes. The new version is actually completely different so will require some testing

Cool

You could possibly null the tag values and then use terraform to filter the tags

# stacks/catalog/abc.yaml
components:
  abc:
    tags:
      my_default_tag: value

# stacks/org/dev/us-east-1.yaml
import:
- stacks/catalog/abc

components:
  abc:
    tags:
      my_default_tag: null

Then in terraform

locals {
  real_tags = {
    for k, v in module.this.tags:
    k => v
    if v != "null"
  }
}

And finally

module "something" {
  # ...
  tags = local.real_tags
}

The base abstracts make more sense in the long run tho. My proposal above is a bit of a hack

This relates to https://github.com/cloudposse/atmos/issues/227, https://github.com/cloudposse/atmos/issues/267

as an alternative, have these tags attached to the steps itself, getting atmos workflow all --skip-tags slow -f dummy (assuming we have an all that calls the main-step-*

nice feature, and can be added

please create an issue in the atmos repo

yea, that would be cool!

https://github.com/cloudposse/atmos/issues/443

Is there a way to see that again using the atmos.yaml configuration?

https://atmos.tools/cli/configuration#logs

Log level “Trace” will show all of that

ahhh! thank you!

or just use

ATMOS_LOGS_LEVEL=Trace atmos terraform ...

@Andriy Knysh (Cloud Posse)

Related question https://sweetops.slack.com/archives/C031KFY9C6M/p1698251886355849

Not yet

Thanks for confirming Erik!

Is it on the road map by chance?

So to be clear, that’s more of a Cloud Posse question than an atmos question. Cloud Posse will probably pursue something like this if/when it pursues BAAs. It’s not on our current roadmap.

ah that makes sense. Thank you

looks like you are trying to create tfstate-backend w/o having a TF state bucket to save the state in

for this to work, you need to comment out all the ref to the backend in the code and use a local backend to create the tfstate-backend component, which will crete an S3 state bucket and a Dynamo table

then enable the S3 backend for the component and run terraform init again

terraform will detect the backend change and will offer you to move the state for the component from the local state to the already created S3 bucket

Thanks @Andriy Knysh (Cloud Posse)! I am assuming local backend was created automatically when I created the accounts. As a matter of fact, I do see the state files on the disk within the “account” component. I do have references to backend : s3 in other components. I am going to comment those references out and try this command again.

you need to create the backend first (tfstate-backend component), and then create anything else with terraform, account and other components

I see. What is my recourse now that I have already executed the commands for account creation and these accounts have been created? That might explain why I am now getting these errors with account_map.

Error: Attempt to get attribute from null value
│ 
│   on ../account-map/modules/roles-to-principals/main.tf line 25, in locals:
│   25:   aws_partition         = module.account_map.outputs.aws_partition
│     ├────────────────
│     │ module.account_map.outputs is null
│ 
│ This value is null, so it does not have any attributes.
╵
╷
│ Error: Attempt to get attribute from null value
│ 
│   on ../account-map/modules/roles-to-principals/main.tf line 25, in locals:
│   25:   aws_partition         = module.account_map.outputs.aws_partition
│     ├────────────────
│     │ module.account_map.outputs is null
│ 
│ This value is null, so it does not have any attributes.
╵

There are more errors - just pasted a couple for reference.

Good morning @Andriy Knysh (Cloud Posse) Any thoughts on the best way to get around this issue? What would happen if I start over (delete local state in account module)? Would the account_map module get the right list of accounts based on current configuration?

you need to provision the components in the correct order:

1. tfstate-backend using local state (comment out the S3 backend)
2. Uncomment the S3 backend and run terraform apply on tfstate-backend to move the state to S3
3. Provision account (this will use S3 backend)
4. Provision account-map (use S3 backend) Note that aacount-map is used in all the other components to get the correct TF role to assume

if you have already created account using local state, you need to move the state to S3 https://medium.com/drmax-dev-blog/migration-of-terraform-state-between-various-backends-d3e0efedd61

so if you have already provisioned account using local state, you need to add S3 backend to the component and run terraform init - Terraform will ask you to migrate the state from local to S3

Thanks @Andriy Knysh (Cloud Posse) Will try this shortly.

It took me a while, but was able to move state for all current components to S3. I followed the following sequence of activities/commands:

atmos vendor pull -c infra/tfstate-backend   
atmos describe component tfstate-backend --stack core-ue1-root   
atmos terraform init tfstate-backend --stack core-ue1-root      
atmos terraform plan tfstate-backend --stack core-ue1-root      
atmos terraform apply tfstate-backend --stack core-ue1-root  
atmos terraform generate backend tfstate-backend --stack core-ue1-root
atmos terraform generate backend account --stack core-gbl-root   
atmos terraform plan account --stack core-gbl-root    
atmos terraform generate backend account-map --stack core-gbl-root

I am now able to plan account and account-map properly. Big lessons learned here is that I should have taken care of the state first before creating the accounts.

if you set auto_generate_backend_file: true in atmos.yaml, Atmos will auto-generate all backends for all components

https://atmos.tools/cli/configuration#components

Thanks for that tip! I was wondering if I need to generate the backend for each component.

I did try to add vpc-flow-logs-bucket as was illustrated in the example after the above steps and am getting the following set of errors for account-map. “local.account_map is null”. Do I need to generate backend of each of the sub-modules of account-map as well? OR would the above flag address these issues?

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Command info:
Terraform binary: terraform
Terraform command: plan
Arguments and flags: []
Component: infra/vpc-flow-logs-bucket
Stack: core-ue1-dev
Working dir: /atmos/components/terraform/infra/vpc-flow-logs-bucket

Executing command:
/usr/bin/terraform workspace select core-ue1-dev

Executing command:
/usr/bin/terraform plan -var-file core-ue1-dev-infra-vpc-flow-logs-bucket.terraform.tfvars.json -out core-ue1-dev-infra-vpc-flow-logs-bucket.planfile
module.iam_roles.module.account_map.data.utils_component_config.config[0]: Reading...
module.iam_roles.module.account_map.data.utils_component_config.config[0]: Read complete after 0s [id=b64bf6d1fbc423e85fa6975d72d99efc578fa53c]
module.iam_roles.module.account_map.data.terraform_remote_state.data_source[0]: Reading...
module.iam_roles.module.account_map.data.terraform_remote_state.data_source[0]: Read complete after 0s

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Attempt to get attribute from null value
│ 
│   on ../account-map/modules/iam-roles/main.tf line 32, in locals:
│   32:   profiles_enabled = coalesce(var.profiles_enabled, local.account_map.profiles_enabled)
│     ├────────────────
│     │ local.account_map is null
│ 
│ This value is null, so it does not have any attributes.
╵
╷
│ Error: Attempt to get attribute from null value
│ 
│   on ../account-map/modules/iam-roles/main.tf line 36, in locals:
│   36:   account_org_role_arn = local.account_name == local.account_map.root_account_account_name ? null : format(
│     ├────────────────
│     │ local.account_map is null
│ 
│ This value is null, so it does not have any attributes.
╵

@Andriy Knysh (Cloud Posse) Any thoughts on the above?

local.account_map is null

looks like you did not provision account-map

or did not specify all the correct context variables to get the account map

the context variables are namespace, tenant, environment, stage

they are defined in top-level Atmos stack manifests, usually in _defaults.yaml per namespace, tenant, environment, stage

@Andriy Knysh (Cloud Posse) What does “provision account-map” mean though? I ran terraform plan and terraform deploy on account _map.

Now I am seeing a different error though:

Error: Invalid index
│ 
│   on ../account-map/modules/iam-roles/main.tf line 38, in locals:
│   38:     local.account_map.full_account_map[local.account_name]
│     ├────────────────
│     │ local.account_map.full_account_map is object with 12 attributes
│     │ local.account_name is "dev"
│ 
│ The given key does not identify an element in this collection value.

@Nate did you find a solution to that issue?

Yes, I did. Thank you @Andriy Knysh (Cloud Posse) I was able to get through all these steps. I documented the steps I took so far – happy to share once I get to a stopping point.

@Nate Hi. Can you help me share the worked version of the Atmos tutorial with the remote backend and how to set up IAM user to be assumed in Atmos? I’m having trouble here.

@Quyen What kind of trouble are you having? Can you elaborate on the specific roadblock?

I got through the above issue (at least I think I did) by updating the variables for account_map as below:

identity_account_account_name: core-identity
dns_account_account_name: core-dns
audit_account_account_name: core-audit
artifacts_account_account_name: core-artifacts

I am now seeing the following error:

Error: Cannot assume IAM Role
│ 
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on providers.tf line 1, in provider "aws":
│    1: provider "aws" {
│ 
│ IAM Role (arn:aws:iam::xxxxxxxxxxxx:role/hct-plat-gbl-dev-terraform) cannot be assumed.
│ 
│ There are a number of possible causes of this - the most common are:
│   * The credentials used in order to assume the role are invalid
│   * The credentials do not have appropriate permission to assume the role
│   * The role ARN is not valid
│ 
│ Error: operation error STS: AssumeRole, https response error StatusCode: 403, RequestID: 33ebd829-b36b-4bbf-b360-d7169dfacf49, api error AccessDenied:
│ User: arn:aws:iam::yyyyyyyyyyyy:user/terraform is not authorized to perform: sts:AssumeRole on resource:
│ arn:aws:iam::xxxxxxxxxxxx:role/hct-plat-gbl-dev-terraform

After some research on current forums, I gathered that I need to “provision” iam roles - except I cannot find any documentation on how to do the “provisioning” of the roles. I see that the account_map output includes the role names, but these roles are not created in the root account or the identity account.

Assuming not provisioning the IAM roles is the primary issue here, can anyone please help with this or provide any guidance on how to get beyond this step?

@Andriy Knysh (Cloud Posse) Any thoughts on this?

you need to use these components to provision the primary roles (which are called teams) and the delegated roles (which are called team-roles)

https://github.com/cloudposse/terraform-aws-components/tree/main/modules/aws-teams

https://github.com/cloudposse/terraform-aws-components/tree/main/modules/aws-team-roles

Thanks @Andriy Knysh (Cloud Posse) I will take a look at those modules today.

@Andriy Knysh (Cloud Posse) I was able to successfully provision aws-teams and aws-team-roles modules. I checked the accounts, and the appropriate groups/users are created when I would expect them. However, when I try to deploy vpc-flow-logs-bucket module, I am still getting the following error. Do I need to add any code to the module/component to grab the roles from account_map and get them created in each of the accounts?

 Error: Cannot assume IAM Role
│ 
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on providers.tf line 1, in provider "aws":
│    1: provider "aws" {
│ 
│ IAM Role (arn:aws:iam::xxxxxxxxxxx:role/hct-plat-gbl-dev-terraform) cannot be assumed.

After digging through the forums, I am starting to wonder if I should be logged in using the <namespace>-gbl-identity-admin while executing the rest of the provisioning. If yes, what is best way to assume this role and when do I do it?

@Nate yes, you should assume the identity-admin role to work with the infra. terraform will then assume the corresponding roles to access the backend and the other accounts

you can use https://www.leapp.cloud/ (we use it) to assume the initial identity-admin role

I am using Leapp tool – based on the initial guidance from getting started docs. I will look into how to assume a role within the tool. I am now logged in using the “terraform” user I manually created in the root account as an IAM user. I probably should be going down the route of aws-sso to create a new account and assume this identity role.

I see….. https://docs.leapp.cloud/latest/configuring-session/configure-aws-iam-role-chained/