#atlantis (2022-09)

usually I like the Cloudposse TF flat structure with component in project that can be consumed as modules too like :

terraform-aws-compliance
    terraform-aws-macie
    terraform-aws-guarddutty
    .......

` and this project/module will instantiate those other submodules to create the compliance

now as a GitOps pipeline I will create the terraform-aws-compliance repo and deploy using atlantis

that that will be deploy to all the account using env vars or input variables to switch per accounts

at the root level components this work fine I think

but I used to do this for applications too and I do not really like it

developers will be using ECS and I was thinking to do a self deploy/serve ECS cluster module

couldn’t you model the spacelift stack names from atmos into atlantis ?

atmos terraform plan vpc --stack ue2-dev

would be ue2-dev-vpc in spacelift and could be the same name in atlantis, no ?

but the deployment process with env vars and such is where I’m focussing and trying to make it easier

why deploy with env vars when you can use tfvars ?

sorry any input type terraform can take, I said env vars thinking TF_VAR…..

I’m trying to decide if I should use workflows or atmos ( It could be hard to sell) etc

why not use both ? use custom workflows with atmos maybe ?

the input can be data lookups to parameter store or tfvars file or json etc

or commit atmos tfvars and use custom workflows with tfvars without atmos ?

the workflow of the developer is what I’m thinking on how to do

the devs will not be deploying vpcs or things like that ( core components)

workflow of dev

atmos terraform plan my-service --stack ue2-sandbox

or

cd components/terraform/my-service
terraform workspace select ue2-sandbox-my-service
terraform plan -tfvars=ue2-sandbox-my-service.tfvars

I’m trying to think how can I do the same without atmos

in case atmos is not an option

that’s what i mean tho. you can setup everything without atmos, you can optionally use atmos to generate the tfvars files

if you choose not to use atmos at all, i would setup atlantis’s terraform mono repo with a list of root modules, and setup the workspaces and tfvars in the same way as if atmos was doing it

so you can have the following directories for instance

components/terraform/my-service
components/terraform/my-service/tfvars/
components/terraform/my-service/tfvars/ue2-dev.tfvars
components/terraform/my-service/tfvars/ue2-sandbox.tfvars

then atlantis.yaml repo config

projects:
- name: my-service-ue2-dev
  dir: components/terraform/my-service
  workspace: ue2-dev
  workflow: myworkflow
  autoplan:
    when_modified: ["**/*.tf", "tfvars/ue2-dev.tfvars"]

- name: my-service-ue2-sandbox
  dir: components/terraform/my-service
  workspace: ue2-sandbox
  workflow: myworkflow
  autoplan:
    when_modified: ["**/*.tf", "tfvars/ue2-sandbox.tfvars"]

# ...etc...

then for workflows

workflows:
  myworkflow:
    plan:
      steps:
      - run: terraform init
      
      - run: terraform workspace select $WORKSPACE

      - run: terraform plan -var-file=tfvars/$WORKSPACE.tfvars

I would recommend generating the repo yaml if possible, something like this

echo $(echo '[' && \ls components/terraform |
  while read d; do \
    \ls components/terraform/$d/tfvars | cut -d'.' -f1 | while read tfvar; do \
        echo '{ name: "'${d}-${tfvar}'", dir: "components/terraform/'$d'", workspace: "'$d-$tfvar'", workflow: myworkflow, autoplan: { when_modified: ["**/*.tf", "tfvars/'${tfvar}'.tfvars"] } },'; \
    done; \
  done
echo ']') | yq -P .

which would return

- name: my-service-ue2-dev
  dir: components/terraform/my-service
  workspace: my-service-ue2-dev
  workflow: myworkflow
  autoplan:
    when_modified:
      - '**/*.tf'
      - tfvars/ue2-dev.tfvars
- name: my-service-ue2-sandbox
  dir: components/terraform/my-service
  workspace: my-service-ue2-sandbox
  workflow: myworkflow
  autoplan:
    when_modified:
      - '**/*.tf'
      - tfvars/ue2-sandbox.tfvars

@jose.amengual what do you think ?

and the state of my-service-ue2-dev bases on the tfvars and the workspace?

or it will be one state for all the components?

I think the state is one in atmos no? state bucket I mean

in atmos, you can have multiple s3 remote state buckets if you want, it just depends on how you configure the backend.

the only tools that can configure a backend on the fly are atmos/terragrunt/terramate and it cannot be done with raw terraform unless you codified it in your custom workflow somehow

if you have a [backend.tf](http://backend.tf) in the root terraform directory, you can use as many workspaces as you’d like and it would create those terraform resources in that workspace, without having to modify the s3 backend

I guess if I wanted to create a state per monorepo and workspace then I will need to manage outside the monorepo and pass it in

I guess you have been thinking on how to run atmos in atlantis

remember in spacelift it will pick the stack changes?

then it could be possible to run the atmos command to generate the atlantis.yaml for the repo

but for that to work the a PR will have to be created under the component repo from within atlantis maybe using the new API……..

@jose.amengual this is all really interesting. We discussed on our ARB call today.

I think one small thing we can do to make this easier is ask @Andriy Knysh (Cloud Posse) to implement an command to generate all the varfiles.

This would also be the “break glass” to show there is zero vendor lockin.

yes or maybe even generate the altlantis.yaml files ON THE FLY!!!!!!

(let me read from the beginning)

yes or maybe even generate the altlantis.yaml files ON THE FLY!!!!!!

that could be done

well if that could be done, then I’m going to start working right away (next week) , we ( atlantis) have discussed integration and we all agree that is the nest for the both projects, if you look at Infracost they did a similar thing

what I mean is Infracost team PR against atlantis to do the integration and modify their app to do it too so Cloudposse/atmos could do the same and it will be freaking awesome

@jose.amengual not sure if you know about it but there is an existing tool that does generate atlantis.yaml files on the fly for projects using terragrunt: https://github.com/transcend-io/terragrunt-atlantis-config

Disclaimer: I maintained this tool for a few years

I’m fully aware of that tool, I’m on eof the Atlantis maintainers

a lot of people uses it

and thanks for your work @David

If you try this out, let us know! Feedback welcome. If you get stuck, that’s okay too - let us know.