#atlantis (2022-09)

atlantis

Discuss the Atlantis (<http://runatlantis.iorunatlantis.io>)

**Archive: ** https://archive.sweetops.com/atlantis/

2022-09-01

jose.amengual avatar
jose.amengual

Hi, so for me it has been a long time setting up a new fresh account on AWS, usually I get accounts already created etc and as you know I’m an Atlantis hardcore believer and I’m trying to thing the PERFECT ( although we know is impossible) pipeline for Infra and app Infra to be created and I think the approach I have in mind could be too old :

1
jose.amengual avatar
jose.amengual

usually I like the Cloudposse TF flat structure with component in project that can be consumed as modules too like :

terraform-aws-compliance
    terraform-aws-macie
    terraform-aws-guarddutty
    .......

` and this project/module will instantiate those other submodules to create the compliance

jose.amengual avatar
jose.amengual

now as a GitOps pipeline I will create the terraform-aws-compliance repo and deploy using atlantis

jose.amengual avatar
jose.amengual

that that will be deploy to all the account using env vars or input variables to switch per accounts

jose.amengual avatar
jose.amengual

at the root level components this work fine I think

jose.amengual avatar
jose.amengual

but I used to do this for applications too and I do not really like it

jose.amengual avatar
jose.amengual

developers will be using ECS and I was thinking to do a self deploy/serve ECS cluster module

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

couldn’t you model the spacelift stack names from atmos into atlantis ?

atmos terraform plan vpc --stack ue2-dev

would be ue2-dev-vpc in spacelift and could be the same name in atlantis, no ?

jose.amengual avatar
jose.amengual

but the deployment process with env vars and such is where I’m focussing and trying to make it easier

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

why deploy with env vars when you can use tfvars ?

jose.amengual avatar
jose.amengual

sorry any input type terraform can take, I said env vars thinking TF_VAR…..

jose.amengual avatar
jose.amengual

I’m trying to decide if I should use workflows or atmos ( It could be hard to sell) etc

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

why not use both ? use custom workflows with atmos maybe ?

jose.amengual avatar
jose.amengual

the input can be data lookups to parameter store or tfvars file or json etc

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

or commit atmos tfvars and use custom workflows with tfvars without atmos ?

jose.amengual avatar
jose.amengual

the workflow of the developer is what I’m thinking on how to do

jose.amengual avatar
jose.amengual

the devs will not be deploying vpcs or things like that ( core components)

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

workflow of dev

atmos terraform plan my-service --stack ue2-sandbox

or

cd components/terraform/my-service
terraform workspace select ue2-sandbox-my-service
terraform plan -tfvars=ue2-sandbox-my-service.tfvars
jose.amengual avatar
jose.amengual

I’m trying to think how can I do the same without atmos

jose.amengual avatar
jose.amengual

in case atmos is not an option

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

that’s what i mean tho. you can setup everything without atmos, you can optionally use atmos to generate the tfvars files

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

if you choose not to use atmos at all, i would setup atlantis’s terraform mono repo with a list of root modules, and setup the workspaces and tfvars in the same way as if atmos was doing it

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

so you can have the following directories for instance

components/terraform/my-service
components/terraform/my-service/tfvars/
components/terraform/my-service/tfvars/ue2-dev.tfvars
components/terraform/my-service/tfvars/ue2-sandbox.tfvars

then atlantis.yaml repo config

projects:
- name: my-service-ue2-dev
  dir: components/terraform/my-service
  workspace: ue2-dev
  workflow: myworkflow
  autoplan:
    when_modified: ["**/*.tf", "tfvars/ue2-dev.tfvars"]

- name: my-service-ue2-sandbox
  dir: components/terraform/my-service
  workspace: ue2-sandbox
  workflow: myworkflow
  autoplan:
    when_modified: ["**/*.tf", "tfvars/ue2-sandbox.tfvars"]

# ...etc...

then for workflows

workflows:
  myworkflow:
    plan:
      steps:
      - run: terraform init
      
      - run: terraform workspace select $WORKSPACE

      - run: terraform plan -var-file=tfvars/$WORKSPACE.tfvars

I would recommend generating the repo yaml if possible, something like this

echo $(echo '[' && \ls components/terraform |
  while read d; do \
    \ls components/terraform/$d/tfvars | cut -d'.' -f1 | while read tfvar; do \
        echo '{ name: "'${d}-${tfvar}'", dir: "components/terraform/'$d'", workspace: "'$d-$tfvar'", workflow: myworkflow, autoplan: { when_modified: ["**/*.tf", "tfvars/'${tfvar}'.tfvars"] } },'; \
    done; \
  done
echo ']') | yq -P .

which would return

- name: my-service-ue2-dev
  dir: components/terraform/my-service
  workspace: my-service-ue2-dev
  workflow: myworkflow
  autoplan:
    when_modified:
      - '**/*.tf'
      - tfvars/ue2-dev.tfvars
- name: my-service-ue2-sandbox
  dir: components/terraform/my-service
  workspace: my-service-ue2-sandbox
  workflow: myworkflow
  autoplan:
    when_modified:
      - '**/*.tf'
      - tfvars/ue2-sandbox.tfvars
RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

@jose.amengual what do you think ?

jose.amengual avatar
jose.amengual

and the state of my-service-ue2-dev bases on the tfvars and the workspace?

jose.amengual avatar
jose.amengual

or it will be one state for all the components?

jose.amengual avatar
jose.amengual

I think the state is one in atmos no? state bucket I mean

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

in atmos, you can have multiple s3 remote state buckets if you want, it just depends on how you configure the backend.

the only tools that can configure a backend on the fly are atmos/terragrunt/terramate and it cannot be done with raw terraform unless you codified it in your custom workflow somehow

RB (Ronak) (Cloud Posse) avatar
RB (Ronak) (Cloud Posse)

if you have a [backend.tf](http://backend.tf) in the root terraform directory, you can use as many workspaces as you’d like and it would create those terraform resources in that workspace, without having to modify the s3 backend

jose.amengual avatar
jose.amengual

I guess if I wanted to create a state per monorepo and workspace then I will need to manage outside the monorepo and pass it in

1
jose.amengual avatar
jose.amengual

I guess you have been thinking on how to run atmos in atlantis

1
jose.amengual avatar
jose.amengual

remember in spacelift it will pick the stack changes?

jose.amengual avatar
jose.amengual

then it could be possible to run the atmos command to generate the atlantis.yaml for the repo

jose.amengual avatar
jose.amengual

but for that to work the a PR will have to be created under the component repo from within atlantis maybe using the new API……..

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@jose.amengual this is all really interesting. We discussed on our ARB call today.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think one small thing we can do to make this easier is ask @Andriy Knysh (Cloud Posse) to implement an command to generate all the varfiles.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This would also be the “break glass” to show there is zero vendor lockin.

1
jose.amengual avatar
jose.amengual

yes or maybe even generate the altlantis.yaml files ON THE FLY!!!!!!

1
Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

(let me read from the beginning)

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)


yes or maybe even generate the altlantis.yaml files ON THE FLY!!!!!!

Andriy Knysh (Cloud Posse) avatar
Andriy Knysh (Cloud Posse)

that could be done

jose.amengual avatar
jose.amengual

well if that could be done, then I’m going to start working right away (next week) , we ( atlantis) have discussed integration and we all agree that is the nest for the both projects, if you look at Infracost they did a similar thing

jose.amengual avatar
jose.amengual

what I mean is Infracost team PR against atlantis to do the integration and modify their app to do it too so Cloudposse/atmos could do the same and it will be freaking awesome

David avatar

@jose.amengual not sure if you know about it but there is an existing tool that does generate atlantis.yaml files on the fly for projects using terragrunt: https://github.com/transcend-io/terragrunt-atlantis-config

Disclaimer: I maintained this tool for a few years

transcend-io/terragrunt-atlantis-config

Generate Atlantis config for Terragrunt projects.

1
jose.amengual avatar
jose.amengual

I’m fully aware of that tool, I’m on eof the Atlantis maintainers

jose.amengual avatar
jose.amengual

a lot of people uses it

jose.amengual avatar
jose.amengual

and thanks for your work @David

2022-09-02

2022-09-06

2022-09-07

jose.amengual avatar
jose.amengual

v1.5.0 what Add support for custom integrations in atmos.yaml Add Atlantis support (Atlantis is an integration) Add atmos terraform generate varfiles and atmos atlantis generate repo-config CLI commands why Support Atlantis Generate the varfiles for all components in all stacks (this is used in Atlantis repo config, and will be used to detect drifts in variables to simplify triggering Spacelift stacks) Automatically generate Atlantis repo config file atlantis.yaml. Using the config, project and… (edited)

Release v1.5.0 · cloudposse/atmos

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you try this out, let us know! Feedback welcome. If you get stuck, that’s okay too - let us know.

2022-09-08

jose.amengual avatar
jose.amengual

v0.19.9-pre.20220908 Pre-release

What's Changed
ignore GitLab Draft MRs by @becjon in #2468
enable hostname (not only IP Address) on Metrics statsd host key by @kangaechu in #2429
Omit type declaration of vars by @estensen in #2476
build(deps): bump go.uber.org/zap from 1.22.0 to 1.23.0 by @dependabot in #2478
build(deps): bump github.com/microcosm-cc/bluemonday from 1.0.19 to 1.0.20 by @dependabot in #2487
build(deps): bump github.com/urfave/cli from 1.22.9 to 1.22.10 by @dependabot in #2488
deps: bump tf to 1.2.9 by @chenrui333 in #2499
Use proper http client for github v4 api by @ysoldak in #2479
deps: bump conftest to 0.34.0 by @chenrui333 in #2500
External Locking DB: Redis by @SudoSpartanDan in #2491
workflows: update runner to use ubuntu-22.04 by @chenrui333 in #2502
fix: allow requests to /api/* without authentication by @vincentgna in #2455
bug: Fix GitHub Markdown for added and deleted resources by @pauloconnor in #2418
Use single github v4 client by @ysoldak in #2480
feat: support parallel execution with order groups by @tufitko in #2403
lint: remove deprecated io/util package by @chenrui333 in #2503
Fixes for multienv step comment by @hatmarch in #2411
fix: Set mergeable correctly when branch protection doesn't require reviewers by @stasostrovskyi in #2470
jose.amengual avatar
jose.amengual

atlantis autoscaling anyone ?????? well now you got it

Release notes from atlantis avatar
Release notes from atlantis
09:18:42 PM

v0.19.9-pre.20220908 What’s Changed ignore GitLab Draft MRs by @becjon in <a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”1349059376” data-permission-text=”Title is private” data-url=”https://github.com/runatlantis/atlantis/issues/2468“…

becjon - Overview

becjon has 5 repositories available. Follow their code on GitHub.

ignore GitLab Draft MRs by becjon · Pull Request #2468 · runatlantis/atlantis

Terraform Pull Request Automation. Contribute to runatlantis/atlantis development by creating an account on GitHub.

2022-09-12

Release notes from atlantis avatar
Release notes from atlantis
01:18:36 AM

v0.19.9-pre.20220912 What’s Changed Adding Redis TLS Option by @SudoSpartanDan in <a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”1368284797” data-permission-text=”Title is private” data-url=”https://github.com/runatlantis/atlantis/issues/2510“…

SudoSpartanDan - Overview

¯_(ツ)_/¯. SudoSpartanDan has 11 repositories available. Follow their code on GitHub.

Adding Redis TLS Option by SudoSpartanDan · Pull Request #2510 · runatlantis/atlantisattachment image

The current implementation works great so far for using Redis as a locking DB. A slight oversight on the initial work, though, was being able to enable TLS for the connection to Redis. This change …

Release notes from atlantis avatar
Release notes from atlantis
01:28:37 AM

v0.19.9-pre.20220912 What’s Changed Adding Redis TLS Option by @SudoSpartanDan in <a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”1368284797” data-permission-text=”Title is private” data-url=”https://github.com/runatlantis/atlantis/issues/2510“…

jose.amengual avatar
jose.amengual

v0.19.9-pre.20220912 Pre-release

What's Changed
Adding Redis TLS Option by @SudoSpartanDan in #2510
build(deps): bump github.com/spf13/viper from 1.12.0 to 1.13.0 by @dependabot in #2517
build(deps): bump runatlantis/atlantis-base from 2022.08.05 to 2022.09.08 by @dependabot in #2515
build(deps): bump github.com/moby/moby from 20.10.17+incompatible to 20.10.18+incompatible by @dependabot in #2516
lint: recursive gofmt by @chenrui333 in #2504
build(deps): bump golang from 1.19.0-alpine to 1.19.1-alpine by @dependabot in #2514

2022-09-13

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@jose.amengual any opinion on using App Runner for a simpler hosting of Atlantis?

    keyboard_arrow_up