#atlantis (2020-04)

atlantis

Discuss the Atlantis (http://runatlantis.io|runatlantis.io) *Archive: * https://archive.sweetops.com/atlantis/

2020-04-29

skel84 avatar
skel84

does anybody here have experience with atlantis multi account setup?

PePe avatar

multi account as ?

PePe avatar

one atlantis many accounts

PePe avatar

or many an atlantis per account

skel84 avatar
skel84

one atlantis many accounts

PePe avatar

and you need to be specific about environments too

PePe avatar

multiple environments per account ?

skel84 avatar
skel84

one account per environment

skel84 avatar
skel84

one environment per account

PePe avatar

and what are your concerns ?

skel84 avatar
skel84

i’m running atlantis in my root account, but i cannot assume the role in my child account

skel84 avatar
skel84

it fails during initialization because it can’t find s3 backend

skel84 avatar
skel84

i’m using the ecs fargate module

skel84 avatar
skel84

and the task role has an administrator policy attached

skel84 avatar
skel84

i can’t figure out if I have to use a credential file, env variables or just the role

PePe avatar

are you specifying the region of the remote backend ?

PePe avatar

instance roles are sufficient

PePe avatar

although I do not recommend one atlantis for all accounts

skel84 avatar
skel84

it’s fargate, so only task role

PePe avatar

blast radius is too big

PePe avatar

fargate task is the same

skel84 avatar
skel84

so you reccomend one atlantis per account?

PePe avatar

yes, or per team if each team have multiple accounts

PePe avatar

you backend config should look like this :

region = "us-east-2"

bucket = "mybucket-state"

key = "terraform.tfstate"

dynamodb_table = "mybucket-state-lock"

encrypt = true
skel84 avatar
skel84

it looks like this

PePe avatar

does it have the region ?

skel84 avatar
skel84

yes

PePe avatar

and are you getting 403 or bucket not found ?

skel84 avatar
skel84

if i run it manually from my laptop it works

skel84 avatar
skel84

bucket not found

PePe avatar

in your local always works

PePe avatar

but what is the error code ?

PePe avatar

does the fargate role have Allow s3:* ?

skel84 avatar
skel84
 Remote state S3 bucket xxxxx-terraform-state does not exist or you don't have permissions to access it. Would you like Terragrunt to create it? (y/n) 
PePe avatar

ok, so it could be a 403

skel84 avatar
skel84

fargate role has admin policy attached

skel84 avatar
skel84

the s3 bucket is in the child account

PePe avatar

child ?

PePe avatar

another account right not the same ?

skel84 avatar
skel84

yes

PePe avatar

you still have to specify the bucket in the policy for the role and in the bucket you need to allow the altantis account to have access

PePe avatar

this is a problem of cross account access of s3 buckets

PePe avatar

no matter if they are in the same org

skel84 avatar
skel84

but i assume the role in the backend definition

skel84 avatar
skel84

or at least i tought so

PePe avatar

is the bucket encrypted ?

PePe avatar

KMS ?

PePe avatar

well anyhow it should work

PePe avatar

if you are assuming the role it should work, so maybe the trust policies are not right

PePe avatar

you can start a t2.micro using the same role as fargate and use the cli

skel84 avatar
skel84

i’ll try

PePe avatar

configure the ~/.aws/config

PePe avatar

using the instance profile

PePe avatar

and see if it works

skel84 avatar
skel84

ok, thanks

skel84 avatar
skel84

is the ~/.aws/config in the container necessary?

PePe avatar

for the aws cli to work yes

PePe avatar
Using Instance Profiles - AWS Identity and Access Management

Use IAM instance profiles to pass a role to an Amazon EC2 instance when the instance starts.

PePe avatar

that one

PePe avatar
credential_source = Ec2InstanceMetadata
skel84 avatar
skel84

the thing is, according to this https://www.runatlantis.io/docs/provider-credentials.html#aws-specific-info, it says that the AWS fargate module has its own way to provide credentials

Provider Credentials | Atlantis

Atlantis: Terraform Pull Request Automation

skel84 avatar
skel84

but i can’t figure out which one it is

PePe avatar

but it should be using the fargate role permissions

PePe avatar

I run it on fargate and I do not have this problem

PePe avatar

but my buckets have bucket policies for cross account access

skel84 avatar
skel84

do you specify the role arn in the provider?

PePe avatar

and for the state it have a polocy allowing the account to have access

PePe avatar

no I do not , I do no assume role

PePe avatar

you can run a script trough atlantis so you could try run commands to list the bucket and see what you get

skel84 avatar
skel84

yeah i guess i’ll have to do some more testing

PePe avatar

but if you try it from an instance and it works, then it should work

PePe avatar

then at least you know is a config in atlantis

skel84 avatar
skel84

yes, i’ll try that

skel84 avatar
skel84

thanks

Jake Lundberg (HashiCorp) avatar
Jake Lundberg (HashiCorp)

Have you looked at the free version of Terraform Cloud?

2020-04-28

joshmyers avatar
joshmyers

@Alex Siegman Hows your Atlantis Anchors working out? May need that real soon!

Alex Siegman avatar
Alex Siegman

Works perfectly. It’s a little janky to have to specify some “fake projects” to use as templates, but it works perfectly and has made the file a lot more readable.

1
joshmyers avatar
joshmyers

Yup, my atlantis.yml just hit 1K lines, gonna give this a whirl, thanks @Alex Siegman!

2020-04-25

2020-04-17

PePe avatar

Is there a way to limit atlantis what environment to run without workspaces ?

PePe avatar

in my case I have one atlantis per aws account = environment

PePe avatar

only selected repos can trigger webhooks

PePe avatar

and only few people can do applies

PePe avatar

using terraform atlantis cloudposse module

PePe avatar

but my terraform project structure is :

./
 [main.tf](http://main.tf)
[variables.tf](http://variables.tf)
staging.tfvars
staging-backend.tfvars
production.tfvars
production-backend.tfvars
PePe avatar

so when we run tf we basically run terraform init -backend-config=staging-backend.tfvars then plan then `apply

PePe avatar

but the account or role used is only for the staging account

PePe avatar

and atlantis is deployed to each specific account

PePe avatar

so I want to limit the scope of what they can run for each atlantis deployment

zeid.derhally avatar
zeid.derhally

We have the same setup where we have an Atlantis in each account. For accounts that handle multiple environments we went with separate directories and try to follow a strict process of only making changes in one environment in a given PR.

PePe avatar

ok I c, same thing we are going to do

PePe avatar

cool

2020-04-16

2020-04-13

RB avatar

i have atlantis working and have

  atlantis_repo_whitelist = [
    "<http://github.com/myorg/*|github.com/myorg/*>",
  ]

  # Atlantis
  # Github repositories where webhook should be created
  atlantis_allowed_repo_names = [
    "myorg/terraform_scripts"
  ]
RB avatar

but almost every pr in my org is receiving comments of atlantis plan when I only want it to show plan for myorg/terraform_scripts

RB avatar

if I set my atlantis_repo_whitelist to only myorg/terraform_scripts, then I see Error: repo not in whitelist or similar as a comment

PePe avatar

format is :

atlantis_repo_whitelist = ["[github.com/org/terraform-xx-xx-ecs-cluster>","<http://github.com/org/terraform-xx-xx-rds|github.com/org/terraform-xx-xx-rds](http://github.com/org/terraform-xx-xx-ecs-cluster|github.com/org/terraform-xx-xx-ecs-cluster>","<http://github.com/org/terraform-xx-xx-rds)"]
PePe avatar

wilcards seem to be used for all repos

PePe avatar

in an org

PePe avatar

I had the same problem

RB avatar

if I don’t set the wildcard, I receive the Error message commented on every pr. did you also see this ? if so, how did you prevent that?

PePe avatar

no, I did not see that

PePe avatar

I allowed just two repos to use altantis so far

PePe avatar

and I’m using cloudposse fork but I think that that matters much

PePe avatar

plus I setted up my worflows as repo config on the the atlantis server

RB avatar

ah interesting. our tf is a mixed bag so i have to do an atlantis.yml per repo to configure it correctly

RB avatar

if i cannot configure the official tf module, ill prob have to go the cloudposse route. which im ok with

PePe avatar

so you kinda want all the repos that matches myorg/terraform_scripts to use atlantis

PePe avatar

and nothing else ?

RB avatar

exactly!

RB avatar

just to test this out, configure it correctly, and then manually start adding new repos to it

RB avatar

we have a number of terraform modules that dont even use a backend so i wouldnt want atlantis to even touch those repos until they are migrated

RB avatar

(their tfstates are unfortunately have to be committed)

PePe avatar

I will recommend to disable auto plan in all repos, it can get overwhelming, I would do that on the repo side

PePe avatar

when I said repo side, repo config side so atlantis config side

PePe avatar

so users are force to comment the pr or branch to get a plan

RB avatar

thats a good idea. let me figure out how to do this

RB avatar

is there an env variable for this on the server level ?

RB avatar

ah it looks like it’s only possible on the repo level https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#use-cases

Repo Level atlantis.yaml Config | Atlantis

Atlantis: Terraform Pull Request Automation

RB avatar
Disable automatic plan server flag · Issue #425 · runatlantis/atlantis

bash-4.4# atlantis version atlantis 0.4.13 Issue: I&#39;d like to be able to disable automatic planning as a server flag. I see this option if using atlantis.yaml files. https://www.runatlantis.io/

RB avatar

yuck… so that means id have to add an atlantis.yml file for hundreds of repos in order to prevent auto planning

PePe avatar

yes repo level

PePe avatar

mmmm no

PePe avatar

you do do a match for all repos with /.*/

PePe avatar

and disable those global configs

PePe avatar

then you do the ones for myorg/terraform_scripts

RB avatar

im not sure if i follow you

RB avatar

what do you mean by and disable those global configs ?

RB avatar

i got around this by creating a new user exclusively for atlantis and only added the user to the private repo

PePe avatar

you can have something like this :

PePe avatar
repos:
  # id can either be an exact repo ID or a regex.
  # If using a regex, it must start and end with a slash
  # Repo ID's are of the form {VCS hostname}/{org}/{repo name}
  - id: /.*/

    # apply_requirements sets the Apply Requirements for all repos that match
    apply_requirements: [approved,mergeable]

    # allowed_overrides specifies which keys can be overridden by this repo in
    # its atlantis.yaml file
    allowed_overrides: [apply_requirements]

    # allow_custom_workflows defines whether this repo can define its own
    # workflows. If false (default), the repo can only use server-side defined workflows
    allow_custom_workflows: false

    # Allow repos to chose one of the workflows created in this file
    allowed_overrides: [workflow]
PePe avatar

or one each repo

PePe avatar

etc

PePe avatar

so for the global id: /.*/

PePe avatar

you could disable auto plan

RB avatar

ah ok but still have to be one for each repo

RB avatar

I think I understand what you’re saying now. Thank you for sending me the config!

RB avatar

i think now that i can test in my org’s terraform_scripts repo, we can now configure our repo level atlantis.yml file without commenting every pr in the org

RB avatar

going forward, i’ll create a new atlantis.yml file in the new repo, then add our bot user to that new repo

PePe avatar

the rule - id: /.*/ will match all repos

PePe avatar

then you can do each repo if you need to

RB avatar

oh! i see. so i could have this user added to all repos and configure it right from the server atlantis yaml

PePe avatar

correct, that is what I think it will do

PePe avatar

I only configure the repos I want with the webhooks

PePe avatar

not all the repos

PePe avatar

and I add the Team were the atlantis user is

PePe avatar

mmm autoplan is not a server level config

PePe avatar

is repo level

PePe avatar

that sucks

RB avatar

yea i saw that. unfortunate. see the ticket i mentioned above. it seems like the maintainer doesnt think it’s a good idea to do but i dont fully understand the reasoning

PePe avatar

you could do that

PePe avatar

although I can see the reason why not to disable autoplan

PePe avatar

I gues the idea is to run plan against to PR to master

PePe avatar

and you enable the webhooks per repo

PePe avatar

not globaly in your VCS

RB avatar

looking at my fargate logs, even tho i only have a single repo whitelisted with a single webhook, the app keeps trying to reach out to every new pr but is getting a 404 (because i havent added the bot to their private repos)

RB avatar

weird that it would still be trying to reach out to each pr. i figured it would know to only look at prs coming from the repos whitelisted

PePe avatar

mmm I do not see that

PePe avatar

the event comes from the repo

PePe avatar

atlantis receives the event

RB avatar

is there a way to configure it for a single repository in an org without it commenting “error” on all other repo prs ?

RB avatar

thanks!

RB avatar

management doesn’t seem to be completely sold on atlantis. besides

• just like we have cicd of code, we can also have cicd of terraform

• auditability of terraform plans

• a number of top companies using it (lyft, shopify, pagerduty, hootsuite, cloudposse) and hashicorp owns it

• locking of modules within prs

terraform workspace support

• only apply changes that are approved notable posts

https://www.runatlantis.io/

https://medium.com/runatlantis/terraform-and-the-dangers-of-applying-locally-543563782a73

https://medium.com/runatlantis/introducing-atlantis-6570d6de7281

https://docs.google.com/presentation/d/1X4VGx-R8UZWE_2s7I8IxcbWsav1kR-QosmHI8kKaIZc/htmlpresent

https://www.reddit.com/r/devops/comments/cakyfp/psa_love_terraform_love_cicd_you_want_to_run/ are there more reasons to use it?

roth.andy avatar
roth.andy

In my experience management will almost never be “sold” on something like this. This has to come from you, as the subject matter expert, as a requirement. Asking for their permission is shifting the risk over to them, and their number one priority is to avoid risk.

https://youtu.be/ecIWPzGEbFc?t=3862

:--1:2
PePe avatar

Dev do not need to set terraform on their machines and TF is run by atlantis using instance profiles that are fully auditable

3
PePe avatar

the changes of infra are in a VCS

PePe avatar

GitOps is awesome

:--1:1
cool-doge1

2020-04-12

2020-04-11

PePe avatar
cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

PePe avatar

I find the doc and this a bit confusing

cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

PePe avatar
cloudposse/terraform-aws-ecs-atlantis

Terraform module for deploying Atlantis as an ECS Task - cloudposse/terraform-aws-ecs-atlantis

PePe avatar

talk about : github_oauth_token and github_webhooks_token that need to be created before the init of the module but then it says that is they are not provided it will be looked up in PS

PePe avatar

and the instruction is that you can write the values with chamber :

chamber write atlantis atlantis_gh_token "..."
  chamber write atlantis github_webhooks_token "...""
PePe avatar

so github_oauth_token and atlantis_gh_token is the same thing

PePe avatar

but called differently

PePe avatar

what I mean is that what is asked to be created in github side does not match the names in Parameter store

PePe avatar

I don’t know I just find it a bit confusing

PePe avatar

working with the cloudposse atlantis module I’m having a problem with the github webhooks

PePe avatar

the error I get after creating the webhooks and then changing the team access to read

PePe avatar
Error: POST <https://api.github.com/repos/xxxxx/terraform-xx-xx-ecs-cluster/hooks>: 404 Not Found []

  on .terraform/modules/repo_webhooks/main.tf line 6, in resource "github_repository_webhook" "default":
   6: resource "github_repository_webhook" "default" {
PePe avatar

it tries to create the webhooks again

PePe avatar

well is trying to look for the webhooks

PePe avatar

but read permission at repository level do not allow you to see webhook configs

PePe avatar

so I don’t know how to fix this

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I vaguely recall there’s no way to update github webhooks using the provider

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

so i would need to taint them and reapply

PePe avatar

but if you taint

PePe avatar

then can you re-apply with read permissions ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

oh no, i never tried changing permissions

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

but i did try changing the webhook settings and that’s when this happened

PePe avatar

the readme says to change to read after the webhook is created

PePe avatar

but that breaks things

PePe avatar

so maybe having another repo for the webhooks maybe a better idea

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Andriy Knysh (Cloud Posse) might recall what to do here… but it’s been a while

PePe avatar

I’m cleaning up my the atlantis setup and I think I got everything pretty much figure out

PePe avatar

using one atlantis for multiple repos in the sam environment

PePe avatar

anyone was able to recall what can be done ?

PePe avatar

taint ?

2020-04-10

2020-04-09

RB avatar

trying to use this module to setup fargate but having issues. anyone using the same module and have it working as expected?

https://github.com/terraform-aws-modules/terraform-aws-atlantis

terraform-aws-modules/terraform-aws-atlantis

Terraform configurations for running Atlantis on AWS Fargate. Github, Gitlab and BitBucket are supported - terraform-aws-modules/terraform-aws-atlantis

PePe avatar

I have not used it , I used the cloudposse one

terraform-aws-modules/terraform-aws-atlantis

Terraform configurations for running Atlantis on AWS Fargate. Github, Gitlab and BitBucket are supported - terraform-aws-modules/terraform-aws-atlantis

RB avatar

my terraform

module "atlantis" {
  source  = "terraform-aws-modules/atlantis/aws"
  version = "~> 2.0"

  name = local.name

  # VPC
  vpc_id             = data.aws_[vpc.selected.id](http://vpc.selected.id)
  
  public_subnet_ids  = local.public_subnet_ids
  # TODO: use private subnets instead of public
  # fargate in private. need to be in same set of AZs as public
  private_subnet_ids = local.public_subnet_ids

  ecs_service_assign_public_ip = true

  route53_zone_name = "[internal.snip.com](http://internal.snip.com)"

  certificate_arn = data.aws_acm_certificate.internal.arn

  # create_github_repository_webhook = true
  # Atlantis
  atlantis_github_user       = "atlantis-bot"
  # TODO: use s3keyring or ssm
  atlantis_github_user_token = "snip"
  atlantis_repo_whitelist    = [
    "<http://github.com/terraform-aws-modules/*|github.com/terraform-aws-modules/*>",
    "<http://github.com/snip/*|github.com/snip/*>",
  ]

  allow_repo_config = "true"

  # Atlantis
  atlantis_allowed_repo_names = [
    "snip/terraform-scripts"
  ]
}
RB avatar

hmmm, perhaps I should switch to the cloudposse one or at least try it side by side

PePe avatar

the cloudposse one is recommended with the use of their fork

PePe avatar

you could use the atlantis from runatlantis but you will have to feed some different config to make it work

RB avatar

ah perhaps thats what im missing. i was having trouble figuring out how to apply the server configuration too

PePe avatar

the cloudposse module uses Parameter store to store the configs and value and I used chamber to populate the Dockerfile

RB avatar

interesting! ok ill focus more on the fork then

RB avatar

do you still require repo specific atlantis.yml configs ?

PePe avatar

I was going to say : to make it easier for you

PePe avatar

here are the file I used

PePe avatar

buildscpec for codebuild :

PePe avatar

atlantis.yaml

PePe avatar

atlantis-repo-config

PePe avatar

there

PePe avatar

that is all you need

PePe avatar

I put this all in the same repo but you do not have to

RB avatar

awesome! you the man pepe

RB avatar

ill give it a go

PePe avatar

I tried it with one repo

PePe avatar

but you could have the Dockerfile+buildspec+atlantisrepo config in one repo and use that as a atlantis ECS cluster for your AWS account and run multiple repos against the same atlantis

PePe avatar

OR

PePe avatar

you could have all this files in all the repos and build atlantis every time from the scratch (codebuild+codepipeline) for every repo that have the webhooks configured

RB avatar

ah so this is a local installation of atlantis then

PePe avatar

this is a relevant thread that explains the why

PePe avatar

no this is for running on ECS+fargate

joshmyers avatar
joshmyers

I’m using https://github.com/terraform-aws-modules/terraform-aws-atlantis , whats the problem with it?

terraform-aws-modules/terraform-aws-atlantis

Terraform configurations for running Atlantis on AWS Fargate. Github, Gitlab and BitBucket are supported - terraform-aws-modules/terraform-aws-atlantis

RB avatar

ive listed my config further up in the thread and i can’t seem to get the github account to post on my pr when i write atlantis plan

RB avatar

first time using fargate so just starting to debug

RB avatar

ah I’ve gotten it to work. I forgot to add the webhook

RB avatar

now im hitting an issue where we use private terraform modules and the user doesn’t have access to those modules and so fails

RB avatar
authenticating for a module with a private github.com repo source · Issue #281 · runatlantis/atlantis

Hello, I&#39;m trying to use a module sourced from a private github repo: module &quot;module-name&quot; { source = &quot;git://github.com/<org>/<repo>.git?ref=0.0.2&quot;> } but …

joshmyers avatar
joshmyers

ATLANTIS_WRITE_GIT_CREDS maybe useful to you

RB avatar

yep that did it, thank you

2020-04-03

Alex Siegman avatar
Alex Siegman

For folks using atlantis.yaml per-repo settings files to define many projects, figured I’d see if I could reduce a lot of repetition in my config file using YAML Anchors and References. Came up with the following anchor system. While atlantis will process the two anchor templates as projects, the directory doesn’t exist so it doesn’t detect any changes and will not run. I couldn’t define the anchors in a separate section as atlantis got mad about it.

projects:
  # Project Definitions
  - &terraform_project
    name: "template-terraform-project"
    dir: '.empty'
    workflow: "make"
    workspace: "default"
    terraform_version: "v0.11.14"
    autoplan:
      when_modified:
        - "Makefile*"
        - "*.tf"
        - "*.tfvars"
        - "*.envrc"
      enabled: true
    apply_requirements:
      - "approved"

  ## Shared Infrastructure
  - <<: *terraform_project
    name: "account-dns"
    dir: "conf/account-dns"

  - <<: *terraform_project
    name: "aws-metrics-role"
    dir: "conf/us-east-1/aws-metrics-role"

  - <<: *terraform_project
    name: "chamber"
    dir: "conf/us-east-1/chamber"

  - <<: *terraform_project
    name: "cloudtrail"
    dir: "conf/cloudtrail"

It’s worked in my limited testing thus far.

2
:--1:1
joshmyers avatar
joshmyers

Could you have used e.g. account-dns as the template project or does it get invoked every time so needed to be a non existent thing? Not tested yet

    keyboard_arrow_up