#geodesic (2021-02)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2021-02-25

2021-02-23

Mathieu SERRA avatar
Mathieu SERRA

Hi everyone, I have a question regarding all modules in aws-terraform-component. In most of this modules you require the module iam-roles. But you call it with few parameters:

module "iam_roles" {    
  source      = "../account-map/modules/iam-roles"    
  stage       = var.stage    
  assume_role = false    
  region      = var.region  
}

But this module need other information to access remote state of account-map. Like the account id, the namespace or the bucket envirnoment_name. I think someting like that can work:

module "iam_roles" {
  source      = "../account-map/modules/iam-roles"
  stage       = var.stage
  tfstate_assume_role = false
  tfstate_bucket_environment_name = var.tfstate_bucket_environment_name
  region      = var.region
  context     = module.this.context
  tfstate_account_id = var.tfstate_account_id
}

Or maybe you have a way in atmos to pass this variables to the module directly. Thanks

2021-02-17

Robert Horrox avatar
Robert Horrox

@Erik Osterman (Cloud Posse) Dropping this here for reference, with the latest version of Docker Desktop for Mac you can run into a high cpu issue when mounting the home directory (https://github.com/docker/for-mac/issues/5200). Disabling the GRPC Fuse filesystem make the CPU issue go away.

com.docker.backend 100% CPU when mounting home directory · Issue #5200 · docker/for-mac

I have tried with the latest version of my channel (Stable or Edge) I have uploaded Diagnostics Diagnostics ID: 7576B821-77FA-4C5C-91C4-3E1ABEA6A7FD/20201230115951 Expected behavior Should have no …

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

hrmmm @Jeremy (Cloud Posse) mentioned something about the problem being introduced. have you seen this workaround?

com.docker.backend 100% CPU when mounting home directory · Issue #5200 · docker/for-mac

I have tried with the latest version of my channel (Stable or Edge) I have uploaded Diagnostics Diagnostics ID: 7576B821-77FA-4C5C-91C4-3E1ABEA6A7FD/20201230115951 Expected behavior Should have no …

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Docker really blew it with the 3.x version and people are pretty steamed. If there were an alternative to Docker, people would be flocking to it, which I am sure has some people working to set up an alternative to it.

Yes, I knew about turning off gRPC Fuse. Sadly, the alternative, OSX Fuse, as used by Docker, seems to have a memory leak. I had it using > 70 GB of swap. But the CPU is manageable and memory leaked to swap is not a big deal since by virtue of it being leaked, it is never paged back in.

Robert Horrox avatar
Robert Horrox

podman is the best alternative I can see https://podman.io/. That could be combined with https://github.com/gyf304/vmcli or https://github.com/machyve/xhyve or https://github.com/moby/hyperkit to make a suitable docker alternative for OSX. You could also use https://developers.redhat.com/blog/2020/02/12/podman-for-macos-sort-of/

gyf304/vmcli

A set of utilities (vmcli + vmctl) for macOS Virtualization.framework - gyf304/vmcli

machyve/xhyve

xhyve, a lightweight OS X virtualization solution. Contribute to machyve/xhyve development by creating an account on GitHub.

moby/hyperkit

A toolkit for embedding hypervisor capabilities in your application - moby/hyperkit

Podman for macOS (sort of) - Red Hat Developer attachment image

See how podman-machine (mostly) lets you run Buildah, Podman, and skopeo on your Mac without having to build your own Linux VM.

2021-02-14

Michael Dizon avatar
Michael Dizon

in a workflow, is it possible to use the outputs from terraform as inputs for helm?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

are we talking about atmos ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

…are you using helmfile with helm?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

…what’s calling helm?

Michael Dizon avatar
Michael Dizon

woops, yeah atmos. i think i read something about helmfile supporting tfstate

Michael Dizon avatar
Michael Dizon

er tfstate-lookup

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

exactly - so you shouldn’t have state based between anything in atmos, you should use the tools ability to access remote state.

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

helmfile can read from SSM or terraform remote state

Michael Dizon avatar
Michael Dizon

loving this so far

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/helmfiles

Comprehensive Distribution of Helmfiles for Kubernetes - cloudposse/helmfiles

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

here’s how to read from SSM in helmfile

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

if you’re using pure helm, then what you’d want is that terraform uses the kubernetes provider to write a ConfigMap or Secret instead

Michael Dizon avatar
Michael Dizon

got it. going to jump on this tonight when i get back. thanks for help!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

also, once you have this stuff working in atmos, you should be able to use spacelift very easily too.

2021-02-12

2021-02-11

2021-02-10

Dan (Cloud Posse) avatar
Dan (Cloud Posse)
03:39:14 PM

@ has joined the channel

RB avatar

im a geodesic noob. how does one run it ?

docker run -it cloudposse/geodesic
docker run -it cloudposse/geodesic bash

both fail

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Oh, so when you run it with no arguments it outputs a bash script installer

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
Run it like docker bash
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

(On my phone

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The result is a script is installed that helps you start it since running long Docker commands is no fun

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

If you just want to kick the tires and skip installation, then add -l

RB avatar

ah ok now it works. ya i just wanted to try it out

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
docker run -it cloudposse/geodesic -l
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

The way we use it is as a base Docker image for our customers

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We also have Debian and CentOS images

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We like to mount $HOME to /localhost so we can access files during development (e.g. tf)

RB avatar

you dont mount it to /conf the root $HOME ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

No…

RB avatar

oh i see i see. got it working

RB avatar
docker run -v "$HOME:/localhost" -it cloudposse/geodesic -l
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Conf is where we copy the configurations for distribution in the Docker file that is FROM geodesic

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
cloudposse/packages

Cloud Posse DevOps distribution of linux packages for native apps, binaries, alpine packages, debian packages, and redhat packages. - cloudposse/packages

1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

All of these tools are easily installed. We update every night automatically

RB avatar

hmmm

⨠ aws-vault exec sso_sre --server
Enter passphrase to unlock /conf/.awsvault/keys/:
Enter passphrase to unlock /conf/.awsvault/keys/:
Enter passphrase to unlock /conf/.awsvault/keys/:
aws-vault: error: exec: fork/exec : no such file or directory
RB avatar

thanks for the help. i’ll keep playing with it

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

I think we do some symlinking for that

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Check out /etc/profile.d

RB avatar

ya i was trying to use the same profiles i have setup locally

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

That should work

RB avatar

i can see my profiles when doing aws-vault list

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Aha are you on a Mac?

RB avatar

lol. yessir

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So I think you need to use the vault backend of file for it to work across OS barriers

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Docker cannot access keychain

1
RB avatar

Used

docker run --env AWS_VAULT_BACKEND=file -v "$HOME:/localhost" -it cloudposse/geodesic -l

then

⨠ env | grep AWS_VAULT
AWS_VAULT_SERVER_ENABLED=false
AWS_VAULT_BACKEND=file
AWS_VAULT_ENABLED=false
 ⧉  geodesic
 √ . [default] ~ ⨠ aws-vault exec sso_sre --server
Enter passphrase to unlock /conf/.awsvault/keys/:
aws-vault: error: exec: fork/exec : no such file or directory
RB avatar

this seems to work tho

aws-vault exec sso_sre -- aws sts get-caller-identity
Enter passphrase to unlock /conf/.awsvault/keys/:

{
    "UserId": "snip",
    "Account": "snip",
    "Arn": "snip"
}
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy (Cloud Posse) may know

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)
aws-vault exec --server sso_sre -- /bin/bash -l

2021-02-09

Mathieu SERRA avatar
Mathieu SERRA

Hi Everyone. I’m trying to reproduce your architecture for a test project. I’m succeded in using atmos. Very helfpful tool. But I’m bit struggling to get the differences between the “iam-primary-role” and “iam-delegated-roles” modules in https://github.com/cloudposse/terraform-aws-components . If I correctly understand , the “iam-primary-role” need to be only run on root account and the “iam-delegated-role” on all other account. Is that correct ? Second question related to the first one. In the “iam-primary-role” you two have two variables delegated_roles_config and primary_roles_config . I’ve noticed that this two variables are merged together to create roles. My question is when I need to use the first one and when I need to use the second one. I see too that delegated_roles_config varaible is use by “iam-delegated-modules” by reading the tfstate output of “iam-primary-role”. Thanks for your help and your work

cloudposse/terraform-aws-components

Catalog of reusable Terraform components and blueprints for provisioning reference architectures - cloudposse/terraform-aws-components

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@ we have some incoming documentation on this from @Matt Gowie

cloudposse/terraform-aws-components

Catalog of reusable Terraform components and blueprints for provisioning reference architectures - cloudposse/terraform-aws-components

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
feat: adds component documentation + pre-commit + rebuild-docs target by Gowiem · Pull Request #297 · cloudposse/terraform-aws-components

what Updates pre-commit to include terraform_docs + bumps to 0.45.0 Updates pre-commit workflow for terraform_docs checking Adds information on installing / using pre-commit Adds rebuild-docs targ…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Should be merged today

1
1
Mathieu SERRA avatar
Mathieu SERRA

Ok thanks for your help. It’s help a lot. One question regarding variables assume_role_restricted in iam-primary-roles. I need to set to false first when I created all the roles for a cold start. But does I need to relaunch terraform process after creation with this variable to true in the identity account ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy (Cloud Posse) @

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Sorry it is so complex, @, but you seem to be getting the hang of it. We recommend creating an identity account separate from the root account and creating the primary roles in that account.

When assume_role_restricted is true, we add limitations on roles to prevent them from assuming other roles in the same account. For example, identity-observer is not allowed to assume identity-ops. However, we cannot create that restriction until after both identity-observer and identity-ops have been created, so when first creating the roles you have to set assume-roles-restricted to false so we do not create rules referencing non-existent roles. Then you need to do terraform apply in order to actually create the roles. After the roles are created, you can set assume-role-restricted back to true and terraform apply to create the restrictions.

1
Mathieu SERRA avatar
Mathieu SERRA

@Jeremy (Cloud Posse) Thanks for yor complete answer. I’ve followed your instruction but I’m facing an issue. I’ve taken your example and put this in var for the module iam-primary-roles :

delegated_roles_config:
          admin:
            role_policy_arns: [ "arn:aws:iam::aws:policy/AdministratorAccess" ]
            role_description: "Role with AdministratorAccess permissions"
            sso_login_enabled: false
            # list of roles in primary that can assume into this role in delegated accounts
            # primary admin can assume delegated admin
            trusted_primary_roles: [ "admin" ]

I take this configuration from you readme. But I juste disable the sso for the moment I will only use aws user. For the first appliance with assume_role_restricted to false no error the role is correctly created. But when I set assume_role_restricted to true and relaunch terraform apply the assume role policy of the admin role is replace with null:

assume_role_policy    = jsonencode(
          ~ {
              ~ Statement = [
                  - {
                      - Action    = [
                          - "sts:TagSession",
                          - "sts:AssumeRole",
                        ]
                      - Effect    = "Allow"
                      - Principal = {
                          - AWS = "arn:aws:iam::x:root"
                        }
                      - Sid       = "IdentityAccountAssume"
                    },
                ] -> null
                Version   = "2012-10-17"
            }
        )

If we look your code it’s seems that you exclude in trusted_primary_roles all the role who has the same name that the role to create or update.

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

Yes, no identity role is allowed to assume the identity-admin role. Also, because of the limitations of role chaining, we do not allow any role to assume itself. We would rather have an error if that situation arises so we can fix whatever is causing that to happen.

Mathieu SERRA avatar
Mathieu SERRA

Thanks for your answer. You said that you not allow any role to assume itself but in readme of iam-primary-roles you put the admin as himself trusted primary role:

delegated_roles_config:
          admin:
            role_policy_arns: [ "arn:aws:iam::aws:policy/AdministratorAccess" ]
            role_description: "Role with AdministratorAccess permissions"
            sso_login_enabled: false
            # list of roles in primary that can assume into this role in delegated accounts
            # primary admin can assume delegated admin
            trusted_primary_roles: [ "admin" ]

And how I can correctly configure the admin role with no trusted_primary_roles and sso disable without having a null assume_role_policy when I set assume_role_restricted to true ? Thanks

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

admin is trusted in delegated roles. This is a shorthand for saying identity-admin can assume otherAcct-admin. When you set assume_role_restricted = true then the roles are automatically restricted from assuming themselves as well.

This is the point of having target_role here

if ! contains([target_role, "cicd"], source_role) 
Mathieu SERRA avatar
Mathieu SERRA

Thanks for your answer. I have know a better view of how it’s work. Do you know I can fix my issue when I put assumed-role-restricted to true. When I set to true the assume role policy of the root role become null. So terraform raise an error:

assume_role_policy    = jsonencode(
          ~ {
              ~ Statement = [
                  - {
                      - Action    = [
                          - "sts:TagSession",
                          - "sts:AssumeRole",
                        ]
                      - Effect    = "Allow"
                      - Principal = {
                          - AWS = "arn:aws:iam::x:root"
                        }
                      - Sid       = "IdentityAccountAssume"
                    },
                ] -> null
                Version   = "2012-10-17"
            }
        )

I have the following config for the delegated-role-config

delegated_roles_config:
          admin:
            role_policy_arns:
              - "arn:aws:iam::aws:policy/AdministratorAccess"
            role_description: "Admin role"
            sso_login_enabled: false
            trusted_primary_roles: [admin]
          terraform:
            role_policy_arns:
              - "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess"
              - "delegated_assume_role"
            role_description: "Role with permissions for terraform automation"
            sso_login_enabled: false
            trusted_primary_roles: ["admin", "terraform"]

Thanks

Jeremy (Cloud Posse) avatar
Jeremy (Cloud Posse)

You need to give me more context. I can’t find jsonencode anywhere. Are you using the latest versions of everything?

    keyboard_arrow_up