#geodesic (2021-03)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic Archive: https://archive.sweetops.com/geodesic/

2021-03-31

2021-03-30

Marcin Brański avatar
Marcin Brański

I’m doing reference architecture with atmos :cloudposse:

Vendir sync need to download repos defined in vendir.yaml to put them to components. There are 16 modules using https://github.com/cloudposse/terraform-aws-components

I think there are two problems slowing me down: • vendir doesn’t have caching layer for modules using same version • every change to vendir.yaml need to redownload all modules and compare them to components even if nothing has changed (.vendir-tmp/git is cleared after every module downloaded) • vendir seems slow… real 4m46.736s I know it’s partially because the repo is quite big but just using git it’s super quick to clone it. vendir lastest 0.17.0

I wonder how it behave with many many modules and upgrades to them. Is there a plan to make it more quick?

cloudposse/reference-architectures

[WIP] Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures

cloudposse/terraform-aws-components

Opinionated, self-contained Terraform root modules that each solve one, specific problem - cloudposse/terraform-aws-components

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Using vendir effectively is blocked by https://github.com/vmware-tanzu/carvel-vendir/issues/37

support ignorePaths to exclude removal of certain files · Issue #37 · vmware-tanzu/carvel-vendir

Describe the problem/challenge you have in some cases it's useful to retain certain temp files among fetched directories. (e.g. temp files generated by tools that were operating on top of fetch…

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

So we’re currently not using it until we can get around to opening a PR for this.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Generally, upgrading of root modules (components) is not feasible in a mature environment.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Root modules are the foundation and upgrading them is often a violent operation.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Instead, our root modules (components) point to upstream modules which can be upgraded.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Our (cloudposse) objective using vendir is primarily to accelerate the coldstart process.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

also, @Marcin Brański our terraform-aws-components there are already quite out of date. We have new ones. If you reach out to me directly, I can probably get you access to newer ones provided you help us upstream them.

Marcin Brański avatar
Marcin Brański

Yeah man. I noticed they are outdated so I had to use tf13 because of that.

Our (cloudposse) objective using vendir is primarily to accelerate the coldstart process. Understand, I thought it’s new pattern where vendir would populate components but it seems we should be putting new modules (manually) to components dir and not execute vendir beside bootstrapping, yeah?

Early access to updated modules would be great.

What I ran into today is that variables (none of them, context as well) from terraform module are not passed to modules

  tfstate-backend:
    vars:
      namespace: terraform
      force_destroy: false
      prevent_unencrypted_uploads: true
      enable_server_side_encryption: true

but global vars works… I need to wrap my head around that…

So for multiaccount setup it’s recommended multi repo?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


t it seems we should be putting new modules (manually) to components dir and not execute vendir beside bootstrapping, yeah?
Correct

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


Early access to updated modules would be great.
Yup, shoot me an email and let’s discuss. This benefits everyone.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


What I ran into today is that variables (none of them, context as well) from terraform module are not passed to modules
Nah, that’s probably just a whitespace issue

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

component vars are passed to the terraform modules.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)


So for multiaccount setup it’s recommended multi repo? (
No - we’ve entirely walked away from this approach. We use a mono repo for infrastructure.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

mult-repo is not gitops friendly - it makes coldstarts expensive and difficult to automate

Marcin Brański avatar
Marcin Brański

I will shoot you an email later today when I will be more familiar with atmos . I need some more time to ask better questions but I love the idea

My bad with not passing context was that I mixed reference architecture, atmos example and atmos tutorial which resulted in wrong yaml definition. I had:

components:
    vpc:
      vars:
        cidr_block: "10.100.0.0/16"

where it should be

components:
  terraform:
    vpc:
      vars:
        cidr_block: "10.100.0.0/16"
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

ya, yes, that ref arch is out-of-date and old/misleading

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

sigh… so many easter eggs! just in time for easter

2021-03-26

2021-03-25

2021-03-24

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
4
Matt Gowie avatar
Matt Gowie

@Erik Osterman (Cloud Posse) just published the updates from https://github.com/cloudposse/docs/pull/531 for this one.

Also, I’m of the mind that we should create a #sweetops or similar channel to announce these docs / discuss the methodology holistically. What do you think?

feat: updates atmos getting started for tutorial updates by Gowiem · Pull Request #531 · cloudposse/docs

what Updates to use date variable in stack instead of unit_of_measurement Updates to language surrounding the same. why We updated the tutorial code to be a bit different, so this corresponds wi…

Marcin Brański avatar
Marcin Brański

Thanks Matt! I followed this one and couldn’t get through docker run . It outputted bash script. I ran make install instead.

Matt Gowie avatar
Matt Gowie

@Marcin Brański* it might have been that you had an old image of geodesic on your local machine? I would delete all older images of Geodesic on your machine and try again — I’ve run into that myself with the new version of geodesic since it’s easy to do.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Marcin Brański that means it’s an old image. Those UX issues are fixed

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

A Docker pull will resolve it

2021-03-12

Markus Muehlberger avatar
Markus Muehlberger

Hi everyone! It’s been a while since I’ve been active in the SweetOps community (after participating in the very first office hours with 3 people), but I’ve been following the excellent work Cloud Posse’s been doing over the years for the Terraform community.

I’m not sure which channel the best place is to ask this question, but I’m recently been trying to wrap my head around atmos and how it can work within companies. There doesn’t seem to be a channel for it yet (and it uses geodesic).

There’s two things that I’m a bit unsure about at the moment (with the old reference-architectures in mind):

  1. Is the atmos approach (with vendir.yml) assuming that all modules have the same version (e.g. if you have a vpc in multiple workflows)
  2. As the reference architectures have been around atlantis for automation, I can see some references in the example files for spacelift. I’m guessing automation would move away from Atlantis towards Jenkins or other services. Is it still viable to do some form of GitOps with atmos?
jose.amengual avatar
jose.amengual

Spacelift is gitops and you can use atmos and Atlantis with a bit of craft

jose.amengual avatar
jose.amengual

In Atmos you can have different version modules per component

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@ we’re slowly releasing the documentation and tutorials

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We haven’t yet adopted vendir due to some missing functionality

support ignorePaths to exclude removal of certain files · Issue #37 · vmware-tanzu/carvel-vendir

Describe the problem/challenge you have in some cases it's useful to retain certain temp files among fetched directories. (e.g. temp files generated by tools that were operating on top of fetch…

Markus Muehlberger avatar
Markus Muehlberger

Thanks @jose.amengual and @Erik Osterman (Cloud Posse). This is tutorial looks really helpful for some things I’m missing.

One final question though: Is atmos designed to be a mono repo, or are you still splitting repositories up per AWS account (as I’ve seen different files for dev, staging and prod in the same example)?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We’ve entirely abandoned one-repo-per-account. Mea culpa. That pattern didn’t make gitops easy.

2
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Our current approach was designed with gitops in mind from the ground up

Markus Muehlberger avatar
Markus Muehlberger

That sounds really amazing! Thanks Erik!

2021-03-07

2021-03-06

2021-03-05

Fernando avatar
Fernando

Hi everyone,

Fernando avatar
Fernando

I’ve been going through the reference-architecture README file and at some point it says:

Update the configuration for this account by editing the configs/master.tfvar file.

But I can’t find any configs folder in the repo, is there something am I missing? Thanks

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

This repo is currently out of date and docs won’t help you too much

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have some fresh documentation up at https://docs.cloudposse.com/

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We archived everything that was no longer relevant and are starting to rollout the current docs

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Geodesic tutorial dropped this week

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

But geodesic is just one once. The ref arch docs are still pending.

Fernando avatar
Fernando

Thanks Erik, I’ll take a look

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
1
joey avatar

odd - in chrome and chrome canary the text is not wrapping

Matt Gowie avatar
Matt Gowie

Wow that’s new…. will look into it.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

fixed!

1
    keyboard_arrow_up