#geodesic (2020-10)
Discussions related to https://github.com/cloudposse/geodesic
Archive: https://archive.sweetops.com/geodesic/
2020-10-07
what’s the place to start with geodesic? example repos like root.cloudposse.co haven’t been updated for a while. is reference-architectures repo the latest example to follow?
It’s deprecated. What is your goal?
So we still use geodesic
in of our engagements, but we don’t practice the “one repo per account” strategy any more
geodesic is really just an image that bundles all the tools we need. That said, even though it’s so simple, we’ve managed to open 500 PRs to get there
We’re actively working on releasing our next generation of reference architecture, but it probably won’t be available until Q1 2021.
we’re releasing bits and pieces as we go. the next important piece we’re releasing is our automation library built on top of #variant
we’re releasing some more opinionated modules around datadog, opsgenie, and terraform cloud too.
https://github.com/cloudposse/reference-architectures is also in transition, so there’s nothing to follow right now
[WIP] Get up and running quickly with one of our reference architecture using our fully automated cold-start process. - cloudposse/reference-architectures
we’ve deprecated the original reference architecture (as we’ve abandoned it)
and we’re launching a new one that will be found here.
Ok, thanks for the clarification. looking for a starter template to follow for a multi-account setup. using something like geodesic
to package all the dependent tools looks like a nice approach to combine with it.
I’m happy to jump on a quick call to give you the lay of the land and pointers to getting started
sure, i appreciate it. i’ll look for some time in the next few days
2020-10-08
2020-10-11
A while ago it was mentioned that geodesic will move to being based on Debian. At least I’m pretty sure I remember that!
Is there a timeline for this yet?
2020-10-12
No timeline, but a big step to being able to support that was migrating our repos to Cloudsmith which is done
It is something we may need to do for one of our current customers, because some things (e.g. teleport shell tsh
) no longer run on alpine
Thanks Erik!
I’d be happy to test it, if you need testers.
ok! follow along in cloudsmith - we’ll be transparent in the process there for adopting deb
2020-10-27
I am trying to use Geodesic and following the cold start documentation (https://docs.cloudposse.com/reference-architectures/cold-start/)
Things were going ok until I got to the PROVISION IAM PROJECT TO CREATE ROOT IAM ROLE
section. I believe that maybe the cd
command here is supposed to be cd root-iam
because the module identified is not used in the iam
directory and the plan identifies nothing to perform.
So I moved over to the root-iam
directory and performed the same steps but when I try to apply I only get 403
errors.
* module.organization_access_group_root.aws_iam_group.readonly: 1 error(s) occurred:
* aws_iam_group.readonly: Error creating IAM Group instinct-root-readonly: InvalidClientTokenId: The security token included in the request is invalid
status code: 403, request id: b06f7a1d-bd11-49d9-9fd9-b0b69dfebd02
....
....
Does anyone have ideas what might be going on here? I have tried all combinations of using the actual root AWS account, the admin account setup at the top of that page, and have disabled MFA in case it was getting in the way. I have been stuck for way to long on this step and can’t seem to find anything in the archives (though i could be missing something)
Looks like your token might have expired?
I thought so as well. I fiddled with reauthenticating, removing the credentials from aws-vault
and replacing them with no luck. I am not 100% clear on exactly how the token is acquired so I may be missing a step to try to re-authenticate. I was able to perform the earlier step and make the S3 bucket and setup the DynamoDB for the TF backend.
FYI, I did get past this after much trial and error. I ended up not running assume-role
before trying to apply and instead waiting for the assume role prompt and entering the role arn.
Sorry! that documentation is out of date. We have a new reference arch coming out by the end of the year that is revamped for 0.13+ (we’ve used this for the past year with our customers but haven’t open sourced it yet)
2020-10-28
Yes, I did see the notice at the top so I did expect to bump into some issues due to outdated docs. After getting stuck with Geodesic I also tried using the reference-architecture repo and got stuck trying to run make root
.
I am very excited to hear about a new release coming out, but unfortunately I am unable to wait to setup this architecture. I am committed to using your tooling as I believe in your design and approach to dev ops and the amazing infrastructure you produce for your clients.
Is there a path forward at this point?
We’re in a similar boat as chris. We’ve used the reference-architecture
repo in the past with success, but we are now in the process of setting up a new project and want to be prepared to transition to the new arch when it’s made available.
Do you have any tips, @Erik Osterman (Cloud Posse), in terms of setting up a new project so that transitioning into the new architecture is as painless as possible? We’re prepared to refactor things when the time comes, but are hesitant to follow the current/previous approach since it would seem the new approach is going to differ significantly.
So we still use geodesic
in of our engagements, but we don’t practice the “one repo per account” strategy any more
i’m sorry - I really wish we could release the new ref arch faster. it leverages entirely new root modules (plus all of our public modules), uses yaml based configuration, runs in geodesic, works with terraform cloud… but unfortunately, we just don’t have the available hands right now. I started working with a community member (@Matt Gowie) to release it faster, but realized there’s too much tacit knowledge required to setup. we are starting a new engagement in the next few weeks. as part of this, we will be taking this opportunity to publish more of the reference architecture.
strategically, this is very important for us - it’s definitely happening - but is a large initiative
Hi Erik, I had Nate reach out on here to find out what the update on the new ref arch was. I’ve been fairly under water, and he just joined us to start work on large a new project, so we wanted to align to the new arch if possible. I 100% understand the tacit knowledge comment, and the overhead of having someone help, but feel free to reach out to us if we can assist (e.g., focused PR reviews, contributions or alpha testing) in any way.
What I really need to do is start a list - because I am losing count
@Andy Miguel let’s discuss setting that up
@Erik Osterman (Cloud Posse) ack
thanks for the update!
@Nathan Margaglio I tried to use reference-architecture
but got stuck there… if you have notes that you could share it would be much appreciated.
I am currently working on figuring out the next steps on moving from setting up the accounts to setting up services (e.g. EKS) and the correct way to structure the repo… lots more reading to do.
In reading my blocker may be a terraform version issue. But I also got parts of the accounts setup going through the cold-start documentation… So I am going to do a lot more research to see if I should continue down the path outlined in cold-start or back out and use the reference-architecture design.
I’m not sure if you saw this, but it was invaluable when we used the reference arch. I don’t think there was much else other than the official docs + this that we needed: https://gist.github.com/joe-niland/b96150bfc13828c2a58751dfca7ffe7e
Thanks that looks like a lot of great information. The next part I have to figure out is going from cold start to deployed services (eg VPC, EKS, etc) so that is what I am digging into now
@Joe Hosteny Thanks for the link it did get me unstuck and I made a lot of progress! Unfortunately, I just bumped into another issue and am digging into it now
terraform init
Copying configuration from "git::<https://github.com/cloudposse/terraform-root-modules.git//aws/accounts?ref=tags/0.58.1>"...
Initializing modules...
- module.audit
Getting source "stage"
- module.corp
Getting source "stage"
- module.data
Getting source "stage"
- module.dev
Getting source "stage"
- module.identity
Getting source "stage"
- module.prod
Getting source "stage"
- module.security
Getting source "stage"
- module.staging
Getting source "stage"
- module.testing
Getting source "stage"
Initializing the backend...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
- Checking for available provider plugins on <https://releases.hashicorp.com>...
- Downloading plugin for provider "aws" (2.70.0)...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.aws: version = "~> 2.70"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
parse error: Invalid numeric literal at line 1, column 2
Goodbye
I don’t recall seeing that, but the last time I did this was probably 8 months ago, with a different version of the provider
I am really at a cross roads of determine what is the best way to get started at this point and get something working that also has some path forward.
2020-10-29
FWIW, I set up one client using the reference-architecture last year. It’s going well, but there are a few limitations with using the full terraform featureset (due to the -from-module approach.)
For all clients since then I’ve simplified things and have been using Terragrunt and Terraform natively. To be fair, all of these clients are on Mac/Linux or I am managing it all for them (on Mac.) None of them need CI/CD with Terraform. So the simple approach is working well.
For authentication, i’m using aws-vault with my AWS account assuming roles in the clients’ and the awscli’s credential_process
option.
That said, I am really looking forward to see what the Cloudposse team have been working on for their updated reference architecture!
This Terragrunt reference architecture could help some of you in the meantime. It’s not exactly how I’m using it, but pretty close.
Terragrunt Reference Architecture (upd: May 2020). Contribute to antonbabenko/terragrunt-reference-architecture development by creating an account on GitHub.
2020-10-30
Thanks for the information. I will look at that as well. I have pushed through and it appears I have got everything setup using the reference-architecture repo. I am going to make a gist of my notes and where I bumped into issues for anyone else that is trying to use it at this time.
@chris it slipped my mind earlier but I created a gist like this myself. Perhaps it’s out of date now. Not sure.
https://gist.github.com/joe-niland/b96150bfc13828c2a58751dfca7ffe7e
Thanks. Someone else also linked this to me… very useful and helped me past a few issues! Thanks again.