Discussions related to https://github.com/cloudposse/geodesic
So, I started using Geodesic on a linux box (Amazon Linux 2 specifically, based on CentOS) rather than my normal OSX setup, and I’m noticing a lot of issues with being root in the container causing files to be root on my local file system and screwing with stuff. there a docker configration i’m missing somewhere? Looking in the startup scripts if you do a
make install I see it passing in USER_ID and GROUP_ID as environment variables in to the docker container on the command line, but not sure how geodesic is consuming those or if it has an effect on file system mounts
yea, no practical way to do that.
in geodesic you want to be able to install packages, so you want to be root.
on the local fs, it’s just mounted and there’s no way to set the uid/mapping
(e.g. like on NFS)
open to suggestions
Yeah, i’ve hit this problem in other scenarios and never solved it, was hoping someone smarter than me had
Hello. Im new to geodesic and attempting to install. During make process “ERROR: Not committing changes due to missing repository tags. Use –force-broken-world to override.” causes process to fail. My tool set is windows docker desktop with a cloned geodesic repo shared from windows dir to alpine Linux container. Testing install from cloned geodesic dir directly on alpine container does complete successfully. Ive attached error with full details. Any assistance is appreciated. Thanks
Wierd, in this case makes me think there’s something up with the line endings (perhaps related to windows?)
see how the
' character is at the beginning of the line? makes me thing something up with
Master successfully built 4 days ago, so I think it’s a windows incompatbility. Our windows support came from community contributions (We don’ thave any way to test it)
Thanks for the quick response Erik. I agree with you that it’s a windows environment problem. I’ll continue working the error out and will update the channel if able to resolve.
that would be great
we accept most PRs too
Is the newest package for terraform really 0.12.10 in the cloudposse packages? Or is it recommended to go back to the community packages for terraform?
We have been in tech debt hell.
GitHub actions changed somethings that broke our automations.
@Maxim Mironenko (Cloud Posse) is working to fix that
Upstream maintainers keep changing their release versions and artifacts breaking our auto updates
I hope we get the auto updates working again in the next day or so. @Maxim Mironenko (Cloud Posse) has been working on it all week.
No worries, was just in the process of updating all my images for a kubernetes update and was going to bump that too. Don’t have a need for it at the moment.
I went from using Geodesic somewhere around version 0.124.0 to the newest 0.129.1 a week or two ago and have noticed that my
assume-role doesn’t seem to be refreshing it’s session.
export AWS_VAULT_SERVER_ENABLED=true in the environment, I do see the message at startup
* assume-role will start EC2 metadata service at <http://169.254.169.254/latest>
But roles time out multiple times daily for me. The max duration for a session on the assumed role is 1 hour, but that used to be auto-refreshed for me. Now I have to backout and reassume the role every hour, rather than every 12.
Something change/break here?
I just saw that shortly after I did that, 0.129.2 came out. Can update to test that, but question still stands.
@Alex Siegman not an expert but the latter behaviour always happened to me since I started using geodesic from about 0.123.0.
I resolved it by setting an env var in ~/.geodesic/preferences.d/aws-vault
You can also set
but honestly I haven’t spent the time to figure out exactly which one has which effect.
Actually, sorry! Just realised you were talking about auto-refresh - I have never seen that working, but would like to!
I had it working; the key being allowing login sessions to your root account to last 12 hours, and assume role sessions being 1-12 hours, it should auto refresh. That’s the whole point of running the aws-vault server is so that it does that. It worked great for me in 0.124.X and before
I know Jeremy from cloudposse is kind of the guru on these things, hoping he chimes in
@Alex Siegman that is good to know. I would love to get that working over here too. Looking at the commits, I don’t see anything obvious around that time.
@Jeremy G (Cloud Posse) anything standout?
Is this related to the deprecation of
There are a few open issues regarding the proxy server already, and I am opening this new one with new information/issues/use cases and will try to consolidate some of the existing issues here. To …
i seem to recall there was something that changed outside of our control.
This is all due to changes in
Big changes in
aws-vault v5 https://github.com/99designs/aws-vault/releases/tag/v5.0.0
A vault for securely storing and accessing AWS credentials in development environments - 99designs/aws-vault
So the trick is probably for @Alex Siegman and @Joe Niland to pin
aws-vault to an earlier release inside of their
We might consider reverting Geodesic to aws-vault 4.7.1 until we and/or they figure out how to get caching and auto-renewing working again.
Thanks @Erik Osterman (Cloud Posse) and @Jeremy G (Cloud Posse) - I’ll try pinning.
I haven’t looked into it much because I haven’t been using aws-vault at all myself lately. I’ve been using aws-google-auth since just after aws-vault moved to v5
There were a lot of complaints about v5, including that it asks for MFA before password and doesn’t cache MFA, and they were going to fix some of that. Been waiting to see.
Thanks, if it’s a known issue with aws-vault I can pin it back to the newest 4.x
Yes, I say go back to 4.7.1 and let us know how it works out. If it’s good, I think we should roll back globally until v5 has the kinks worked out.
I am using the latest release of AWS Vault I have provided my .aws/config (redacted if necessary) I have provided the debug output using aws-vault –debug (redacted if necessary) When aws-vault ass…
Added the following to my Dockerfile:
# Pin aws-vault to a version <5.0 # There are bugs with aws credential caching that make version 5 more annoying to use; see: # <https://github.com/99designs/aws-vault/issues/552> RUN apk del aws-vault && apk add [email protected]==4.7.1-r0
Will see if that helps
@Alex Siegman before rolling back aws-vault to v4, make sure you have set up the new TTL variables
in v5? i can try that
I assume you mean these?
AWS_SESSION_TOKEN_TTL: Expiration time for the GetSessionToken credentials. Defaults to 1h AWS_CHAINED_SESSION_TOKEN_TTL: Expiration time for the GetSessionToken credentials when chaining profiles. Defaults to 8h AWS_ASSUME_ROLE_TTL: Expiration time for the AssumeRole credentials. Defaults to 1h AWS_FEDERATION_TOKEN_TTL: Expiration time for the GetFederationToken credentials. Defaults to 1h
The session token one is probably the big hoser here, should be 12h to operate like it did previously
That said, if the credentials aren’t cached, if assume_role_ttl is 1h, it will still ask every hour, so you’d have to change all of your profiles in AWS to allow >1h timing
But maybe bumping the session is enough
Yes, I was looking for where they were defined. You need to set them all, except maybe FEDERATION
I’ll play with those before reverting to 4.7.1 and report back. I just logged in with them set, should know in an hour or so I’d imagine
The idea with aws-vault is that you log in and it caches your session, including your MFA, so you can keep using it to assume roles.
Yeah, but if it doesn’t cache your session, it can’t re-assume roles without new MFA input I’d imagine
case in point, i just did an
aws-vault --debug list it decided my existing session was expired and deleted it, but my session is still valid (can use awscli) yet aws-vault shows no sessions for my profile
seems funky, now that I know where to look though I’ll play with it
I was excited about aws-vault v5 because the fixed some of the problems with running
aws-vault exec directly, which running
role-server was a workaround. But as long as we have role-server, if changing the TTLs in the env vars doesn’t work, then yes, go back to 4.7.1 and role-server.
Just a note - I couldn’t pin because –duration is an unsupported option in 4.7 (https://github.com/cloudposse/geodesic/blob/59d9637c32c29d3097859ad3d2c76fef0fb802b9/rootfs/etc/profile.d/aws-vault.sh#L132)
Geodesic is a cloud automation shell. It's the fastest way to get up and running with a rock solid, production grade cloud platform built on top of strictly Open Source tools. ★ this repo! h…
@Alex Siegman @Joe Niland Please check if you have
AWS_CHAINED_SESSION_TOKEN_TTL set as well.
@Alex Siegman @Joe Niland @Erik Osterman (Cloud Posse) PR for Geodesic to re-enable support for
aws-vault version 4, plus clean up some issues that may have been causing problems with
aws-vault version 5. I suggest trying the new Geodesic (when it is released) with v5 before reverting to v4. https://github.com/cloudposse/geodesic/pull/579
what [aws-vault] Simultaneous support for aws-vault v4 and v5 [helmfile] update to v0.111.0 Update other tools per cloudposse/packages why aws-vault released a major upgrade, going from version …
@Jeremy G (Cloud Posse) thank you - this is great!
OK, Geodesic version 0.130.0 is released. Give it a try and please report back.
Using just 0.130.0 with no changes, I still had timeouts due to aws-vault still not properly caching credentials it seems.
Will use your method to revert to aws-vault v4 and retest as able
How’s it is going?
I haven’t been able to get v5 to work refreshing stuff. I have not yet reverted as it was the weekend and I didn’t bother to test it yet
I just did the reversion, and initial debugging looks like it will work fine. I can see it caching sessions properly as well. Will use over next day or two to be sure. Should know later today if it properly renews sessions after 1hr timeout
Still doesn’t look like it’s working quite right. I have a valid session, the first hour has gone by so my role assumption should be expired and thus need refreshing, but using something that needs it to be refreshed does not cause it to be.
√ : [spoton-staging-admin] customer-auth-service ⨠ aws-vault --debug list 2020/04/28 04:58:35 [keyring] Considering backends: [file] 2020/04/28 04:58:35 Using AWS_CONFIG_FILE value: /localhost/.aws/config 2020/04/28 04:58:35 Loading config file /localhost/.aws/config 2020/04/28 04:58:35 Parsing config file /localhost/.aws/config 2020/04/28 04:58:35 [keyring] Expanded file dir to /conf/.awsvault/keys/ 2020/04/28 04:58:35 Looking up all keys in keyring 2020/04/28 04:58:35 [keyring] Expanded file dir to /conf/.awsvault/keys/ 2020/04/28 04:58:35 Session "session,c3BvdG9u,YXJuOmF3czppYW06OjE0ODQ4NzQwMjkxOTptZmEvYXNpZWdtYW5Ac3BvdG9uLmNvbQ,1588089054" expires in 10h52m18.485944511s Profile Credentials Sessions ======= =========== ======== default - spoton spoton 1588089054 (mfa) spoton-corp-admin spoton spoton-staging-admin spoton ⧉ staging √ : [spoton-staging-admin] customer-auth-service ⨠ aws s3 ls Unable to locate credentials. You can configure credentials by running "aws configure".
Container startup seemed normal
# Exposing port 34060 * aws-vault: overriding AWS_SESSION_TTL () with AWS_VAULT_SESSION_TTL (12h) * assume-role will start EC2 metadata service at <http://169.254.169.254/latest>
It does not, however, reask for MFA to get new session credentials if I exit out and rerun
assume-role so it is properly caching that.
Hi all. Can i ask how much will it cost to deploy
reference-architectures monthly assuming nothing ran/deployed on it ?
i will play with that with my personal account
so i can get a heads up
Just read Erik’s comment about reference-architecture on this channel
Hi all, I am new to
geodesic . Does anyone know where can I find some video tutorial/docs I can get myself up and running more in-depth?