#geodesic (2023-10)

geodesic https://github.com/cloudposse/geodesic

Discussions related to https://github.com/cloudposse/geodesic

Archive: https://archive.sweetops.com/geodesic/

2023-10-04

Matt Gowie avatar
Matt Gowie

Hey @Jeremy G (Cloud Posse) + other Geodesic users – I’m struggling to get Geodesic running on an engineers Windows machine. Is that something anyone here has gotten working? We’re running into this error when starting via the wrapper script:

# Starting new toolbox session from toolbox:latest
# Exposing port 32440
docker: Error response from daemon: mkdir C:\Users\<USERNAME>;C: Access is denied.
See 'docker run --help'.

I know that sadly isn’t a lot to go on… but just reaching out in case. When searching around GH issues I didn’t see anything. Wondering if anyone here has seen that and has any advice to debug. We don’t have a windows machine at Masterpoint so we’re struggling.

Matt Gowie avatar
Matt Gowie

It looks like this may be due to permissions issues on this engineers windows box.

Having them run these two commands pointed me to that:

  1. docker run -it --rm --volume "$HOME:/localhost" toolbox:latest -c bash
  2. docker run -it --rm toolbox:latest -c bash The first results in the same error as above. The second does not.
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy White (Cloud Posse)

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

It should work on WSL

this1
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

We have customers using it, but there are definitely issues on some machines. For example, if users are using onedrive for their home

Matt Gowie avatar
Matt Gowie

There is also this: https://github.com/docker-archive/toolbox/issues/673#issue-242634430


if users are using onedrive for their home
I will check up on this…

We also weren’t pushing multi-platform images, so we just started doing that as well.

Jeremy White (Cloud Posse) avatar
Jeremy White (Cloud Posse)

I’ll get my notes together. I’ve gone through this a few times.

Jeremy White (Cloud Posse) avatar
Jeremy White (Cloud Posse)

First off, @Matt Gowie, can you answer a few of these (all would be great, but at least three should help a lot):

  1. is this on WSL v1, WSL v2, or from a Powershell/cmd on the host machine?
  2. is this Windows 10 or Windows 11?
  3. if they are not running on WSL v2, is there any reason they cannot?
  4. often the intent of the volume mount is to help with development workflow. Is that the case here? Or is this just to run geodesic for essential commands? A common way of working on Windows while targeting Linux is to use WSL v2 and then have your code editor use remote development mode… pointed at that environment. Here’s some docs on this for vscode and also some docs on this for anything JetBrains. Note that the JetBrains solution costs a license. Another alternative to WSL v2 is just to run with a remote connection to a docker host. Some folks even prefer this since it can speed up Terraform init considerably when your host is downloading binaries straight out of the datacenter pipes.

I wouldn’t say remote development is always ideal, but often local access to the files is not as important as a network connection.

Dan Miller (Cloud Posse) avatar
Dan Miller (Cloud Posse)

Also, we do have this bit of docs on setting up Geodesic on Windows within our reference architecture: https://docs.cloudposse.com/reference-architecture/fundamentals/#docker-on-windows

Matt Gowie avatar
Matt Gowie

Hey folks – we ended up getting this working by using multi-platform builds AND the MSYS_NO_PATHCONV=1 env var. That enabled us to at least start our toolbox with -c bash and then we are directing users to change directory to wherever the project is, which is totally fine. Thanks for the follow ups though and appreciate the help!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Wow, MSYS_NO_PATHCONV=1 looks like it might fix one of the things we’ve been struggling with!!

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)
	elif [[ $(uname -r) =~ Microsoft$ ]]; then

		# Lookup correct mount path for WSL
		mount_path=$(dirname $(findmnt -S C: -t drvfs -no target))

		windows_user_name=$($mount_path/c/Windows/System32/cmd.exe /c 'echo %USERNAME%' | tr -d '\r')
		user_local_app_data=$(cmd.exe /c echo %LOCALAPPDATA% | tr -d '\r' | sed -e 's/\\/\//g')

		if [ -d "$mount_path/c/Users/${windows_user_name}/AppData/Local/lxss/" ]; then
			local_home=${user_local_app_data}/lxss${HOME}
		else
			local restore_nullglob=$(shopt -p nullglob)
			shopt -s nullglob
			for dir in $mount_path/c/Users/${windows_user_name}/AppData/Local/Packages/CanonicalGroupLimited.Ubuntu*; do
				folder_name=$(basename ${dir})
				local_home=${user_local_app_data}/Packages/${folder_name}/LocalState/rootfs${HOME}
				break
			done
			$restore_nullglob
		fi

		if [ -z "${local_home}" ]; then
			echo "ERROR: can't identify user home directory, you may specify path via LOCAL_HOME variable"
			exit 1
		else
			echo "Detected Windows Subsystem for Linux, mounting $local_home instead of $HOME"
		fi
Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Trying to find the real local home directory

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@Matt Gowie Thank you for letting us know about MSYS_NO_PATHCONV. I’m wondering if, without that set, this command works for you:

docker run -it --rm --volume "/$HOME:/localhost" toolbox:latest -c bash
Matt Gowie avatar
Matt Gowie

I believe we tried that command, but without the / in front of $HOME and it didn’t work. It has been a bit now and I actually don’t have access to the windows machines in question to test, so I can’t say, but I think we tried that and it failed.

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

I’ve read that adding the initial /, causing the expanded value to start with //, would prevent the problematic conversion, but I don’t have a system to test it on.

Matt Gowie avatar
Matt Gowie

Ah I was wondering that fronting slash made a difference. I can ask someone to try next time we’re walking their devs through something. If that happens, I’ll let you know.

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

NOTE: This appears to be specific to using “Git Bash” . We recommend you use WSL instead. (WSL appears to have been shipping pre-installed on Windows since 2004.) Personally, I recommend Ubuntu under WSL, as it is more user friendly, but we do use Debian for Geodesic, so that’s a reasonable choice for WSL as well if you are mainly going to be using Geodesic.

Git for Windows

We bring the awesome Git VCS to Windows

Install WSLattachment image

Install Windows Subsystem for Linux with the command, wsl –install. Use a Bash terminal on your Windows machine run by your preferred Linux distribution - Ubuntu, Debian, SUSE, Kali, Fedora, Pengwin, Alpine, and more are available.

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

Apparently there’s a wslpath command we can use to also simplify finding paths.

Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@Erik Osterman (Cloud Posse) This particular problem is about using “Git Bash” (a descended of Cygwin) to give you a bash command line without using WSL. Git Bash by default converts POSIX paths to Windows paths where it things arguments are paths so it can interoperate with Windows commands. This is just a hacky solution suited to the 1990’s and given that WSL is free with Windows, not something we should spend time supporting. That said, we have 2 solutions available:

  1. export MSYS_NO_PATHCONV=1
  2. Add an extra / to the beginning of the directory path

2023-10-06

2023-10-08

Release notes from geodesic avatar
Release notes from geodesic
01:50:41 AM

v2.6.0 Update alpine Docker tag to v3.18.4 @renovate (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”1918367176” data-permission-text=”Title is private” data-url=”https://github.com/cloudposse/geodesic/issues/879“…

Release v2.6.0 · cloudposse/geodesicattachment image

Update alpine Docker tag to v3.18.4 @renovate (#879) This PR contains the following updates:

Package Type Update Change

alpine final patch 3.18.3 -> 3.18.4

alpine stage patch 3.18.3 -> 3.1…

Update alpine Docker tag to v3.18.4 by renovate[bot] · Pull Request #879 · cloudposse/geodesicattachment image

This PR contains the following updates:

Package Type Update Change

alpine final patch 3.18.3 -> 3.18.4

alpine stage patch 3.18.3 -> 3.18.4

Configuration Schedule: Branch creation - A…

2023-10-11

2023-10-12

2023-10-16

Release notes from geodesic avatar
Release notes from geodesic
07:10:41 PM

v2.7.0 Update Debian to v11.8 @renovate (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”1938615645” data-permission-text=”Title is private” data-url=”https://github.com/cloudposse/geodesic/issues/881” data-hovercard-type=”pull_request”…

Release v2.7.0 · cloudposse/geodesic

Update Debian to v11.8 @renovate (#881) This PR contains the following updates:

Package Type Update Change

debian final minor 11.7-slim -> 11.8-slim

Included Tools

Update AWS CLI …

Update debian Docker tag to v11.8 by renovate[bot] · Pull Request #881 · cloudposse/geodesic

This PR contains the following updates:

Package Type Update Change

debian final minor 11.7-slim -> 11.8-slim

Configuration Schedule: Branch creation - At any time (no schedule defined)…

2023-10-21

Release notes from geodesic avatar
Release notes from geodesic
08:40:38 PM

v2.7.1 Included Tools

Update AWS CLI packages @renovate (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”1955221298” data-permission-text=”Title is private” data-url=”https://github.com/cloudposse/geodesic/issues/883“…

Release v2.7.1 · cloudposse/geodesicattachment image

Included Tools

Update AWS CLI packages @renovate (#883) This PR contains the following updates:

Package Change Age Adoption Passing Confidence

awscli (source, changelog) ==1.29.63 -> ==…

Update AWS CLI packages by renovate[bot] · Pull Request #883 · cloudposse/geodesicattachment image

This PR contains the following updates:

Package Change Age Adoption Passing Confidence

awscli (source, changelog) ==1.29.63 -> ==1.29.68

boto3 ==1.28.63 -> ==1.28.68

Release No…

2023-10-27

Hans D avatar

any plans on updating geodesic to debian 12 ?

Erik Osterman (Cloud Posse) avatar
Erik Osterman (Cloud Posse)

@Jeremy G (Cloud Posse) let’s get a task on the backlog, cc @Gabriela Campana (Cloud Posse)

1
Jeremy G (Cloud Posse) avatar
Jeremy G (Cloud Posse)

@Hans D You can try updating Geodesic yourself. If you did that and tested it out and let us know what you found, that would help us make sure the upgrade goes smoothly for everyone else. To update Geodesic, check out the repo and run

$ export DEBIAN_VERSION=12.2-slim
$ ARGS=DEBIAN_VERSION make debian.build

(Spoiler alert: it does not work at the moment because the netcat package has been removed and replaced with netcat-openbsd and netcat-traditional. You can fix this by editing packages-debian.txt and replacing netcat with netcat-openbsd.)

cloudposse/geodesic

2023-10-31

Release notes from geodesic avatar
Release notes from geodesic
08:20:42 AM

v2.7.2 Included Tools

Update dependency cryptography to v41.0.5 @renovate (<a class=”issue-link js-issue-link” data-error-text=”Failed to load title” data-id=”1959701836” data-permission-text=”Title is private” data-url=”https://github.com/cloudposse/geodesic/issues/884“…

Release v2.7.2 · cloudposse/geodesicattachment image

Included Tools

Update dependency cryptography to v41.0.5 @renovate (#884) This PR contains the following updates:

Package Change Age Adoption Passing Confidence

cryptography (changelog)…

Update dependency cryptography to v41.0.5 by renovate[bot] · Pull Request #884 · cloudposse/geodesic

This PR contains the following updates:

Package Change Age Adoption Passing Confidence

cryptography (changelog) ==41.0.4 -> ==41.0.5

Release Notes

pyca/cryptography (cryptography) v…

    keyboard_arrow_up