#docker (2020-09)

We’re using it in our CI pipelines. Primary reason for switching was to use the --secret flag so we can run npm install to get private packages without having NPM_TOKEN baked into the image. That part works great.

We have run into some issues with caching though. Specifically https://github.com/moby/moby/issues/41219. The good news is the fix has been merged to master, but they have not yet released a stable version with the fix - only betas.

So for now I’m still doing docker pull for the images I want to use as --cache-from values. Once the next version of docker is released that should be unnecessary.

Interesting, thanks! So far we’re only using BuildKit in a very specific case where I need the feature from https://github.com/moby/moby/issues/12886, and I hope this bug won’t hit me. The --secret flag will also be quite nice, so maybe I’ll dip my toes in further

It turns out that the feature mentioned in this issue is kinda flaky, so I will hold off on using BuildKit. Just an FYI in case anyone else has similar problems.

Relevant Dockerfile snippet

WORKDIR /app

RUN --mount=type=secret,id=build-secrets source /run/secrets/build-secrets  && \
    npm install                                                             && \
    rm -rf /root/.npm

VOLUME /app/node_modules

Using a volume to load code into container for hot-reloading while working:

docker run -v "$(pwd)":/app imageName

Instead of doing that, you can first COPY your package.json, then RUN npm install, then copy in your app code. That way your node_modules will only change if your package.json changes

Yeah, I’m doing that.

relying on node_modules from a mount means the container isn’t self-contained, which is an anti-pattern

Exactly

The dockerfile in question is only used locally. I want to load the code into the container so the developers can have hot-reloading (this works).

The problem is that even if I don’t have a node_modules folder in the directory I mount, after the docker run command, the host has the node_modules folder. Which makes sense, docker copies it into place at runtime.

The problem is it is owned by root instead of my user that executed the docker run command.

This means I have to sudo rm -rf node_modules if I want to get rid of it. And I’d like to avoid my devs getting in the habit of using sudo to delete stuff.

It will be owned by whichever user is being used inside the dockerfile. you can use USER change to a different user, but that probably won’t fix your problem.

Hmm, thought I tried that. But apparently not. I’ll give that a try.

Nope Inside the container ownership is correct but not outside.

• When you did the USER command, did you assign by (string) username or by (numeric) uid? • Do you do USER in the dockerfile, or do you do --user={uid}:{gid} in your docker run command?

• Is this under Linux, or do you need to also support Mac/Windows/WSL/WSL2?

Sorry was out yesterday and not checking slack.

The docker file has this:

ARG UID=1000
ARG GID=1000
RUN usermod -u $UID node && groupmod -g $GID node
USER node

I’m setting UID and GID using --build-arg on the docker run command.

I need to support Linux and Mac.

Does the “host machine user” have indeed an ID of 1000? Keep in mind that Ubuntu sets it to 1000 by default but on Mac it was something like 501. You should resolve that through a script programmatically and pass it to the docker build command.

It’s something I’m already doing for a project.

I suggest you try to change the UID of the “root” user itself inside the container, instead of changing the user to “node”. That is, unless the standard NodeJS docker image uses the “node” user. In that case, disregard my comment.

Yeah, my docker run command includes --build-arg UID=$(id -u) and --build-arg GID=$(id -g)

Amusingly, the one person on our team who has run into issues with this is using Ubuntu. For whatever reason his user id is 1001, it’s a dell so I blame them.

There’s apparently something in docker for mac that does some sort of translation, cause those on mac have never had issues editing files inside containers without any special handling.

And yeah, the nodejs docker image uses the node user.