docker build with private credentials — the issue with intermediate layer secrets

tl;dr: Do never use COPY or ADD with docker credentials but use a static temporary server.

Although I am a Frontend Engineer I do have the feeling that I am spending nearly as much time with Docker as with everything else. Not because it consumes that much time but because it just helps any workflow.

Docker is an amazing tool but without knowing the architecture it can easily become a security issue when the files you need for building are private secrets. Let’s take a basic example: git clone something@my_private_repo_that_i_need_to_build . You definitely want this repository at build time. But obviously you neither want the files in that repo nor the private keys to be in the docker image. The very (!) wrong way of credential management in Docker

FROM image
COPY id_rsa
#do something in between
RUN git clone ...
RUN rm id_rsa

The COPY command is an atomic action. This will only ensure that your private key will definitely be available. This is due to the fact that every single command in a Dockerfile is a new layer — which is an image. So after calling COPY id_rsa there will be a parent layer of the resulting image which will contain your private key.

One could argue: Wait, there is docker secrets ( https://docs.docker.com/engine/swarm/secrets/). True but it applies to services, so running containers and not the build task of docker. So how can I add secrets to the docker build without exposing it to the public?

Spin up a secret server, fetch it, run your commands, get rid of the secrets. Sounds complicated? Well it’s not:

1. Go to d=/whatever/directory/you/like
2. Create your Dockerfile that shall work with build-time secrets e.g.:

FROM debian:latest
RUN apt-get update && apt-get install -y curl# IMPORTANT NOTE: Reading, storing and deleting secrets MUST be in one single RUN command!RUN curl http://secrets:8043/my-secret > secret-stored.txt && \
   FIRST2CHARS_OF_SECRET=$(cat secret-stored.txt | cut -c1–2) && \
   echo "${FIRST2CHARS_OF_SECRET}" > partial-secret && \
   echo 'The first 2 chars of the secret are: "'${FIRST2CHARS_OF_SECRET}'"' && \
   rm secret-stored.tx

3. Create a $d/secrets folder put a file named my-secret into it and spin up a server for it: docker run -d -p 8043 -v $d/secrets:/srv/http — name static-secrets-server pierrezemb/gostatic. Okay so you got that secrets server running.
4. Save the ip of the container we span up so that we can use it at build-time: secrets_ip=$(docker inspect -f ‘{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}’ static-secrets-server)
5. Let’s build the image with secrets: docker build . -t my-build-with-secrets --add-host=secrets:${secrets_ip}
6. Verify that not the full secret is left but only the part that we wanted: docker run test-build-with-secrets bash -c 'ls -al && echo "---" && cat partial-secret'

Works. Gotcha!

There are different solutions out there, feel free to check out this list from a GitHub Issue: https://github.com/moby/moby/issues/13490