Dockerignore: convert to whitelist

[alxndrsn]

No description provided.

[matthew-white]

What will happen to the version files if we make this change? I'm thinking that they'll identify the excluded files as changes to the working directory and append -dirty to the version, which probably isn't what we want.

What do you think about running git clean instead at some point during the build? That way we could remove things like node_modules without causing a change that would show up in the version files.

[matthew-white]

Also required to generate version.txt in the nginx container (files/prebuild/write-version.sh I believe).

[alxndrsn]

Thanks, added to comment

[matthew-white]

If we already had node_modules in here, how was it copied through during the build of that one server? Does .dockerignore work differently from .gitignore in that way?

[alxndrsn]

Yes 😞

.dockerignore

the patterns /foo/bar and foo/bar both exclude a file or directory named bar in the foo subdirectory of PATH or in the root of the git repository located at URL. Neither excludes anything else.
-- https://docs.docker.com/engine/reference/builder/#dockerignore-file

.gitignore

If there is a separator at the beginning or middle (or both) of the pattern, then the pattern is relative to the directory level of the particular .gitignore file itself. Otherwise the pattern may also match at any level below the .gitignore level.
-- https://git-scm.com/docs/gitignore#_pattern_format

Probably good to include the preceding / in .dockerignore to make the intent more explicit.

[matthew-white]

Could we make it something like this:

/server/node_modules
/server/npm-debug.log

/client/node_modules
/client/npm-debug.log

Actually, could that change be all we need? Then an errant node_modules directory wouldn't be copied through.

[alxndrsn]

This would be an alternative approach, but I think with a blacklisting approach there is more scope for accidentally including unwanted files in the future.

[alxndrsn]

What will happen to the version files if we make this change? I'm thinking that they'll identify the excluded files as changes to the working directory and append -dirty to the version, which probably isn't what we want.

What do you think about running git clean instead at some point during the build? That way we could remove things like node_modules without causing a change that would show up in the version files.

Very interesting point. I don't think we can reliably keep .dockerignore and .gitignore in sync, especially using a whitelist in one and not the other.

Maybe the real issue in the node_modules case is that we carefully populate that directory:

central/service.dockerfile

Lines 12 to 14 in 2eea364

	COPY server/package*.json ./
	RUN npm install --production
	RUN npm install pm2 -g

and then subsequently overwrite that with whatever the user has locally:

central/service.dockerfile

Line 16 in 2eea364

COPY server/ ./

😰

[alxndrsn]

central/service.dockerfile

Line 16 in 2eea364

COPY server/ ./

might be better replaced with:

COPY server/config config/
COPY server/lib lib/

[matthew-white]

Maybe the real issue in the node_modules case is that we carefully populate that directory:

central/service.dockerfile

Lines 12 to 14 in 2eea364

	COPY server/package*.json ./
	RUN npm install --production
	RUN npm install pm2 -g

and then subsequently overwrite that with whatever the user has locally:

central/service.dockerfile

Line 16 in 2eea364

COPY server/ ./

central/service.dockerfile

Line 16 in 2eea364

COPY server/ ./

might be better replaced with:

COPY server/config config/
COPY server/lib lib/

That change sounds sensible to me. I worry a little about the day when we add a new directory besides config/ and lib/, but it'd be easy enough to add it here if/when we do.

Another option to consider: what if copy everything, but do so before the npm clean-install? Something like:

-COPY server/package*.json ./
+COPY server/ ./
 
 RUN npm clean-install --omit=dev --legacy-peer-deps
 RUN npm install pm2@5.2.0 -g
 
-COPY server/ ./

Then even if node_modules is initially copied, it will be replaced by npm clean-install.

[alxndrsn]

Then even if node_modules is initially copied, it will be replaced by npm clean-install.

I don't think this would pick up manual changes made in node_modules, so still leaves scope for unpredictable builds. Also including node_modules in the first place will increase the size of the build context.