Avoid copying image files - use dockerignore instead

[plajjan]

So we currently copy the image file(s) for a virtual router before initiating the docker build. This is an optimization since the entire content of the docker build directory is sent in the build context to the docker engine. With multiple virtual machine images available locally, all of them will be sent for every invocation of the docker build. Thus, we instead copy the image to the docker/ directory and use that as our build context directory. This reduces the total amount of copying.

For example, in the most naive approach, with 3 images, say version 1, 2, 3 of virtual router FOO we would build three times (for ver 1, 2 & 3) and each time, all three images would be included in the docker build context. With a 1GB image, that is 33GB = 9GB copied. The current version instead does one copy + docker context send = 3(1GB+1GB) = 6GB.

We should be able to reduce this to 3GB by dynamically generating a dockerignore file. Thus we avoid the initial copy and instead send the local directory as the docker build context but ignore all the image files except the one we are building, thus 3*1GB = 3GB in total.

[bdreisbach]

is this kind of what you were thinking?

index ecb5a3f..d91e570 100644
--- a/makefile.include
+++ b/makefile.include
@@ -12,9 +12,24 @@ docker-image:
        done
 endif

+docker-clean-ignore:
+       @if [ -f .dockerignore ]; then \
+           rm .dockerignore; \
+       fi
+
+docker-ignore:
+       $(MAKE) docker-clean-ignore
+       echo "*$(IMAGE_FORMAT)" > .dockerignore
+       echo "!$(IMAGE)*" >> .dockerignore
+
 docker-clean-build:
        -rm -f docker/*.qcow2* docker/*.tgz* docker/*.vmdk* docker/*.iso

+docker-clean:
+       $(MAKE) docker-clean-ignore
+       (cd docker; rm healthcheck.py vrnetlab.py)
+
 docker-pre-build: ;

 docker-build-image-copy:
@@ -25,8 +40,12 @@ docker-build-common: docker-clean-build docker-pre-build
        @if [ "$(IMAGE)" = "$(VERSION)" ]; then echo "ERROR: Incorrect version string ($(IMAGE)). The regexp for extracting version information is likely incorrect, check the regexp in the Makefile or open an issue at https://github.com/plajjan/vrnetlab/issues/new including the image file name you are using."; exit 1; fi
        @echo "Building docker image using $(IMAGE) as $(REGISTRY)vr-$(VR_NAME):$(VERSION)"
        cp ../common/* docker/
-       $(MAKE) IMAGE=$$IMAGE docker-build-image-copy
-       (cd docker; docker build --build-arg http_proxy=$(http_proxy) --build-arg https_proxy=$(https_proxy) --build-arg IMAGE=$(IMAGE) -t $(REGISTRY)vr-$(VR_NAME):$(VERSION) .)
+       @if [ -z "$$IMAGE_FORMAT" ]; then \
+           $(MAKE) IMAGE=$$IMAGE IMAGE_FORMAT=$(IMAGE_FORMAT) docker-ignore; \
+       fi
+       docker build --build-arg http_proxy=$(http_proxy) --build-arg https_proxy=$(https_proxy) --build-arg IMAGE=$(IMAGE) -t $(REGISTRY)vr-$(VR_NAME):$(VERSION) -f docker/Dockerfile .
+       $(MAKE) docker-clean

 docker-build: docker-build-common

diff --git a/sros/docker/Dockerfile b/sros/docker/Dockerfile
index d419307..41bb313 100644
--- a/sros/docker/Dockerfile
+++ b/sros/docker/Dockerfile
@@ -15,7 +15,7 @@ RUN apt-get update -qy \

 ARG IMAGE
 COPY $IMAGE* /
-COPY *.py /
+COPY docker/*.py /

 EXPOSE 22 161/udp 830 5000 10000-10099
 HEALTHCHECK CMD ["/healthcheck.py"]