allow explicit volumes to override exportDir

[jgangemi]

i'm trying to build a data volume for an nginx and based on the directions, i need to place content in the following two places

/usr/nginx/share/html
/etc/nginx

to do this i set my / and manage the output directories in the assembly.xml file. the problem is / is turned into a volume which makes the container unusable.

i'd like to propose the addition of a volumes element that takes an explicit list of volumes to expose and if this is tag has values, it overrides exposing exportDir as a volume.

thoughts?

[rhuss]

I see your point. An export of the assembly doesn't make much sense if the assembly is intended to be used only internally. In fact since if we are able to export dedicated directories one could even argue to not export exportDir at all (and then of course should rename this property).

But while we are at it, my idea was also to support Dockerfiles directly, probably a bit similar as the spotify/docker-maven-plugin does. In contrast to the run case we should not reinvent everything a Dockerfile provides. Im still thinking about how to combine the assembly mechanism (which is unique to this plugin) with a user provided Dockerfile (where all this volume exports can be declared directly in the "docker language").

So some of the ideas are:

• Specify a directory with a regular Dockerfile in the <build> configuration
• Substitute maven properties
• Have a comment-tag or a placeholder where to put in the directory with the content specified via an assembly. This would be replaced on-the-fly when building an image
• Other goodies like code blocks with a list of shell commandos, given line by line, which are concatenated before the build to with && to avoid extra image layers. That's probably more a syntactic sugar, but other pre-processing things might prove useful here, too.

At the end there could be two use case: For building a simple data image or a simple microservice everything can be specified in the build configuration section directly in the pom.xml. For more complex needs (like your example with a specific volume export), a real Dockerfile can be provided (along with support files directly in the same directory and which can be added directly in the Dockerfile). The latter mode then should still support assemblies for sucking in dependencies and artifacts without providing ugly pathes into some target directory.

Give me a day or two, I think we can come up with a good and powerful solution for covering most (if not all) use cases when building images.

[jgangemi]

i like not having to use a Dockerfile for what i'm doing. in most cases, the exportDir works just fine for my needs b/c i'm either deploying a webapp and i override exportDir to be the volume tomcat will look for or i'm deploying a set of jars/fat-jar which can live in whatever the default is (in this case, maven).

i have somewhat of an odd use case here b/c i'm using the maven-frontend-plugin to run a angular/grunt build and then i'm using the docker plugin to build/deploy the container it will live in. the assembly descriptor works just fine for me b/c in addition to creating the container, i can use it to produce a binary file via the assembly plugin directly.

either way, having the ability to export a volume as part of building the image and adding a check to see if the exportDir is / to skip exposing it (instead of just ignoring it entirely b/c i can see the need to want to expose additional volumes other then just that one) will cover my needs.

i'm all for supporting the Dockerfile if you want to build a complex container via this process but if someone is happy w/ the current mechanism (like me), i don't think they should be forced to use a more complex process if they don't have to. of course, someone may argue that the assembly descriptor is more complex, so...

[rhuss]

Completely agreed.

So the final question is how to determine whether exportDir should be exported or not. What do you think about the following idea:

• exportDir gets deprecated but keeps its bevahior: root of the maven assembly and being exported. An implicit behaviour to not export this directory if specified as / could be added, because this should be save (as you said it won't anyway the other way round).
• A new property assemblyRoot which is introduced to specify the root directory but without exporting it.
• In addition an extra boolean property exportAssembly is introduced for finally deciding whether to export the assembly or not (whose default could be true for backwards compability).

[rhuss]

I'll add the implicit behaviour to not export an exportDir of / for 0.10.5, the rest is then scheduled fo 0.10.6

[jgangemi]

i agree w/ those options. i can tackle this as part of finishing off the container options given i have an immediate need for it unless you want to handle it.

[rhuss]

Ok, go ahead.

BTW, in other issue #53 Thomas has problems with the UID of the exported directory. I tried to changed from ADD to COPY but this didn't helped.

My idea is to use an extra configuration (like 'assemblyUser') and then do the required 'chown' on our own by inserting it into the on-the-fly generated Dockerfile. Since this is yet another assembly related paremeter, what do you think about combining this into extra configuration section like in

<build>
   ...
   <assembly>
      <descriptor>src/.../assembly.xml</descriptor>
      <directory>/maven</directory>
      <export>true</export>
      <uid>www-data</uid>
   </assembly>
</build>

For backwards compatibility one would still support exportDir, assemblyDescriptor and assemblyDescriptorRef for some time ?

I think this is much cleaner leading to a better structured configuration, what do you think ?

[jgangemi]

i would change uid to user b/c that's what docker calls it and i'd probably call directory basedir b/c that's more maven-ish

[rhuss]

The stuff with support for Dockerfiles can be found here #13

[rhuss]

Yes, user it better then uid.

I think before introducing volume (at "build" level) we should directly move it to this dedicated assembly section in order to keep migration pain to a minimum (luckily we are still pre 1.0 so a decent refactoring in the configuration is ok ;-).

I will take care about #64 (and include your volume export there)

[jgangemi]

i don't think that's a good idea actually - what if i just want to expose a volume w/o actually putting anything in it? i shouldn't need the assembly section to do that and in fact, i was going to try and make that be completely optional if you are ok w/ the defaults used (i'm mid-typing this out in the other ticket).

[rhuss]

Hmmm, why would you like to do this ? I mean, exposing something without putting in anything ?

An example would be helpful to get me on track ....

[rhuss]

(BTW, have to leave soon, but I will setup an IRC channel on freenode tomorrow, so we don't have to chat via issues ;-)

[chris-snider]

I can see wanting to do a VOLUME for /var/logs/application_name as a volume since the contained application would perhaps do all logging to this location. I would then mount this exposed volume to my host somewhere such that I could search all running container logs in a single location on the host. Or I have to have the logs for audit purposes going back in time XX amount regardless whether the container still exists or not.

[jgangemi]

yeah, what @chris-snider said is what i was about to type out.

i think the assembly should focus solely on what ends up in the container from the project and not control anything related to how the container is constructed, etc except for offering the convenience of automatically exposing whatever destination directory this ended up in.

and a irc channel sounds good.

[rhuss]

Ok, see your point now. Let's keep outside the assembly and the section deals only about the data from the assembly descriptor (including the ownership and whether the assembly root dir should be added to the list of exposed volumes or not).

[jgangemi]

sounds good - i'll have a preview of things to come documented for feedback in #64 very shortly.

[jgangemi]

released in 0.11.0