Remove strict validation of external volume name (#7024)

* Removed strict validation of external volume name
* Added parameter 'shared' on external volumes
* If shared is true, unique validation for the volume name is excluded

JIRA issues: MARATHON-8681

(cherry picked from commit faceeb0)

Co-Authored-By: Aleksey Dukhovniy <alex.dukhovniy@googlemail.com>

README.md

@@ -371,3 +371,4 @@ and <a href="https://www.yourkit.com/.net/profiler/index.jsp">YourKit
 .NET Profiler</a>,
 innovative and intelligent tools for profiling Java and .NET
 applications.
+

changelog.md

@@ -1,4 +1,64 @@
-## Changes from 1.8.194 to 1.8.xxx
+## Changes from 1.8.218 to 1.8.xxx
+
+### External Volume Validation changes
+
+#### Relaxed name validation
+
+As there are some external volume providers which require options in the volume name, the strict validation of the name on the external volume is now removed.
+
+As the uniqueness check is based on the volume name, this may lead to some inconsistencies, for the sake of uniqueness, the following volumes are distinct:
+
+```json
+"volumes": [
+      {
+        "external": {
+          "name": "name=volumename,option1=value",
+        },
+      }
+    ],
+```
+
+```json
+"volumes": [
+      {
+        "external": {
+          "name": "option1=value,name=volumename",
+        },
+      }
+    ],
+```
+
+#### Optional uniqueness check
+
+Previously, Marathon would validate that an external volume with the same name is only used once across all apps. This was due to the initial implementation being focused on Rexray+EBS. However, multiple external volume providers now
+allow shared access to mounted volumes, so we introduced a way to disable the uniqueness check:
+
+A new field, `container.volumes[n].external.shared` which defaults to `false`. If set to true, the same volume name can be used
+by multiple containers. The `shared` flag has to be set to `true` on all external volumes with the same name, otherwise a conflict is reported on the volume without the `shared=true` flag.
+
+```json
+  "container": {
+    "type": "MESOS",
+    "volumes": [
+      {
+        "external": {
+          "size": 5,
+          "name": "volumename",
+          "provider": "dvdi",
+          "shared": "true",
+          "options": {
+            "dvdi/driver": "pxd",
+            "dvdi/shared": "true"
+          }
+        },
+        "mode": "RW",
+        "containerPath": "/mnt/nginx"
+      }
+    ],
+  }
+```
+
+## Changes from 1.8.194 to 1.8.218
 
 ### Revive and Suppress Refactoring
 

docs/docs/external-volumes.md

@@ -122,6 +122,33 @@ If you scale your app down to 0 instances, the volume is detached from the agent
 
 The default implicit volume size is 16 GB. If you are using the original Mesos containerizer or the UCR, you can modify this default for a particular volume by setting `volumes[x].external.size`. For the Mesos and Docker containerizers, you can modify the default size for all implicit volumes by [modifying the REX-Ray configuration][11].
 
+#### Shared Volumes
+
+By default, external volumes can not share the same name. As there are providers that allow this, there is a flag that prevents the uniqueness check on the volume name. `volumes[x].external.shared` can be set to true. In this case, this volume is not included when checking if the volume name already exists. It still verifies that no other external volumes with `volumes[x].external.shared=false` exist, so all volumes with the same name must have this flag set.
+
+```
+  "container": {
+    "type": "MESOS",
+    "volumes": [
+      {
+        "external": {
+          "size": 5,
+          "name": "volumename",
+          "provider": "dvdi",
+          "shared": "true",
+          "options": {
+            "dvdi/driver": "pxd",
+            "dvdi/shared": "true"
+          }
+        },
+        "mode": "RW",
+        "containerPath": "/mnt/nginx"
+      }
+    ],
+  }
+```
+
+
 ### Potential Pitfalls
 
 *   If one or more external volumes are declared for a Marathon app, and the Docker image specification includes one or more `VOLUME` entries, Docker may create anonymous external volumes. This is default Docker behavior with respect to volume management when the `--volume-driver` flag is passed to `docker run`. However, anonymous volumes are not automatically deleted and will accumulate over time unless you manually delete them. To prevent Docker from creating anonymous volumes, you can either use a Mesos container with a Docker image or follow these steps:

docs/docs/rest-api/public/api/v2/types/volumes.raml

@@ -25,6 +25,10 @@ types:
         type: label.KVLabels
         (pragma.omitEmpty):
         description: Provider specific volume configuration options
+      shared?:
+        type: boolean
+        default: false
+        description: If true, excludes this volume from the global unique volume name check
   PersistentVolumeType:
     type: string
     description: |