Compose config adds quotes to tmpfs size, docker rejects 'must be a integer'

[carlhodder]

Output from docker compose adds quote marks around tmpfs size parameter, and then docker rejects:

steps to reproduce:

• docker compose -f example.yml config > interim.yml
• cat <(echo "version: '3.9'") interim.yml > compiled.yml
• docker stack deploy -c compiled.yml testing

Example config:

services:
  test:
    image: "nginx"
    ports:
      - "8080:8080"
    volumes:
        - type: tmpfs
          target: /dev/shm
          tmpfs:
              size: 512000000

Output from docker compose config:

name: backend
services:
  test:
    image: nginx
    networks:
      default: null
    ports:
    - mode: ingress
      target: 8080
      published: "8080"
      protocol: tcp
    volumes:
    - type: tmpfs
      target: /dev/shm
      tmpfs:
        size: **"512000000"**
networks:
  default:
    name: backend_default

docker stack -c {file} {destination} then fails with error:
services.test.volumes.1.tmpfs.size must be a integer

NOTE: Can supply non-bytes value, output is calculated to correct value in bytes and is quoted as a string.

Expected behaviour: Output from docker compose config is integer.

Version:
Docker: 20.10.14, build a224086

Docker info:
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc., v0.8.2)
compose: Docker Compose (Docker Inc., v2.4.1)
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
scan: Docker Scan (Docker Inc., v0.17.0)
Server:
Containers: 4
Running: 4
Paused: 0
Stopped: 0
Images: 94
Server Version: 20.10.14
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
NodeID: 44pjigzt9pxjithhv3jmtn2cl
Is Manager: true
ClusterID: 2z407xi24syndhj1snfsm7mt7
Managers: 1
Nodes: 1
Default Address Pool: 10.0.0.0/8
SubnetSize: 24
Data Path Port: 4789
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
Election Tick: 10
Dispatcher:
Heartbeat Period: 5 seconds
CA Configuration:
Expiry Duration: 3 months
Force Rotate: 0
Autolock Managers: false
Root Rotation In Progress: false
Node Address: 192.168.65.3
Manager Addresses:
192.168.65.3:2377
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
runc version: v1.0.3-0-gf46b6ba
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.102.1-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 24.98GiB
Name: docker-desktop
ID: OCXV:42LW:KJJZ:EQH5:XQQV:DEXW:MVL3:5FEB:IXTV:JPPD:UEPM:55PV
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5000
127.0.0.0/8
Live Restore Enabled: false

[ndeloof]

According to the compose specification, tmpfs.size can be either a number or a string https://github.com/compose-spec/compose-spec/blob/master/spec.md#long-syntax-4

docker stack command is not compliant with the Compose Specification, you should not try to mix compose and stack commands.

[ndeloof]

Can you please let me know why you use docker compose config in combination with docker stack? Could be interesting we understand your expectation and maybe offer a "stack-way" to do the same to avoid confusion

[carlhodder]

We use it to populate the env vars in the compose file as docker stack doesn't support substitution.

Honestly, thought that was the correct approach, whoops. In retrospect it has a bit of a smell - I should've just used envsubst or set the shell flags in a step prior instead. My apologies for taking up your time, and thanks for getting me to the solution!

[ndeloof]

thanks, created docker/cli#3574

[ndeloof]

Ran a quick test and stack deploy fully supports variable expansion, not sure what makes you think it's not the case.
Anyway, proposed docker/cli#3575 to workaround confusion regarding compose vs stack commands

[carlhodder]

I just tried it on my dev machine with the unsubbed compose file and it worked! I haven't seen any notes about it being supported and the docker page still says it isn't:

The .env file feature only works when you use the docker-compose up command and does not work with docker stack deploy.

So the workaround was for docker compose config to read the .env file and add them to the output for consumption by docker stack deploy.

I think we considered docker secrets, but secrets are only available to swarm service not standalone containers.

Also dug up moby/moby#29133 ticket which seems familiar enough that it's possibly where I got this idea.

[ndeloof]

oh ok, so this is about .env file support, not variable substitution (which is supported by stack command but only for os.Env variables.