ddev is not mounting sites/default/files into the web container

[cstuart]

Is there an existing issue for this?

• I have searched the existing issues

Output of ddev debug test

Expand `ddev debug test` diagnostic information

Running bash [-c /var/folders/wf/l_07dlmj3bq_hy88rz0n74x00000gn/T/test_ddev.sh] 
======= Existing project config =========
These config files were loaded for project :[ddev project][/Users/[user]/www/[ddev project]/.ddev/config.yaml]
name: [ddev project]
type: drupal10
docroot: web
php_version: 8.1
webserver_type: apache-fpm
webimage: ddev/ddev-webserver:v1.22.7
router_http_port: 80
router_https_port: 443
additional_hostnames: []
additional_fqdns: []
database: {mariadb 10.4}
project_tld: ddev.site
use_dns_when_possible: true
composer_version: 2
nodejs_version: 18
default_container_timeout: 120
web_extra_exposed_ports: [{node-vite 3000 9998 9999}]
======= Creating dummy project named  tryddevproject-14468 in ../tryddevproject-14468 =========
OS Information: Darwin Cullums-Mac-mini.local 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:49 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6020 arm64
ProductName:            macOS
ProductVersion:         14.4.1
BuildVersion:           23E224
User information: uid=501([user]) gid=20(staff) groups=20(staff),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),33(_appstore),100(_lpoperator),204(_developer),250(_analyticsusers),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh),400(com.apple.access_remote_ae),701(com.apple.sharepoint.group.1)
DDEV version:  ITEM             VALUE                                   
 DDEV version     v1.22.7                                 
 architecture     arm64                                   
 db               ddev/ddev-dbserver-mariadb-10.4:v1.22.7 
 ddev-ssh-agent   ddev/ddev-ssh-agent:v1.22.7             
 docker           24.0.2                                  
 docker-compose   v2.24.5                                 
 docker-platform  docker-desktop                          
 mutagen          0.17.2                                  
 os               darwin                                  
 router           ddev/ddev-traefik-router:v1.22.7        
 web              ddev/ddev-webserver:v1.22.7             
PROXY settings: HTTP_PROXY='' HTTPS_PROXY='' http_proxy='' NO_PROXY=''
======= DDEV global info =========
Global configuration:
instrumentation-opt-in=true
omit-containers=[]
performance-mode=mutagen
router-bind-all-interfaces=false
internet-detection-timeout-ms=3000
disable-http2=false
use-letsencrypt=false
letsencrypt-email=
table-style=default
simple-formatting=false
use-hardened-images=false
fail-on-hook-fail=false
required-docker-compose-version=v2.24.5
use-docker-compose-from-path=false
project-tld=ddev.site
xdebug-ide-location=
no-bind-mounts=false
router=traefik
wsl2-no-windows-hosts-mgt=false
router-http-port=80
router-https-port=443
mailpit-http-port=8025
mailpit-https-port=8026
traefik-monitor-port=10999

======= DOCKER info =========
docker location: lrwxr-xr-x@ 1 root  wheel  54 Jun 30  2023 /usr/local/bin/docker -> /Applications/Docker.app/Contents/Resources/bin/docker
Docker Desktop Version: Docker Desktop for Mac 4.21.0 build 113844
docker version: 
Client:
 Cloud integration: v1.0.35
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/arm64
 Context:           default

Server: Docker Desktop 4.21.0 (113844)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:50:59 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
DOCKER_DEFAULT_PLATFORM=notset
======= Mutagen Info =========
Mutagen is installed in ddev, version=0.17.2
--------------------------------------------------------------------------------
Name: [ddev project]
Identifier: sync_FFSTP6z1LTSZZ0mT10ElMBV9AiCsrFfZDI1NpJw70kT
Labels:
        com.ddev.config-hash: c212281329fb24762a96e1b7a37a19ed84868866
        com.ddev.volume-signature: Users-[user]-docker-run-1711597404
Configuration:
        Synchronization mode: Two Way Resolved
        Hashing algorithm: Default (SHA-1)
        Maximum allowed entry count: Default (2⁶⁴−1)
        Maximum staging file size: Default (18 EB)
        Symbolic link mode: POSIX Raw
        Ignore VCS mode: Default (Propagate)
        Ignores:
                /.git
                /.tarballs
                /.ddev/db_snapshots
                /.ddev/.importdb*
                .DS_Store
                .idea
                /web/sites/default/files
        Permissions mode: Default (Portable)
Alpha:
        URL: /Users/[user]/www/[ddev project]
        Configuration:
                Watch mode: Default (Portable)
                Watch polling interval: Default (10 seconds)
                Probe mode: Default (Probe)
                Scan mode: Default (Accelerated)
                Stage mode: Neighboring
                File mode: Default (0600)
                Directory mode: Default (0700)
                Default file/directory owner: Default
                Default file/directory group: Default
        Connected: Yes
        Synchronizable contents:
                11240 directories
                77531 files (501 MB)
                41 symbolic links
Beta:
        URL: docker://ddev-[ddev project]-web/var/www/html
                DOCKER_HOST=unix:///Users/[user]/.docker/run/docker.sock
        Configuration:
                Watch mode: Default (Portable)
                Watch polling interval: Default (10 seconds)
                Probe mode: Default (Probe)
                Scan mode: Default (Accelerated)
                Stage mode: Neighboring
                File mode: Default (0600)
                Directory mode: Default (0700)
                Default file/directory owner: Default
                Default file/directory group: Default
                Compression: Default (DEFLATE)
        Connected: Yes
        Synchronizable contents:
                11240 directories
                77531 files (501 MB)
                41 symbolic links
Status: Watching for changes
--------------------------------------------------------------------------------
======= Docker Info =========
Docker platform: docker-desktop 
Using Docker context: default (unix:///Users/[user]/.docker/run/docker.sock) 
docker-compose: v2.24.5 
Using DOCKER_HOST=unix:///Users/[user]/.docker/run/docker.sock 
Docker version: 24.0.2 
Able to run simple container that mounts a volume. 
Able to use internet inside container. 
Docker disk space:
Filesystem                Size      Used Available Use% Mounted on
overlay                  58.4G     26.2G     29.1G  47% /

 Container ddev-[ddev project]-db  Stopped 
 Container ddev-[ddev project]-web  Stopped 
 Container ddev-[ddev project]-db  Stopped 
 Container ddev-[ddev project]-web  Stopped 
 Container ddev-[ddev project]-db  Removed 
 Container ddev-[ddev project]-web  Removed 
 Network ddev-[ddev project]_default  Removed 
Project [ddev project] has been stopped. 
Stopped Mutagen daemon 
The ddev-ssh-agent container has been removed. When you start it again you will have to use 'ddev auth ssh' to provide key authentication again. 
Network ddev_default removed 
Existing docker containers: 
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
Network ddev_default created 
Starting tryddevproject-14468... 
Network ddev-tryddevproject-14468_default created 
 Container ddev-ssh-agent  Created 
 Container ddev-ssh-agent  Started 
ssh-agent container is running: If you want to add authentication to the ssh-agent container, run 'ddev auth ssh' to enable your keys. 
Building project images... 
Project images built in 0s. 
 Container ddev-tryddevproject-14468-db  Created 
 Container ddev-tryddevproject-14468-web  Created 
 Container ddev-tryddevproject-14468-web  Started 
 Container ddev-tryddevproject-14468-db  Started 
You have Mutagen enabled and your 'php' project type doesn't have `upload_dirs` set. 
For faster startup and less disk usage, set upload_dirs to where your user-generated files are stored. 
If this is intended you can disable this warning with `ddev config --disable-upload-dirs-warning`. 
Starting Mutagen sync process... 
Mutagen sync flush completed in 1s.
For details on sync status 'ddev mutagen st tryddevproject-14468 -l' 
Waiting for web/db containers to become ready: [web db] 
Starting ddev-router if necessary... 
 Container ddev-router  Created 
 Container ddev-router  Started 
Waiting for additional project containers to become ready... 
All project containers are now ready. 
Successfully started tryddevproject-14468 
Project can be reached at https://tryddevproject-14468.ddev.site https://127.0.0.1:49497 
======== Curl of site from inside container:
HTTP/1.1 200 OK
Server: nginx
Date: Fri, 29 Mar 2024 00:30:22 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding

======== curl -I of http://tryddevproject-14468.ddev.site from outside:
HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Date: Fri, 29 Mar 2024 00:30:22 GMT
Server: nginx
Vary: Accept-Encoding

======== full curl of http://tryddevproject-14468.ddev.site from outside:
Success accessing database... db via TCP/IP
ddev is working. You will want to delete this project with 'ddev delete -Oy tryddevproject-14468'
======== Project ownership on host:
drwxr-xr-x@ 4 [user]  staff  128 Mar 29 11:29 ../tryddevproject-14468
======== Project ownership in container:
drwxr-xr-x 4 [user] dialout 4096 Mar 29 00:30 /var/www/html
======== In-container filesystem:
Filesystem     Type 1K-blocks     Used Available Use% Mounted on
/dev/vda1      ext4  61202244 29013516  29047404  50% /var/www
======== curl again of tryddevproject-14468 from host:
Success accessing database... db via TCP/IP
ddev is working. You will want to delete this project with 'ddev delete -Oy tryddevproject-14468'
Thanks for running the diagnostic. It was successful.
Please provide the output of this script in a new gist at gist.github.com
Running ddev launch in 5 seconds

Expected Behavior

Expected the ddev import-files command to mount the files to the container as they are not accessible unless the container gets restarted.

Actual Behavior

Files are not being mounted to container.

Steps To Reproduce

1. Start DDEV with ddev start
2. Ensure files directory/compressed files, you wish to import e.g. latest.files.tar.gz.
3. Ceate junk file in files directory touch web/sites/default/files/junk.txt
4. Find junk.txt listed in files directory ls -l web/sites/default/files
5. Find junk.txt listed in files directory in the container ddev exec ls -l web/sites/default/files
6. Run ddev import-files --source=latest.files.tar.gz
7. Can no longer find junk.txt listed in files directory as the directory was successfully imported ls -l web/sites/default/files
8. Ceate junk file in files directory touch web/sites/default/files/junk.txt
9. Find junk.txt listed in files directory ls -l web/sites/default/files
10. Can NOT find junk.txt in files directory in the container as the files have not been remounted to container after import ddev exec ls -l web/sites/default/files

[tyler36]

I thought the flag was --source

ddev import-files --source=/path/to/files.tar.gz

[cstuart]

Apologies it is --source. I've updated the issue

[stasadev]

Hi @cstuart, please provide the output for ddev debug test in the first post.

I assume you are using macOS, and this has something to do with Mutagen.

[rfay]

Let us know when you've updated this and we'll be happy to help, but we need more information. We're happy to help in #support in Discord too, https://discord.gg/5wjP76mBJD

[rfay]

FYI, ddev import-files only brings your files into sites/default/files on the host side. If it's working correctly, you'll find that those files are in sites/default/files. However, if you are importing a tarball with an unexpected format or unexpected content, they won't be there.

As far as how they end up inside the container: On a normal Drupal project they will be bind-mounted by Docker into the container.

[cstuart]

@stasadev Issue has been updated with ddev debug test output. And yes, I'm on a Mac.

@rfay The files get uncompressed and moved over correctly, it's just they don't get bind-mounted into the container which is the issue.

[rfay]

@cstuart you say

Try to view an asset form the /web/sites/default/files directory in running container

But they would not be in /web.

The assets will be in /var/www/html/web/sites/default/files

If you see them in your project's web/sites/default/files folder and you do not see them in /var/www/html/web/sites/default/files then please show the output of this:

ls -l web/sites/default/files
ddev exec ls -l web/sites/default/files

And please attach a file with .ddev/.ddev-docker-compose-full.yaml as a text file.

If you have edited the .ddev/mutagen/mutagen.yml file, please remove that file and ddev start

[rfay]

BTW, per your description, this has nothing to do with ddev import-files right? Your files get imported, your problem is that you're not seeing them inside the container. You should be able to touch web/sites/default/files/junk.txt and see it show up inside the container at web/sites/default/files/junk.txt.

[cstuart]

@rfay I've updated the Steps to reproduce to make it as explicit as I can.

[rfay]

Just so you know, when you ddev import-files it will overwrite and remove the junk.txt, so you're not understanding the steps here. Just stop thinking about import-files and do these things that I already asked for:

1. touch web/sites/default/files/junk.txt
2. ddev exec ls -l web/sites/default/files/junk.txt
3. Please attach a file with .ddev/.ddev-docker-compose-full.yaml as a text file.
4. Please delete your .ddev/mutagen/mutagen.yml file

[rfay]

Here's an idea for you.

1. Create a plain-vanilla D10 site using the quickstart, https://ddev.readthedocs.io/en/stable/users/quickstart/#__tabbed_2_1
2. Download the files tarball from https://github.com/ddev/d10simple-artifacts (it's at https://github.com/ddev/d10simple-artifacts/raw/main/files.tgz )
3. Do the above steps and see if you're getting junk.txt to show up in the container.
4. Try importing the files.tgz that you downloaded and see what happens. ddev import-db --source=path/to/files.tgz

[richardbporter]

This sounds like an issue we are experiencing as well. Does ddev import-files delete the entire web/sites/default/files directory? If that happens, it seems like the only way to get it back in the web container is with ddev restart. I can replicate this using the quickstart D10 project and sample files tarball referenced in #6026 (comment):

➜  my-drupal-site ls -l web/sites/default/files
total 8
-rw-r--r--  1 user  group  5 Apr 22 13:20 foo.txt
➜  my-drupal-site ddev exec ls -l web/sites/default/files
total 4
-rw-r--r-- 1 user dialout 5 Apr 22 18:20 foo.txt
➜  my-drupal-site ddev import-files --source .ddev/.downloads/files.tgz
Successfully imported files for my-drupal-site
➜  my-drupal-site ls -l web/sites/default/files
total 3624
-rw-rw-r--  1 user  group   66852 Apr 22 13:21 borscht-with-pork-ribs-umami.jpg
-rw-rw-r--  1 user  group  100481 Apr 22 13:21 chili-sauce-umami.jpg
-rw-rw-r--  1 user  group   30431 Apr 22 13:21 chocolate-brownie-umami.jpg
-rw-rw-r--  1 user  group   62307 Apr 22 13:21 crema-catalana-umami.jpg
drwxrwxr-x  6 user  group     192 Apr 22 13:21 css
-rw-rw-r--  1 user  group   36372 Apr 22 13:21 heritage-carrots.jpg
-rw-rw-r--  1 user  group  114984 Apr 22 13:21 home-grown-herbs.jpg
drwxrwxr-x  4 user  group     128 Apr 22 13:21 js
drwxrwxr-x  3 user  group      96 Apr 22 13:21 media-icons
-rw-rw-r--  1 user  group   70160 Apr 22 13:21 mediterranean-quiche-umami.jpg
-rw-rw-r--  1 user  group   49938 Apr 22 13:21 mojito-mocktail.jpg
-rw-rw-r--  1 user  group   92740 Apr 22 13:21 mushrooms-umami.jpg
-rw-rw-r--  1 user  group   76171 Apr 22 13:21 oatmeal-fruit-syrup-topping.jpg
drwxrwxrwx  3 user  group      96 Apr 22 13:21 php
-rw-rw-r--  1 user  group   81301 Apr 22 13:21 pizza-umami.jpg
drwxrwxr-x  5 user  group     160 Apr 22 13:21 styles
-rw-rw-r--  1 user  group  100645 Apr 22 13:21 supermarket-savvy-umami.jpg
drwxr-xr-x  3 user  group      96 Apr 22 13:21 sync
-rw-rw-r--  1 user  group   38674 Apr 22 13:21 thai-green-curry-umami.jpg
drwxrwxr-x  3 user  group      96 Apr 22 13:21 translations
-rw-rw-r--  1 user  group   30490 Apr 22 13:21 umami-bundle.png
-rw-rw-r--  1 user  group  261638 Apr 22 13:21 vegan-brownies-hero-umami.jpg
-rw-rw-r--  1 user  group  109982 Apr 22 13:21 vegan-chocolate-nut-brownies.jpg
-rw-rw-r--  1 user  group   65650 Apr 22 13:21 vegan-chocolate.jpg
-rw-rw-r--  1 user  group  225830 Apr 22 13:21 veggie-pasta-bake-hero-umami.jpg
-rw-rw-r--  1 user  group  102082 Apr 22 13:21 veggie-pasta-bake-umami.jpg
-rw-rw-r--  1 user  group   65418 Apr 22 13:21 victoria-sponge-umami.jpg
-rw-rw-r--  1 user  group   31072 Apr 22 13:21 watercress-soup-umami.jpg
➜  my-drupal-site ddev exec ls -l web/sites/default/files
total 0

You can get the same result if you delete the web/sites/default/files directory manually on the host and then recreate it with different contents. If I run ddev config --performance-mode=none && ddev restart, everything seems to work correctly.

[rfay]

Does ddev import-files delete the entire web/sites/default/files directory

It deletes the contents of web/sites/default/files

@richardbporter I think it would be best for you to open a new issue and please do the requested ddev debug test so we know what your setup is. Please make sure you're using current stable DDEV v1.23.0, thanks!

And if you could try the suggestions in #6026 (comment) first, that would be great.

[cstuart]

@richardbporter

Or if you want to save yourself the hassle you can use rm -r "${DRUPAL_FILES_DIR}/*" && tar -xzf backup/latest.files.tar.gz -C $DRUPAL_FILES_DIR

[rfay]

Agreed, there is nothing all that magical about ddev import-files. It just tries to untar your source tarball into your files directory.

[richardbporter]

That is essentially the workaround we are going to take for our provider import_files functionality, which is where our team is running into this.

I'll create that other issue though when I can.