hdiutil create fails with permission denied on file in DMG target, only on some machines

[apjanke]

I'm now doing the AS build on a different machine, scone, and I'm getting some "permission denied" errors during the DMG creation. In the hdiutil create command for the "create interstitial DMG" step. (I switched to scone since eilonwy was having problems.)

Blocking #243.

TODO

• Use sudo for the create-dmg call.
• Add a --no-sudo option to skip the new sudo behavior.

Description

Creating DMG...
Creating dist DMG at ./build/Octave-8.4.0_beta1.dmg
running: /Users/janke/repos/create-dmg/create-dmg --verbose --volname Octave 8.4.0_beta1 --volicon ./build/Octave-8.4.0_beta1.app/Contents/Resources/applet.icns --window-size 550 500 --icon-size 48 --icon Octave-8.4.0_beta1.app 125 180 --hide-extension Octave-8.4.0_beta1.app --app-drop-link 415 180 --eula /Applications/Octave-8.4.0_beta1-BUILT.app/Contents/Resources/usr/Homebrew/Library/Taps/octave-app/homebrew-octave-app/assets/COPYING/COPYING.rtf --add-file COPYING ./build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/octave-octapp@8.4.0/8.4.0/COPYING 126 300 --add-file VERSIONS ./build/Octave-8.4.0_beta1.app/Contents/Resources/VERSIONS 415 300 --disk-image-size 7500 --background ./assets/background.tiff ./build/Octave-8.4.0_beta1.dmg ./build/Octave-8.4.0_beta1.app
Creating disk image...
Creating interstitial DMG at: /Users/janke/repos/octave-app-bundler/build/cdmg_temp.rw.83531.Octave-8.4.0_beta1.dmg
running: hdiutil create -srcfolder /Users/janke/repos/octave-app-bundler/build/Octave-8.4.0_beta1.app -volname Octave 8.4.0_beta1 -fs HFS+ -fsargs -c c=64,a=16,e=16 -format UDRW -size 7500m /Users/janke/repos/octave-app-bundler/build/cdmg_temp.rw.83531.Octave-8.4.0_beta1.dmg
...could not access /Volumes/Octave 8.4.0_beta1/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.so - Permission denied

hdiutil: create failed - Permission denied
janke@scone:~/repos/octave-app-bundler $

I do not know what's going on here.

Diagnosis

I've run it a few times, and the error is always on that same file, usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.so.

It's talking about a location under /Volumes and that Octave 8.4.0_beta1 is the name of the volume in the DMG, so I think it's talking about the target location in the mounted DMG there. Once this error happens, I don't see any mounted DMGs under /Volumes, so I can't examine it to e.g. look at the file permissions in the DMG's file system.

Permissions on the source for that file seem normal. It's a symlink pointing to an extant file next to it. I can read it.

hdiutil: create failed - Permission denied
janke@scone:~/repos/octave-app-bundler $ ls -l /Applications/Octave-8.4.0_beta1-BUILT.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.so
lrwxr-xr-x@ 1 janke  admin  19 Jan 31 02:09 /Applications/Octave-8.4.0_beta1-BUILT.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.so -> libp2edrvlplot.0.so
janke@scone:~/repos/octave-app-bundler $ ls -l build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.so
lr-xr-xr-x@ 1 janke  admin  19 Jan 31 02:09 build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.so -> libp2edrvlplot.0.so
janke@scone:~/repos/octave-app-bundler $ ls -l build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/*
-r--r--r--@ 1 janke  admin   85360 Jan 31 02:09 build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.0.so
lr-xr-xr-x@ 1 janke  admin      19 Jan 31 02:09 build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.so -> libp2edrvlplot.0.so
-r--r--r--@ 1 janke  admin  649680 Jan 31 02:09 build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvstd.0.so
lr-xr-xr-x@ 1 janke  admin      17 Jan 31 02:09 build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvstd.so -> libp2edrvstd.0.so

janke@scone:~/repos/octave-app-bundler $ f=build/Octave-8.4.0_beta1.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib/pstoedit/libp2edrvlplot.so
janke@scone:~/repos/octave-app-bundler $ readlink $f
libp2edrvlplot.0.so
janke@scone:~/repos/octave-app-bundler $ head $f | od -c | head
0000000  317 372 355 376  \f  \0  \0 001  \0  \0  \0  \0  \b  \0  \0  \0
0000020  022  \0  \0  \0  \0  \a  \0  \0 205 200 001  \0  \0  \0  \0  \0
0000040  031  \0  \0  \0   x 002  \0  \0   _   _   T   E   X   T  \0  \0
0000060   \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000100   \0 200  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0  \0
0000120   \0 200  \0  \0  \0  \0  \0  \0 005  \0  \0  \0 005  \0  \0  \0
0000140   \a  \0  \0  \0  \0  \0  \0  \0   _   _   t   e   x   t  \0  \0
0000160   \0  \0  \0  \0  \0  \0  \0  \0   _   _   T   E   X   T  \0  \0

Progress summary

References

• Similar-ish issue: hdiutil failures when creating DMGs on macOS 13 runner actions/runner-images#7522
  • comment about XProtectBehaviorService. Suggests running with sudo, or killing a service.
• Similar-ish issue: Operation not permitted during dmg building on macOS 11 Big Sur electron-userland/electron-builder#5465
  • They worked around it by not using HFS+
• https://eclecticlight.co/2023/05/10/how-macos-now-tracks-the-provenance-of-apps/

[apjanke]

I was able to recreate it with an isolated hdiutil create command on a test directory that I populated from the pstoedit lib dir copied from the app bundle.

2024-02-01 08:56:30.925 diskimages-helper[87961:3162155] _postflightMountPointsAfterDAMount: disk4 aborting because no mount point found.
2024-02-01 08:56:30.934 diskimages-helper[87961:3162157] *useRealIDs******** euid/egid changed to 504,20 (uid/gid is 504,20)
Copying…
2024-02-01 08:56:30.935 diskimages-helper[87961:3162157] *useEffectiveIDs**** euid/egid changed to 504,20 (uid/gid is 504,20)
2024-02-01 08:56:30.941 copy-helper[87976:3162243] copying /Users/janke/tmp/repro_hdiutil_error/source1 to /Volumes/source1
2024-02-01 08:56:30.941 copy-helper[87976:3162243] About to copy "/Users/janke/tmp/repro_hdiutil_error/source1".
2024-02-01 08:56:30.944 copy-helper[87976:3162243] copy error (canceling): /Volumes/source1/lib/pstoedit/libp2edrvstd.so: Permission denied
2024-02-01 08:56:30.944 copy-helper[87976:3162243] Copy finished with error 13 (Permission denied).
..Error 13 (Permission denied).
could not access /Volumes/source1/lib/pstoedit/libp2edrvstd.so - Permission denied

Finishing…
2024-02-01 08:56:32.102 diskimages-helper[87961:3162190] -processKernelRequest: flush received
2024-02-01 08:56:32.110 diskimages-helper[87961:3162157] *useEffectiveIDs**** euid/egid changed to 504,20 (uid/gid is 504,20)
DIHLDiskImageCreate() returned 13
hdiutil: create: returning 13
hdiutil: create failed - Permission denied
janke@scone:~/tmp/repro_hdiutil_error $

But when I moved that lib dir around and re-copied it in to the source2 directory, the error went away.

2024-02-01 09:07:34.925 copy-helper[90294:3170989] About to copy "/Users/janke/tmp/repro_hdiutil_error/source2".
2024-02-01 09:07:34.931 copy-helper[90294:3170989] Copy finished.
....
2024-02-01 09:07:36.090 diskimages-helper[90279:3170937] -processKernelRequest: flush received
2024-02-01 09:07:36.097 diskimages-helper[90279:3170903] *useRealIDs******** euid/egid changed to 504,20 (uid/gid is 504,20)
Finishing…
2024-02-01 09:07:36.098 diskimages-helper[90279:3170903] *useEffectiveIDs**** euid/egid changed to 504,20 (uid/gid is 504,20)
created: /Users/janke/tmp/repro_hdiutil_error/build/repro_hdiutil_temp.rw.90275.source2.dmg
hdiutil: create: returning 0
janke@scone:~/tmp/repro_hdiutil_error $ open .
janke@scone:~/tmp/repro_hdiutil_error $

Re-created the test dir again:

janke@scone:~/tmp/repro_hdiutil_error $ mkdir source3
janke@scone:~/tmp/repro_hdiutil_error $ cp source1/*.txt source3
janke@scone:~/tmp/repro_hdiutil_error $ cd source3
janke@scone:~/tmp/repro_hdiutil_error/source3 $ cp -R /Applications/Octave-8.4.0_beta1-BUILT.app/Contents/Resources/usr/Cellar/pstoedit/3.78/lib .
janke@scone:~/tmp/repro_hdiutil_error/source3 $ find lib -ls
10799838        0 drwxr-xr-x    6 janke            staff                 192 Feb  1 09:10 lib
10799839        0 drwxr-xr-x    6 janke            staff                 192 Feb  1 09:10 lib/pstoedit
10799840     1272 -rw-r--r--    1 janke            staff              649680 Feb  1 09:10 lib/pstoedit/libp2edrvstd.0.so
10799841        0 lrwxr-xr-x    1 janke            staff                  19 Feb  1 09:10 lib/pstoedit/libp2edrvlplot.so -> libp2edrvlplot.0.so
10799842        0 lr--r--r--    1 janke            staff                  17 Feb  1 09:10 lib/pstoedit/libp2edrvstd.so -> libp2edrvstd.0.so
10799843      168 -r--r--r--    1 janke            staff               85360 Feb  1 09:10 lib/pstoedit/libp2edrvlplot.0.so
10799844        0 drwxr-xr-x    3 janke            staff                  96 Feb  1 09:10 lib/pkgconfig
10799845        8 -r--r--r--    1 janke            staff                 372 Feb  1 09:10 lib/pkgconfig/pstoedit.pc
10799846        0 lrwxr-xr-x    1 janke            staff                  19 Feb  1 09:10 lib/libpstoedit.dylib -> libpstoedit.0.dylib
10799847      800 -r--r--r--    1 janke            staff              407648 Feb  1 09:10 lib/libpstoedit.0.dylib
janke@scone:~/tmp/repro_hdiutil_error/source3 $

And yep, the error happens again.

2024-02-01 09:11:19.328 copy-helper[90353:3172922] copying /Users/janke/tmp/repro_hdiutil_error/source3 to /Volumes/source3
2024-02-01 09:11:19.328 copy-helper[90353:3172922] About to copy "/Users/janke/tmp/repro_hdiutil_error/source3".
2024-02-01 09:11:19.331 copy-helper[90353:3172922] copy error (canceling): /Volumes/source3/lib/pstoedit/libp2edrvstd.so: Permission denied
2024-02-01 09:11:19.331 copy-helper[90353:3172922] Copy finished with error 13 (Permission denied).
...Error 13 (Permission denied).
could not access /Volumes/source3/lib/pstoedit/libp2edrvstd.so - Permission denied

Finishing…
2024-02-01 09:11:20.546 diskimages-helper[90338:3172871] -processKernelRequest: flush received
2024-02-01 09:11:20.553 diskimages-helper[90338:3172824] *useEffectiveIDs**** euid/egid changed to 504,20 (uid/gid is 504,20)
DIHLDiskImageCreate() returned 13
hdiutil: create: returning 13
hdiutil: create failed - Permission denied
janke@scone:~/tmp/repro_hdiutil_error $

That libp2edrvstd.so file is different from the libp2edrvlplot.so file in the original error. But it's the same arrangement: a symlink to a .so file right next to it.

I don't see any difference between these files and directories. Same file contents, same order in the directories, same permissions AFAICT. Maybe it's an extended attribute (xattr) difference of some sort?

janke@scone:~/tmp/repro_hdiutil_error $ diff -r source2/lib source3/lib
janke@scone:~/tmp/repro_hdiutil_error $
janke@scone:~/tmp/repro_hdiutil_error $ find source2/lib -ls
10799339        0 drwxr-xr-x    6 janke            staff                 192 Feb  1 09:04 source2/lib
10799340        0 drwxr-xr-x    6 janke            staff                 192 Feb  1 09:04 source2/lib/pstoedit
10799341     1272 -rw-r--r--    1 janke            staff              649680 Feb  1 09:04 source2/lib/pstoedit/libp2edrvstd.0.so
10799342        0 lrwxr-xr-x    1 janke            staff                  19 Feb  1 09:04 source2/lib/pstoedit/libp2edrvlplot.so -> libp2edrvlplot.0.so
10799343        0 lrw-r--r--    1 janke            staff                  17 Feb  1 09:04 source2/lib/pstoedit/libp2edrvstd.so -> libp2edrvstd.0.so
10799344      168 -r--r--r--    1 janke            staff               85360 Feb  1 09:04 source2/lib/pstoedit/libp2edrvlplot.0.so
10799345        0 drwxr-xr-x    3 janke            staff                  96 Feb  1 09:04 source2/lib/pkgconfig
10799346        8 -r--r--r--    1 janke            staff                 372 Feb  1 09:04 source2/lib/pkgconfig/pstoedit.pc
10799347        0 lrwxr-xr-x    1 janke            staff                  19 Feb  1 09:04 source2/lib/libpstoedit.dylib -> libpstoedit.0.dylib
10799348      800 -r--r--r--    1 janke            staff              407648 Feb  1 09:04 source2/lib/libpstoedit.0.dylib
janke@scone:~/tmp/repro_hdiutil_error $
janke@scone:~/tmp/repro_hdiutil_error $ find source3/lib -ls
10799838        0 drwxr-xr-x    6 janke            staff                 192 Feb  1 09:10 source3/lib
10799839        0 drwxr-xr-x    6 janke            staff                 192 Feb  1 09:10 source3/lib/pstoedit
10799840     1272 -rw-r--r--    1 janke            staff              649680 Feb  1 09:10 source3/lib/pstoedit/libp2edrvstd.0.so
10799841        0 lrwxr-xr-x    1 janke            staff                  19 Feb  1 09:10 source3/lib/pstoedit/libp2edrvlplot.so -> libp2edrvlplot.0.so
10799842        0 lr--r--r--    1 janke            staff                  17 Feb  1 09:10 source3/lib/pstoedit/libp2edrvstd.so -> libp2edrvstd.0.so
10799843      168 -r--r--r--    1 janke            staff               85360 Feb  1 09:10 source3/lib/pstoedit/libp2edrvlplot.0.so
10799844        0 drwxr-xr-x    3 janke            staff                  96 Feb  1 09:10 source3/lib/pkgconfig
10799845        8 -r--r--r--    1 janke            staff                 372 Feb  1 09:10 source3/lib/pkgconfig/pstoedit.pc
10799846        0 lrwxr-xr-x    1 janke            staff                  19 Feb  1 09:10 source3/lib/libpstoedit.dylib -> libpstoedit.0.dylib
10799847      800 -r--r--r--    1 janke            staff              407648 Feb  1 09:10 source3/lib/libpstoedit.0.dylib
janke@scone:~/tmp/repro_hdiutil_error $

But xattrs look the same too:

janke@scone:~/tmp/repro_hdiutil_error $ xattr -l source3/lib/pstoedit/*
source3/lib/pstoedit/libp2edrvlplot.0.so: com.apple.provenance:
source3/lib/pstoedit/libp2edrvlplot.so: com.apple.provenance:
source3/lib/pstoedit/libp2edrvstd.0.so: com.apple.provenance:
source3/lib/pstoedit/libp2edrvstd.so: com.apple.provenance:
janke@scone:~/tmp/repro_hdiutil_error $ xattr -p com.apple.provenance source3/lib/pstoedit/*
source3/lib/pstoedit/libp2edrvlplot.0.so:
source3/lib/pstoedit/libp2edrvlplot.so:
source3/lib/pstoedit/libp2edrvstd.0.so:
source3/lib/pstoedit/libp2edrvstd.so:
janke@scone:~/tmp/repro_hdiutil_error $
janke@scone:~/tmp/repro_hdiutil_error $
janke@scone:~/tmp/repro_hdiutil_error $ xattr -l source2/lib/pstoedit/*
source2/lib/pstoedit/libp2edrvlplot.0.so: com.apple.provenance:
source2/lib/pstoedit/libp2edrvlplot.so: com.apple.provenance:
source2/lib/pstoedit/libp2edrvstd.0.so: com.apple.provenance:
source2/lib/pstoedit/libp2edrvstd.so: com.apple.provenance:
janke@scone:~/tmp/repro_hdiutil_error $ xattr -p com.apple.provenance source2/lib/pstoedit/*
source2/lib/pstoedit/libp2edrvlplot.0.so:
source2/lib/pstoedit/libp2edrvlplot.so:
source2/lib/pstoedit/libp2edrvstd.0.so:
source2/lib/pstoedit/libp2edrvstd.so:
janke@scone:~/tmp/repro_hdiutil_error $

[apjanke]

Hmm. I tried clearing the xattrs on my source dir, but that didn't seem to work. Doesn't seem like I can clear the com.apple.provenance xattr at all. And the error is still happening when I ran that. And there's the same xattr on the source2 which is working. So maybe it's not this xattr.

janke@scone:~/tmp/repro_hdiutil_error/source4-xattr-clear $ xattr -c -r lib
xattr: [Errno 13] Permission denied: 'lib/pstoedit/libp2edrvlplot.so'
xattr: [Errno 13] Permission denied: 'lib/pstoedit/libp2edrvlplot.0.so'
xattr: [Errno 13] Permission denied: 'lib/pkgconfig/pstoedit.pc'
xattr: [Errno 13] Permission denied: 'lib/libpstoedit.dylib'
xattr: [Errno 13] Permission denied: 'lib/libpstoedit.0.dylib'
janke@scone:~/tmp/repro_hdiutil_error/source4-xattr-clear $ sudo xattr -c -r lib
Password:
janke@scone:~/tmp/repro_hdiutil_error/source4-xattr-clear $ xattr -l -r .
./lib/pstoedit/libp2edrvstd.0.so: com.apple.provenance:
./lib/pstoedit/libp2edrvlplot.so: com.apple.provenance:
./lib/pstoedit/libp2edrvstd.so: com.apple.provenance:
./lib/pstoedit/libp2edrvlplot.0.so: com.apple.provenance:
./lib/pstoedit: com.apple.provenance:
./lib/pkgconfig/pstoedit.pc: com.apple.provenance:
./lib/pkgconfig: com.apple.provenance:
./lib/libpstoedit.dylib: com.apple.provenance:
./lib/libpstoedit.0.dylib: com.apple.provenance:
./lib: com.apple.provenance:
./hello-link.txt: com.apple.provenance:
./hello.txt: com.apple.provenance:
.: com.apple.provenance:
janke@scone:~/tmp/repro_hdiutil_error/source4-xattr-clear $ sudo xattr -c -r .
janke@scone:~/tmp/repro_hdiutil_error/source4-xattr-clear $
janke@scone:~/tmp/repro_hdiutil_error/source4-xattr-clear $
janke@scone:~/tmp/repro_hdiutil_error/source4-xattr-clear $ xattr -l -r .
./lib/pstoedit/libp2edrvstd.0.so: com.apple.provenance:
./lib/pstoedit/libp2edrvlplot.so: com.apple.provenance:
./lib/pstoedit/libp2edrvstd.so: com.apple.provenance:
./lib/pstoedit/libp2edrvlplot.0.so: com.apple.provenance:
./lib/pstoedit: com.apple.provenance:
./lib/pkgconfig/pstoedit.pc: com.apple.provenance:
./lib/pkgconfig: com.apple.provenance:
./lib/libpstoedit.dylib: com.apple.provenance:
./lib/libpstoedit.0.dylib: com.apple.provenance:
./lib: com.apple.provenance:
./hello-link.txt: com.apple.provenance:
./hello.txt: com.apple.provenance:
.: com.apple.provenance:

[apjanke]

Hmm. Doing a tar on these files is resulting in a "failed to restore metadata" error. That sounds xattr-ish. A sudo fixes it. Maybe the hdiutil create needs to be done with sudo.

janke@scone:~/tmp/repro_hdiutil_error/expand $ tar cf - -C .. source3 | tar x
source3/lib/pstoedit/libp2edrvstd.so: Failed to restore metadata
tar: Error exit delayed from previous errors.
janke@scone:~/tmp/repro_hdiutil_error/expand $ tar cf source3.tar -C .. source3
janke@scone:~/tmp/repro_hdiutil_error/expand $ tar xf source3.tar
source3/lib/pstoedit/libp2edrvstd.so: Failed to restore metadata
tar: Error exit delayed from previous errors.
janke@scone:~/tmp/repro_hdiutil_error/expand $
janke@scone:~/tmp/repro_hdiutil_error/expand $ sudo tar xf source3.tar
Password:
janke@scone:~/tmp/repro_hdiutil_error/expand $

Yeah... adding sudo to the repro-err-hdiutil source3 call results in success.

I'm not a fan of this, but maybe we just need to sudo the hdiutil create by sudo-ing the create-dmg call.

[apjanke]

Added sudo to the create-dmg call here: octave-app/octave-app-bundler@b9a004a.

Need to add a --no-sudo option so people can get the old behavior in case they're running it in a headless environment, don't have admin access, or can't use sudo for some other reason. It'll break sometimes, but at least they can make the attempt on their machine and try to figure it out.

[apjanke]

Hasn't recurred recently. I think the sudo did fix it.

Changing this ticket's milestone from 8.4.0 to The Future, since all that's left is probably adding the --no-sudo option, and that's not urgent.