Warewulf rpm install fails if /var/lib/warewulf/* is read-only

[griznog]

Steps to reproduce

I'm not entirely sure this is a "bug", but it does break my attempts to use a single master which replicates its chroots to other sites via ZFS send/recv to a filesystem with readonly=on.

I can leave readonly=off set on the given filesystems; future send/recv will just roll back any local changes. But that does introduce a small window for site-specific image drift to accumulate which ideally I'd like to prevent. If this failure during install could be a warning that would solve this specific problem, presumably the permissions and contexts are being set correctly when I upgrade the upstream WW master and already replicated to the local site.

Note this is the lowest of low on priorities, I just wanted to note this as it's potentially going to disrupt some HA approaches that want to distribute read-only copies of the node images using ZFS send/recv, an approach that I may be the only one interested in.

1. Make /var/lib/warewulf/chroot read-only.
2. Try to install warewulf rpm.

Error message

Last metadata expiration check: 0:20:43 ago on Mon 29 Apr 2024 07:44:47 AM PDT.
Package warewulf-4.5.0rc3+119.gad2992c4-1.el9.x86_64 not installed, cannot reinstall it.
No match for argument: /root/rpmbuild/RPMS/x86_64/warewulf-4.5.0rc3+119.gad2992c4-1.el9.x86_64.rpm
Error: No packages marked for reinstall.
Last metadata expiration check: 0:20:44 ago on Mon 29 Apr 2024 07:44:47 AM PDT.
Dependencies resolved.
=============================================================================================================================================================================================================
 Package                                     Architecture                              Version                                                         Repository                                       Size
=============================================================================================================================================================================================================
Upgrading:
 warewulf                                    x86_64                                    4.5.0rc3+119.gad2992c4-1.el9                                    @commandline                                     25 M

Transaction Summary
=============================================================================================================================================================================================================
Upgrade  1 Package

Total size: 25 M
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                     1/1 
  Running scriptlet: warewulf-4.5.0rc3+119.gad2992c4-1.el9.x86_64                                                                                                                                        1/2 
  Upgrading        : warewulf-4.5.0rc3+119.gad2992c4-1.el9.x86_64                                                                                                                                        1/2 
error: lsetfilecon: (44 /var/lib/warewulf/chroots, system_u:object_r:var_lib_t:s0) Read-only file system
error: Plugin selinux: hook fsm_file_prepare failed

Error unpacking rpm package warewulf-4.5.0rc3+119.gad2992c4-1.el9.x86_64
  Verifying        : warewulf-4.5.0rc3+119.gad2992c4-1.el9.x86_64                                                                                                                                        1/2 
  Verifying        : warewulf-4.5.x-1.git_96e37278.el9.x86_64                                                                                                                                            2/2 

Failed:
  warewulf-4.5.x-1.git_96e37278.el9.x86_64                                                            warewulf-4.5.0rc3+119.gad2992c4-1.el9.x86_64                                                           

Error: Transaction failed
Failed to install rpm.

Information on your system

• Rocky Linux 9.3
• Building warewulf from latest main

General information

• I have run wwctl version and reported the contents of /etc/os-release
• I have searched the issues of this repo and believe this is not a duplicate
• I have captured and reported relevant error messages and logs

[anderbubble]

@griznog I like the idea of Warewulf being able to install with an empty /var/lib/warewulf. I expect for things in provision and even chroots this would just be fine; but I think, to do this, we'd need to get /var/lib/warewulf/overlays split into something like /usr/share/warewulf/overlays and /etc/warewulf/overlays first.