WFLYCTL0056: Could not create a timestamped backup of current history standalone/configuration/standalone_xml_history/current

[slominskir]

When running Wildfly in a container making configuration changes with jboss-cli then restarting Wildfly results in the following Exception:

[org.jboss.as.controller] (Controller Boot Thread) WFLYCTL0056: Could not rename /opt/jboss/wildfly/standalone/configuration/standalone_xml_history/current to /opt/jboss/wildfly/standalone/configuration/standalone_xml_history/20221003-195729566: java.nio.file.DirectoryNotEmptyException: /opt/jboss/wildfly/standalone/configuration/standalone_xml_history/current
#12 17.20       at java.base/sun.nio.fs.UnixCopyFile.ensureEmptyDir(UnixCopyFile.java:384)
#12 17.20       at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:484)
#12 17.20       at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)
#12 17.20       at java.base/java.nio.file.Files.move(Files.java:1421)
#12 17.20       at org.jboss.as.controller@18.1.2.Final//org.jboss.as.controller.persistence.ConfigurationFile.forcedMove(ConfigurationFile.java:817)
#12 17.20       at org.jboss.as.controller@18.1.2.Final//org.jboss.as.controller.persistence.ConfigurationFile.createHistoryDirectory(ConfigurationFile.java:699)
#12 17.20       at org.jboss.as.controller@18.1.2.Final//org.jboss.as.controller.persistence.ConfigurationFile.successfulBoot(ConfigurationFile.java:542)
#12 17.20       at org.jboss.as.controller@18.1.2.Final//org.jboss.as.controller.persistence.BackupXmlConfigurationPersister.successfulBoot(BackupXmlConfigurationPersister.java:95)
#12 17.20       at org.jboss.as.controller@18.1.2.Final//org.jboss.as.controller.AbstractControllerService.finishBoot(AbstractControllerService.java:637)
#12 17.20       at org.jboss.as.server@18.1.2.Final//org.jboss.as.server.ServerService.finishBoot(ServerService.java:456)
#12 17.20       at org.jboss.as.server@18.1.2.Final//org.jboss.as.server.ServerService.boot(ServerService.java:417)
#12 17.20       at org.jboss.as.controller@18.1.2.Final//org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:473)
#12 17.20       at java.base/java.lang.Thread.run(Thread.java:834)
#12 17.20
#12 17.20 19:57:29,568 WARN  [org.jboss.as.controller] (Controller Boot Thread) WFLYCTL0414: Could not create a timestamped backup of current history dir /opt/jboss/wildfly/standalone/configuration/standalone_xml_history/current, so it may still include versions from the previous boot.

This is reported in a similar context here: https://issues.redhat.com/browse/CLOUD-2195

That issue is marked as closed, but I'm not sure what the resolution is. It seems the workaround for now is to just ignore the warning. The description of that issue suggests the issue is related to limitations of the Overlay Filesystem in Docker and EXDEV error with the rename command. It seems if Wildfly is going to be Docker friendly it should fall back to the copy and unlink strategy for renames as suggested. Note: mounting a volume may work as a temporary workaround, but it's likely worse than just ignoring the warning as you'll possibly trigger a nasty race condition in the 9P file server client.

[slominskir]

Note: This only happens if you create your own Docker image and bake-in config changes in different layers.

See also:

• https://issues.redhat.com/browse/CLOUD-3499
• https://issues.redhat.com/browse/WFCORE-4135
• https://issues.redhat.com/browse/WFCORE-3937

Looks like workarounds include:

1. Just ignore the WARNING log message
2. Use the --read-only-server-config command line argument
3. In your own Dockerfile create an additional final stage that just copies everything from the Wildfly install dir from the previous final stage. This ensures all Wildfly files are in the same top layer.

I've only tried workaround 1 so far

[slominskir]

Actually, looks like just removing the history directory in the same layer as your setup script that modifies the history is sufficient as a workaround to avoid the WARNING:

RUN /setup.sh \
        && rm -rf /opt/jboss/wildfly/standalone/configuration/standalone_xml_history