Deleted wal space is not returned.

[gywndi]

Hi,

I am currently using gridgain version 8.8.20, but there is an issue where the deleted WAL space is not returned.
I checked what file makes this problem with lsof. As shown below, the already deleted WAL files are still being referenced by the gridgaion process, so it was not returned normally.

$ lsof -p $(ps -ef             \
            | grep java        \
            | grep ignite      \
            | grep -v grep     \
            | awk '{print $2}' \
            | head -n 1)       \
  | grep wal

java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000034.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000035.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000036.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000037.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000038.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000039.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000040.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000042.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000041.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000043.wal (deleted)
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000044.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000045.wal

I set walPath and walArchivePath as same in the gridgain configuration as below, this means WAL archiving was disabled.

<property name="dataStorageConfiguration">
  <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
    <property name="defaultDataRegionConfiguration">
      <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
        <property name="name" value="DefaultRegion"/>
        <property name="metricsEnabled" value="true"/>
        <property name="persistenceEnabled" value="true"/>
        <property name="maxSize" value="#{32L * 1024 * 1024 * 1024}"/>
        <property name="checkpointPageBufferSize" value="#{2L * 1024 * 1024 * 1024}"/>
      </bean>
    </property>
    <property name="metricsEnabled" value="true"/>
    <property name="storagePath" value="storage"/>
    <property name="walSegmentSize" value="#{512 * 1024 * 1024}"/>
    <property name="walPath" value="/data/ignite-wal"/>
    <property name="walArchivePath" value="/data/ignite-wal"/>
    <property name="writeThrottlingEnabled" value="false"/>
  </bean>
</property>

I couldn't solve this problem with any other configuration, except setting walPath and walArchivePath parameter different path.

<property name="dataStorageConfiguration">
  <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
    <property name="defaultDataRegionConfiguration">
      <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
        <property name="name" value="DefaultRegion"/>
        <property name="metricsEnabled" value="true"/>
        <property name="persistenceEnabled" value="true"/>
        <property name="maxSize" value="#{32L * 1024 * 1024 * 1024}"/>
        <property name="checkpointPageBufferSize" value="#{2L * 1024 * 1024 * 1024}"/>
      </bean>
    </property>
    <property name="metricsEnabled" value="true"/>
    <property name="storagePath" value="storage"/>
    <property name="walSegmentSize" value="#{512 * 1024 * 1024}"/>
    <property name="walPath" value="/data/ignite-wal"/>
    <property name="walArchivePath" value="/data/ignite-arc"/>
    <property name="writeThrottlingEnabled" value="false"/>
  </bean>
</property>

And the problem was solved as below.

$ lsof -p $(ps -ef             \
            | grep java        \
            | grep ignite      \
            | grep -v grep     \
            | awk '{print $2}' \
            | head -n 1)       \
  | grep wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000008.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000001.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000000.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000000.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000002.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000003.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000004.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000005.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000006.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000007.wal
java ..skip.. /data/ignite-wal/node00-7e8..1a4/0000000000000008.wal

Currently, an alternative has been prepared, but I hope this phenomenon does not occur when WAL is disabled.
(This phenomenon also occurs on 8.8.21.)

Thanks.
Chan.