Allow to configure HSQLDB DEFRAG value

[sjames-au]

• Added 'defrag' database param
• Proposes a default of 90%
• Added GUI for setting percentage
• read, update and initialize setting
• requires session restart for change to take effect

[kingthorin]

This would definitely be easier to review if all the unnecessary formatting changes were suppressed 😉

[kingthorin]

To address the DCO requirement you'll need to sign-off the commit(s):

• https://github.com/zaproxy/zaproxy/blob/main/CONTRIBUTING.md#developer-certificate-of-origin
• https://git-scm.com/docs/git-commit#Documentation/git-commit.txt---signoff

[kingthorin]

Thanks that's much easier to review now. I think it's largely good, I want to pull the branch and see what the UI and tooltip look like. I'll try to do that tomorrow.

[sjames-au]

Thank you, your calm approach to my mess helped me dive in further. I still have a decision point stuck in my head on the "default" position for this; while the DB software defaults to zero they hint that this should be between 30 and 60, 30 would help keep the file size smallish and zap has a max file size of 128Gb.

I pitched 90% simply because it seemed as close to the current default of never defrag (though zero is still a viable option).
I am leaning towards taking a 30% approach for myself as if I am running a scan, I would rather the potential performance hit of a DB defrag than maxing out the DB size.

Also as a random contributor, while I happy to raise this as a decision point, I am not the right person to make such a choice. If it is purely a me call, I would probably default to 0 just so it is aligned with what has been the zaps history.

[kingthorin]

I need to read about this option/feature. If zero is off and also close to 90 then what's 30? If the suggestion is 30 to 60 then 45 seems logical 🤷‍♂️

[sjames-au]

from: http://www.hsqldb.org/doc/2.0/guide/management-chapt.html
SET FILES DEFRAG

set files defrag statement

::= SET FILES DEFRAG

Sets the threshold for performing a DEFRAG during a checkpoint. The is the percentage of abandoned space in the *.data file. When a CHECKPOINT is performed either as a result of the .log file reaching the limit set by SET FILES LOG SIZE m, or by the user issuing a CHECKPOINT command, the amount of space abandoned since the database was opened is checked and if it is larger than the specified percentage, a CHECKPOINT DEFRAG is performed instead of a CHECKPOINT. As the DEFRAG operation uses a lot of memory and takes a long time with large databases, setting the threshold well above zero is suitable for databases that are around than 500 MB or more.

The default is 0, which indicates no DEFRAG. Useful values are between 30 to 60.

Only a user with the DBA role can execute this statement.

This is equivalent to the connection property hsqldb.defrag_limit.

[kingthorin]

In that case I think my original suggestion of 45 is probably a good balance between performance and disk usage.
When the DB is just about half empty tiddy it up 😉

[sjames-au]

That sounds completely reasonable to me. :)

I will get that updated :)

[kingthorin]

You could wait till tomorrow, others might chime in early Monday.

[sjames-au]

Sure. I have a local workaround, so this isnt blocking me from anything and it will be nice to have helped :) I am already dealing with Mondayitis at my location, the weather is nice though. ;)

I am a little curious about a code quality issue, specifically, I mirrored/copied others work and have ended up with the default being defined in two locations. OptionsDatabasePanel.jave, which sets default within the spinner and DatabaseParam.java which does the real work. Ideally the default should be defined centrally and just referenced where its needed.

If someone could point me to a preferred pattern for that, I would be happy to incorporate it into this change :)