os/bluestore: Recompression, part 4. Scanner, Estimator and core recompression.

[aclamk]

This work is a partial solution to the problem bluestore compression quality on RBD.

Original (v1) compression has following characteristics:

• is as fast as possible on write
• does not care for read speed
• relaxed disk space management
• ignores snaps

New (v2) compression is doing:

• has a (fast - best compresison) dial
• keeps high read speeds
• is very conscious of used space
• is snap aware

#54075 Nice debugs.
#54504 New write path.
#57448 Segmented onode.
#56975 Main
#57450 Test

Initial engineering tests of the feature:
old: https://docs.google.com/spreadsheets/d/1ZRfTYFlc8gdDoU76qrgTFswzViho4ameabIk9TIx6Uw/edit#gid=560728694
moved, no restructions: https://docs.google.com/spreadsheets/d/1M_k2huVDlqZOPC5lz5JoKM1jrPSP9K43ErBoo5cMHDo/edit#gid=1978643008

Below extracted summary:
ref - reference
1.1 - most agressive recompression, it is enough to predict 1.1x gain to do recompress
..
1.7 - relaxed recompression, it requires 1.7x gain estimation to do recompress

Allocated space:

Write IOPs:

CPU used for writes:

Read IOPs: (Ref degradation is not a fluke. It is real and repeatable.):

CPU used for reads: (Ref degradation is not a fluke. It is real and repeatable.):

Contribution Guidelines

• To sign and title your commits, please refer to Submitting Patches to Ceph.

• If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

• When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Review checklist

• new write path is run-time optional
  Implemented.
  Tested by randomly selecting write path on mount: https://github.com/aclamk/ceph/tree/wip-aclamk-bs-compression-recompression-test-writerandom
• recompression is run-time optional
• onode segmentation is run-time optional
  Tested by randomly selecting segment size on mount: https://github.com/aclamk/ceph/tree/wip-aclamk-bs-compression-recompression-test-randomsegment
• upgradeable - new code can work on data produced by old code
  • regular data
  • snaps
  • sharded onodes
• downgradeable - old code can accept data produced by new code
  • regular data
  • snaps
  • sharded onodes
• perf counters are updated in comparable logic
• unit tests that relay on perf counters adapted to both cases
• zero-detection compatibility

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox
• jenkins test windows
• jenkins test rook e2e

[markhpc]

@aclamk Very nice! Digging those 1.7 results especially. They are nearly as good as ref in write IOPS and significantly better in all other metrics.

[aclamk]

jenkins test make check