[WIP] mon/OSDMonitor: force-remove-snap --purged-snaps-only

[Matan-B]

Blocked By: #57548

---

    /*
     *  Forces removal of snapshots in the range of
     *  [lower_snapid_bound, upper_snapid_bound) on pool <pool>
     *  in order to cause OSDs to re-trim them.
     *  * purged-snaps-only: Adding this flag will result
     *    in the reremoval of snapids in the given range.
     *    Only the snapids which are *already* marked as purged in the
     *    Monitor will be removed again. This may be useful for cases in
     *    which we would like to trigger OSD snap trimming again.
     */

Reremove snap ids in the range which are already marked as purged.

---

Notes:

• `scrub_purged_snaps` can also be used to cause the OSD to re-trim
  purged snapshots. However, those will only get re-trimmed if they were
  marked as purged (PSN_ keys) in the OSD store.
  Using the command introduced here, the snapshots that were marked as
  purged (purged_snaps_ keys) in the mon's store will be also marked,
  correspondingly, in the OSD store.
  That way `scrub_purged_snaps` will be able to re-trim the snapshots that weren't
  marked as purged in the OSD side (for some reason).
  

• The re-removed snapshots will be inserted to new_purged_snaps which
  will be used when handling the new MOSDMap. These new_purged_snaps
  are passed to SnapMapper::record_purged_snaps to be added to the OSD store.
  

---

Initial testing looks to be stable:

2023-09-19T14:33:10.222 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,16) from pool 3. reremoved snapids: 2,3,6,8,14
2023-09-19T14:36:28.131 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,206) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,189,192,193,195,196,197,199,200,201,205
2023-09-19T14:36:42.319 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,220) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,189,192,193,195,196,197,199,200,201,205,206,207,208,212,213,215,218,219
2023-09-19T14:37:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,234) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,157,189,190,218,219,220,221,222,225,226,229,232,233
2023-09-19T14:42:13.540 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,443) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,157,189,190,218,219,220,221,222,223,224,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,283,284,285,286,287,347,348,349,350,383,384,385,386,387,388,389,394,395,396,426,427,428,429,432,433,434,438,441,442
2023-09-19T14:44:49.541 INFO:teuthology.orchestra.run.smithi084.stderr:force reremoving snap ids in the range of [1,563) from pool 3. reremoved snapids: 2,3,4,6,7,8,9,11,46,47,49,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,94,95,96,97,98,99,100,101,102,103,104,105,106,107,109,110,111,112,128,129,130,131,132,133,146,147,148,149,150,151,152,153,154,155,156,157,189,190,218,219,220,221,222,223,224,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,283,284,285,286,287,347,348,349,350,383,384,385,386,387,388,389,394,395,396,426,427,428,429,492,493,494,495,496,497,498,499,526,527,528,529,530,536,537,538,539,543,552,554,561

---

Contribution Guidelines

• To sign and title your commits, please refer to Submitting Patches to Ceph.

• If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

• When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox
• jenkins test windows
• jenkins test rook e2e