Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lep): auto-balance pressured disks #7576

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

c3y1huang
Copy link
Contributor

@c3y1huang c3y1huang commented Jan 8, 2024

Which issue(s) this PR fixes:

Issue #4105

What this PR does / why we need it:

Propose to enhance replica auto-balance (best-effort) with replica-auto-balance-disk-pressure-percentage setting to allow automatically rebuild replica on another disk when the defined threshold is reached.

Special notes for your reviewer:

None

Additional documentation or context

None

@c3y1huang c3y1huang self-assigned this Jan 8, 2024
@c3y1huang c3y1huang force-pushed the lep-feat-auto-balance-node-disks branch 2 times, most recently from 101af10 to 66b6aed Compare January 30, 2024 05:27
longhorn-4105

Signed-off-by: Chin-Ya Huang <chin-ya.huang@suse.com>
@c3y1huang c3y1huang force-pushed the lep-feat-auto-balance-node-disks branch from 66b6aed to 3e6c271 Compare January 30, 2024 05:28
@c3y1huang c3y1huang marked this pull request as ready for review January 30, 2024 05:29
@c3y1huang c3y1huang requested a review from a team as a code owner January 30, 2024 05:29
Copy link
Contributor

@ejweber ejweber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC correctly, there is no mechanism to select which replica on a disk would be best (whatever that means) to autobalance. Instead, it is essentially random. Whichever volume happens to reconcile first when a disk is under pressure is balanced to a different disk.

Do you think there is a significant advantage to choosing one replica versus another? If so, how difficult do you think it would be to add that kind of choice to the implementation? (My initial thought is that it would add significant complexity, since right now all replica scheduling decisions are made with only the replica to be scheduled and the aggregate available space on nodes in disks in mind.)

string target = 2;
}
```
1. Then the sync-agent server is responsible for copying the files using `req.SyncFileInfoList` from source to the target directory when `req.FilesSyncRequest.LocalSync` is provided.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How exactly are the files copied? I have experienced some strange behavior when experimenting with copying replica sparse files before, even when using options meant to preserve sparseness. The checksums always matched (and IIRC matched even if sparseness was not preserved), but it the actual space usage was different by a little bit between the source and target file. I think the exact behavior varied between copy utilities (e.g. rsync vs cp, etc.).

@derekbit
Copy link
Member

derekbit commented Apr 25, 2024

Some questions before diving into the codes:

  1. There is no lock mechanism in replica scheduler. Is it possible that all volumes are triggered to be rebuilt on the same disk while reaching to the user-defined threshold?

  2. Set replica-soft-anti-affinity to enabled is a requirement for enabling the feature. Will it introduce a side effect that multiple replicas of a volume are created on the same node?

  3. Snapshot files are sparse. Will the copy method handle holes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants