Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PR1] allow parallel lookups #1866

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[PR1] allow parallel lookups #1866

wants to merge 1 commit into from

Conversation

sethiay
Copy link
Collaborator

@sethiay sethiay commented Apr 26, 2024

Description

Allow parallel lookups by
(a) passing FUSE_PARALLEL_DIROPS option which allows parallel dirops (readdir and lookup calls) from FUSE driver. In my experiments i noticed that with this flag, readdir becomes parallel for same directory and lookup becomes parallel for files under same mount. Also, the even without this option, the lookup calls are parallel on repeat lookups of same files. GCSFuse takes locks on dir inode during readdir operations hence with this option. for user only the lookups are parallelized.
(b) take readonly lock on dir inode instead of exclusive lock while doing lookup.

Link to the issue in case of a bug fix.

NA

Testing details

  1. Manual - NA
  2. Unit tests - Added in [PR2] Composite tests for Parallel dirops test. #1906
  3. Integration tests - Run1, Run2, Run3, Run 4, Run 5 (More e2e tests to be added in next PR: [PR3] E2E tests for parallel dirops #1907)
  4. Perf test (Run 4)
    image

@sethiay sethiay added execute-perf-test Execute performance test in PR execute-integration-tests Run only integration tests labels Apr 26, 2024
@sethiay sethiay force-pushed the fix_inode_lock branch 2 times, most recently from 135b862 to cad3614 Compare April 26, 2024 18:50
@sethiay sethiay force-pushed the fix_inode_lock branch 2 times, most recently from 21f3b41 to 91d449f Compare May 13, 2024 10:50
@sethiay sethiay changed the title Changes to allow parallel dirops and lookups [PR1] allow parallel dirops and lookups May 13, 2024
@sethiay sethiay changed the title [PR1] allow parallel dirops and lookups [PR1] allow parallel lookups May 14, 2024
@sethiay sethiay marked this pull request as ready for review May 14, 2024 12:24
@sethiay sethiay requested a review from a team as a code owner May 14, 2024 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
execute-integration-tests Run only integration tests execute-perf-test Execute performance test in PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant