Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error in count inode glusterfs #4347

Open
alvic2000 opened this issue May 3, 2024 · 1 comment
Open

Error in count inode glusterfs #4347

alvic2000 opened this issue May 3, 2024 · 1 comment

Comments

@alvic2000
Copy link

alvic2000 commented May 3, 2024 •
edited

GlusterFS inodes count show 97% IUse%
but all physical discs in this volume, inodes count show only 5% IUse%

df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
......
/dev/nvme13n1 1500292096 69419375 1430872721 5% /data/brick_nvme13n1
/dev/nvme8n1 1500292096 67343419 1432948677 5% /data/brick_nvme8n1
/dev/nvme22n1 1500292096 66027751 1434264345 5% /data/brick_nvme22n1
/dev/nvme6n1 1500292096 69619855 1430672241 5% /data/brick_nvme6n1
/dev/nvme18n1 1500292096 68580148 1431711948 5% /data/brick_nvme18n1
/dev/nvme23n1 1500292096 66027751 1434264345 5% /data/brick_nvme23n1
/dev/nvme14n1 1500292096 69419375 1430872721 5% /data/brick_nvme14n1
/dev/nvme11n1 1500292096 67343419 1432948677 5% /data/brick_nvme11n1
/dev/nvme16n1 1500292096 68580148 1431711948 5% /data/brick_nvme16n1
/dev/nvme1n1 1500292096 72355100 1427936996 5% /data/brick_nvme1n1
/dev/nvme5n1 1500292096 69619855 1430672241 5% /data/brick_nvme5n1
/dev/nvme19n1 1500292096 68580148 1431711948 5% /data/brick_nvme19n1
/dev/nvme15n1 1500292096 69419375 1430872721 5% /data/brick_nvme15n1
/dev/nvme12n1 1500292096 69419375 1430872721 5% /data/brick_nvme12n1
/dev/nvme2n1 1500292096 72355100 1427936996 5% /data/brick_nvme2n1
/dev/nvme3n1 1500292096 72355100 1427936996 5% /data/brick_nvme3n1
/dev/nvme21n1 1500292096 66027751 1434264345 5% /data/brick_nvme21n1
/dev/nvme20n1 1500292096 66027751 1434264345 5% /data/brick_nvme20n1
/dev/nvme0n1 1500292096 72355100 1427936996 5% /data/brick_nvme0n1
/dev/nvme9n1 1500292096 67343419 1432948677 5% /data/brick_nvme9n1
/dev/nvme4n1 1500292096 69619855 1430672241 5% /data/brick_nvme4n1
/dev/nvme17n1 1500292096 68580148 1431711948 5% /data/brick_nvme17n1
/dev/nvme7n1 1500292096 69619855 1430672241 5% /data/brick_nvme7n1
/dev/nvme10n1 1500292096 67343419 1432948677 5% /data/brick_nvme10n1
......
localhost:/nfs_volume 9001752576 8643902291 357850285 97% /mnt/nfs_volume
......

df -HT
Filesystem Type Size Used Avail Use% Mounted on
.......
/dev/nvme13n1 xfs 16T 470G 15T 4% /data/brick_nvme13n1
/dev/nvme8n1 xfs 16T 472G 15T 4% /data/brick_nvme8n1
/dev/nvme22n1 xfs 16T 491G 15T 4% /data/brick_nvme22n1
/dev/nvme6n1 xfs 16T 528G 15T 4% /data/brick_nvme6n1
/dev/nvme18n1 xfs 16T 465G 15T 4% /data/brick_nvme18n1
/dev/nvme23n1 xfs 16T 491G 15T 4% /data/brick_nvme23n1
/dev/nvme14n1 xfs 16T 470G 15T 4% /data/brick_nvme14n1
/dev/nvme11n1 xfs 16T 472G 15T 4% /data/brick_nvme11n1
/dev/nvme16n1 xfs 16T 465G 15T 4% /data/brick_nvme16n1
/dev/nvme1n1 xfs 16T 618G 15T 5% /data/brick_nvme1n1
/dev/nvme5n1 xfs 16T 528G 15T 4% /data/brick_nvme5n1
/dev/nvme19n1 xfs 16T 465G 15T 4% /data/brick_nvme19n1
/dev/nvme15n1 xfs 16T 470G 15T 4% /data/brick_nvme15n1
/dev/nvme12n1 xfs 16T 470G 15T 4% /data/brick_nvme12n1
/dev/nvme2n1 xfs 16T 618G 15T 5% /data/brick_nvme2n1
/dev/nvme3n1 xfs 16T 618G 15T 5% /data/brick_nvme3n1
/dev/nvme21n1 xfs 16T 491G 15T 4% /data/brick_nvme21n1
/dev/nvme20n1 xfs 16T 491G 15T 4% /data/brick_nvme20n1
/dev/nvme0n1 xfs 16T 618G 15T 5% /data/brick_nvme0n1
/dev/nvme9n1 xfs 16T 472G 15T 4% /data/brick_nvme9n1
/dev/nvme4n1 xfs 16T 528G 15T 4% /data/brick_nvme4n1
/dev/nvme17n1 xfs 16T 465G 15T 4% /data/brick_nvme17n1
/dev/nvme7n1 xfs 16T 528G 15T 4% /data/brick_nvme7n1
/dev/nvme10n1 xfs 16T 472G 15T 4% /data/brick_nvme10n1
........
localhost:/nfs_volume fuse.glusterfs 31T 1.4T 30T 5% /mnt/nfs_volume
.......

Expected result:

IUsed of glusterfs = 6xIUsed of disc (where 6 - number of data disc in subvolume)
IFree of glusterfs = 6xIFree of disc (where 6 - number of data disc in subvolume)
IUse% of glusterfs = 6xIUse% of disc (where 6 - number of data disc in subvolume)

Size of glusterfs = 6xSize of disc (where 6 - number of data disc in subvolume)
Used of glusterfs = 6xUsed of disc (where 6 - number of data disc in subvolume)
Avail of glusterfs = 6Avail of disc (where 6 - number of data disc in subvolume)

*** Mandatory information ***

gluster volume info nfs_volume

Volume Name: nfs_volume
Type: Distributed-Disperse
Volume ID: 9c039b2e-ebd7-4159-b807-6b2e56f64db7
Status: Started
Snapshot Count: 0
Number of Bricks: 6 x (8 + 4) = 72
Transport-type: tcp
Bricks:
Brick1: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme0n1
Brick2: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme1n1
Brick3: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme2n1
Brick4: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme3n1
Brick5: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme0n1
Brick6: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme1n1
Brick7: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme2n1
Brick8: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme3n1
Brick9: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme0n1
Brick10: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme1n1
Brick11: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme2n1
Brick12: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme3n1
Brick13: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme4n1
Brick14: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme5n1
Brick15: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme6n1
Brick16: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme7n1
Brick17: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme4n1
Brick18: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme5n1
Brick19: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme6n1
Brick20: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme7n1
Brick21: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme4n1
Brick22: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme5n1
Brick23: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme6n1
Brick24: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme7n1
Brick25: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme8n1
Brick26: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme9n1
Brick27: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme10n1
Brick28: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme11n1
Brick29: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme8n1
Brick30: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme9n1
Brick31: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme10n1
Brick32: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme11n1
Brick33: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme8n1
Brick34: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme9n1
Brick35: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme10n1
Brick36: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme11n1
Brick37: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme12n1
Brick38: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme13n1
Brick39: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme14n1
Brick40: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme15n1
Brick41: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme12n1
Brick42: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme13n1
Brick43: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme14n1
Brick44: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme15n1
Brick45: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme12n1
Brick46: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme13n1
Brick47: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme14n1
Brick48: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme15n1
Brick49: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme16n1
Brick50: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme17n1
Brick51: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme18n1
Brick52: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme19n1
Brick53: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme16n1
Brick54: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme17n1
Brick55: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme18n1
Brick56: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme19n1
Brick57: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme16n1
Brick58: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme17n1
Brick59: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme18n1
Brick60: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme19n1
Brick61: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme20n1
Brick62: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme21n1
Brick63: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme22n1
Brick64: p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme23n1
Brick65: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme20n1
Brick66: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme21n1
Brick67: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme22n1
Brick68: p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme23n1
Brick69: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme20n1
Brick70: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme21n1
Brick71: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme22n1
Brick72: p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme23n1
Options Reconfigured:
performance.parallel-readdir: on
ctime.noatime: on
performance.stat-prefetch: on
network.inode-lru-limit: 400000
features.cache-invalidation: on
features.cache-invalidation-timeout: 600
performance.cache-invalidation: on
performance.md-cache-timeout: 600
cluster.readdir-optimize: on
performance.readdir-ahead: on
performance.io-cache: on
performance.cache-size: 1GB
client.event-threads: 4
server.event-threads: 4
features.quota-deem-statfs: on
features.inode-quota: on
features.quota: on
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
cluster.enable-shared-storage: enable
nfs-ganesha: enable

gluster volume status nfs_volume

Status of volume: nfs_volume
Gluster process TCP Port RDMA Port Online Pid

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me0n1 49153 0 Y 8396
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me1n1 49154 0 Y 8407
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me2n1 49155 0 Y 8422
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me3n1 49156 0 Y 8432
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me0n1 49153 0 Y 4171
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me1n1 49154 0 Y 4179
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me2n1 49155 0 Y 4187
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me3n1 49156 0 Y 4197
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me0n1 49153 0 Y 4360
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me1n1 49154 0 Y 4381
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me2n1 49155 0 Y 4379
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me3n1 49156 0 Y 4380
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me4n1 49157 0 Y 8451
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me5n1 49158 0 Y 8468
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me6n1 49159 0 Y 8473
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me7n1 49160 0 Y 8485
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me4n1 49157 0 Y 4207
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me5n1 49158 0 Y 4222
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me6n1 49159 0 Y 4230
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me7n1 49160 0 Y 4237
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me4n1 49157 0 Y 4404
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me5n1 49158 0 Y 4411
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me6n1 49159 0 Y 4430
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me7n1 49160 0 Y 4443
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me8n1 49161 0 Y 8505
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me9n1 49162 0 Y 8515
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me10n1 49163 0 Y 8527
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me11n1 49164 0 Y 8548
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me8n1 49161 0 Y 4249
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me9n1 49162 0 Y 4261
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me10n1 49163 0 Y 4274
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me11n1 49164 0 Y 4286
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me8n1 49161 0 Y 4459
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me9n1 49162 0 Y 4458
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me10n1 49163 0 Y 4482
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me11n1 49164 0 Y 4497
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me12n1 49165 0 Y 8557
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me13n1 49166 0 Y 8569
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me14n1 49167 0 Y 8584
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me15n1 49168 0 Y 8593
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me12n1 49165 0 Y 4293
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me13n1 49166 0 Y 4305
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me14n1 49167 0 Y 4324
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me15n1 49168 0 Y 4336
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me12n1 49165 0 Y 4511
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me13n1 49166 0 Y 4522
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me14n1 49167 0 Y 4532
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me15n1 49168 0 Y 4549
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me16n1 49169 0 Y 8607
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me17n1 49170 0 Y 8617
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me18n1 49171 0 Y 8644
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me19n1 49172 0 Y 8652
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me16n1 49169 0 Y 4344
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me17n1 49170 0 Y 4360
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me18n1 49171 0 Y 4372
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me19n1 49172 0 Y 4383
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me16n1 49169 0 Y 4566
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me17n1 49170 0 Y 4572
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me18n1 49171 0 Y 4589
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me19n1 49172 0 Y 4602
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me20n1 49173 0 Y 8663
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me21n1 49174 0 Y 8692
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me22n1 49175 0 Y 8694
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nv
me23n1 49176 0 Y 8716
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me20n1 49173 0 Y 4404
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me21n1 49174 0 Y 4407
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me22n1 49175 0 Y 4427
Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nv
me23n1 49176 0 Y 4444
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me20n1 49173 0 Y 4610
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me21n1 49174 0 Y 4624
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me22n1 49175 0 Y 4633
Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nv
me23n1 49176 0 Y 4643
Self-heal Daemon on localhost N/A N/A Y 4535
Quota Daemon on localhost N/A N/A Y 4834
Self-heal Daemon on p1-nx-fsr0003-be.mydomain.com N/A N/A Y 4707
Quota Daemon on p1-nx-fsr0003-be.mydomain.com N/A N/A Y 4887
Self-heal Daemon on p1-nx-fsr0001-be.mydomain.com N/A N/A Y 9000
Quota Daemon on p1-nx-fsr0001-be.mydomain.com N/A N/A Y 9513

Task Status of Volume nfs_volume

There are no active volume tasks

gluster volume heal nfs_volume info
Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme0n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme1n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme2n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme3n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme0n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme1n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme2n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme3n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme0n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme1n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme2n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme3n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme4n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme5n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme6n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme7n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme4n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme5n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme6n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme7n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme4n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme5n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme6n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme7n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme8n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme9n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme10n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme11n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme8n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme9n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme10n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme11n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme8n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme9n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme10n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme11n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme12n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme13n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme14n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme15n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme12n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme13n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme14n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme15n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme12n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme13n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme14n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme15n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme16n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme17n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme18n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme19n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme16n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme17n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme18n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme19n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme16n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme17n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme18n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme19n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme20n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme21n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme22n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0001-be.mydomain.com:/data/brick_nvme23n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme20n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme21n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme22n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0002-be.mydomain.com:/data/brick_nvme23n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme20n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme21n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme22n1
Status: Connected
Number of entries: 0

Brick p1-nx-fsr0003-be.mydomain.com:/data/brick_nvme23n1
Status: Connected
Number of entries: 0

grep '] E [' /var/log/glusterfs/glusterd.log
[2024-05-03 10:35:59.973676] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for transport.socket.own-thread key
[2024-05-03 10:35:59.975461] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:35:59.975477] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.975485] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:35:59.975488] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.975493] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:35:59.975495] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.975500] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:35:59.975503] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.975507] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:35:59.975510] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.975515] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:35:59.975517] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.975522] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:35:59.975524] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.975529] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:35:59.975532] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.976201] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for limit-set key
[2024-05-03 10:35:59.980767] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for nfs.transport-type key
[2024-05-03 10:35:59.983032] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for nfs.log-level key
[2024-05-03 10:35:59.986613] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for xattr-user-namespace-mode key
[2024-05-03 10:35:59.989246] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.989260] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for config key
[2024-05-03 10:35:59.989268] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for config key
[2024-05-03 10:35:59.989880] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for op-version key
[2024-05-03 10:35:59.989901] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for max-op-version key
[2024-05-03 10:35:59.991201] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.991212] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:35:59.995471] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-seckey key
[2024-05-03 10:35:59.995543] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-keyid key
[2024-05-03 10:35:59.995601] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-bucketid key
[2024-05-03 10:35:59.995659] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-hostname key
[2024-05-03 10:36:22.885397] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for nufa key
[2024-05-03 10:36:22.886475] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for switch key
[2024-05-03 10:36:22.890405] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for strict-readdir key
[2024-05-03 10:36:22.893165] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for readdir-failover key
[2024-05-03 10:36:22.897031] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for least-rate-limit key
[2024-05-03 10:36:22.899258] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.899269] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.899278] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.899803] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for transport.socket.own-thread key
[2024-05-03 10:36:22.901527] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:36:22.901536] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.901546] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:36:22.901549] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.901554] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:36:22.901557] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.901561] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:36:22.901564] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.901569] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:36:22.901571] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.901576] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:36:22.901579] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.901583] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:36:22.901586] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.901591] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 10:36:22.901593] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.902237] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for limit-set key
[2024-05-03 10:36:22.906725] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for nfs.transport-type key
[2024-05-03 10:36:22.909475] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for nfs.log-level key
[2024-05-03 10:36:22.913172] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for xattr-user-namespace-mode key
[2024-05-03 10:36:22.915801] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.915822] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for config key
[2024-05-03 10:36:22.915831] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for config key
[2024-05-03 10:36:22.916452] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for op-version key
[2024-05-03 10:36:22.916466] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for max-op-version key
[2024-05-03 10:36:22.917824] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.917838] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 10:36:22.922104] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-seckey key
[2024-05-03 10:36:22.922176] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-keyid key
[2024-05-03 10:36:22.922235] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-bucketid key
[2024-05-03 10:36:22.922292] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-hostname key
[2024-05-03 11:20:15.635887] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for nufa key
[2024-05-03 11:20:15.636789] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for switch key
[2024-05-03 11:20:15.640027] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for strict-readdir key
[2024-05-03 11:20:15.643036] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for readdir-failover key
[2024-05-03 11:20:15.646432] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for least-rate-limit key
[2024-05-03 11:20:15.648956] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.648970] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.648983] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.649665] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for transport.socket.own-thread key
[2024-05-03 11:20:15.651461] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 11:20:15.651471] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.651477] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 11:20:15.651480] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.651485] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 11:20:15.651488] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.651492] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 11:20:15.651495] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.651500] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 11:20:15.651502] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.651507] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 11:20:15.651510] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.651514] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 11:20:15.651517] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.651525] E [MSGID: 101097] [xlator.c:218:xlator_volopt_dynload] 0-xlator: dlsym(xlator_api) missing: /usr/lib64/glusterfs/6.0/rpc-transport/socket.so: undefined symbol: xlator_api
[2024-05-03 11:20:15.651527] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.652129] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for limit-set key
[2024-05-03 11:20:15.656638] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for nfs.transport-type key
[2024-05-03 11:20:15.658884] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for nfs.log-level key
[2024-05-03 11:20:15.662529] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for xattr-user-namespace-mode key
[2024-05-03 11:20:15.665164] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.665183] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for config key
[2024-05-03 11:20:15.665190] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for config key
[2024-05-03 11:20:15.665802] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for op-version key
[2024-05-03 11:20:15.665817] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for max-op-version key
[2024-05-03 11:20:15.667074] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.667087] E [MSGID: 106434] [glusterd-utils.c:13426:glusterd_get_value_for_vme_entry] 0-management: xlator_volopt_dynload error (-1)
[2024-05-03 11:20:15.671348] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-seckey key
[2024-05-03 11:20:15.671422] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-keyid key
[2024-05-03 11:20:15.671480] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-bucketid key
[2024-05-03 11:20:15.671537] E [MSGID: 106332] [glusterd-utils.c:13437:glusterd_get_value_for_vme_entry] 0-management: Failed to get option for s3plugin-hostname key
[2024-05-03 12:00:23.329796] E [MSGID: 106301] [glusterd-syncop.c:1322:gd_stage_op_phase] 0-management: Staging of operation 'Volume Heal' failed on localhost : This command is supported for only volume of replicated type. Volume nfs_volume is not of type replicate

*** Additional information ***

cat /etc/os-release
NAME="Red Hat Enterprise Linux"
VERSION="8.8 (Ootpa)"

rpm -qa | grep gluster
glusterfs-selinux-1.0-5.el8rhgs.noarch
glusterfs-libs-6.0-63.el8rhgs.x86_64
glusterfs-cli-6.0-63.el8rhgs.x86_64
glusterfs-server-6.0-63.el8rhgs.x86_64
glusterfs-ganesha-6.0-63.el8rhgs.x86_64
glusterfs-client-xlators-6.0-63.el8rhgs.x86_64
glusterfs-geo-replication-6.0-63.el8rhgs.x86_64
python3-gluster-6.0-63.el8rhgs.x86_64
glusterfs-api-6.0-63.el8rhgs.x86_64
glusterfs-fuse-6.0-63.el8rhgs.x86_64
glusterfs-6.0-63.el8rhgs.x86_64
nfs-ganesha-gluster-3.4-10.el8rhgs.x86_64
@anon314159
Copy link

anon314159 commented May 3, 2024 •
edited

FYI, rghs will be EOL December of this year. I would consider migrating to the latest stable maintained version of 10.x or 11.x. I am not sure if the version you currently use (6.0-63) has any of the current bug fixes backported to it.

Similarly related issue:
#4157

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alvic2000 @anon314159