Gluster - Repair volume - Extend Cluster

General support questions
Post Reply
User avatar
penguinpages
Posts: 42
Joined: 2015/07/21 13:58:05

Gluster - Repair volume - Extend Cluster

Post by penguinpages » 2019/09/27 15:14:18

I am trying to become more competent with gluster. There is not any active IRC / Slack forums that are responsive so sorry.. but dumping into general OS community.

What I have today is a two node cluster.. one brick on each active. Now trying to add new node and get away from when crash / reboot happens a split brain mod.

Each server now has a new 1TB SSD.

New server can't start glusterd viewtopic.php?f=47&t=71815 working on that.. fresh install and still issues ..Ugh...

But... I also seem to have some clean up to do first:

[root@thor ~]# gluster vol info gv1

Volume Name: gv1
Type: Replicate
Volume ID: 0d366584-4757-4bb8-be2d-e8c617e50553
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: thorrp.penguinpages.local:/data/thorvdo_bay1/gv1
Brick2: odinrp.penguinpages.local:/data/odinvdo_bay1/gv1
Options Reconfigured:
features.trash: on
cluster.self-heal-daemon: enable
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
server.allow-insecure: on
cluster.enable-shared-storage: enable
[root@thor ~]# gluster volume heal gv1 info
Brick thorrp.penguinpages.local:/data/thorvdo_bay1/gv1
<gfid:1034336e-7f26-4e8b-b27f-3cc43fbeb477>
<gfid:e0bbd39e-fa02-4014-b17d-017e746f49d8>
<gfid:157dd98d-edad-4ef6-b8f4-e34199492308>
Status: Connected
Number of entries: 3

Brick odinrp.penguinpages.local:/data/odinvdo_bay1/gv1
<gfid:b2f14d96-18e3-4201-b5fe-c6052d193e13>
<gfid:42caf9aa-09c3-49f8-98fb-1bf3fab81dd5>
<gfid:f4537b55-e197-482f-8280-734406b9688a>
<gfid:6d6222ef-6277-4869-bb52-70b4acf4fca3>
<gfid:ea89eeb7-82ff-4cde-b1de-f7fb9ffb1cba>
<gfid:b6358ff7-22e5-410d-b24b-313569d6babd>
<gfid:1e50b9ba-5bdd-4a90-af24-73952fda3b75>
<gfid:05cb813c-e07e-4092-8a38-f06184fdec45>
<gfid:0923e784-d01a-46dd-b2d1-006268debc6b>
<gfid:56e4c0d8-2999-430b-8a19-59227e8684fd>
<gfid:bc180fde-f5c3-48c4-95d8-9c02aed2a9cd>
<gfid:b1fe3c0c-90bb-4ba7-b1a3-62eac87fd4bb>
<gfid:e21690b4-600d-45d7-ab19-e1f7d004e1ca>
/vms/ns02.qcow2
Status: Connected
Number of entries: 14

[root@thor ~]#



So a bunch of files are not syncing / happy. Not sure how to repair:


Tried to move up one level to see if bricks ok... and not sure what / where the "ss_brick" came from.. but can't remove brick when not bound to volume.... So not sure if this is related to the above inability to repair.

[root@thor ~]# gluster v status
Status of volume: gluster_shared_storage
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick odinrp.penguinpages.local:/var/lib/gl
usterd/ss_brick 49152 0 Y 7613
Brick thorrp.penguinpages.local:/var/lib/gl
usterd/ss_brick N/A N/A N N/A

Self-heal Daemon on localhost N/A N/A Y 14053
Self-heal Daemon on odin N/A N/A Y 20932

Task Status of Volume gluster_shared_storage
------------------------------------------------------------------------------
There are no active volume tasks

Status of volume: gv1
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick thorrp.penguinpages.local:/data/thorv
do_bay1/gv1 49152 0 Y 7304
Brick odinrp.penguinpages.local:/data/odinv
do_bay1/gv1 49153 0 Y 7622
Self-heal Daemon on localhost N/A N/A Y 14053
Self-heal Daemon on odin N/A N/A Y 20932

Task Status of Volume gv1
------------------------------------------------------------------------------
There are no active volume tasks

[root@thor ~]#


Question:
1) ss_brick -> those seem to be bricks... maybe left over from some initial testing to setup I did... how do I remove them?
2) How do I force a clean up on those GUIDs which are not replicating
3) I want to add third node, I want it to also have and host data, but also be arbiter node. Can all three be peers for data as well as one primary role as arbiter? All with new drives .. get things set then move data to new volume and recommission old drives ... I think command would be:
gluster volume create gv0 \
replica 3 \
gfs01:/gluster/bricks/1/brick \
gfs02:/gluster/bricks/2/brick \
gfs03:/gluster/bricks/2/brick

But recommendations appreciated. I have live VMs on this so don't want to "wipe and start over.. that may be easier at one level but i need to learn debug / root cause, as well as how to repair and extending.

Thanks,

Post Reply

Return to “CentOS 7 - General Support”