I've been working at creating a highly available set of host servers for a linux diskless boot cluster. Each host machine is fitted with redundant power supplies and two 1TB drives in a Raid-1 configuration.

When I first started this project they each had twin 160GB drives. In my original setup I had both nodes acting as primary systems so that I could mount /dev/drbd0 to my /data directory and see changes immediately. It had worked in the past where I could make a file in /data on the first machine and it would show up in the /data directory on the second machine. One day this had stopped working where if I made a file on the first machine it would display on the second one until I unmounted /data and remounted it to the /dev/drbd0 disk. This is when I bought the new disks and decided to start from scratch. The weird thing is that if I make a file on the primary and remount on the secondary to see it and then delete it from the secondary and THEN delete it on the primary, the primary throws no error however it goes into read-only mode because it knows that file was already gone.

Below is my configuration file, sda8 is an 820GB partition which is used for all of the data I want to replicate, currently populated with ~10GB of data. Sda7 is my metadisk partition which is 500MB large clearly more than needed by drbd.

Also you will notice "incon-degr-cmd "halt -f";" is commented out. This is because when I go to actually use it drbd throws an error when reloading the configuration and I'm not sure why that is either


resource r0 {
protocol C;
#incon-degr-cmd "halt -f";
net {
allow-two-primaries;
after-sb-0pri discard-zero-changes;
after-sb-1pri discard-secondary;
after-sb-2pri disconnect;
}
startup {
become-primary-on both;
}
disk {
on-io-error detach;
}
syncer {
rate 128M; # Note: 'M' is MegaBytes, not MegaBits
}
on dh1.domain.com {
device /dev/drbd0;
disk /dev/sda8;
address 192.168.1.140:7789;
meta-disk /dev/sda7[0];
}
on dh2.domain.com {
device /dev/drbd0;
disk /dev/sda8;
address 192.168.1.141:7789;
meta-disk /dev/sda7[0];
}
}

Perhaps I misunderstand your problem, but unless you are running GFS, you may not mount /dev/drbd0 on both machines.

I WAS using the ext3 filesystem. After posting this I started looking for sync problems with ext3 and ext2 filesystems even though I wasn't using ext2. One topic came up and the guy had the same exact problem. Anyway the result of the post was that ext3 is not a shared filesystem and he had to use one of a specific list of shared filesystems in order for it to work. It makes sense that it will not work with ext3.

A list of shared filesystems and block devices were listed:

Shared Block Devices:
SAN
Dual attached SCSI
GNBD
DRBD
iSCSI

Shared Filesystem:
gfs
gfs2
gpfs
ocfs
ocfs2

I choose to go with the ocfs2 filesystem and found a complete guide to setting it up with my current configuration of DRBD without having to invalidate all my data on the second machine and resync. Here is a link to the guide http://www.drbd.org/users-guide/ch-ocfs2.html

Just for the record my problem is solved. For anyone else that comes across this I hope the information above suffices for a resolution for your problem.

reference : DRBD guide *Introduction* : http://www.drbd.org/users-guide-emb/s-dual-primary-mode.html