Cluster Filesystems - Printable Version
+- IRIX Network Forums (//forums.irixnet.org)
+-- Forum: Other Platforms (//forums.irixnet.org/forum-5.html)
+--- Forum: Other UNIX (//forums.irixnet.org/forum-30.html)
+--- Thread: Cluster Filesystems (/thread-4041.html)
+- IRIX Network Forums (//forums.irixnet.org)
+-- Forum: Other Platforms (//forums.irixnet.org/forum-5.html)
+--- Forum: Other UNIX (//forums.irixnet.org/forum-30.html)
+--- Thread: Cluster Filesystems (/thread-4041.html)
Cluster Filesystems - jpstewart - 08-21-2023
Does anyone here have any experience with cluster filesystems? I've got a a fibre channel SAN setup here (IBM DS3512) and I thought it would be neat to share it between a couple of my main Linux boxes with a cluster filesystem.
A little research suggested that OCFS2 would be the simplest for me. The ocfs2-tools package on Debian contains everything needed to configure the cluster heartbeat and all that stuff. Setting up the two hosts was indeed straightforward. I managed to get them accessing one filesystem on the SAN array without any problems. But the performance of OCFS2 was very disappointing. An rsync backup job to the array seemed vastly slower than before the change to OCFS2, but that was hard to quantify due to the nature of the data being backed up. The weekly job to tar up the array's contents and write it out to tape (LTO5) was also quite slow, running at 58 MB/sec.
I've reverted back to XFS on the array and sharing it via NFS for now. (Which is how it was accessed before the conversion to OCFS2.) The rsync to the array takes about half as much time. Backing up from the array now runs over 100 MB/sec and completes in 3 hours and 21 minutes vs 6 hours and 7 minutes under OCFS2. The difference in performance is huge, for both reading and writing.
So that's got me wondering if there was something wrong with my OCFS2 configuration or if that's just the nature of the beast. And, is it OCFS2 that is slow or all cluster filesystems?
Before I go through the process of configuring GFS2, does anyone here have experience with it? Or with SGI's CXFS? CXFS is commercially licensed so not really a viable option for my little hobby project, but I wonder how it compares with traditional XFS.
I'd love to hear your thoughts and comments if you have experience with clustered filesystems. Are there other open source options I should be looking at beyond OCFS2 and GFS2?
RE: Cluster Filesystems - Linux-RISC - 08-21-2023
Very, very interesting question, I'm cooking pop corn.
You could try to test Proxmox cluster file systems (https://pve.proxmox.com/wiki/Storage, https://pve.proxmox.com/wiki/Proxmox_Cluster_File_System_(pmxcfs)) and compare performances.
I tried Proxmox CFS in the past but no success, I always have installed a NFS server as "SAN controller", converting a SAN to a NAS if a DFS was necessary.