[jan-jaap]

This is a repost from Nekochan, from December 1st 2010. It was partially in the nekonomicon. I thought it was worth preserving

I added 4TB of local storage to my Origin 350: two 2TB SATA disks attached to an LSI SAS3041X-R, striped using XVM. These are “eco” (5400RPM) models. The SAS3041 is in the lowest PCI slot of the second Origin 350, where you would normally find the IO9. It has the PCI bus to itself so it runs at 100 or 133MHz PCI-X speed. I had to order some special 70cm SATA cables with an angled connector at the drive end: there’s insufficient between the drives and the nearest fan for anything else.


Angled SATA connector


Power cable


LSI SAS3041X-R, underneath a QLA2342 dual 2Gb FC board

Needed for this modification are:

• Two SATA disks. I used 2TB Samsung disks
  
• Two O350 drive sleds or the Intel / SUN equivalent
  
• Two 70cm SATA cables with a straight connector on one end and an angled connector on the other end
  
• A SATA power cable with two vertically attached SATA power plugs and a Molex
  
• Another misc power cable with at least a Molex plug
  
• One TYCO ELECTRONICS / AMP – 3-640426-4 – HOUSING, 18AWG, 4WAY
  

Of course you cannot boot from this, and the drive bay must not have the SCSI drive backplane either. So this works only in a system with at least two modules.
Space between the disks and the blower is limited. I looked around for a suitable SATA backplane but found nothing. Actually, I suspect that the Altix 350 has it. But that’s why the cables have to be just right. I had the right SATA power cable in my junk bin, but not the extra long, angled SATA cables. They have to be the ‘left’ / ‘straight’ model.
There are some clips on the outside of the Origin 350 chassis, and the space to route the cable around the fan, which is why I chose the left hand cable. The more common cables with vertical attachment won’t work on the lower drive (not enough space).

Benchmarking

The disks installed in the Origin 350:

Code:

# diskpatch -v
sc0d1l0:  Disk        SEAGATE ST373453LC      9507  Serial: 3HW2BVHV
sc5d1l0:  Disk        ATA    SAMSUNG HD204UI 0001  Serial: S2HGJ1BZ
sc5d2l0:  Disk        ATA    SAMSUNG HD204UI 0001  Serial: S2HGJ1BZ

Some results with the SATA XVM stripe:

Code:

# diskperf -W -D -n "SATA stripe 4TB (2*HD204UI)"  /xvm0/junk
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name    : SATA stripe 4TB (2*HD204UI)
# Test date    : Fri Nov  5 10:57:17 2010
# Test machine  : IRIX64 speedo 6.5 07202013 IP35
# Test type    : XFS data subvolume
# Test path    : /xvm0/junk
# Request sizes : min=16384 max=16777216
# Parameters    : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 1341112320 bytes
#---------------------------------------------------------
# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      16384  55.88  70.44  60.45  14.16    8.35    1.81
      32768  108.69  100.48  94.44  16.10  15.45    3.61
      65536  149.80  150.16  135.75  19.38  29.30    6.90
    131072  195.81  198.17  194.69  26.10  29.54  10.76
    262144  198.04  197.83  196.17  33.64  49.47  20.15
    524288  191.79  197.42  189.57  54.66  77.68  36.58
    1048576  199.66  197.79  188.56  67.87  105.20  63.45
    2097152  195.73  197.21  190.51  100.51  134.21  95.33
    4194304  200.74  197.27  191.49  139.87  160.75  128.47
    8388608  199.87  197.80  196.63  162.92  177.47  155.89
  16777216  201.08  197.08  194.55  169.71  198.21  174.91

Compare this to the system disk, a Seagate 15K.3:

Code:

# diskperf -W -D -n "Seagate 15K.3 ST373453LC" /tmp/junk
#---------------------------------------------------------
# Disk Performance Test Results Generated By Diskperf V1.2
#
# Test name    : Seagate 15K.3 ST373453LC
# Test date    : Fri Nov  5 11:14:22 2010
# Test machine  : IRIX64 speedo 6.5 07202013 IP35
# Test type    : XFS data subvolume
# Test path    : /tmp/junk
# Request sizes : min=16384 max=16777216
# Parameters    : direct=1 time=10 scale=1.000 delay=0.000
# XFS file size : 1073741824 bytes
#---------------------------------------------------------
# req_size  fwd_wt  fwd_rd  bwd_wt  bwd_rd  rnd_wt  rnd_rd
#  (bytes)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)  (MB/s)
#---------------------------------------------------------
      16384  23.32  65.05  13.64  22.25    6.37    3.28
      32768  60.58  67.06  21.84  27.76  11.57    6.19
      65536  61.83  67.00  31.23  30.94  19.33  11.09
    131072  60.91  67.04  40.32  30.13  29.41  19.10
    262144  61.99  67.25  49.14  34.72  39.77  29.26
    524288  61.85  67.55  55.04  50.05  46.37  40.16
    1048576  61.62  67.03  55.29  55.64  46.08  49.00
    2097152  61.96  67.37  59.26  59.52  53.08  40.21
    4194304  28.18  37.33  48.09  60.36  43.42  61.29
    8388608  61.24  63.95  60.65  65.78  58.89  64.25
  16777216  61.34  67.52  62.85  66.93  61.51  65.58

As was to be expected, the 5400 RPM disks loose to the 15K disk for small random I/O, but for everything else the SCSI disk takes a severe beating. Both tests use direct I/O, all disks have the (disk) write cache enabled. The system is in use, which probably explains the dip at block size 4194304 for the SCSI disk.