Performance tuning small Xsan
I just set up a small Xsan and I'm pretty disappointed with the performance.
I have 2 promise E610FD's that were direct attached to two Red Hat Linux boxes running Autodesk Flame and Flare. Both have the latest firmware. One is an apple unit and the other is dark gray.
I used Apple's script to configure them for Xsan so they are identical. One has metadata and one has journal on it's raid1. They have two raid5's, one on each controller. They're full of stock 1TB hitachi enterprise drives. There are two spare drives per chassis.
I plugged both of them into a Sanbox 5600 switch. The switch is zoned to send each channel of FC from the raids directly to one controller port in the workstations. There is also a zone that allows the two channel MCD's to see all of the raid controllers.
Streamguard is configured for all of the computers but not for the arrays. The switch reports that all of the links are 4G. I am using copper cables for the arrays and fibre for the computers.
Both linux machines are connected with 4x FC ports to the switch.
My MDC's are minis with promise Sanlinks.
The Autodesk software used to report ~725MB/second reads with one raid direct attached to 4 ports. Now it reports ~650MB/second using both arrays. I was really counting on getting a bit of a speed boost by using two arrays but that didn't happen.
Any ideas where to look for the bottleneck? I have never direct attached both raids to one linux box but I had always assumed that two would be much faster than one. Slower is not what I had in mind when I stared this project.
-One idea that strikes me would be to upgrade the drives. Those 1TB drives are pretty long in the tooth. I believe they are all 7k1000's.
I've seen the suggestion on this site to use SSD's for MDC and Journal. How big would they need to be? I'm assuming they would need to go in the Promise Array, but maybe I could just get by with one pair....
Thanks in advance for your ideas.