Xsanity Forums

Troubleshooting :: RE: No More 10.6.2 + Xsan 2.2 for me....

pixel — Thu, 11 Mar 2010 00:32:11 GMT

Author: pixel
Posted: Wed Mar 10, 2010 7:32 pm (GMT -5)

Just started seeing this, very random though. Testing maya renders and it seems hit and miss, and only with Renderman renders...

will post again if i figure anything out.

Troubleshooting :: RE: File Corruption

lotte — Wed, 10 Mar 2010 23:16:31 GMT

Author: lotte
Posted: Wed Mar 10, 2010 6:16 pm (GMT -5)

And you�re running the latest quicktime version with the appropiate imx codec?

Lotte

Hardware :: Anyone set up the Promise VTrak E-Class non-apple style?

Julius — Wed, 10 Mar 2010 20:26:29 GMT

Author: Julius
Subject: Anyone set up the Promise VTrak E-Class non-apple style?
Posted: Wed Mar 10, 2010 3:26 pm (GMT -5)

Hey all!

Currently were running a 14TB raid using the old apple xserve raids, and naturally were looking at upgrading out of them. After looking around it looks like getting a Promise VTrak E-Class with redundant controllers costs around 6k without drives, which is much better then 12k to buy the same thing form apple. I called and talked to the guys at promise and they told me that the apple version of the Promise VTrak E-Class comes with a special apple firmware and anther piece of hardware that is used to connect the drives to both controllers. I was wondering if anyone has an experience with using the NON-apple version of the promise VTrak E-Class? I've also looked at jumping into the new Active Storage XRAID�ES, but they are a bit more on the expensive side. We are on a tight budget and any input would be greatly appreciated!

Current system:
4 x xserve raids : 16tb
3 x SANBox 5200 : inter-connected via the 10gb cable
1 x netgear gigbit : for privet net
3 x netgear gigbit : for Public net
1 x Intel mac : MDC
1 x PowerPC G5 : MDC
1 x Intel mac : render server
9 x Intel macpro : running : FCP/After effects : san enabled

Running RED footage and Proress 422(HQ) primarily.

Running 10.5.8 on all & xsan 2.1.1

Troubleshooting :: RE: Xserve Firmware

aldo1234 — Wed, 10 Mar 2010 20:12:51 GMT

Author: aldo1234
Posted: Wed Mar 10, 2010 3:12 pm (GMT -5)

Hi Mjsanders

The reason i ask is the raid array with firmware 1.5.1f2/1.51b has no issues the other with the latest firmware restarts when put under any pressure. I can get it to restart if i create a file over 2GB.

I must contact apple as it has made the second raid array virtually useless.

Thanks again for your help.

Aldo

Troubleshooting :: RE: File Corruption

abstractrude — Wed, 10 Mar 2010 17:20:18 GMT

Author: abstractrude
Posted: Wed Mar 10, 2010 12:20 pm (GMT -5)

is this bad public atom?

General Xsan Talk :: RE: My recent experience with Xsan2 and smaller file sizes

ravi — Wed, 10 Mar 2010 16:49:00 GMT

Author: ravi
Posted: Wed Mar 10, 2010 11:49 am (GMT -5)

SS wrote:

I haven't experimented with metadata LUNs set to write-back cache, so I can't speak to this; I'd imagine it would make less of an impact if this was the case. However - you are correct - the load from the metadata IO will be spread across two controllers (assuming that you have the LUN affinities set correctly) and two LUNs, thereby increasing performance.

Hi Steve

This paragraph picked my curiosity. As far as the metadata LUNs set to write-back, if you look at StorNext tuning guidelines all the way from their draft document at ADIC to various definitive versions at Quantum, it is consistent and they state (see the RAID Cache Configuration section):

"For example, write-back caching is absolutely essential for metadata stripe groups to achieve high metadata operations throughput.

However, there are a few drawbacks to consider as well. For example, read-ahead caching improves sequential read performance but might reduce random performance. Write-back caching is critical for small write performance but may limit peak large I/O throughput."

The second part of your paragraph is not clear to me. We are talking about random I/O pattern of small transactions, so seek time vs transfer time comes into play.

Troubleshooting :: RE: File Corruption

jsiegers — Wed, 10 Mar 2010 09:10:45 GMT

Author: jsiegers
Posted: Wed Mar 10, 2010 4:10 am (GMT -5)

lotte wrote:

I�ve seen this behavior before but I can�t remember what the solution was...
I would suggest it has something to do with Quicktime, but I�m a little bit unsure....

Lotte

Hmm... A couple of days have past. Hopefully you'll remember it now?

Troubleshooting :: Changing the Metadata Network

sith33 — Tue, 09 Mar 2010 19:50:15 GMT

Author: sith33
Subject: Changing the Metadata Network
Posted: Tue Mar 09, 2010 2:50 pm (GMT -5)

Hey All - I've just been informed that I need to move my metadata network to a new address range.

The san is currently 1.4.2, but I have the upgrade to 2 (2.2.1) scheduled for next week.

First off, any reason to do this with 1.4.2 instead of 2.2.1? Otherwise my preference is to do the full upgrade, and then worry about changing IPs, in case I end up blowing up the volumes anyways.

Otherwise, do I just follow this apple KB article, or do I manually edit fsmlist etc?
http://support.apple.com/kb/HT3898

I'm cool with going down to a single controller and having all volume stopped during this process. Are there other things I should be thinking about?

General Xsan Talk :: RE: My recent experience with Xsan2 and smaller file sizes

abstractrude — Tue, 09 Mar 2010 17:48:59 GMT

Author: abstractrude
Posted: Tue Mar 09, 2010 12:48 pm (GMT -5)

there is a guy snfsguy on here, i wonder what he thinks. I have used a metadata SAS solution in a lab but not in production. It seemed faster under multiple requests but didnt get to spend enough time to actually benchmark it.

General Xsan Talk :: RE: My recent experience with Xsan2 and smaller file sizes

SS — Tue, 09 Mar 2010 17:20:59 GMT

Author: SS
Posted: Tue Mar 09, 2010 12:20 pm (GMT -5)

[quote="staze"]

SS wrote:

daver wrote:

This is simple. Create two RAID1's during initial setup. When dragging LUNs into the Metadata pool, simply drag both of these RAID1's into it and you're good to go. Obviously, you'll have to do away with the two spares in the top chassis for this. You'll still have the protection of the spares located in the JBOD as they can span enclosures. This definitely strays from Apple's recommended practices but then again, this particular configuration isn't exactly posted on their website either. If you'd like help scripting the LUN creation when you decide to try this, get in touch with me and I'll whip something up for you.

Since I haven't tried this, or seen anything about it, I'm curious. Isn't this basically creating a RAID 10 array for the MD, or does Xsan just seen that if there are two RAID1's, it sticks the journal on one, and the MD on the other?

Has there been any testing with using SAS drives for MD rather than the standard SATA drives? Of course, you'd want a J class to be 13 SATA's, and 3 SAS drives to allow a spare for each type of LUN (Data and MD, respectively). But, 15k SAS drives seem like they'd be faster than the dual RAID1 idea, as well as the fact that putting MD on a 1TB RAID1 seems kinda, overkill. =P

Hi staze,
It's my understanding by default, the journal gets placed on the first LUN in the metadata storage pool. The metadata gets striped across both.
As for testing with a SAS MDL, we don't do it in-house here [at Promise]. However, there may be a few people on this forum that may have tried it.....abstractrude ?

Hardware :: RE: Qlogic Stacking

sammya — Tue, 09 Mar 2010 12:47:17 GMT

Author: sammya
Posted: Tue Mar 09, 2010 7:47 am (GMT -5)

The 5802v switches come with 8 8/4gb ports active and the 10gb stacking already licensed.

It's worth looking at the cost difference.

Hardware :: RE: Qlogic Stacking

brianwells — Tue, 09 Mar 2010 01:29:34 GMT

Author: brianwells
Posted: Mon Mar 08, 2010 8:29 pm (GMT -5)

As already stated, simply "cross patching" devices is not enough - the two Fibre Channel switches need to be connected together in order for the network to function. This could be done with 10Gb interconnects or 4Gb jumpers. Obviously, the 4Gb connections will be a bottleneck to your data throughput.

For ideal data throughput and redundancy, I would enable the 10Gb ports and have two stacking cables between the switches. This is what we did when we were using the QLogic 5200 switches and it worked just great.

The peace of mind was worth the extra money to enable the 10Gb ports. We were able to reuse most of the stacking cables when we upgraded to QLogic 5802V switches last year.

-- Brian Wells

Hardware :: RE: Qlogic Stacking

morphenine — Mon, 08 Mar 2010 23:04:24 GMT

Author: morphenine
Posted: Mon Mar 08, 2010 6:04 pm (GMT -5)

In order to "cross patch" each device, you need to have the switches linked anyway. However I would suggest that you link the two switches together with at least 2 10gig connections AND cross patch the devices. This will create redundancy along with the high speeds. If you really don't want to license the 10gb ports, and you dont mind the reduced speed, then you can link the switches with a couple of normal 4gb connections.

Hardware :: Qlogic Stacking

mrtubz — Mon, 08 Mar 2010 22:14:30 GMT

Author: mrtubz
Subject: Qlogic Stacking
Posted: Mon Mar 08, 2010 5:14 pm (GMT -5)

Hi All

Wonder if anyone has any ideas on this? Currently looking at buying 2 x Qlogic 5600 switches to setup our first Xsan.

Im debating as there are two switches whether to go for licensing the 10GB ports and using the stacking cables or, not licensing the ports and then 'cross patching' i.e one FC Cable to Switch 1 and the other to Switch 2. This would be done with all devices.

I was also wondering if I could use a combination of both methods? Simply as the 'cross patching' approach gives me better redundancy, obviously should we go to a 3 switch stack we would have to license the 10GB ports. It just seems like a waste of money to license stacking ports if by cross patching it wont use them.

Any ideas on this would be appreciated.

Thanks

Dan.

Troubleshooting :: RE: Xserve Firmware

mjsanders — Mon, 08 Mar 2010 22:04:14 GMT

Author: mjsanders
Posted: Mon Mar 08, 2010 5:04 pm (GMT -5)

Apple Firmware updates do not allow 'downgrades', so the only way is to stay at current version or update (and never back... )
And the firmware is tied to the version of the Xserve Raid Admin tool.
so the 1.5.1 version of the utility has the 1.5.1 firmware
the 1.5 version is here 1.5
Older versions I cannot find easily. Contact me offline if you need them

maurits

Troubleshooting :: RE: drop frame

ravi — Mon, 08 Mar 2010 22:03:08 GMT

Author: ravi
Posted: Mon Mar 08, 2010 5:03 pm (GMT -5)

abstractrude wrote:

word. sorry ravi if i offended.

No offense taken, but my point is that there is a lot of misinformation floating out here on pools, LUNs, stripe breadths etc. Things are not simple and cut-and-dried.

To elaborate on MattG's point on "non-blessed" storage, let me throw in some practical numbers. There are detailed technical explanations for a lot of scenarios, but I will not go there now. It is fundamentally very simple: The number of pools and number of LUNs in a pool are a function of one or more of these: your average I/O request sizes, how you want to balance writes vs reads, the internal architecture of the storage boxes you use, etc., etc.

I.

Assume you have, say for example, a DDN S2A9550 couplet with dual independent fabric, and assume all the storage and fibre parameters are configured as they should be. Assume you use 4gig Apple dual fibre cards for Xsan clients in a StorNext environment. Assume you need 4 StorNext volumes, with a main high throughput volume for video and the rest for general purpose and misc. purposes. Assume you have carved out 1 RAID1 LUN for metadata and 10 RAID6 LUNs for video data for the main volume with the rest of the storage assigned to the other three volumes. Assume a filesystem block size of 16384 in bytes, ThreadPool size 128, stripe breadth 512K for the data pool, and the rest typical StorNext parameters. Assume you have 10 to 12 edit stations, 4-6 ingest stations, ProRes 422 style predominant workflow with some uncompressed SD thrown in:

With two pools of 5 RAID6 LUNs each, you can typically sustain close to 640MB/sec without dropping frames.

With one pool of 10 RAID6 LUNs, you can easily sustain close to 1.2 GB/sec without dropping frames.

II

A second example: Assume you are using Infortrends, say 4 A16F-R2431s for Xsan video data LUNs. Assume you have a separate 1U dual controller Infortrend for Xsan metadata LUN. Assume you have a Qlogic fabric. Assume all the storage and fibre parameters are configured as they should be. Assume you use ATTO dual port 4gig fibre cards and not Apple dual port 4gig fibre cards. Assume you want 4 stations for 1080i uncompressed 10 bit 4:4:4, 1 stream each, two stations of 1080i uncompressed 10 bit 4:2:2, 1 stream each, and a station for uncompressed SD. Assume you have an Xsan file system block size of 16K and a RAID stripe size 128K for data LUNs (you can use higher file system block sizes like 64K or 128 and/or higher RAID stripe size 1024K, but that is a separate discussion). Assume you have configured a stripe breadth of 8 blocks (why 8 blocks for stripe breadth? That is a separate discussion).

With two pools of 4 RAID5 LUNs each, you can typically sustain close to 640MB/sec without dropping frames.

With one pool of 8 RAID5 LUNs, you can easily sustain close 1.2GB/sec without dropping frames.

III.

Here is a third example: Xsan, Active Storage, 20 RAID5 LUNs out of 10 boxes in a single video data pool, 3.2GB/sec sustained throughput, and not a single drop of frame over 8 months. With that many LUNs, personally I would have balanced out the writes with two pools, but I am not going to tell the guy who installed it that he didn't know what he was doing. The client had a fundamental sustained throughput requirement with respect to various reads and writes combinations with a mixture of DVCPROHD and uncompressed SD and the results far exceeded those requirements.

I can go on and on, eg., with enterprise monsters like Hitachi or boxes like Nexsan in the other end of the spectrum, but you get the idea. You cannot configure LUNs and pools and stripe breadths without taking into account the characteristics and internal architecture of the underlying storage.

Even in the case of Promise, there is simply no reason why you should stick to a RAID stripe size of 64 in the Promise boxes as in the Apple scripts. It is a configurable parameter, 128 is better in certain workflows. You can't tell me "Apple says this, so that number is set in stone." Not so.

When I hear the 4 LUNs per pool theory, the first thing that comes to my mind is E=2, J=2, E+J= 4, what a coincidence!

Troubleshooting :: RE: drop frame

abstractrude — Mon, 08 Mar 2010 19:49:03 GMT

Author: abstractrude
Posted: Mon Mar 08, 2010 2:49 pm (GMT -5)

word. sorry ravi if i offended.

Troubleshooting :: RE: drop frame

MattG — Mon, 08 Mar 2010 18:00:12 GMT

Author: MattG
Posted: Mon Mar 08, 2010 1:00 pm (GMT -5)

All:

Not all Xsan configurations conform to the standards that Apple outlines in its documentation, especially when non-blessed storage is being used.

Let's be respectful of other opinions.

MG

Troubleshooting :: RE: drop frame

abstractrude — Mon, 08 Mar 2010 14:37:30 GMT

Author: abstractrude
Posted: Mon Mar 08, 2010 9:37 am (GMT -5)

he has too many LUNS in his storage pool he is trying to write to. its like a broken record. the problem i believe is the software has to spread the data across too many LUNS. also, the number of those LUNS are uneven. i know even numbers are your safe bet.

Troubleshooting :: Xserve Firmware

aldo1234 — Mon, 08 Mar 2010 13:48:22 GMT

Author: aldo1234
Subject: Xserve Firmware
Posted: Mon Mar 08, 2010 8:48 am (GMT -5)

Hi

I have one working raid array with firmware 1.5.1f2/1.51b the other has the latest firmware. I cant find the old firmware version on apples support site does anyone have a copy or a link?

Thanks

Troubleshooting :: RE: Unable to start FSS of the volume after power outage

AL — Mon, 08 Mar 2010 13:30:34 GMT

Author: AL
Posted: Mon Mar 08, 2010 8:30 am (GMT -5)

Sorry that I haven't replied earlier and also thanks to lotte for pointing me to right direction. After I run cvfsck according to the instructions we got the system up and running. We lost only few files so the biggest issue after all was couple of days downtime. Thanks for the tip of the automated shutdown although in this case it would not have saved us since the UPS didn't run out of battery but the whole unit shutdown. But that's entirely different story and I'll save you from that. Anyways, thanks a lot for all help.

Troubleshooting :: RE: Problem with Swedish special chars on XSAN connected storage

NEO — Mon, 08 Mar 2010 12:54:34 GMT

Author: NEO
Posted: Mon Mar 08, 2010 7:54 am (GMT -5)

Mkay. Seems like this isn't something local and unique to us but a wide spread "problem". The issue here seems to be related to unicode normalization. It seems like HFS+ enforce filenames to be in decomposed unicode (Unicode Normalization Form D). Other OSes like windows prefer (but don't enforce) precomposed unicode (unicode normalization form c). So I guess we would have to make the OSX client to write to the SAN in precomposed mode instead of decomposed mode. How we would do that is beyond my knowledge and I'm not sure if this is an Xsan, SNFS or OSX "issue".

Some interesting read on the subject.
http://code.google.com/p/macfuse/issues/detail?id=139

In there there's a link to a linux utility for converting between the too. Guess that could be an option for us.

Troubleshooting :: Problem with Swedish special chars on XSAN connected storage

NEO — Mon, 08 Mar 2010 10:51:38 GMT

Author: NEO
Subject: Problem with Swedish special chars on XSAN connected storage
Posted: Mon Mar 08, 2010 5:51 am (GMT -5)

Morning guys!

I have some issues over here with a SNFS based SAN from Bright Systems. On the Mac clients it's connected with XSAN 1.5.x and on the Windows clients it uses StoreNext File System Client to mount the system.

The problem I have is with all the files that have the Swedish funny AAO's (��, AAO with dots over them) in the filenames. When copied to the SAN it almost feels and looks like something changes the character coding. On the Mac side it looks ok but on a windows client that looks on the SAN the AAOs are "crippled"/replaced. Worth to mention is that files from windows with AAO in them looks ok on the OSX client.

This is a screenshot of some AAO foldernames in varios environments. Local on OSX,
http://temp.irry.com/neo/mekaniken/bright/brightInternationalCharacters.v3.jpg

And this is how a path looks in command line on windows:
http://temp.irry.com/neo/mekaniken/bright/brightInternationalCharacters.v4.jpg

So it basically takes the swedish AAO and replaces them with regular AAO and adds the dots as a seperate character after them. Really strange.

Have any one seen this before? Has any one solved it?

I looked in OSX and the character encoding and it looks like it on OS-level maps the characters OK. Atleast the codes are the same on win and OSX. So it feels like something "on the way" messes with it.

Cheers and hope you understood what I wrote and that someone have a theory.

General Xsan Talk :: RE: My recent experience with Xsan2 and smaller file sizes

staze — Mon, 08 Mar 2010 06:23:59 GMT

Author: staze
Posted: Mon Mar 08, 2010 1:23 am (GMT -5)

[quote="SS"]

daver wrote:

Troubleshooting :: RE: drop frame

ravi — Mon, 08 Mar 2010 06:04:26 GMT

Author: ravi
Posted: Mon Mar 08, 2010 1:04 am (GMT -5)

abstractrude

The number of LUNs in a pool is a function of your I/O request sizes, how you want to balance writes vs reads, the architecture of the storage boxes you use and so forth.

In this case, Perseo needs either one data pool or two. Two is better for balance in writes.