Finder crashes after xsan finder copy, Apple is stumped

[mw10dot1]

Hi Tom

We have seen a number of scenarios on 10.6 san's that cause the files to become fragmented. It does not seem to be limited to afp.

Exports from adobe software, pushing(pulling seems fine) files to the san over afp or smb and duplicating a file on the san using ruby all seem to cause fragmentation.


Michael

[jsiegers]

With us fragmentation is mostly caused when you write a file and it's not sure what the file size would be from the beginning.
So in other terms when an application does not reserve all the blocks it needs at once but keeps requesting more blocks until it's finished.

For example we use an application called AddMovie to make one movie out of many XDCAM clips. This program doesn't preallocate all the blocks it needs but keeps requesting blocks. This is not caused by the application BTW but by the QuickTime framework.
Now when you run AddMovie on at least two sets simulataniously you'll get the situation where both sets keep requesting blocks. So then a set of blocks is given to one set and the next to other, etc.
This is causing some heavy defragmented files in our environment.

[tomwelch]

@mw10dot1 - how did you know it was adobe files? (it usually is though I guess!)

@jsiegers - we're mainly seeing it with psd / psb files, and occasionally quicktimes too, so I wonder if Photoshop requests blocks in the way you mentioned.

In the end, I took lots, and lots! of backups and then ran the 10.6.8 and 2.2.2 updates. A bit of a long night. Everything came back up OK and we've been using the volume for a couple of hours this morning. I'll post back with some updates to let people know how things go.

Tom

[jsiegers]

@tom

Please note that the 2.2.2 update can still create fragmented files.
The 2.2.2 update only makes sure your computer won't crash when it stumbles upon these files.

[tomwelch]

Hi jsiegers

I figured as much Still I guess if apps are not reserving space there is nothing Xsan can do.

I know continually defragmenting a Xsan volume is not a good idea, but I don't really see any other options. If I only defrag fragmented files of say over 100 extends, using your script, on say a biweekly basis do you think that is going to do more harm than good?

We're going to free up some more space and rebuild the volume too, but as its only 3 months old now, this problem is obviously going to come back.

Thanks!

Tom.

[jsiegers]

It's normal for volumes to get fragmented over time.
I'd say leave the files with 100 extents at rest.
I always defray files with more than 1000 extents. And look at the files with more than 500 extents to see what would be best.
For example when I know that people are going to work with a file with around 700 extents I'd defrag only that one. So that they could work without a spinning beach ball. But since Xsan 2.2.2 that might not even be necessary anymore.

[tomwelch]

Thanks jsiegers. It's interesting to get someone else's thoughts on the thresholds of extents

[lucasnap]

Just installed 2.2.2 in a production environment.

The fragmentation problems are less, but not gone. I had (after a big fragmentation test) a hanging Finder with the same symptoms except that the Mac could restart itself after a while (no hard reset needed).

But overall less hangs and less corrupted files.

I'm not convinced that we can go back to the workflow we had in the old days (ingesting/saving large files on the san volume).

[Starbytes]

Any news on the subject? We are having the same problems with a customer..

[tomwelch]

Hi Starbytes

We upgraded to 2.2.2 back in September 2011, and we haven't experienced the problems we were having almost daily since. Fingers crossed, that's it.

Hope this helps


Tom

[lucasnap]

Hi Starbytes,

need some help?

Lucas