Data, Metadata, and Fragmentation

Thursday, April 28 2005 @ 01:59 PM EDT

Contributed by: aaron

Question: I have a simple Xsan: two-seats, one Xserve RAID, one storage pool. After copying a whole bunch of tiny files, via the Finder, to the SAN, any write creates a file with at least 30 extents, sometimes as much as 500 on a single file. I tried defragmenting the whole volume, deleting the directories that had the zillion little files, I even upgraded the clients to OS 10.3.9/Xsan 1.0.1. The large-extent render files are not playing back unless I defrag them. My questions are: Are 30-500 extents for single files (100-500MB render files for example) normal? Do you know of anything to cure this poor file system?

Answer: We need to understand several behaviors of Xsan:

Read more for more info.

In your configuration, the volume's metadata is stored on the same disks as the data. So after every file is written to the SAN, a little piece of metadata is written. That metadata is fragmented (not a big problem).

But if you then delete these small Finder files, you have thousands of small block-or-two-sized holes of unused space in-between that metadata.

Copying any new file will use the first available free space, which would be completely fragmented.

Defragmenting would move those pieces of data to contiguous space further down the volume. But defragmenting does not move metadata blocks. This would leave the free space very fragmented. The next new file also fills up those fragmented holes. The problem repeats.

That is why we make the following recommendations: