The command cvfsdb in Xsan has not been discussed much. There is man page, but it is more of a placeholder than anything else.

cvfsdb is mainly a debugging tool for Xsan volumes. It can normally be invoked in interactive mode as:

cvfsdb <XsanvolumeName>

Using the show subcommand with appropriate arguments within cvfsdb one can see a wealth of information about the Xsan volume. Here are some examples (assuming you are in a terminal on the metadata controller as root user):

To see the details of the Info Control Block:

cvfsdb> show icb

Info Control Block
    icb_fs_marker    = 0x4376467324243232 [CvFs$$22]
    icb_fsname       = Testvol
    icb_fsbsize      = 0x4000
    icb_numparts     = 0x2
    icb_meta_root    = 0x0
    icb_journal_root = 0x0
    icb_inode_offs   = 0x0
    icb_sb_offs      = 0x20
    icb_arb_offs     = 0x21
    icb_cfgi_offs    = 0x22
    icb_diski_offs   = 0x26
    icb_parti_offs   = 0x36
    icb_fl_offs      = 0x46
    icb_journal_offs = 0x47
    icb_data_offs    = 0x48
    icb_sum          = 0x1f37ef87 (okay)

To see the details of the Super Block:

cvfsdb> show sb

Super Block
    sb_marker             = 0x5375506552243230 [SuPeR$20]
    sb_Epoch              = 0x4494780331ca5
    sb_FsStatus           = 0x102
    sb_NumDinodes         = 0x2400
    sb_FreeDinodes        = 0x23ec
    sb_InodeVersion       = 0x203
    sb_FreeListBlocks     = 0x1
    sb_NextInodeChunk     = 0xd90
    sb_FreeDinodeRotor    = 0x16
    sb_NTSecurityIdxInode = 0x5
    sb_NTSecurityDatInode = 0x6
    sb_QuotaInode         = 0x7
    sb_IDDBInode          = 0xc
    sb_ClientWOpens       = 0x8
    sb_sum                = 0x3660496577 (okay)

One of the most useful features of cvfsdb is not only the ability to show information about a given inode corresponding to a file, but also to save the contents of that inode to another location. In the example below the file with inode 15 is MacPorts-1.6.0-10.5-Leopard.dmg, its file permission is 644, the owner has uid 501 and gid 0 etc.

cvfsdb> show inode 15
ShowInode 15
  idi_marker_st     = 0x4e6f [No]
  idi_attr_flags    = 0x10
  idi_user_flags    = 0x0
  idi_expandsize    = 0x0
  idi_flags         = 0xa00
  idi_mode          = 0100644
  idi_nchildren     = 0x1
  idi_nsubdirs      = 0x0
  idi_uid           = 0x1f5
  idi_gid           = 0x0
  idi_gen           = 0x0
  idi_dm_events     = 0x0
  idi_atime.tv_sec  = 0x47eb9ea7
  idi_atime.tv_nsec = 0x2ef69540
  idi_mtime.tv_sec  = 0x47de558a
  idi_mtime.tv_nsec = 0x0
  idi_ctime.tv_sec  = 0x47eb9ea8
  idi_ctime.tv_nsec = 0x9896418
  idi_crttime = 0x47de558a
  idi_affinity      = 0x0
  idi_size          = 0x67329
  idi_blocks        = 0x1a
  idi_nextiel       = 0xd00	(0x340000 / 3407872)
  idi_seqno         = 0xa
  idi_xattr_blk     = 0x0

  Extended attribute area =
000:  0100 8000 3f04 0327 5250 4c00 0000 0000  |....?..'RPL.....
010:  0000 024d 6163 506f 7274 732d 312e 362e  |...MacPorts-1.6.
020:  302d 3130 2e35 2d4c 656f 7061 7264 2e64  |0-10.5-Leopard.d
030:  6d67 0404 084e 5453 4400 0000 0000 0000  |mg...NTSD.......
040:  0000 0000 0000 0000 0000 0000 0000 0000  |................
050:  0000 0000 0000 0000 0000 0000 0000 0000  |................
060:  0000 0000 0000 0000 0000 0000 0000 0000  |................
070:  0000 0000 0000 0000 0000 0000 0000 0000  |................

  Extents =
        [ 0 ] flags     = 0x1
        [ 0 ] sg        = 0x1
        [ 0 ] depth     = 0x0
        [ 0 ] frblock   = 0x0   		( 0 - 25 = 26 )
        [ 0 ] base     *= 0x6
        [ 0 ] end      *= 0x1f

  idi_marker_en     = 0x4465 [De]

We can save this file to another location this way:

cvfsdb> save 15 /var/root/MacPorts-1.6.0-10.5-Leopard-copy.dmg
Saving data from inode 0xf to file "/var/root/MacPorts-1.6.0-10.5-Leopard-copy.dmg"
Saved       26 of       26 blocks (100%)
cvfsdb> 

This feature of saving the file to another location comes very handy in disaster recovery scenarios. A short and highly simplified example is given below.

The scenario is where a low level hardware error is developing in one or more of the disks somewhere in the underlying RAID LUNs, yet the cvfsck command in Xsan reports that the file system is fine.

For example, the customer cannot see a particular folder, say, Projects, under the Xsan volume through the Finder. When the customer does an ls in the terminal, he can see an entry Projects, yet when he types ls –al Projects, he gets an Input/Output error. Yet a file system check using cvfsck shows no problems.

The following assumes that the administrator is operating from the metadata controller. The name of the Xsan volume is Testvol.

The first step is to unmount the volume from the clients and then stop the volume. Then, executing the following command (as root or using sudo if not root) should generate and save a list of all files and folders Xsan can “see:”

cvfsck –x Testvol > allfilesfolders

The file allfilesfolders generated above will have comma separated entries with the following information

Inode#, Mode, Size, Block Count, Affinity, Path, Extent Count, Extent Number, Storage pool, File Relative Block, Base, End, Depth, Breadth

Example entries:

3c4dcd,16877,2048,1,Data,Projects/sunshine,1,0,0,0,3858,3858,0,256 3c4de8,33188,0,0,none,Projects/moonshine,0,0,0,0,0,0,0,0 3ee3ddb,16893,2048,1,none,Projects3/>,1,0,0,0,49948,49948,0,4

Note: Despite what the man page for cvfsck says, the Inode# is reported in hex and not in decimal. The rest of the numerical values are in decimal.

Of particular interest is the second value, ie., mode. For example, if the mode value is 16877, then in octal it is 040775, so the entry corresponds to a directory with 755 permissions, if it is 33206, then in octal it is 100666, so a file with permissions 666, if it is 33188, then in octal it is 100644, so a file with permissions 644. If the mode is 41471, then it is 0120777 in octal, so a symbolic link.

The only interest is in the "missing" Projects folder, so we can extract all such entries to a file called alllostProjects.

Based on the mode value we can then create directory paths in a restore area. We can then use the inode values of the files in conjunction with cvfsdb to restore the files along the appropriate directory paths.

In the case of the example entries above, we can simply do:

mkdir –p /Volumes/restore/Projects/sunshine
echo “save 0x3c4de8 /Volumes/restore/Projects/moonshine” | cvfsdb Testvol

Based on our experience in the field, a somewhat detailed exposition of this example, taken from a real world scenario, along with a sample script will soon be posted in our web site http://www.consol.com/apple under "Tips & Tricks." Please note that Apple may not officially support solutions like this, so you are at your own risk if you decide to deploy something like this in a live environment. Please make sure you absolutely test everything in a lab environment!