The command cvfsdb in Xsan has not been discussed much. There is man page, but it is more of a placeholder than anything else.
cvfsdb is mainly a debugging tool for Xsan volumes. It can normally be invoked in interactive mode as:
Using the show subcommand with appropriate arguments within cvfsdb one can see a wealth of information about the Xsan volume. Here are some examples (assuming you are in a terminal on the metadata controller as root user):
To see the details of the Info Control Block:
cvfsdb> show icb Info Control Block icb_fs_marker = 0x4376467324243232 [CvFs$$22] icb_fsname = Testvol icb_fsbsize = 0x4000 icb_numparts = 0x2 icb_meta_root = 0x0 icb_journal_root = 0x0 icb_inode_offs = 0x0 icb_sb_offs = 0x20 icb_arb_offs = 0x21 icb_cfgi_offs = 0x22 icb_diski_offs = 0x26 icb_parti_offs = 0x36 icb_fl_offs = 0x46 icb_journal_offs = 0x47 icb_data_offs = 0x48 icb_sum = 0x1f37ef87 (okay)
To see the details of the Super Block:
cvfsdb> show sb Super Block sb_marker = 0x5375506552243230 [SuPeR$20] sb_Epoch = 0x4494780331ca5 sb_FsStatus = 0x102 sb_NumDinodes = 0x2400 sb_FreeDinodes = 0x23ec sb_InodeVersion = 0x203 sb_FreeListBlocks = 0x1 sb_NextInodeChunk = 0xd90 sb_FreeDinodeRotor = 0x16 sb_NTSecurityIdxInode = 0x5 sb_NTSecurityDatInode = 0x6 sb_QuotaInode = 0x7 sb_IDDBInode = 0xc sb_ClientWOpens = 0x8 sb_sum = 0x3660496577 (okay)
One of the most useful features of cvfsdb is not only the ability to show information about a given inode corresponding to a file, but also to save the contents of that inode to another location. In the example below the file with inode 15 is MacPorts-1.6.0-10.5-Leopard.dmg, its file permission is 644, the owner has uid 501 and gid 0 etc.
cvfsdb> show inode 15 ShowInode 15 idi_marker_st = 0x4e6f [No] idi_attr_flags = 0x10 idi_user_flags = 0x0 idi_expandsize = 0x0 idi_flags = 0xa00 idi_mode = 0100644 idi_nchildren = 0x1 idi_nsubdirs = 0x0 idi_uid = 0x1f5 idi_gid = 0x0 idi_gen = 0x0 idi_dm_events = 0x0 idi_atime.tv_sec = 0x47eb9ea7 idi_atime.tv_nsec = 0x2ef69540 idi_mtime.tv_sec = 0x47de558a idi_mtime.tv_nsec = 0x0 idi_ctime.tv_sec = 0x47eb9ea8 idi_ctime.tv_nsec = 0x9896418 idi_crttime = 0x47de558a idi_affinity = 0x0 idi_size = 0x67329 idi_blocks = 0x1a idi_nextiel = 0xd00 (0x340000 / 3407872) idi_seqno = 0xa idi_xattr_blk = 0x0 Extended attribute area = 000: 0100 8000 3f04 0327 5250 4c00 0000 0000 |....?..'RPL..... 010: 0000 024d 6163 506f 7274 732d 312e 362e |...MacPorts-1.6. 020: 302d 3130 2e35 2d4c 656f 7061 7264 2e64 |0-10.5-Leopard.d 030: 6d67 0404 084e 5453 4400 0000 0000 0000 |mg...NTSD....... 040: 0000 0000 0000 0000 0000 0000 0000 0000 |................ 050: 0000 0000 0000 0000 0000 0000 0000 0000 |................ 060: 0000 0000 0000 0000 0000 0000 0000 0000 |................ 070: 0000 0000 0000 0000 0000 0000 0000 0000 |................ Extents = [ 0 ] flags = 0x1 [ 0 ] sg = 0x1 [ 0 ] depth = 0x0 [ 0 ] frblock = 0x0 ( 0 - 25 = 26 ) [ 0 ] base *= 0x6 [ 0 ] end *= 0x1f idi_marker_en = 0x4465 [De]
We can save this file to another location this way:
cvfsdb> save 15 /var/root/MacPorts-1.6.0-10.5-Leopard-copy.dmg Saving data from inode 0xf to file "/var/root/MacPorts-1.6.0-10.5-Leopard-copy.dmg" Saved 26 of 26 blocks (100%) cvfsdb>
This feature of saving the file to another location comes very handy in disaster recovery scenarios. A short and highly simplified example is given below.
The scenario is where a low level hardware error is developing in one or more of the disks somewhere in the underlying RAID LUNs, yet the cvfsck command in Xsan reports that the file system is fine.
For example, the customer cannot see a particular folder, say, Projects, under the Xsan volume through the Finder. When the customer does an ls in the terminal, he can see an entry Projects, yet when he types ls –al Projects, he gets an Input/Output error. Yet a file system check using cvfsck shows no problems.
The following assumes that the administrator is operating from the metadata controller. The name of the Xsan volume is Testvol.
The first step is to unmount the volume from the clients and then stop the volume. Then, executing the following command (as root or using sudo if not root) should generate and save a list of all files and folders Xsan can “see:”
cvfsck –x Testvol > allfilesfolders
The file allfilesfolders generated above will have comma separated entries with the following information
Inode#, Mode, Size, Block Count, Affinity, Path, Extent Count, Extent Number, Storage pool, File Relative Block, Base, End, Depth, Breadth
3c4dcd,16877,2048,1,Data,Projects/sunshine,1,0,0,0,3858,3858,0,256 3c4de8,33188,0,0,none,Projects/moonshine,0,0,0,0,0,0,0,0 3ee3ddb,16893,2048,1,none,Projects3/>,1,0,0,0,49948,49948,0,4
Note: Despite what the man page for cvfsck says, the Inode# is reported in hex and not in decimal. The rest of the numerical values are in decimal.
Of particular interest is the second value, ie., mode. For example, if the mode value is 16877, then in octal it is 040775, so the entry corresponds to a directory with 755 permissions, if it is 33206, then in octal it is 100666, so a file with permissions 666, if it is 33188, then in octal it is 100644, so a file with permissions 644. If the mode is 41471, then it is 0120777 in octal, so a symbolic link.
The only interest is in the "missing" Projects folder, so we can extract all such entries to a file called alllostProjects.
Based on the mode value we can then create directory paths in a restore area. We can then use the inode values of the files in conjunction with cvfsdb to restore the files along the appropriate directory paths.
In the case of the example entries above, we can simply do:
mkdir –p /Volumes/restore/Projects/sunshine echo “save 0x3c4de8 /Volumes/restore/Projects/moonshine” | cvfsdb Testvol
Based on our experience in the field, a somewhat detailed exposition of this example, taken from a real world scenario, along with a sample script will soon be posted in our web site http://www.consol.com/apple under "Tips & Tricks." Please note that Apple may not officially support solutions like this, so you are at your own risk if you decide to deploy something like this in a live environment. Please make sure you absolutely test everything in a lab environment!