NEED HELP Xsan volume is not mounted (strange problem)

AnMAX's picture

Hi Xsan guru :)
I ask the help in solving the volume mount problem. (advance I am sorry for my english)
All began with the fact that the MDC is rebooted and on the two LUNs were gone XSAN label. I relabeled these LUNs using commands:
[code]
cvlabel -c >label_list/code
then in the file label_list I corrected unknown disks on the label with a same names that were. Then I ran the [i]cvlabel label_list/i command. Finally I got the correct label on all drives.

[img]https://dl.dropboxusercontent.com/u/5920358/LUNS.png/img/img

[code]

  1. cvlabel -l

/dev/rdisk14 [Raidix meta_i 3365] acfs-EFI "META_I"Sectors: 3906830002. Sector Size: 512. Maximum sectors: 3906975711.
/dev/rdisk15 [Raidix QSAN_I 3365] acfs-EFI "QSAN_I"Sectors: 7662714619. Sector Size: 4096. Maximum sectors: 7662714619.
/dev/rdisk16 [Raidix meta_ii 3365] acfs-EFI "META_II"Sectors: 3906830002. Sector Size: 512. Maximum sectors: 3906975711.
/dev/rdisk17 [Raidix 2k_I 3365] acfs-EFI "2K_I"Sectors: 31255934943. Sector Size: 512. Maximum sectors: 31255934943.
/dev/rdisk18 [Raidix 2k_II 3365] acfs-EFI "2K_II"Sectors: 31255934943. Sector Size: 512. Maximum sectors: 31255934943.
/dev/rdisk19 [Raidix QSAN_II 3365] acfs-EFI "QSAN_II"Sectors: 7662714619. Sector Size: 4096. Maximum sectors: 7662714619.
/code

The volume [2K] starts successfully.

[img]https://dl.dropboxusercontent.com/u/5920358/VOLUME.png/img

but not mounted on the MDC and client.
I ran a volume check
[code]
sh-3.2# cvfsck -wv 2K
Checked Build disabled - default.

BUILD INFO:

  1. !@$ Revision 4.2.2 Build 7443 (480.8) Branch Head
  2. !@$ Built for Darwin 12.0
  3. !@$ Created on Mon Jul 29 17:01:44 PDT 2013

Created directory /tmp/cvfsck3929a for temporary files.

Attempting to acquire arbitration block... successful.

Creating MetadataAndJournal allocation check file.
Creating Video allocation check file.
Creating Data allocation check file.

Recovering Journal Log.

Super Block information.
FS Created On : Wed Oct 2 23:59:20 2013
Inode Version : '2.7' - 4.0 big inodes + NamedStreams (0x207)
File System Status : Clean
Allocated Inodes : 4022272
Free Inodes : 16815
FL Blocks : 79
Next Inode Chunk : 0x51a67
Metadump Seqno : 0
Restore Journal Seqno : 0
Windows Security Indx Inode : 0x5
Windows Security Data Inode : 0x6
Quota Database Inode : 0x7
ID Database Inode : 0xa
Client Write Opens Inode : 0x8

Stripe Group MetadataAndJournal ( 0) 0x746ebf0 blocks.
Stripe Group Video ( 1) 0x746ffb60 blocks.
Stripe Group Data ( 2) 0xe45dfb60 blocks.

Inode block size is 1024

Building Inode Index Database 4022272 (100%).
4022272 inodes found out of 4022272 expected.

Verifying NT Security Descriptors
Found 13 NT Security Descriptors: all are good

Verifying Free List Extents.

Scanning inodes 4022272 (100%).

Sorting extent list for MetadataAndJournal pass 1/1
Updating bitmap for MetadataAndJournal extents 21815 ( 0%).
Sorting extent list for Video pass 1/1
Updating bitmap for Video extents 3724510 ( 91%).
Sorting extent list for Data pass 1/1
Updating bitmap for Data extents 4057329 (100%).

Checking for dead inodes 4022272 (100%).

Checking directories 11136 (100%).

Scanning for orphaned inodes 4022272 (100%).

Verifying link & subdir counts 4022272 (100%).

Checking free list. 4022272 (100%).
Checking pending free list.

Checking Arbitration Control Block.

Checking MetadataAndJournal allocation bit maps (100%).
Checking Video allocation bit maps (100%).
Checking Data allocation bit maps (100%).

File system '2K'. Blocks-5784860352 free-3674376793 Inodes-4022272 free-16815.

File System Check completed successfully.
/code
check not helping :(
[code]sh-3.2# cvadmin
Xsan Administrator

Enter command(s)
For command help, enter "help" or "?".

List FSS

File System Services (* indicates service is in control of FS):
1>*2K[0] located on big.local:64844 (pid 5217)

Select FSM "2K"

Created : Wed Oct 2 23:59:20 2013
Active Connections: 0
Fs Block Size : 16K
Msg Buffer Size : 4K
Disk Devices : 5
Stripe Groups : 3
Fs Blocks : 5784860352 (86.20 TB)
Fs Blocks Free : 3665561306 (54.62 TB) (63%)

Xsanadmin (2K) > show
Show stripe groups (File System "2K")

Stripe Group 0 [MetadataAndJournal] Status:Up,MetaData,Journal,Exclusive
Total Blocks:122088432 (1.82 TB) Reserved:0 (0.00 B) Free:121753961 (1.81 TB) (99%)
MultiPath Method:Rotate
Primary Stripe [MetadataAndJournal] Read:Enabled Write:Enabled

Stripe Group 1 [Video] Status:Up
Total Blocks:1953495904 (29.11 TB) Reserved:270720 (4.13 GB) Free:129179 (1.97 GB) (0%)
MultiPath Method:Rotate
Primary Stripe [Video] Read:Enabled Write:Enabled

Stripe Group 2 [Data] Status:Up
Total Blocks:3831364448 (57.09 TB) Reserved:270720 (4.13 GB) Free:3665432127 (54.62 TB) (95%)
MultiPath Method:Rotate
Primary Stripe [Data] Read:Enabled Write:Enabled

/code
I checked the availability of LUNs on MDC and client, there also all right.
[img]https://dl.dropboxusercontent.com/u/5920358/disk_util.png/img

But, unfortunately, the Volume is not mounted :(

[code]sh-3.2# xsanctl mount k2
mount command failed: Unable to mount volume `k2' (error code: 3)
/code

[b]Please help me to figure out this situation, I will be grateful for any information./b
Thanks

AnMAX's picture

I made ​​a mistake in the mount command but it does not change the situation
[code]
xsanctl mount 2K
mount command failed: Unable to mount volume `2K' (error code: 5)
/code
just attach logs:

[img]https://dl.dropboxusercontent.com/u/5920358/2K_LOG.png/img

nrausch's picture

very basic, and I hope not over obvious....

go into the /Volumes directory on one of the machines not mounting the volume.

If there is a folder in there with the name of your volume, trash it.
The volume may mount right away after that. If not, try mounting with Xsan admin.

If that works, try on the other machines.