Xsanity - Sanity for Apple's Xsan and Final Cut Server.

Sanity for Apple's Xsan and Final Cut Server.

Home . Advanced Search . Contribute

Friday, May 24 2013 @ 08:16 PM EDT

Topics

Home

Final Cut Server (42)

Storage (39)

People (1)

Xsan (103)

How To (26)

Tips & Tricks (33)

Site Info (39)

Forum

Features

RSS

User Functions

Don't have an account yet? Sign up as a New User

Who's Online

Guest Users: 9

Sponsorship

Xsanity is proudly sponsored by:

The Old Reliable Mac Shop

MDC failover crashes clients with CVFS ASSERTION FAILED

Xsanity Forums Forum Index -> Troubleshooting

View previous topic :: View next topic

Author

Message

xsanguy
Xsan Master

Joined: 24 Sep 2012
Posts: 79

Posted: Thu Jan 24, 2013 5:32 pm Post subject: MDC failover crashes clients with CVFS ASSERTION FAILED

Simply failing over a volume from one MDC to the next results in approx. 1/2 to 1/4th of the clients crashing hard with CVFS ASSERTION FAILED panics. Crashes seem to (but not always) involve clients actively transferring a lot of data at the time.

matx
Xsan Master

Joined: 15 Apr 2005
Posts: 378

Posted: Tue Jan 29, 2013 4:55 pm Post subject:

What's the setup? Describe the software/hardware involved

wrstuden
Xsan Master

Joined: 04 Jan 2008
Posts: 99

Posted: Tue Jan 29, 2013 8:58 pm Post subject:

What assertion? There are many, and they each have a unique log message.

xsanguy
Xsan Master

Joined: 24 Sep 2012
Posts: 79

Posted: Wed Jan 30, 2013 4:19 am Post subject:

Here's the complete crash report

Code:

Interval Since Last Panic Report: 1967519 sec
Panics Since Last Report: 1
Anonymous UUID: 3248A8EA-D63B-47B2-B654-0E5879376CD7

Mon Oct 8 18:26:34 2012
panic(cpu 16 caller 0xffffff7f81eeef75): "CVFS ASSERTION FAILED: f_rwlck->rw_state & RW_WRITE line 1879 file lock.c"@nomad/md_debuglog.c:358
Backtrace (CPU 16), Frame : Return Address
0xffffff838eefbc40 : 0xffffff8000220792
0xffffff838eefbcc0 : 0xffffff7f81eeef75
0xffffff838eefbd10 : 0xffffff7f81ef87d0
0xffffff838eefbd60 : 0xffffff7f81ee0795
0xffffff838eefbda0 : 0xffffff7f81ee44b4
0xffffff838eefbdc0 : 0xffffff7f81f0be5e
0xffffff838eefbdf0 : 0xffffff7f81f20cc7
0xffffff838eefbe40 : 0xffffff7f81f20b73
0xffffff838eefbe70 : 0xffffff7f81f0ea2d
0xffffff838eefbee0 : 0xffffff7f81f0f8e7
0xffffff838eefbf40 : 0xffffff7f81ef53f3
0xffffff838eefbfb0 : 0xffffff8000820057
Kernel Extensions in backtrace:
com.apple.filesystems.acfs(457.7)[68FF12CD-B6D6-3D9A-9810-D99AEF0FFB63]@0xffffff7f81e6b000->0xffffff7f81fb3fff
dependency: com.apple.iokit.IOStorageFamily(1.7.1)[5DF14080-C47C-3393-9176-6A2CD3CFE1CB]@0xffffff7f80785000

BSD process name corresponding to current thread: kernel_task

Mac OS version:
11E53

Kernel version:
Darwin Kernel Version 11.4.0: Mon Apr 9 19:32:15 PDT 2012; root:xnu-1699.26.8~1/RELEASE_X86_64
Kernel UUID: A8ED611D-FB0F-3729-8392-E7A32C5E7D74
System model name: MacPro5,1 (Mac-F221BEC8)

System uptime in nanoseconds: 547895591426323
last loaded kext at 50342042120: com.apple.filesystems.smbfs 1.7.2 (addr 0xffffff7f8081b000, size 241664)
last unloaded kext at 172841006954: com.apple.driver.AppleIntel8254XEthernet 2.1.3b1 (addr 0xffffff7f80c76000, size 110592)
loaded kexts:
com.ATTO.driver.ATTOExpressSASRAID2 2.1.1
com.ATTO.driver.ATTOCelerityFC8 2.0.1
com.apple.filesystems.smbfs 1.7.2
com.apple.driver.AppleHWSensor 1.9.5d0
com.apple.filesystems.autofs 3.0
com.apple.driver.AudioAUUC 1.59
com.apple.driver.AppleHDA 2.2.0f3
com.apple.driver.AppleTyMCEDriver 1.0.2d2
com.apple.driver.AGPM 100.12.42
com.apple.driver.AppleMikeyHIDDriver 122
com.apple.driver.AppleMikeyDriver 2.2.0f3
com.apple.driver.AppleUpstreamUserClient 3.5.9
com.apple.driver.AppleMCCSControl 1.0.26
com.apple.kext.ATIFramebuffer 7.1.8
com.apple.iokit.IOUserEthernet 1.0.0d1
com.apple.iokit.IOBluetoothSerialManager 4.0.5f11
com.apple.Dont_Steal_Mac_OS_X 7.0.0
com.apple.driver.AudioIPCDriver 1.2.2
com.apple.driver.ACPI_SMC_PlatformPlugin 5.0.0d0
com.apple.ATIRadeonX3000 7.1.8
com.apple.driver.AppleLPC 1.5.8
com.apple.driver.XsanFilter 404
com.apple.driver.BroadcomUSBBluetoothHCIController 4.0.5f11
com.apple.iokit.SCSITaskUserClient 3.2.0
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0d1
com.apple.BootCache 33
com.apple.driver.AppleFWOHCI 4.8.9
com.apple.driver.AppleRAIDCard 209
com.apple.iokit.IOAHCISerialATAPI 2.0.3
com.apple.driver.AppleUSBHub 4.5.0
com.apple.driver.AirPort.Brcm4331 530.4.20
com.apple.driver.Intel82574L 2.2.0b2
com.apple.driver.AppleAHCIPort 2.3.0
com.apple.driver.AppleUSBEHCI 4.5.8
com.apple.driver.AppleUSBUHCI 4.4.5
com.apple.driver.AppleEFINVRAM 1.5.0
com.apple.driver.AppleRTC 1.5
com.apple.driver.AppleHPET 1.6
com.apple.driver.AppleACPIButtons 1.5
com.apple.driver.AppleSMBIOS 1.8
com.apple.driver.AppleACPIEC 1.5
com.apple.driver.AppleAPIC 1.5
com.apple.driver.AppleIntelCPUPowerManagementClient 193.0.0
com.apple.nke.applicationfirewall 3.2.30
com.apple.security.quarantine 1.3
com.apple.driver.AppleIntelCPUPowerManagement 193.0.0
com.apple.filesystems.acfsctl 457.7
com.apple.filesystems.acfs 457.7
com.apple.kext.triggers 1.0
com.apple.driver.DspFuncLib 2.2.0f3
com.apple.driver.AppleSMBusController 1.0.10d0
com.apple.iokit.IOSurface 80.0.2
com.apple.iokit.IOSerialFamily 10.0.5
com.apple.driver.AppleHDAController 2.2.0f3
com.apple.iokit.IOHDAFamily 2.2.0f3
com.apple.iokit.IOAudioFamily 1.8.6fc17
com.apple.kext.OSvKernDSPLib 1.3
com.apple.driver.AppleSMBusPCI 1.0.10d0
com.apple.driver.ApplePolicyControl 3.0.16
com.apple.driver.AppleGraphicsControl 3.0.16
com.apple.driver.AppleSMC 3.1.3d8
com.apple.driver.IOPlatformPluginLegacy 5.0.0d0
com.apple.iokit.IONDRVSupport 2.3.2
com.apple.driver.IOPlatformPluginFamily 5.1.0d17
com.apple.kext.ATI5000Controller 7.1.8
com.apple.kext.ATISupport 7.1.8
com.apple.iokit.IOGraphicsFamily 2.3.2
com.apple.iokit.IOFireWireIP 2.2.4
com.apple.driver.AppleUSBBluetoothHCIController 4.0.5f11
com.apple.iokit.IOBluetoothFamily 4.0.5f11
com.apple.iokit.IOSCSIMultimediaCommandsDevice 3.2.0
com.apple.iokit.IOBDStorageFamily 1.6
com.apple.iokit.IODVDStorageFamily 1.7
com.apple.iokit.IOCDStorageFamily 1.7
com.apple.iokit.IOFireWireFamily 4.4.5
com.apple.iokit.IOSCSIBlockCommandsDevice 3.2.0
com.apple.iokit.IOUSBUserClient 4.5.8
com.apple.iokit.IO80211Family 420.3
com.apple.iokit.IONetworkingFamily 2.1
com.apple.iokit.IOAHCIFamily 2.0.8
com.apple.iokit.IOSCSIParallelFamily 2.5.1
com.apple.iokit.IOSCSIArchitectureModelFamily 3.2.0
com.apple.iokit.IOUSBFamily 4.5.8
com.apple.driver.AppleEFIRuntime 1.5.0
com.apple.iokit.IOHIDFamily 1.7.1
com.apple.iokit.IOSMBusFamily 1.1
com.apple.security.sandbox 177.5
com.apple.kext.AppleMatch 1.0.0d1
com.apple.security.TMSafetyNet 7
com.apple.driver.DiskImages 331.6
com.apple.iokit.IOStorageFamily 1.7.1
com.apple.driver.AppleKeyStore 28.18
com.apple.driver.AppleACPIPlatform 1.5
com.apple.iokit.IOPCIFamily 2.6.8
com.apple.iokit.IOACPIFamily 1.4
Model: MacPro5,1, BootROM MP51.007F.B03, 12 processors, 6-Core Intel Xeon, 2.66 GHz, 32 GB, SMC 1.39f11
Graphics: ATI Radeon HD 5770, ATI Radeon HD 5770, PCIe, 1024 MB
Memory Module: DIMM 1, 4 GB, DDR3 ECC, 1333 MHz, 0x80CE, 0x4D33393142353237334348302D4348392020
Memory Module: DIMM 2, 4 GB, DDR3 ECC, 1333 MHz, 0x80CE, 0x4D33393142353237334348302D4348392020
Memory Module: DIMM 3, 4 GB, DDR3 ECC, 1333 MHz, 0x80CE, 0x4D33393142353237334348302D4348392020
Memory Module: DIMM 4, 4 GB, DDR3 ECC, 1333 MHz, 0x80CE, 0x4D33393142353237334348302D4348392020
Memory Module: DIMM 5, 4 GB, DDR3 ECC, 1333 MHz, 0x80CE, 0x4D33393142353237334348302D4348392020
Memory Module: DIMM 6, 4 GB, DDR3 ECC, 1333 MHz, 0x80CE, 0x4D33393142353237334348302D4348392020
Memory Module: DIMM 7, 4 GB, DDR3 ECC, 1333 MHz, 0x80CE, 0x4D33393142353237334348302D4348392020
Memory Module: DIMM 8, 4 GB, DDR3 ECC, 1333 MHz, 0x80CE, 0x4D33393142353237334348302D4348392020
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x8E), Broadcom BCM43xx 1.0 (5.106.198.4.20)
Bluetooth: Version 4.0.5f11, 2 service, 18 devices, 1 incoming serial ports
Network Service: Internet, Ethernet, en0
Network Service: Meta, Ethernet, en1
PCI Card: ATTO ExpressSAS R680, sppci_raid, Slot-3
PCI Card: Apple RAID Card, sppci_raid, Slot-4
PCI Card: ATI Radeon HD 5770, sppci_displaycontroller, Slot-1
PCI Card: ATTO Celerity FC-82EN, sppci_fibrechannel, Slot-2
PCI Card: ATTO Celerity FC-82EN, sppci_fibrechannel, Slot-2
Serial ATA Device: HL-DT-ST DVD-RW GH41N
Fibre Channel Device: SCSI Target Device @ 4
Fibre Channel Device: SCSI Target Device @ 7
Fibre Channel Device: SCSI Target Device @ 8
Fibre Channel Device: SCSI Target Device @ 13
Fibre Channel Device: SCSI Target Device @ 17
Fibre Channel Device: SCSI Target Device @ 18
Fibre Channel Device: SCSI Target Device @ 19
Fibre Channel Device: SCSI Target Device @ 20
Fibre Channel Device: SCSI Target Device @ 21
Fibre Channel Device: SCSI Target Device @ 22
Fibre Channel Device: SCSI Target Device @ 23
Fibre Channel Device: SCSI Target Device @ 24
Fibre Channel Device: SCSI Target Device @ 25
Fibre Channel Device: SCSI Target Device @ 26
Fibre Channel Device: SCSI Target Device @ 28
Fibre Channel Device: SCSI Target Device @ 29
Fibre Channel Device: SCSI Target Device @ 33
Fibre Channel Device: SCSI Target Device @ 34
Fibre Channel Device: SCSI Target Device @ 35
Fibre Channel Device: SCSI Target Device @ 36
Fibre Channel Device: SCSI Target Device @ 37
Fibre Channel Device: SCSI Target Device @ 38
Fibre Channel Device: SCSI Target Device @ 39
Fibre Channel Device: SCSI Target Device @ 40
Fibre Channel Device: SCSI Target Device @ 41
Fibre Channel Device: SCSI Target Device @ 42
Fibre Channel Device: SCSI Target Device @ 43
Fibre Channel Device: SCSI Target Device @ 44
Fibre Channel Device: SCSI Target Device @ 45
Fibre Channel Device: SCSI Target Device @ 46
Fibre Channel Device: SCSI Target Device @ 47
Fibre Channel Device: SCSI Target Device @ 48
Fibre Channel Device: SCSI Target Device @ 49
Fibre Channel Device: SCSI Target Device @ 50
Fibre Channel Device: SCSI Target Device @ 51
Fibre Channel Device: SCSI Target Device @ 52
Fibre Channel Device: SCSI Target Device @ 53
Fibre Channel Device: SCSI Target Device @ 54
Fibre Channel Device: SCSI Target Device @ 55
Fibre Channel Device: SCSI Target Device @ 56
Fibre Channel Device: SCSI Target Device @ 57
Fibre Channel Device: SCSI Target Device @ 58
Fibre Channel Device: SCSI Target Device @ 59
Fibre Channel Device: SCSI Target Device @ 60
Fibre Channel Device: SCSI Target Device @ 61
Fibre Channel Device: SCSI Target Device @ 62
Fibre Channel Device: SCSI Target Device @ 63
Fibre Channel Device: SCSI Target Device @ 64
Fibre Channel Device: SCSI Target Device @ 65
Fibre Channel Device: SCSI Target Device @ 66
Fibre Channel Device: SCSI Target Device @ 67
Fibre Channel Device: SCSI Target Device @ 68
Fibre Channel Device: SCSI Target Device @ 69
Fibre Channel Device: SCSI Target Device @ 70
Fibre Channel Device: SCSI Target Device @ 71
Fibre Channel Device: SCSI Target Device @ 4
Fibre Channel Device: SCSI Target Device @ 7
Fibre Channel Device: SCSI Target Device @ 8
Fibre Channel Device: SCSI Target Device @ 13
Fibre Channel Device: SCSI Target Device @ 17
Fibre Channel Device: SCSI Target Device @ 18
Fibre Channel Device: SCSI Target Device @ 19
Fibre Channel Device: SCSI Target Device @ 20
Fibre Channel Device: SCSI Target Device @ 21
Fibre Channel Device: SCSI Target Device @ 22
Fibre Channel Device: SCSI Target Device @ 23
Fibre Channel Device: SCSI Target Device @ 24
Fibre Channel Device: SCSI Target Device @ 25
Fibre Channel Device: SCSI Target Device @ 26
Fibre Channel Device: SCSI Target Device @ 27
Fibre Channel Device: SCSI Target Device @ 29
Fibre Channel Device: SCSI Target Device @ 30
Fibre Channel Device: SCSI Target Device @ 34
Fibre Channel Device: SCSI Target Device @ 35
Fibre Channel Device: SCSI Target Device @ 36
Fibre Channel Device: SCSI Target Device @ 37
Fibre Channel Device: SCSI Target Device @ 38
Fibre Channel Device: SCSI Target Device @ 39
Fibre Channel Device: SCSI Target Device @ 40
Fibre Channel Device: SCSI Target Device @ 41
Fibre Channel Device: SCSI Target Device @ 42
Fibre Channel Device: SCSI Target Device @ 43
Fibre Channel Device: SCSI Target Device @ 44
Fibre Channel Device: SCSI Target Device @ 45
Fibre Channel Device: SCSI Target Device @ 46
Fibre Channel Device: SCSI Target Device @ 47
Fibre Channel Device: SCSI Target Device @ 48
Fibre Channel Device: SCSI Target Device @ 49
Fibre Channel Device: SCSI Target Device @ 50
Fibre Channel Device: SCSI Target Device @ 51
Fibre Channel Device: SCSI Target Device @ 52
Fibre Channel Device: SCSI Target Device @ 53
Fibre Channel Device: SCSI Target Device @ 54
Fibre Channel Device: SCSI Target Device @ 55
Fibre Channel Device: SCSI Target Device @ 56
Fibre Channel Device: SCSI Target Device @ 57
Fibre Channel Device: SCSI Target Device @ 58
Fibre Channel Device: SCSI Target Device @ 59
Fibre Channel Device: SCSI Target Device @ 60
Fibre Channel Device: SCSI Target Device @ 61
Fibre Channel Device: SCSI Target Device @ 62
Fibre Channel Device: SCSI Target Device @ 63
Fibre Channel Device: SCSI Target Device @ 64
Fibre Channel Device: SCSI Target Device @ 65
Fibre Channel Device: SCSI Target Device @ 66
Fibre Channel Device: SCSI Target Device @ 67
Fibre Channel Device: SCSI Target Device @ 68
Fibre Channel Device: SCSI Target Device @ 69
Fibre Channel Device: SCSI Target Device @ 70
Fibre Channel Device: SCSI Target Device @ 71
USB Device: BRCM2046 Hub, 0x0a5c (Broadcom Corp.), 0x4500, 0x5a100000 / 2
USB Device: Bluetooth USB Host Controller, apple_vendor_id, 0x8215, 0x5a110000 / 3
FireWire Device: built-in_hub, 800mbit_speed

[/quote]

xsanguy
Xsan Master

Joined: 24 Sep 2012
Posts: 79

Posted: Fri Feb 08, 2013 6:50 am Post subject:

Any ideas? Half the clients crash whenever a volume moves form MDC1 to MDC2... exceptionally frustrating, and the only time I've ever seen this.

xsanguy
Xsan Master

Joined: 24 Sep 2012
Posts: 79

Posted: Wed Feb 13, 2013 9:37 pm Post subject:

upgraded MDCs from 10.7.3 on Mac Minis (overloaded) to 09 xserves and 10.8.2. No change.

wrstuden
Xsan Master

Joined: 04 Jan 2008
Posts: 99

Posted: Wed Feb 27, 2013 12:47 pm Post subject:

Unfortunately no shipping version of Xsan has a fix for this issue. Do you have AppleCare? Note: if the mac in question was purchased recently-enough, you may have it from that purchase. Alternatively if you were a developer, you could probably ask about it on the Developer Forums.

Display posts from previous:

	Xsanity Forums Forum Index -> Troubleshooting	All times are GMT - 5 Hours
Page 1 of 1

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum

Best Viewed on a Mac | Suggested Browser: Whatever floats yer boat.