Recent Comments

  • Reply to: Types of Servers   1 day 8 hours ago

    Quantum has their own NAS appliance: http://www.quantum.com/serviceandsupport/softwareanddocumentationdownloa...

    Any medium grade Windows server with a couple NICs would probably fulfill your requirement.  We handle more users and larger files on an old Xserve with a 3x 1Gbit bond.

     

  • Reply to: XSAN Maverick FSMPM die randomly on 2 MDC.   3 days 16 hours ago

    Pete,

    Our network is all enterprise Cisco based. I'm not managing the network, so I can't give you too much details there. However, we didn't have failover issues this frequently (every 1-2 weeks) before upgrading to OS X 10.9, so I can't see how this could be an external network issue.

    Considering we see the same issues in all our five separate Xsan environments, and the fact that everyone I have talked to who are running Xsan on OS X 10.9 have these issues, I can't see how this can be anything else than a bug in OS X.

    Our MDCs consists of Xserves, Mac Pros and Mac minis. The hardware for the MDC doesn't matter, they all have failover issues.

    The Mac mini MDCs are running in a lab environment with a simple HP ProCurve switch for the Metadata Network. We have failovers happen there too, just not as frequently, which I guess is because of the limited activity in our lab environment.

     

  • Reply to: XSAN Maverick FSMPM die randomly on 2 MDC.   3 days 17 hours ago

    Thomas,

    You're right about the secondary MDC.  I've been rebooting them whenever one acts up, and that seems to stabilize it for a bit.

    You mentioned that you thought it was a network issue in OS X.  Just out of curiosity, what ethernet switches are you using?  We recently had some OS X network issues that traced back to our Cisco gear, which apparently doesn't play nice like all the other switches on the block.  This was just with a PEG6 card that we were trying to bond, though--completely unrelated to Xsan.  I can't say that I've had any other issues with our Ciscos, but I'm not willing to rule them out completely.  I haven't been able to find any telltail log messages about networking either.

    Another possibility might be our ethernet configuration on the MDCs.  What hardware are you using?  Xserves?  Minis?  Pros?  I've got two Mac Minis, each with a Promise SANLink adapter and a Thunderbolt to GigE adapter.  The GigE adapters daisychain through the Promise adapter back to the single Thunderbolt port on the Minis.  We're currently using the GigE adapters for our metadata network, but I'm considering flipping them around.  If anyone is having this issue with non-Mini MDCs, then I'd scratch that idea.

    Pete

  • Reply to: XSAN Maverick FSMPM die randomly on 2 MDC.   4 days 9 hours ago

    Sorry to say this, but I really doubt that this issue is isolated to your primary MDC. If you haven't seen it happen to your secondary MDC yet, you will soon enough unfortunately.

    We'll hopefully see a fix from Apple for these issues soon.

  • Reply to: XSAN Maverick FSMPM die randomly on 2 MDC.   4 days 9 hours ago

    Sorry to say this, but I really doubt that this issue is isolated to your primary MDC. If you haven't see it happen to your secondary MDC yet, you will soon enough unfortunately.

    We'll hopefully see a fix from Apple for these issues soon.

  • Reply to: Fast cvfsck question   4 days 9 hours ago

    What do you mean with "SAN definitions"?

    Failovers do not necessarily damage the file system, but running a check doesn't hurt.

    All Xsan environments running OS X 10.9 see issues with failovers, and it's a bug in OS X, that hopefully will be fixed soon. The best thing you can do in the meantime is to make sure you reboot the MDC that was in control of the volumes before the failover, to make sure it's ready for the next failover.

    Last time I ran cvfsck on a 70TB Xsan volume where the majority of files were large video files, a read-only check took about 5-10 minutes. The file system needed repair, and the repair took about 10-15 minutes. The time required depends a lot on the amount of files on the volume.

    What is the total capacity of your Xsan volumes? How much space is available?

    Have you read Apple's articles about cvfsck? Read both carefully.

    Xsan: How to repair the filesystem
    http://support.apple.com/kb/ht1081

    Xsan: File System Status reported by cvfsck does not indicate need for repair
    http://support.apple.com/kb/TA24607

    Info from "man cvfsck"

    -j Execute journal recovery and then exit. Running journal recovery will ensure all operations have been  committed  to disk, and that the metadata state is up to date. It is recommended that cvfsck is run with the -j flag before any read-only checks or volume reports are run.

    -n This  option allows a volume to be checked in a read-only mode. The modifications that would have happened are described but are not actually performed. A read-only volume check may display errors if there  are  journaled volume  transactions  which have not yet been committed. It is recommended that cvfsck is run with the -j flag before a read-only check is run.

    -v Use verbose reporting methods.

    NOTE: On  large  file  systems  cvfsck may requires 100s of megabytes or more of local system disk space for working files.

     

  • Reply to: RIP Active Storage - CLOSED   5 days 8 hours ago

    I wouldn't put my money in the NEW Active Storage. They are putting out a real bad vibe and I can't trust it.

  • Reply to: NAB 2014   1 week 1 day ago

    I got jumped by the Object Matrix "1 Petabyte" dudes while standing at the Archiware booth.

  • Reply to: NAB 2014   1 week 4 days ago

    How is everyone finding the show? I agree with Trevor on last year and it's echoing into this year as well but nevertheless some interesting bits coming out.
     

  • Reply to: XSAN Maverick FSMPM die randomly on 2 MDC.   1 week 4 days ago

    I'm also seeing something similar to this on 10.9.2, though I'm pretty sure our volumes have been running fine on 10.9.2 for several weeks.  This issue cropped up on the primary MDC yesterday.  If PortMapper reported a hiccup from FSM on the primary MDC, the volume would fail over to the secondary MDC.  Both my volumes run fine on the secondary MDC, so I've kept them like that until I can sort out what's wrong with the primary.  I watched the primary and saw a few more of these hiccups throughout the day yesterday, so on a hunch, I rebooted it.  I haven't seen any more yet, and if it holds steady for another day or so, I'll fail one of the volumes back to it and see how it goes.

  • Reply to: Some LUNs invisible to some clients but not others?   2 weeks 5 days ago

    Hi,

    Just wanted to update this thread, and tell you that we ended up disabling the zoning again because of issues with replicating zoning changes accross all 19 switches.

    Also, the faulty switch that caused the LUN visibility issues had to be repaired because of an uncommon problem with the DDR Memory (not a user problem).

    We have also found that there is a CLI shutdown command, that I would highly recommend using instead of just cutting the power, when you need to turn off a switch. Even Qlogic support say they rarely use the shutdown command, but after having three 5800 switches fail in less then a year, we have become quite cautious about it.

  • Reply to: NAB 2014   3 weeks 3 days ago

    If you guys work our a meetup, post a photo!

  • Reply to: Xsan 2 on a Xserve2,1 (10.6.8) and Marvericks   3 weeks 3 days ago

    You can technically use different OS versions, but you might run into an awful lot of problems if you do (see my threads for examples.  . .) You'll need to upgrade your metadata controller to Mavericks with Xsan 3.1, and the older clients will need Xsan 2.2.2 to function correctly with a Maverick's controller.

    My personal experience with 3 Mavericks Metadata controllers and 11 clients (2 Mavericks clients and 9 Snow Leopard) over 3 months is that this configuration will work most of the time, but you'll have swarms and swarms of Console error messages, a higher rate of failovers and general connectivity hiccups and problems. I could have other factors that are making my life hell at the moment, but that is my experience with introducing Mavericks into an Xsan envrionment thus far.

  • Reply to: Users not able to log in after recreating ldap in Mavericks   3 weeks 3 days ago

    I am about to do the same thing, upgrade a clients Xsan to 3.1 and use OD from the new Mavericks Server, although reading this maybe I should create a OD on the final cut server server (10.6.8) and have this as the primary OD, the FCSvr then resolves to this OD (bound) and have replication to the new server running Mavericks as a fallback perhaps...

  • Reply to: NAB 2014   3 weeks 4 days ago

    I will return this year as well.

    Sorry to hear you won't be there Aaron, we had a great time last year.