Notes on Xsan4

wrstuden's picture

Notes on Xsan 4
As others have noted, Xsan 4's administration model is notably different from the model in all previous versions of Xsan. Here are some notes I have on the changes.

We have coined a new term, "Activation." "Upgrade" and "Migration" are two ways of taking the base operating system and configuration from a previous OS to Yosemite. "Promotion" is the act of updating Server.app to Server 4.0. We coined a corresponding term, "Activation," to describe taking an Xsan 3 or 2 configuration and moving it into Xsan 4. SAN Activation happens after Upgrade or Migration and also after Server Promotion. Activation happens first on the previous Xsan Primary MDC, and then happens on the other MDCs.

Xsan 4 no longer directly manages clients. We ran into too many issues where SAN operations would fail because one client was off-line. To address this, Xsan 4 uses ldap to store the SAN configuration. Now instead of having Xsan Admin update (push) the configuration on all machines, we have Server.app store a changed configuration in ldap and we inform all the clients that they need to re-parse (pull) the configuration.

We now run Open Directory on all the MDCs and they act as an OD cluster to replicate this information. Clients do not need to bind to these severs. If you previously had Xsan managing Users & Groups, Xsan will use that OD for its storage. If you have an external source for your Users and Groups, say AD, just use that for Users&Groups and don't worry about this OD cluster.

There are two direct consequences of this change.

First, Xsan 4 does not support Xsan 3 or earlier systems in the SAN. We do not lock them out of the fsm's, and you can perform a zero-down-time upgrade. The rub is that the older clients do not understand the messages we now send out when the configuration changes, nor do they understand the new message instructing clients to unmount a volume (as it is about to be stopped). In this respect they are akin to Linux or Windows clients in the SAN. If you want to stop a volume, these clients will not automatically unmount it. If you destroy a volume, these cients will not correctly forget it.

Second, we use Transport Level Security (TLS) in ldap when querying the configuration. As such, we need certificates to anchor the TSL trust. Certificates need DNS host names. So Xsan 4 requires DNS be configured on the Metadata Network. We expect many sites already had it, but we now will require it.

Unfortunately the error message you get today is unclear about this issue. So be forewarned.

A thrid issue is that all of the MDCs need to be in the same OD cloud. If you had a SAN before where Xsan was NOT managing users & groups but you had OD running on some of the MDCs, you need to ensure the Xsan 3 primary controller is in that OD cluster as the primary (ODM) before activating the SAN. Otherwise Xsan will see that OD is not running on the former primary, create it, and then you won't be able to activate MDCs which are in a different OD cloud; we require all SAN MDCs have the same OD master.

Comments

23
abstractrude's picture

Thanks wrstuden

-Trevor Carlson
THUMBWAR

aaron's picture

Yes, thanks for keeping the flame alive, Bill!

Aaron Freimark
CEO, GroundControl

R. Gates's picture

We are running Xsan 3.1 on a couple of Mac minis as dual MDC's, and have new Mac Pro computer that has Yosemite on it, and we want to add it to the San but can't seem to add it. (We continually get the old "unreachable or offline" warning. We have no plans to upgrade our MDC's to Yosemite for now, or is the only way to add a Yosemite client?

abstractrude's picture

There are a lot of notes on this site about your issue. Metadata controllers should/must be newer than clients. Xsan4 makes that more strict. You can "hack" this by manually configuring clients and pointing to the FSM. 10.9 is your best bet until your ready to move your whole environment to Yosemite.

http://support.apple.com/kb/HT200135?viewlocale=en_US&locale=en_US

-Trevor Carlson
THUMBWAR

thomasb's picture

Has anybody figured out if there a way to specify what volumes to mount in the configuration profiles created by Xsan 4 in the Server app?

We have more than 16 Xsan volumes in one MultiSAN configuration. It is highly undesirable to mount every volume on every client every time we push out configuration profiles. Different volumes are used by different departments and client machines, hence we would really like to be able to push out configuration profiles without mounting any volumes. The best would be an option to specify volumes to mount, and that this could be specified to "none".

Note, we have not upgraded to Xsan 4 yet, as we have over 120 client systems that all need to be upgraded at the same time :-/

Testing all the different production software and workflows required for broadcast production is a huge undertaking on this many systems. Not having the ability to move slowly towards Xsan 4, like we have been able to do with previous Xsan versions, all the way back to 1.4.2, is a big disappointment to be honest.

I have a feeling many larger Xsan customers will consider moving over to Quantum StorNext MDCs, rather than upgrading to Xsan 4, just to be able to move slowly towards OS X 10.10 and Xsan 4 on the client side.

Thomas

abstractrude's picture

Thomas, funny you bring up Stornext because you have to manually manage the mount process as Stornext controllers won't remotely change mounts for OS X clients. That said, you will have to manually manage mounts using the automount.plist preferences on the client in Xsan4 or Stornext.

Since you're running one of the biggest SANS I have ever heard of, I'm sure you can have someone on your team cook up a quick mounting app or script to solve this problem. When a machine is installed choose the package that chooses your mount options and writes the preference to the client. You could actually also manage all of this using remote desktop.

Also, remember although not supported you can run previous versions of Xsan pointing to new Xsan controllers. Check out wrsturden notes. The important thing to understand, controllers no longer manage the clients in xsan4.

This was said to be done to prevent bad clients from ruining SAN configs. This is closer to the stornext model than the xsan model. Funny how we take for granted Xsan ease of use.

p.s. I didn't answer your original question, but I believe the answer is Xsan4 controllers no longer manage clients dynamically.

-Trevor Carlson
THUMBWAR

thomasb's picture

Thanks Trevor. I am fully aware that changing the automount.plist is easily scriptable after pushing the config file. It would still be very useful in large Xsan environments to be able decide wether you want the configuration profile to modify the automount.plist or not.

It's true that controllers no longer manage clients like they do in Xsan 3.1 and earlier, but clients pull the configuration profile from the OD LDAP every time you change the SAN config, e.g. add or remove an Xsan volume, so in a sense, clients are still "partially managed".

Xsan 3.1 and earlier also mount every new volume you create on all clients that are online, which is really annoying in large environments. This really should be an option in Xsan 4, is what I'm trying to say.

Over to something else though.

Is there any other important changes to Xsan 4, other than OD/LDAP/configuration profile support, that makes it a "requirement" to run Xsan 4 on both controllers and clients?

The client revisions, are the same, but it's a different build. Why not make it possible to install Xsan 4 on OS X 10.9? It would make the transition so much easier!

OS X 10.9 - cvversions
Client Revision 4.3.2 Build 508.4[30118] Branch Head
Built for Darwin 13.0 x86_64
Created on Thu May 29 22:01:43 PDT 2014
Built in /SourceCache/XsanFS/XsanFS-508.4/buildinfo

OS X 10.10 - cvversions
Client Revision 4.3.2 Build 546[30118] Branch Head
Built for Darwin 14.0 x86_64
Created on Tue Sep 9 14:57:57 PDT 2014
Built in /SourceCache/XsanFS/XsanFS-546/buildinfo

Thomas

morphenine's picture

So if I'm understanding this correctly, the Yosemite MDCs are acting a little more like Stornext MDCs, in that they dont directly manage the client, but rather tell the client that it should go grab an updated config.

Assuming that you push the volume config and fsnameservers and such manually (stornext style), is there any reason that the 10.9 clients shouldn't be able to mount an Xsan 4 volume? It sounds like the 10.9 client would just act as though it were connecting to a Stornext SAN. He states that they Don't lock the older clients out of the fsm, so it really should work, shouldn't it?

kekorreo's picture

Hi there

Our scenario is Stornext MDC 4.3.1 with XSan clients on 10.6, 10.7 and 10.8 OSX. There are any incompatability between Yosemite clients and Stornext MDC??

abstractrude's picture

This would be unsupported but earlier clients are not locked out of the FSM. So things should work no problem. Of course the software on the client has to support the volume featureset.

Yosemite is running 4.3.2 though, let me ask someone for a better answer.

-Trevor Carlson
THUMBWAR

kekorreo's picture

Thanks for the quick response.

I'm very interested to know when will be available to use Yosemite Clients under our Stornext SAN Metadata enviroinment.

When you say that Yosemite is running on 4.3.2 are you meaning over a metadata controlers in that version?? So in fact I only need to upgrade my Stornext system to that version?? And finally this will works with the preview Xsan Clients (2, 2.2, 3,...)

Thanks a lot.

abstractrude's picture

For moving a volume configuration around? Have you thought about setting the mounts via the fibre channel system preference pane, then copying it around to the clients...

-Trevor Carlson
THUMBWAR

cutmoney's picture

Quick question on Xsan 4. Under Xsan 3 it was my understanding that it was the best practice (or necessary) to set static IPs for the client machines under Xsan Admin. I have my current machines all on static IPs for the metadata network and the primary network and everything is running successfully under Xsan 4. However, I'm looking to add four more devices to my SAN and am wondering, with the new profile method, whether static IPs are necessary on the primary network. I have no problem setting statics on the metadata network, but the IT department at this location is always reluctant to give out additional static IPs so if it's not necessary I would prefer to just let the new clients use DHCP on the primary network. Thanks for the help, there's just so little documentation on Xsan 4 I'm a little more in the dark than I was with previous versions.

abstractrude's picture

With the new Xsan 4 admin model, you don't need static addresses on the public network.

-Trevor Carlson
THUMBWAR

cutmoney's picture

abstractrude wrote:

With the new Xsan 4 admin model, you don't need static addresses on the public network.

/quote

Thank you for the information! That's what I assumed since it pulls rather than the server pushing the configuration, but I just wanted to make sure.

SquarePixel's picture

We too have a large multisan with 13 volumes hosted on 5 mdc pairs. With the new Yosemite version, I don't see any way of creating a multisan setup. It appears that all volumes will be hosted by all mdc. Has anyone found a way to specify which volumes are hosted on a given mdc pair?

Thanks,
Glenn

abstractrude's picture

I don't know the answer to this question. Have you tested an upgrade of multisan to see what happens?

-Trevor Carlson
THUMBWAR

wrstuden's picture

An upgrade of a MultiSAN SAN should continue working after upgrading to Xsan 4. Note: upgrading will currently require DNS on the metadata network.

10.10.2 servers and clients will not need DNS on the metadata network after the MDCs have been "Activated."

SquarePixel's picture

Thanks for the reply. Unfortunately, our test san has only one volume and 2 mdc units. Was hoping someone had run across this. Will have to dig further. Thanks again.
Glenn

thomasb's picture

I have the exact same problem. Our test environment at work only has one volume and two MDCs.

I would think manually setting up the fsmlist on each MDC in a MultiSAN would work, but I have not been able to test. We have multiple MultiSAN environments, and our largest one consists of 7 MDCs, 16 volumes and about 120 clients.

There are some interesting discussions about StorNext and Xsan 4 over at the StorNext Forum:

StorNext 5.1 and MacOS X Yosemite (10.10)
http://stornextforum.com/forum/topics/stornext-5-1-and-macos-x-yosemite-10-10

StorNext V4.4 & OS X 10.10
http://stornextforum.com/forum/topics/stornext-v4-4-os-x-10-10

Apple say the following now at their Xsan/StorNext matrix page:
http://support.apple.com/en-is/HT201256

"Xsan 4 in OS X Yosemite is not currently qualified for use with StorNext clients or metadata controllers. Please contact Quantum support for additional information."

mnoga's picture

If I upgrade my Apple MDC's to Yosemite with Xsan 4, will my RedHat and Solaris clients running snfs 4.7.2 work without any issues?

csanchez's picture
csanchez's picture

Apple's Xsan-StorNext compatibility matrix has been updated:

https://support.apple.com/en-us/HT201256