Notes on Xsan 4
As others have noted, Xsan 4's administration model is notably different from the model in all previous versions of Xsan. Here are some notes I have on the changes.
We have coined a new term, "Activation." "Upgrade" and "Migration" are two ways of taking the base operating system and configuration from a previous OS to Yosemite. "Promotion" is the act of updating Server.app to Server 4.0. We coined a corresponding term, "Activation," to describe taking an Xsan 3 or 2 configuration and moving it into Xsan 4. SAN Activation happens after Upgrade or Migration and also after Server Promotion. Activation happens first on the previous Xsan Primary MDC, and then happens on the other MDCs.
Xsan 4 no longer directly manages clients. We ran into too many issues where SAN operations would fail because one client was off-line. To address this, Xsan 4 uses ldap to store the SAN configuration. Now instead of having Xsan Admin update (push) the configuration on all machines, we have Server.app store a changed configuration in ldap and we inform all the clients that they need to re-parse (pull) the configuration.
We now run Open Directory on all the MDCs and they act as an OD cluster to replicate this information. Clients do not need to bind to these severs. If you previously had Xsan managing Users & Groups, Xsan will use that OD for its storage. If you have an external source for your Users and Groups, say AD, just use that for Users&Groups and don't worry about this OD cluster.
There are two direct consequences of this change.
First, Xsan 4 does not support Xsan 3 or earlier systems in the SAN. We do not lock them out of the fsm's, and you can perform a zero-down-time upgrade. The rub is that the older clients do not understand the messages we now send out when the configuration changes, nor do they understand the new message instructing clients to unmount a volume (as it is about to be stopped). In this respect they are akin to Linux or Windows clients in the SAN. If you want to stop a volume, these clients will not automatically unmount it. If you destroy a volume, these cients will not correctly forget it.
Second, we use Transport Level Security (TLS) in ldap when querying the configuration. As such, we need certificates to anchor the TSL trust. Certificates need DNS host names. So Xsan 4 requires DNS be configured on the Metadata Network. We expect many sites already had it, but we now will require it.
Unfortunately the error message you get today is unclear about this issue. So be forewarned.
A thrid issue is that all of the MDCs need to be in the same OD cloud. If you had a SAN before where Xsan was NOT managing users & groups but you had OD running on some of the MDCs, you need to ensure the Xsan 3 primary controller is in that OD cluster as the primary (ODM) before activating the SAN. Otherwise Xsan will see that OD is not running on the former primary, create it, and then you won't be able to activate MDCs which are in a different OD cloud; we require all SAN MDCs have the same OD master.