Data Archive for XSAN

Bryson · Could work for Apple Joined: 31 May 2005 Posts: 43

I agree... LTO has been trouble free for us generally as well. The restore times aren't bad and even with a single drive (which I don't recommend) you can get by if you're not creating more than a few GB a day.

(a 400GB tape can last a few days/weeks in a small shop so you swap less than you might think)

Autoloaders are under $10k now (I've seen them under $5k).

I agree with the "green team" on here. Most admins aren't paying the light bill on a shop when they say, "let's add spinning drives for backup!"

I am looking into a plan to put the "most active" couple of TB on some spinning drives for "just in case" situations.

Please someone, give us tape virtualization for small shops! I want to drop a tape in and have it searchable and browse-able as a volume. (no, I don't want a Quantum "A" series lol) I want the ability to replace video tape with LTO in a paradigm that my video guys can understand for small backups and archiving video to tape.

I bring it up to companies and they all act like tape is dead. Video will revive it, I believe, as video tape decks die. The guys on this board alone are creating petabytes a year of data that can't spin forever... at least I don't wanna pay the power bills if it does.

Ernesto Sanchez · Xsan Master Joined: 31 Jul 2006 Posts: 74

Bryson, I am looking for the same kind of solution. On that does not force my clients to learn a backup application. I feel this is the main reason creative shops do not use their tape libraries. It seems that Final Cut Server along with a HSM solution such as Quantum's Storage Management Solution fits the bill. It allows for data to be moved to tape libraries without end-user interaction. The client only interacts with FCS to access assets. I am in the process of setting up this solution in our lab and will report what I find.

Cheers,
Ernesto Sanchez
Senior SAN Engineer - The DR Group
O - 323.896.3416
C - 949.690.3680
ernesto@thedrgroup.com

Bryson · Could work for Apple Joined: 31 May 2005 Posts: 43

Thanks man, I'll look a little deeper into my developer pool too. FCS might help us if they get it rollin.

I'm gonna show your post to some of the software guys who claimed "tape is dead" lol.

peace

mark raudonis · Xsan Master Joined: 23 Sep 2005 Posts: 123

It's important to note that "back up" means different things to different people. There's a huge difference between "archive" and just "back up".

If I had to summarize all of the solutions that I've explored to date, it's that they're all too complex and too expensive for our needs. Sure, some people need absolute mission critical back up and intstantaneous restore, but in my world (video post production), that isn't critical. Often times the original master tapes are available as a "source of last resort" in the event of a total drive meltdown. So, I already have a "back up"... it's just too slow to restore in a crunch. So, we're really treading the fine line between convenience and necessity.

Also, as the recent fire at Universal Studios illustrated, you can't just assume that the vault (or your entire building) is secure. So, I'm exploring the concept of "fire insurance" back up. Ideally, this is a spindle based solution that physically lives in another building. It doesn't have to be fast, have raid redundancy, or even be elegant. It just has to be cheap and BIG! (75 terabytes) I only need it to "survive" for the duration of a project's post production schedule ( 6 months), and then it gets written over.

This kind of manual system isn't gonna excite any of the code monkeys on this board, but it will get the job done. Until Google opens up it's "G Drive" concept to this scale of data warehousing, I'm stuck in the "sneakernet" world.

Mark

Ernesto Sanchez · Xsan Master Joined: 31 Jul 2006 Posts: 74

Mark,

I found the following solution when I was researching offsite backup for disaster recovery. StorageDNA, http://www.storagedna.com , provides asynchrous backup. It is designed for post-production environments. I have not implemented it but I like the concept. I talked to them at NAB and they sound like the understand the needs of video post-production.

ogminlo · Xsan Master Joined: 29 May 2008 Posts: 149

I know I've been an LTO guy on this thread, but an alternative worth examining is StorNext 3.X's new storage disk de-duplication capability. They don't play it up much on their product page, but one of the new features Quantum added to Storage Manager for version 3 was a software de-duplication engine. I haven't had my SAN analyzed yet to see if my video data is a good candidate for de-dupe, but if it is and the ratio is right, I would absolutely do a mezzanine disk tier before my Scalar. I figure the de-dupe disk would be nearline and the Scalar could be smaller than it otherwise would be and could be used for vaulting the oldest data that still has value. For some of you guys, this might be the trick that makes it practical to have nothing but disk.

I plan to have Quantum analyze my SAN to see what my de-dupe ratio is. I'll post results afterward.

ogminlo · Xsan Master Joined: 29 May 2008 Posts: 149

fritz. · Xsan Master Joined: 03 Oct 2007 Posts: 177

Funny, I was discussing the possibility of FCSvr and Storage Manager yesterday. I haven't had anyone bite at it yet but it's comforting to know that it can be done!

MattG · Xsan Master Joined: 15 Apr 2005 Posts: 456

It's a lovely combo:

Two issues to consider:

1) SNSM is !(&*ing expensive.

2) The Restore Feature in FCSvr deletes the file from the Archive device upon successful transfer of the file back to its original location. Hopefully, in a future version, Final Cut Server will be able to account for an archived representation regardless of whether the Primary Representation is there or not.

ogminlo · Xsan Master Joined: 29 May 2008 Posts: 149

vidiot · Knows DNS is the answer Joined: 16 May 2006 Posts: 38

Chalk me up for SNSM and FCSvr too. We've had the 2 running together well for about a month now. It's fantastic that I don't have to go hunting and loading of files and tapes. The users just click and restore as needed. Brilliant!

To MattG's #2 point, this is really a problem. Once you've restored an asset, FCS deletes the file from the Archive Device, in this case SNSM. Which means your asset is now once again unprotected. For us, that is usually a P2 MXF camera original file.

Sure it still lives on tape in a manner of speaking, unless you clean up all the tapes.

I've asked Apple for a quote from their Professional Services team to write me a different Archive process that does *not* remove the file from the Asset device.

Ernesto Sanchez · Xsan Master Joined: 31 Jul 2006 Posts: 74

Well gents, I'm glad to see there are real deployments of FCSvr and SNMS. I'm now in the shoes of the client. I need to implement an archive solution for in the next 6-12 months. I really want to see a demonstration of the this solution. Anyone know anyone at Quantum who could set this up in the LA area?

Cheers,
Ernesto
Toybox Entertainment

om_nick · JBOD Joined: 20 Apr 2009 Posts: 1

There are also plugins that allow seamless archiving to a disk based archive.

Benefits:

- Assets AND metadata protected.
- Archived assets remain protected upon restore.
- Entire Productions and associated metadata can be archived with a single click.
- Scripts provided to backup FCSvr DB and proxies.

To the subject of disk based archiving:

- 2TB disks now available using same or slightly less electricity than 1TB
- New disks spin down when not in use. Green and getting greener.
- Your data is available 24/7, when you need it.
- some clustered systems automate the most error prone and labourious tasks, are self-healing, self managing. < One admin per Pb+. Whats the manual resource count for the same amount of tape?
- Metadata kept with the assets not on a separate DB on a separate Tape.

The reality is that some customers are asking for disk only, some are asking to disk nearline/tape mix. Enabling customer choice is good which is why we have created a similar integration with CatDV.

Contact me to chat or find out more.

(I have edited this post to respect the non-PR bull policy Xsantiy is known for, thanks Nick)

bjr2 · partially protected Joined: 25 Sep 2009 Posts: 8

Hi All! Me again!

So, our FCS system is running smoothly. I finally got approval to tier our SAN system. So we're ordering a G-Tech XL 16tb FC (quad) for editing, demoting our current XServe Raid SAN for broadcast and hosting, and I'm currently mulling over a quote for a Quantum LTO4 Autoloader for tape backup.

So, I've read in this thread that some are using the Quantum Autoloaders for their backup. My only question is if there's any possibility of designating the Quantum as an archive device in FCS so that the rest of the editors don't have to worry about running some script and can just archive assets within the FCS client.