The Cost of Perpetuity. Can You Afford Forever?
I’m at the National Association of Broadcasters (NAB) conference in Las Vegas this week – and I’ve been coming here annually for over 20 years. It’s me and 100,000 of my closest friends who have anything to do with media, entertainment, or technology related to those two things.
The belle of the ball this year is “cloud.” It’s cloud . . . and then all the things you do to make media. Cloud broadcast, distribution, rendering, special effects, analytics, storage, etc., etc. If you’ve done it before on the ground somewhere, you can now do it in the cloud.
As a lifelong technologist, the notion of cloud is a little silly to me, just as a concept. It basically means computer things happening where you’re not. But today if we access them a certain way, with a certain set of protocols, we call it “cloud.” Fair enough. The problem is that the term itself can be deceiving.
Because clouds are big and airy, and mostly just pass over your head. And they don’t bother you – unless they dump out rain or snow. But even then, we don’t blame the clouds. We curse the rain – not the clouds.
The deception is that cloud computing does in fact take place on the ground – always. It’s nothing but data centers, computers, disk drives, power couplings, cooling units, in boxy buildings stuffed with metal and flashing lights. Don’t get me wrong, I LOVE data centers. But I’m also a pilot. I fly through actual clouds. And a data center is nothing like any actual cloud. Data centers are solid, and real, and decidedly terrestrial.
Which brings me to media, and saving things, and perpetuity, and why cloud should be a pretty difficult sell when the term “forever” is in the conversation.
In the professional media world, anything you make is called an “asset” – and you don’t want to throw away assets that have value – even if that value might be long in the future.
Granted, some assets have more value than others. But when, for example, you shoot a movie, you have what winds up in the movie, and lots of what is termed B-roll — extra stuff that doesn’t make the final product. And even if you perform triage on the B-roll and get rid of lots of it, there is always a lot left.
Which brings us to forever, or as we say in the storage business, perpetuity.
Older media technologies were not built for the concept of forever.
One of the first media projects I worked on was to help CNN make digital copies of their aging analog library. Back then, many, many years ago, past shows were on old fashioned video tape, but those analog video tapes were not built to last for many years. Especially if they were not held in perfectly climate controlled conditions, literally the oxide might have been pulling off some of the tapes. Indeed, you can find analog tapes (video and audio) that have one, last play on them.
Therefore, about 25 years ago, the media industry began many digitization projects, converting analog content to digital content – for the most part digitizing analog tape and placing it into terrestrial repositories. So that was then. And as problematical as some of those earlier repositories were, requiring massive redundancy to prevent against component failure (e.g. a disk dying), the problem today has less to do with older repositories than it does to the veritable tsunami of media/data that is “born digital.” Lots of it. Every day.
Here at NAB, every cloud provider has a solution for your mass of media data – just put it in “the cloud.” Sounds easy. What they mean is, just put it in my terrestrial data centers, sending it over communication lines, and I will get it out of your face – as long as you keep paying for it every month.
Many of the cloud vendors, I’m guessing, have never been in the media business. Because media people know that there is hot media material, and cool material, and cold material. Hot – you need it immediately every day. Cool – you need it with relative frequency, but not all the time. Cold – you may not look at it for years and years. But that cold data? There is a LOT of it, and it’s the stuff nobody wants to throw out.
(See my article Your Data Is Yours, Right? )
No denying cloud is great for hot media files needed day to day, for broadcast, for a project or documentary, etc.
If it’s cold, you need it in a place where it is both safe, and costs you least. Which is not in somebody else’s data center incurring monthly charges – forever.
With Data Tape today, especially LTFS formatted LTO tape, we can reliably predict what it might cost to keep our files for decades and decades to come
Here is your checklist for pricing perpetuity in the cloud:
1. Can you be free of relying on the fact that a given company might be in business 20, 30, 40, 50 years from now? Have you heard of Compaq? Atari? Digital Equipment Corporation? They were all big deals 30 years ago. Are you happy you didn’t bet the future of your data on them? Yes you are.
2. Can you get cost guarantees that go past the next 5 or 10 years? No, you can’t. Problem: people doing things as a business are always going to reserve the right to charge you more if their costs increase. (Can you predict the cost of energy 10 years from now? No you can’t. Neither can they.) No business makes promises out that far. Read the fine print.
3. Can you get costs down considerably during the years when your data/media is doing very little, but you still want it in a secure place?
4. Can you get your data out from your cloud provider – that is, can you get back what is yours, and can you get it fast, and at no cost? Just ask. Most cloud providers let you load your data for free, and then charge you for getting it out. And determine how much you might need in a short time. Even pretty pricey connections will let you bring down about 1 -2 TB per day. Is that enough for you?
The above items make cloud a problem for long term use cases – unless you can get a really sweet deal on forever. The problem is that perpetuity means that there is no time limit – and perpetuity is a long, long time.
The solution is to look at hybrid architectures that allow you to move data from a hot cloud environment to a sensible terrestrial solution.
Cards on the table – I’m an advocate of tape repositories as the very best way to store archival data. Today, with LTFS, you have an ISO standard format with tapes that are ever more dense – allowing you to store more and more data in the same footprint. LTO7 allowed us to put over 6 TB in a single cartridge. LTO8 is in plan that will double a tape density to 12 TB uncompressed. The plan for LTO10? 120TB per tape!
Because LTFS is file based, the job of moving to a next generation of tape is almost trivial – you just pour the old data into the new container.
And tapes, though rated to last over 10 years, really can last as long as 25.
Significantly – your data in the cloud is, by and large, still burning energy all the time, it’s just that it’s not in your back yard. Overall carbon footprint is a global thing, however, not a local thing. So hands down, your data on tape has an energy advantage of over 200X against disk systems. (Actually 256X based on a recent Clipper Group report.)
I advocate Data Tape because it is both the least costly way for any organization to store their valuable archives – hands down, and because it is also the most energy efficient by a large factor. You or your organization can wear those carbon footprint savings proudly!
In addition, with a Fundamental Storage Unit of LTO tape, formatted in LTFS (the international standard,) with predicable costs for decades and decades, Data Tape is really the only sensible way to store things that require the label Perpetuity.