M+E Connections

By Adrian Herrera, Vice President, Marketing, Caringo –

When it comes to consuming content, convenience is now king and your storage strategy must adapt. From dwindling cable subscriptions and declining movie attendance to the shift from print advertising to video advertising, the trend is clear: You need to create and deliver digital content faster, in more formats, and to various screens — and then you need to keep it online and accessible for rapid reuse and long-tail enablement. The foundation for it all is storage. But these days, some types of “storage” do much more than just provide a data repository. Organizations that understand this are already using storage as a competitive advantage.

Here we’ll briefly describe the commonalities in challenges we have heard from those in the M&E space during the production, post-production and delivery process in this new “cord cutting” world; and then describe how different storage technologies solve them.

The changes in consumption patterns discussed in the opening paragraph are leading to specific issues consistent across post houses, studios, and broadcasters. All are struggling with:

Economics: Storing what is needed in the face of increasing file sizes

Performance: Satisfying different performance requirements during various workflows

Accessibility: Digital assets that need to remain accessible (even though they may never be accessed)

Search: Enabling search and instant delivery

Cloud-enabled: Asset access from new cloud-based services

Security: Control of access

The main types of storage used by M&E organizations are Tape, Network Attached Storage (NAS), Storage Area Network (SAN), Cloud Storage and Object Storage. All satisfy different performance and functional needs. All also use one of three different methods to store data on media with each one bringing certain benefits.

The three ways data is stored are block, file and object.

*Block-based storage is like a raw hard drive; you install software on top of it and data is stored as segments. Most of your transactional data (i.e., databases) utilize block storage.

*File-based storage, a file system manages writing data to disk, is what you are probably most familiar with. In file-based storage, a file is split into many segments and each is associated with an inode. To get the file back, those segments must be stitched back together. The file system also manages file locking.

*Object-based storage is what enables all major cloud storage services. Each piece of content (or content segments) is given a unique ID.

It’s important to look at the underlying method of how the data is actually stored because each method results in different functionality through the storage device. The diagrams shown are an overly simplified representation, but you can get a sense for what the enabling software must do to access and deliver content.

In object storage, there is (usually) a one-to-one relationship in content ID to the asset.

Present the system with the ID, get the asset. As you can see, it isn’t that straight forward in file- and block-based storage since you need to go through the operating system. However, both file- and block-based storage are optimized for performance.

Many of the devices discussed utilize gateways and/or translation to support different use cases on different architectures. For instance, there are many NAS solutions that have object interfaces and many object solutions that have file system interfaces. This is where the rubber hits the road for applicability of one device versus another, especially when there are incumbent performance characteristics trying to be met or exceeded. In the following table, the typical storage devices and methods are evaluated using the criteria extracted from the change in viewer patterns.

Historically, M&E workflows were serviced by SAN, NAS and tape. However, the velocity of growth in both file size and file amount (driven by increasing resolutions and variation in formats) and new access requirements have exposed the limits in these technologies. SAN and NAS are expensive and don’t scale in a practical manner for today’s use cases, and tape lacks instant accessibility.

The ideal mix of economy and accessibility is where on-premise object storage and cloud shine, but strict adherence to legacy file system functionality (like POSIX compliance) while still delivering high performance remains an issue for many existing workflows.

As a result, for the foreseeable future, all the aforementioned storage technologies will continue to play a role in M&E workflows. The good news? Reliance on expensive/proprietary storage appliances is being reduced and a move to software-defined storage on commodity hardware is in full swing.

Significant advancements in file-to-object translation and the integration of the S3 API by many content creation/distribution and asset management platforms are allowing all organizations to benefit from both legacy/ traditional and new storage architectures.

The importance of metadata (for search, rapid reuse and painless royalty calculations) can’t be ignored. For most current M&E workflows, metadata is locked in a proprietary database managed by an asset manager for various reasons.

As asset libraries exponentially grow, the only method for practical scale of metadata is to leverage a NoSQL or document-oriented approach.

For many rapidly scaling M&E use cases, storage and compute won’t be the issue. It will be database scale, protection and access that will reduce your ability to compete. These are issues that can be solved at the storage layer with coordination between your asset managers and those object or cloud storage vendors that utilize a NoSQL-like key/value pair approach.

What sets organizations apart that use storage as a competitive advantage from others is their ability to move away from infrastructure management and to focus on usability and creating content. These forward-thinking organizations also understand that the right storage can solve a lot more than just storage issues.

New storage technologies can significantly reduce backups and databases, directly distribute and deliver content, and be used to make assets and their associations instantly searchable and accessible. Is your storage technology providing you with a competitive advantage, or is it inhibiting the growth of your business?

—-

Click here to translate this article
Click here to download the complete .PDF version of this article
Click here to download the entire Winter 2018 M&E Journal