Connections

By Matt Ashton, Technical Marketing Engineer, Qumulo –

Storage infrastructure is an integral ingredient in the success of today’s media and entertainment organizations. File-based data including 4K video, broadcasting, animation rendering and visual effects (VFX) are the core assets of these companies, and the ability to effectively store them is vital to production. If something causes the data storage to slow or fail, the digital factory grinds to a screeching halt and the result can be crushing.

To keep up in the competitive M&E landscape, customers need incredibly scalable file storage with the ability to store tens of billions of files and deliver maximum performance with real-time visibility and control at the user’s fingertips.

Current challenges

New storage challenges facing media and entertainment companies include animation rendering, managing live action virtual reality applications and keeping up with capacity demands related to 2D compositing and VFX.

Large-scale animation rendering requires the use of massive numbers of compute cores, or what are commonly known as animation render farms. These machines complete thousands of compute jobs that generate millions to billions of files that require high performance storage systems at petabyte scale. The universal- scale solutions of today are ideal for these workloads.

Similarly, immersive live action virtual reality applications generate tremendous amounts of digital files, requiring storage that is highly scalable and delivers high performance to ingest and process data quickly.

We will look at how one of the world’s largest film animation studios and a boutique VFX studio employed universal-scale file storage architectures to accelerate their digital content pipelines, tackle massive projects, increase efficiency and meet business objectives.

Accelerating the digital content pipeline

One of the world’s largest film animation studios was struggling to keep pace with the vast amount of small file data generated from animation rendering workflows. The studio’s film process requires an exceptional amount of digital manufacturing, making file-based data one of the core assets of its business. For any given film, the studio may generate more than half a billion files. Having the ability to support that demand was vital to keeping its digital content pipeline humming. Any slowing or failure of the storage architecture would negatively impact everything.

The animation studio faced significant challenges with its existing storage system, including insufficient scalability and write performance for large numbers of small files, lack of data visibility and limited APIs for custom integrations with important media workflows. Realizing that its current legacy storage system was beginning to hold it back, the studio upgraded to a flash-first hybrid storage architecture that delivers billion-file scale with high performance to meet the demands of the digital content factory.

The new solution includes a modern code base and architecture, write scalability and integrated file systems analytics that provide real-time visibility and control at scale for unparalleled business value.

2D compositing for a major Hollywood film

Universal-scale file storage solutions aren’t only remedying the storage hurdles facing large studios. A boutique visual effects studio that’s earned a reputation in the 2D compositing world for its work on countless hit films and shows found itself struggling with the opening scene of a major hit movie.

At more than five minutes long and over eight thousand frames, the film’s opening sequence dwarfs the industry average of two to five seconds and a hundred frames between cuts. The scene included the rendering of CG cars, added dancers, transitions between multiple takes, wardrobe adjustments, removal of set equipment and much more; a process that required hundreds of renders and terabytes of data. Generally speaking, 2D rendering doesn’t tax compute power, and working with a limited number of frames is usually straightforward. But scale that single shot by an order of magnitude, and suddenly the wheels can come off.

With its existing storage architecture, moving from one frame to the next took 20 seconds, and the studio had over eight thousand frames to go through. Speed is directly related to the quality of its work; if artists are waiting for shots to play back, then output drops and it can put an entire project in jeopardy. Pressures to keep production on schedule were mounting, and it was time to make a change.

The studio transitioned to a universal-scale file storage solution capable of storing and managing hundreds of terabytes of file data. This massive capacity and scalability is critical, as the studio’s volume of data continues to grow rapidly every year. With a new storage architecture in place, the studio could deliver the opening VFX sequence easily and with no delays to production. Having a universal-scale file storage system that is reliable and scales affordably has the studio poised for continued success in the future.

Universal-scale file storage

As media and entertainment organizations continue to create exponentially more data and take on larger projects, the ability to efficiently store, manage and access their file data at scale will be critical for success. Universal-scale file storage infrastructure ensures that these companies gain real-time visibility into their data footprint, meet business objectives and stay competitive.

—-

Click here to translate this article
Click here to download the complete .PDF version of this article
Click here to download the entire Winter 2018 M&E Journal