By Erik Weaver, Global Director, M&E Strategy and Market Development, Data Center Solutions, Western Digital
Hollywood has its fair share of name dropping. But have you heard the latest four-letter word in M&E: NVMe?
NVMe is the new kid on the block for accessing high-speed next-gen SSDs that store huge volumes of media content. As studios grapple with the dual challenges of larger content and shorter deadlines, everyone is looking at NVMe to understand how it addresses growing bandwidth needs and how to leverage its incredible speed.
NVMe brings many advantages compared to legacy SATA or SAS protocols, including up to 45% improvement in IOPs and throughput with up to 30% lower latency over SAS in all-flash arrays, letting studios do more with data, faster.
There are three major challenges facing IT in media production right now:
Data Rates are Going Up
We all know data in M&E is growing at an incredible pace and there’s more of it! 8K is starting to become a genuine fear and a potential reality. Next-generation cameras now shoot in 8K, and then may compress to 4K for editing. Sony Pictures Studios’ video production playback of multiple 4K streams at 24 frames per second requires throughput of 1.2 GB/second. Just as frame sizes are increasing, so are frame rates: from 24 to 48 to 60 FPS. The current pipeline can’t handle this level of data. Many studios can’t even play content back on an 8K monitor because they’d need a very advanced infrastructure to even create those pixels, a fast storage medium that can feed that type of data and a network to push that data through.
New Standards in Broadcast Driving Race to Retool IT Infrastructure
New standards such as ATSC 3.0 and HDR10+ in broadcast are emerging and bring lots of data, causing a race to retool IT infrastructures. In this rush to resolve how to change infrastructures to meet these new standards, studios have to think about how to deliver packages and stay relevant as they manage this onslaught of data and deliver on time content to these advanced formats. In addition, you have a business to run every day, so integrating new data infrastructure has to be carefully planned. The advantage of NVMe is the impact to infrastructure is minimal.
New Generation of Video Editing Solutions Brings LOTS of Data
Along with the increase in frame rates and new standards there is a new generation of video editing solutions that handle insane amounts of data. In just one example, Avid Media Composer 2019 was announced at NAB with 16K, 32-bit color depth. Post-production editors are faced with larger, more complex files and shorter, tighter deadlines. It’s simply not feasible to wait 12- 24 hours to edit content captured because it physically can’t get to the editing room fast enough.
It’s the same with VFX where hundreds of artists are working on special effects. Using traditional protocols may only allow throughput of around 600 MB/second depending on hardware and software. Using NVMe all-flash arrays to accelerate data access at the speed of memory enables scale-out situations where 100 to 200 users all on 4K can simultaneously work with the content. One example that helps enable this is utilizing Excelero with Western Digital NVMe all-flash arrays. By harnessing Excelero’s award-winning NVMesh software-defined block storage solution along with Western Digital’s NVMe IntelliFlash hardware, the companies are enabling unprecedented storage scalability, performance, and agility.
The biggest use case for NVMe in M&E is mastering, where editing, color grading and cleansing happen. The new UHD standard means broadcast content has much more processing to do, in a shorter period of time.
All of these use cases translate into the need to store larger content, more of it, and meet increasingly faster deadlines. The only way studios can keep pace with this level of throughput is with NVMe.
NVMe, “The New Fast”
NVMe is a new data access protocol that enables lightning-fast flash drives to talk to storage systems exponentially faster than traditional protocols like SAS or SATA. Without NVMe, the benefits of flash-based storage are compromised by bottlenecks on the proverbial highway. NVMe is “The New Fast” because it delivers incredible speeds and a multi-lane approach (vs. a one lane approach like SAS or SATA) to push data faster.
Top 5 Reasons to Consider NVMe in M&E
- SAS and SATA have reached their limits
Traditional protocols such as SAS or SATA were designed for HDD needs, not for flash. It’s like driving on a single lane country road compared to the multi-lane freeway that NVMe brings. Flash is fast but using SAS or SATA does not allow it to realize its full potential when it comes to throughput. NVMe, which was designed for flash, provides direct access to flash memory with latency as low as 250 microseconds. In addition, SAS and SATA are limited to serial commands, whereas NVMe allows parallel commands, enabling fast data access at the speed of memory. NVMe accelerates applications, cutting data analysis times in half as compared to SAS.
- NVMe unlocks bottlenecks
With NVMe, production houses are able to deploy NVMe drives within shared storage, shifting the bottleneck that occurs from SAS/SATA’s single-threading I/O requests and ultimately eliminating it. Better shared storage performance means more resources are available for data services. In the case of an editing team, an NVMe solution can scale to support hundreds of special effects team members working on effects simultaneously as they deliver content on increasingly tight deadlines.
- Speed up your workflow
Only NVMe can handle the increasing frame rates, frame sizes, new standards and next generation technologies that continue to evolve. Its fast performance enables M&E workflows to keep pace.
- Improve ROI
IT organizations in M&E can drive ROI through the increased density that NVMe enables. The latest flash technologies are more efficient, allowing increased throughput of more terabytes and virtual machines in the same rack space. Moving from traditional SATA or SAS to NVMe enables this high-density, high-performance reality.
For example, Western Digital IntelliFlash N-series arrays combine the performance of NVMe, software-defined flash management and rich data services to deliver high-performance shared storage that accelerates enterprise applications. They deliver ultra-low latency with up to 3x better performance than non-NVMe all-flash arrays and as low as 200 microseconds latency. Western Digital Intelliflash N-series arrays bring substantial TCO savings with up to 10:1 data consolidation efficiency depending on workloads.
- Future-ready your assets
NVMe paves the way for NVME-over-fabrics (NVMe-oF) and future composable infrastructures. A composable, shareable high-performance storage solution provides a flexible architecture that is extensible to support emerging future M&E data center workloads.
NVMe-oF, a series of NVMe appliances that can use shared storage, can be significantly faster than traditional network-attached storage (NAS) arrays. A composable infrastructure allows NVMe-oF to dramatically improve compute and storage utilization, performance and agility in the data center. Western Digital’s OpenFlex architecture, based on NVMe-oF, and products allow for the independent scaling of compute, storage and network resources, with orchestration and management enabled through an open API to create an application environment that provides needed resources to applications in real-time to meet workload demands.
NVMe Plays a Leading Role in M&E
2019 is the year of NVMe. The diversity of formats, tools, data types and metadata usage is creating major workflow challenges in M&E. NVMe delivers consistent and predictable high-speed performance that is interoperable and allows M&E to future-proof IT infrastructure. NVMe is the storage interface protocol to help studios more efficiently managing the data deluge and speed productivity. NVMe, with its high performance and low latency, is the only way to keep pace with increasingly large frames, new standards and the latest technologies that keep Hollywood abuzz.
Erik Weaver is Global Director, Media & Entertainment Market Development at Western Digital, focused on the intersection of cloud and the media and entertainment industry – from object storage and all-flash arrays to hybrid cloud storage solutions and archive.