The Rock & Roll Hall of Fame in Cleveland has a mission to engage, teach and inspire through the power of music, but that mission faced a challenge because its video content was trapped on decaying tape media and obsolescent infrastructure, according to Tape Ark COO Kyle Evans.
The Rock Hall came to Tape Ark to overcome that challenge with its tape-to-cloud migration service, he said July 2 during the data breakout session “We Will Rock You | Preserving the Richest History of Rock and Roll Footage for Future Generations” at the Global Media & Entertainment Day event presented live, virtually, from London.
With partner Seagate, Tape Ark was able to safely restore more than 2,800 hours (350 TB) of preservation-level video files for the Rock Hall’s Library & Archives and ingested the content into the public cloud.
It was a “really critical transformation project, where we restored some previously unreachable, really unique preservation-level files that were stuck on backup tape,” Evans told viewers. The files included “born-digital oral histories, concerts, ceremonies, and they were absolutely unique – they couldn’t be captured again,” he recalled.
“We’re a fairly young company, but we’re addressing a really contemporary issue,” he pointed out, adding: “It’s an issue that plagues nearly every industry and, in particular, media and entertainment. For the last five decades, magnetic backup tape has been the … option for really low cost-reliable archive. And there’s over a billion of these in off-site storage facilities today alone.”
But now, “with the falling cost of cloud storage and the advent of machine learning” and artificial intelligence (AI), “our customers are really wanting to connect with their legacy content and derive a lot more value from that content,” he said.
The infrastructure that was used to create these tapes “really isn’t applicable for using for a mass migration project, he told viewers. “Our customers are in [a] position of saying, ‘Well, I created these tapes 15 years ago. Do I now use that 15-year-old technology to try and push it all to the cloud?’ And that just doesn’t work. So we’ve taken a really different approach. We built a technology stack that helps our customers really transform their data and then quickly and efficiently get it out of tape and put it to the cloud with zero data loss.”
Evans chose the Rock Hall project to discuss because it was an “interesting case study to really showcase what we do, and everyone can relate to music — I think it’s a really good uniter of people,” said Evans, who also mentioned his own rock and roll story from 1999. He was a Beastie Boys fan and he saw them when they toured in Perth and Sydney that year, he recalled, adding he won a competition and got to meet them, which was an experience he said he would never forget.
They were inducted into the Rock Hall in 2012, which indicates the wide breadth of genres supported by the Hall, which he noted is a not-for-profit organization.
The museum’s groundbreaking ceremony was in 1993 and it was clear that the Hall was going to be substantial when attendees at that event include Billy Joel and Pete Townshend of The Who, he noted.
The facility opened two years later in Cleveland. There was a concert nearby to celebrate the opening and it ran for many hours with about 60 acts including Bruce Springsteen and Chuck Berry. “It really solidified the Rock & Roll Hall of Fame as the go-to place for preserving rock’s history,” Evans said.
However, “as the organization grew, they needed to expand their capabilities,” he pointed out, noting its Library & Archives were built “just down the road from the museum” and, in that Library & Archives, “they preserve transcript, they preserve different forms of music and documents.”
Like many organizations, “when they’re preserving born-digital content, they put that onto magnetic tape and, in this case, they were using LTO tape written out through an IBM TSM library,” he said, adding: “In a very prudent measure, these were written in triplicate and held at separate sites, each copy. And the information that was held on there [included] some really unique content: Aretha Franklin telling her life history. We had unique interviews with Run-DMC. So it was really important this content was safe.”
Over time, however, “that tape backup system failed …. the hardware failed and then the database failed,” he pointed out. The Rock Hall was, as a result, “in this position where the data was unreachable and they had to take some steps to bring that back, and when they thought of that project, there was the prospect of putting in a new tape library,” he noted. But that was ruled out because “they really saw that would just repeat history” and they would potentially not be able to access the content yet again down the road, he said.
“What they did is take a different approach,” he said, explaining: “They reflected on their mission and their mission is to engage, teach and inspire through the power of rock and roll. And that first part – the engagement piece – is really important because if they were to keep that content on tape, they wouldn’t really be able to get out to their stakeholders. And so that was a critical decision point, where they said: ‘No. We’re going to take this to the cloud.’ And so, in going to the cloud, they partnered with AWS and, in that conversation, AWS reached out to us because we’re an AWS partner.”
One challenge that Tape Ark faced on the project was the IBM TSM tape used by the Rock Hall, he said, noting it is “really difficult to work with that format outside of its native environment. But we said, ‘Send us your tapes.’” he recalled, adding: “It was part of their risk profile that they actually were fine to send tapes because they were using an off-site storage facility already and moving tapes around between their facility and the off-site storage facility was part of their day-to-day process.”
The tapes were sent to Seagate’s Oklahoma facility and that is “where the strength of our partnership with Seagate Technology comes in,” he noted. Out of Oklahoma City and Europe, “there are two fantastic centers that are designed to migrate masses of tape to the cloud,” he pointed out, noting that the partnership combines Seagate’s expertise in manufacturing hard discs and data management with Tape Ark’s tape migration expertise.
It is an Internet of Things-based system, “where we can essentially scale beyond all of those legacy systems to migrate the biggest data collections in the world,” he said, noting “some customers have millions of tapes [that run] into hundreds of petabytes, so “we need to have a process that’s really scalable and reliable.”
The Rock Hall warned Tape Ark it could not afford to lose a single bit of data and “that really stuck with us,” he recalled.
He went on to describe the migration process, explaining the tapes were first received at its facility, where they were photographed and given unique QR codes. They restored the content to discs and then collected the raw data and metadata out of the proprietary formatting. “We use some of the AWS tools to securely push that across to cloud storage,” and then use Amazon S3 storage, Evans noted, adding that, throughout the process, integrity checks are conducted by Tape Ark.
Once the content is in the cloud and the “preservation-level files are sitting in S3, that’s where the pretty funky stuff happens,” he said, adding: “With their cloud-based media asset management system, an access copy was made and then those preservation-level files were pushed down into AWS’s dedicated archive tier of storage called Glacier Deep Archive,” which is “ideally designed as an archive tier for low access and it’s reflected in the cost, so it comes out around $1,000 per petabyte per month.”
Another partner, GrayMeta then uses AI and machine learning to extract info including celebrity and logo data so that it is all searchable, he told viewers.
In the end, the project aligned with the Rock Hall’s mission and, now, “instead of the data being trapped on tape, it’s available and accessible in the cloud,” he said. “The planning process was really critical,” and so was the speed in which it was done and the Hall’s desire to innovate, he noted.
As an additional bonus, Tape Ark found 109 preservation-level videos that the Rock Hall didn’t even know about, he concluded.
The fourth annual M&E Day event, presented by the Media & Entertainment Services Alliance (MESA), featured mainstage panels and more than 15 breakout sessions, covering the latest it data, cloud, IT and security across the media and entertainment technology ecosystem.
The event was presented by Caringo, with sponsorship by Convergent Risks, Cyberhaven, Richey May Technology Solutions, RSG Media, Signiant, Whip Media Group, Zendesk, Seagate Powered by Tape Ark, Sony New Media Solutions, 5th Kind, ATMECS, Eluvio, Tamr, the Audio Business Continuity Alliance (ABCA), the Entertainment Identifier Registry (EIDR) and The Trusted Partner Network (TPN).