Hammerspaceâs Hyperscale NAS technology sits on an industry-standard server between a customerâs GPU-based front-end servers and whatever storage they use from hard drive to NVMe flash as a way to scale file services as needed, particularly for massive data stores associated with AI workloads.
Hammerspace, which develops global data file systems that work across data centers and large public clouds, Wednesday introduced its new Hyperscale NAS as a way to provide high-performance, highly-scalable file services to burgeoning AI workloads.
Hammerspaceâs Hyperscale NAS is nothing less than a new storage architecture, said Eric Bassier, senior director of solutions marketing for the Los Altos, Calif.-based company.
âIt is effectively combining the properties of an HPC file system with an enterprise NAS system for the first time ever,â Bassier told CRN. âIt is the best architecture for general AI training, GPU computing, and other forms of unstructured data process.â
[Related: Storage Vendors Evolve With The Times: 2023 Storage 100]
There is a broad-based adoption of different forms of GPU processing going on in every major industry, and one of the AI infrastructure challenges is around assembling and sourcing data, Bassier said.
âHammerspace can help with that, as well as with managing these large sets of unstructured data that are typically distributed across multiple sites, and maybe stuck in different storage silos,â he said. âWhat we’ve really learned in the last year working with some large customers is there is an unmet need for a storage architecture that can provide the extreme parallel performance to feed GPUs. And that’s what we’re introducing now.â
When it comes to high-performance storage, enterprises can generally use one of two types, scale-out NAS and HPC (high performance computing) systems, Bassier said.
âNeither one meets all of their needs,â he said. âScale out NAS, although it’s widely deployed in the enterprise and meets all the enterprise feature sets, it simply can’t provide the throughput and low latency performance to feed these GPU clusters. And while HPC file systems have the performance, they’re incredibly complex to manage, and require proprietary software and unique networking requirements. Thatâs prevented HPC parallel file systems from really being a viable option in the enterprise.â
To power enterprise AI initiatives in GPU computing requires a new architecture, which Hammerspace is introducing with its Hyperscale NAS, which combines a parallel file system with enterprise NAS, Bassier said.
The fact that hyper-tuned, purpose-built supercomputer parallel file systems like Lustre, GPFS, and Weka exist is kind of proof that enterprise NAS, the traditional enterprise file environments, have not been able to address performance needs, said Hammerspace CEO David Flynn (pictured).
âTheir very existence is proof that there is a an opportunity and a missing piece,â he said. âHow do we get that level of performance, but with the ease of use, with the reliability, with the enterprise rich feature set of NAS because frankly, it won’t scale if it requires the kind of care and feeding that those exotic file systems take in the world of supercomputing. You have to have engineers on staff. You have to tweak and tune them. They fall apart and break easily.â
Hammerspace Hyperscale NAS sits on a separate standard server between GPU-based servers running standard Linux and the customersâ choice of storage from hard drives to NVMe flash storage, Bassier said. Hyperscale NAS leverages the NFS 4.2 client in the Linux kernel to do a couple of important functions including separate out the metadata and the data path and establish a direct data path between the Linux clients and the underlying storage.
âAny client can directly read and write data to any volume on any one of those storage systems,â he said. âEverything is standards-based. Everything is plug and play. In some ways, it looks like a parallel file system architecture with the benefits of extreme parallel performance and being able to scale performance linearly just by adding more GPU nodes or adding more storage nodes. But unlike those parallel file systems, everything’s standards-based. There’s no proprietary software. And you have your choice of any type of networking.â
No other storage vendors have built a hyperscale NAS system similar to what Hammerspace has done because building file systems is incredibly difficult, Flynn said.
âThere’s a risk in using new protocols,â he said. âIf you don’t have somebody like our [Chief Technology Officer] Trond Myklebust, who’s the kernel maintainer of this, it’s rather impossible to do what we do. Folks couldn’t do what we did because we basically we acquired the talent that is very limited in the industry. There’s only one Linux, and there’s only one maintainer of this part of Linux. And he’s our CTO.â
What Hyperscale NAS Means To Channel
Hammerspace Hyperscale NAS gives channel partners an all-new architecture to meet the needs of generative AI training and GPU computing, Bassier said.
âHammerspace is providing that software, but everything else can be the hardware of their choice,â he said. âAnd that is Hyperscale NAS in a nutshell, for the first time combining the performance of HPC file systems with standards-based simplicity and the reliability, availability, and serviceability of enterprise NAS.â
Hammerspace currently has hundreds of channel partners engaged with many enterprise and web-scale accounts, Flynn said.
âFor this technology, we have found that some of the cloud-focused partners, the cloud-first partners, are better than the ones that are focused on on-prem infrastructure,â he said. âThe reason for that is the cloud-first partners understand the challenges of getting file data into the cloud. You need folks who are actually trying to solve hybrid cloud or multi-data center things which require data orchestration, or partners who are trying to solve building hyper-scalable infrastructure HPC.â
Hammerspace has proven to be a good partner to the channel, said Dave Donald, CEO of Keeper Technology, an Ashburn, Va.-based solution provider which has worked with the company for over two years.
Donald told CRN that Hyperscale NAS is a big deal for his companyâs customer base.
âIt really demonstrates operations at scale,â he said. âThis is a key concern for our customer base, including the Depart of Defense and intel agencies. Having scalable front-end processing with NFS storage is exactly what our customers need. And none of our customers start out at the hyperscale level, but itâs important to know they can grow into it.â
Donald said one very large high data velocity client of his company is working with 30 petabytes to 40 petabytes of data in billions of files, and can really use Hammerspace Hyperscale NAS.
âSuch customers are using Quantum StorNext now,â he said. âBut Hyperscale NAS is a differentiator when they want to consolidate other NAS systems, and scale to the cloud. Now they can do so without migrating from StorNext. Hyperscale NAS can sit alongside Quantum StorNext. No vendor likes to see someone in their space. But for clients, itâs important to have multi-vendor implementations.â
Hammerspace, which was founded in 2018, has yet to be cash flow positive, and that is by design and now only temporary, Flynn said.
âTechnically speaking, if we wanted to grow organically and stay cashflow positive, we could,â he said. âBut it would come at the cost of the rate at which we can capture market share. So it’s always a balance between going into the negative to hire and ramp up the sales team in advance or growing organically. If we wanted to operate cashflow positive, we could today, but that would mean we don’t put as many new resources in play as we can when we have access to investor capital. ⦠By the end of the year, where we couldn’t go negative if we wanted to, we’re going to be having so much cash flow that it won’t require much more.â
Wanda Parisien is a computing expert who navigates the vast landscape of hardware and software. With a focus on computer technology, software development, and industry trends, Wanda delivers informative content, tutorials, and analyses to keep readers updated on the latest in the world of computing.