AMD will rely on advancements in high-bandwidth memory (HBM) in its bid to unseat Nvidia as the industry leader for making the components that power generative AI systems.
Building on the theme of processor-in-memory (PIM), Xilinx, which is owned by AMD, showcased its Virtex XCVU7P card, in which each FPGA had eight accelerator-in-memory (AiM) modules. The firm showcased this at OCP Summit 2023, alongside SK Hynix’s HBM3E memory unit, according to Serve the Home.
Essentially, by performing compute operations directly in memory, data won’t need to move between components on systems, meaning performance increases and the overall system becomes more energy efficient. Using PIM, with SK Hynix’s AiM, led to ten times shorter server latency, five times lower energy consumption, and half the costs in AI inference workloads.
The latest twist in the ongoing AI arms race
Nvidia and AMD make most of the best GPUs between them, and one may assume that efforts to improve the quality of these components are key to improving AI performance. But it’s actually by tinkering with the relationship between compute and memory do these firms see there are huge advantages to be made in power and efficiency.
Nvidia is also racing ahead with its own plans to incorporate HBM technology into its line of GPUs, including the A100, H100 and GH200, which are among the best graphics cards out there. It struck a deal with Samsung last month for incorporate its HBM3 memory technology into its GPUs, for example, and will likely extend this to include the new HBM3e units.
PIM has been something several companies have pursued in recent months. Samsung, for example, showcased its processing-near-memory (PNM) in September. The CXL-PNM module is a 512GB card with up to 1.1TB/s of bandwidth.
This follows a prototype for an HBM-PIM card, which was made in collaboration with AMD. The addition of such a card boosted performance by 2.6% while boosting energy efficiency by 2.7% against existing GPU accelerators.
More from TechRadar Pro
Denial of responsibility! TechCodex is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, and all materials to their authors. For any complaint, please reach us at – [email protected]. We will take necessary action within 24 hours.
Alex Smith is a writer and editor with over 10 years of experience. He has written extensively on a variety of topics, including technology, business, and personal finance. His work has been published in a number of magazines and newspapers, and he is also the author of two books. Alex is passionate about helping people learn and grow, and he believes that writing is a powerful tool for communication and understanding.
